Posts Tagged ‘statistics’

ABOUT MY BOOK TFO&&&

August 21, 2021

UPDATES ON A LIVING BOOK

This post is so titled because it stands for “THINKING FOR OURSELVES-ADDITIONS” where “Thinking for Ourselves” refers to my book with that title originally published in 2020 and updated in 2021 (For details see elsewhere under ‘My Books’ Category or under Tags on ‘Thinking’.) But from now on I want the book to become live, so that it can be continually updated here on line. Here you will find Exercises with Answers, corrections, images, calculations, supporting data, more detailed and improved arguments, readers comments with my responses to what is intended to become what I call ‘A LIVING BOOK’.See at the bottom of this Post how to make such Comments.

All the additions are shown below, mostly under a Chapter number and page number in the paperback book, version 2021.

GENERAL

EXERCISES WITH ANSWERS can be found at;

at the following url:

https://mjdisney.org/wp-content/uploads/2020/10/tfoexans.pdf

AUTHOR’S MIXED FEELINGS ABOUT TFO (as of Aug 21)

         I finished the book 3 years ago with the surprising but triumphal discovery of Categorical Inference – which connects the whole scheme for Common Sense Thinking so naturally and necessarily with Animal Thinking and Evolution. And IF it’s right it could change the world.

         At that point I sometimes get struck with what  I believe they call ‘Imposter Syndrome’– how could little me have unearthed a powerful scheme entirely missed by giants such as Francis Bacon, Isaac Newton and Albert Einstein? It doesn’t seem likely does it?

         But then I look at some of its manifest achievements such as:

  • Explaining Humankind’s dramatic leap in mental capability around 1000 BC.
  • Its unique mechanism for balancing conflicting evidence, as illustrated with its success with Hidden Galaxies.
  • A first transparent and convincing explanation for Ockham’s Razor.
  • Its powerful mechanism (PAW) for dealing with Systematic Errors, which have kept us back so many times  for so long.
  • It’s perfect dovetailing into Animal Thinking and Darwinian Evolution.
  • The multiple new insights which spring from it – see this blog and my other book “History of the Brits’ [HOB ch.5]. For instance  it comes up with the keys to human Progress, what I call ‘The Seven Pillars of Wisdom’ .

So then I am reassured. But, but……Why little me again? All I can say to myself, and to potential readers is :

 “It was bloody minded doggedness more than anything else. I started out with the modest ambition to find out what I believed was already known  –  the Scientific Method, only to find to my surprise that it was not, but that it probably had something to do with Common Sense, but that hadn’t been defined either. So I asked myself a different question: ‘How could animals think?’ and thereafter progress became relatively rapid  because now I could entirely  ignore Philosophy, Mathematics and Religion.

So I didn’t have to be a genius, which I definitely am not. And one doesn’t have to be a genius to make a great discovery. Look at Darwin – he spent the first  30 years  of his idle life slaughtering wild creatures for fun. Basically he was an illiterate lout – but he stumbled upon the greatest scientific discovery of all because he happened to be in the right place at the right time – the Galapagos Islands  in 1838. But he was only there because his exasperated father had sent him out there as a punishment, saying “You wouldn’t even make a decent rat-catcher.” Indeed there’s little evidence of ‘genius’ in science more generally [ See Chapter 3 of TFO to see how great discoveries have been made in history] – so even if I’m not a genius , TFO  could still be right.”

CORRECTIONS.

As of 21/8/21 there are only 2 because I have just made two dozen corrections to the original paperback edition.They are

P 302: replace ‘Sherman’ with ‘Pershing’.

P 456, line 7: replace 13 with 23.

But the most important of those for purchasers of the older editions are at:

https://mjdisney.org/wp-content/uploads/2020/10/tfocorrsjun20.docx

ADDITIONS AND MODIFICATIONS BY CHAPTER

CHAPTER 1 (‘Can we learn to think better?’) p 15

CHAPTER 2 (‘Different kinds of Thinking’) p25

CHAPTER 3 (‘How do Scientists Think?) p46

CHAPTER 4 (‘Natural Thinking and Bayes’ Rule’) p95

There are several Posts on the fascinating subject of ‘Galaxies’ , including ‘Hidden Galaxies’, in the ‘Astronomy’ Category here, with many images.

CHAPTER 5 (‘The Detective’s Equation’) p132

CHAPTER 6 (‘Numbers and Thinking’) p154

CHAPTER 7 (‘Woolly Thinking and Ockham’s Razor’) p170

There are several posts here on ‘Big Bang Cosmology’ — which I use as a case study in dodgy thinking, under the ‘Astronomy’ Category’.

CHAPTER 8 (‘Common Sense’) p198

CHAPTER 9 (‘Error Analysis’) p236

CHAPTER 10 (‘Systematic Errors, The Elephants in the Room’) p268

CHAPTER 11 (‘Statistics – or Terror Analysis’) p294

Statisticians turned themselves from humble clerks into a dogmatic priesthood based on several misunderstandings, on their part. They need to be put firmly back on their stools. Having spent 30 years trying to teach Statistics at university, I gradually came to realise that the profession has got itself hopelessly lost in the No-man’s land between Induction and Deduction. Look what confusing advice they have given to the government over the Covid pandemic, They’re not scientists, they’re mostly priests, who hide behind higher mathematics when they are challenged. See Post “Statistics: exposed at last” under ‘Thinking’ Category.

CHAPTER 12 (‘Persuasion’) p342 t

CHAPTER 13 (‘Poor Thinking’) p357

CHAPTER 14 (‘The Extraordinary History of Thinking’) p407

CHAPTER 15 (‘The Peculiarities of Science’) p451

In Sect (15:12) ‘What about Mathematics’ I only gave some modest examples because I didn’t want to frighten off non-mathematical readers but on this site its maybe worth drawing attention to some more spectacular examples. For instance on pp 471-472 I then failed to recognise the full and dramatic implications of mathematics when applied to immigration: basically because immigrants arrive every year, while children arrive only a couple of times or so in a female’s life, immigration is no less than 160 times more significant than natural birthrate when it comes to population increase! Thus immigration into the UK at present is equivalent to 3 British mothers out of 4 having an extra child! If you don’t believe me, and I found it very difficult to believe it myself, you should consult the url:

https://mjdisney.org/wp-content/uploads/2020/10/immigmaths-copy.pdf

Then the modern world, including radio, broadcasting, television, Relativity, satellites, mobile phones, the internet…. were all implicit in a set of equations derived by two Brits in the 19th century, James Clerk Maxwell and Oliver Heaviside. You don’t have to understand the equations in detail but one can certainly admire a human artefact millions of times more momentous than either The Rosetta Stone or Tutenkamun’s Tomb. See:

https://mjdisney.org/wp-content/uploads/2021/04/MAXWELLSEQUATIONS.pdf

CHAPTER 16 (‘Consequences and the Ascent of Mankind’) p476

On p486 there is a very brief discussion of Time. If you want to see a deeper discussion of a profound topic see the Post “WHAT IS TIME?” under the Category ‘Thinking’. Those who want to look deeper into TIME can look at the Post ‘MAXWELL’S EQUATIONS’ (under ‘Thinking’ Category )which explains why Relativity has changed our view that Time is absolute; it’s not, according to physicists. Even so Time is still a great mystery; there seem to be several different kinds of time. all mistakenly labelled with the same four lettered word.

GLOSSARY p513

REFERENCES p526

APPENDICES pp 547 to 604

INDEX p612

Tags:, , , , , ,
Posted in My Books | Leave a Comment »

STATISTICS: EXPOSED AT LAST

October 28, 2020

Why can’t all those professors of medical statistics give governments sound, or at least consistent advice about the Covid pandemic? I am sorry to say it is because Statistics is deeply flawed in its very foundations.

I taught Statistics at university for 30 years, at first with zeal, then with growing puzzlement, finally with disillusion. Towards the end I couldn’t bring myself to teach the students “Hypothesis Testing” — the central ambition of the whole enterprise.

Collecting data is fine: the more the merrier. Analysing that data in search of useful information is essential. But turning information into a wise recommendation for action turns out to be fiendishly difficult. Why? Because the real world is far more complicated than the artificial world of card-playing, from which Probability Theory evolved long ago. And at its heart Statistics is the application of Probability Theory to real situations — like outbreaks of Corona Virus.

The problem is this: in a card game there are 52 cards so that all possible combinations of cards can be imagined — and calculated. But in the real world the combinations are infinite and so incalculable. Faced with this absolute road-block professional statisticians try to fudge their way round it by “making approximations” that is to say by pretending that arcane mathematical results drawn from card-play still apply approximately to a world of awkward germs, and bloody awkward people.

But mostly they don’t, We scientists know that there are such things in the real world as Systematic Errors, that is to say misconceptions which no amount calculation, or approximation, can ever surmount. Take one example: earthquake waves travel through the Earth arguing that it must be rigid. The great guru of geophysics at Cambridge University, Harold Jeffreys ,used it to maintain that therefore Continental Drift must be impossible — holding back the subject for 50 years. But he was making a Systematic error in assuming that because rock was rigid on a timescale of seconds (waves) it must likewise be rigid on a timescale of millions of years. Had he gone for a walk on a beach in say Pembrokeshire, and seen the dramatic folding of the rocks, he would have realised he was talking nonsense.

Ironically, in his case, one ghastly mistake led to another. Sir Harold, as he became, morphed alas into an even greater guru — on the subject of The Scientific Method, and founder of the school of “Objective Bayesian Statistics” — which is highly fashionable in academic circles today. But wrong, as Henri Poincare’ argued in the nineteenth century.

Once one knows what to look for (but only then) it’s not difficult to spot the flaws in the all-too-many (to be healthy,) text-books of Statistics, . For instance:

a) They pretend that Systematic Errors do not exist.

b) They use mathematical notions such as “The Normal Distribution”, and misapply them to the real world, justifying what they are doing by appealing to the ‘Central Limit Theorem’ — which most appear not to understand.

c) They disagree violently among themselves, and divide into many schools — which explains those all-too-numerous textbooks on the subject

d) They hardly ever come up with sound insights which couldn’t have been reached anyway using plain Common Sense (e.g. Smoking and Lung Cancer).

WHAT SHOULD WE DO ABOUT IT?

It’s all very well criticising Statistics, but what are we to do about it in the present crisis? I suggest:

1) We should listen to Statistician’s advice, but grant it only the same degree of respect we would accord to Economist’s predictions. Neither are remotely scientists.

2) We should disregard all academic titles like Professor or Doctor because they have become meaningless nowadays. Shameless grade inflation in British academe is a scam for demanding unearned respect from the public and unearned rents from the young and vulnerable. No one fails a doctorate nowadays whilst you can now become a professor by filling in a form and have your colleagues (who all want to become ‘Professors’ too of course) countersign it.

3) We should all take a crash course in Common Sense Thinking so that we could do that very needful Hypothesis Testing for ourselves, but soundly.[I have just produced a very short book (60 pages) entitled Common Sense Thinking whose details are in the ‘my books’ Category elsewhere in this blog.]

All these matters are covered in considerable detail in Thinking For Ourselves (described elsewhere on this site. It supplies many worked examples and some exercises with answers.) Practically anyone literate should be able to understand it while technical types will benefit from learning why they don’t need to learn Statistics.

Meanwhile there are two addenda attached to this post. A more technical survey with references, on the weaknesses of Statistics at:

https://mjdisney.org/wp-content/uploads/2020/10/statsdead.pdf

And a shortish extract from my book Thinking for Ourselves explaining why we have all, me especially, struggled so long with this tendentious and difficult subject, at :

https://mjdisney.org/wp-content/uploads/2020/10/apolstat.pdf

Tags:, , , ,
Posted in Thinking | Leave a Comment »

ABOUT MY BOOK ‘THINKING FOR OURSELVES’.

September 29, 2020

Why not learn to think like Einstein? He attributed his success entirely to common sense. But what is Common Sense Thinking (CST) ? They don’t tell us in class because scholars don’t know how it works; it’s buried too deep inside us. So the author, who is a Space astronomer, has spent a lifetime unearthing it. Common Sense turns out to have nothing whatever to do with IQ, or with passing exams; it has to do with Curiosity, with the deliberate gathering of evidence, with a willingness to change one’s mind, and with the patience to wait until the Odds look decisive. But these common sense thinking skills are the very skills of the detective as well as the scientist; we can all learn them if we want to, whatever our educational attainments.

 To think efficiently for ourselves we require the right information and the right tools. At last the Internet is bringing an increasing flood of the kind of information we need to make crucial decisions for ourselves, and not have to rely any longer on expensive and often unreliable experts. But do we have the right mental tools? We do and we don’t. We do because Nature has embedded deep within us the sophisticated mental mechanisms  needed to survive in the perilous Stone-age out of which we evolved. We don’t, in the sense that we don’t know how those tools work, and so we cannot adapt them for best use in a modern world full of culture and technology. The author has unearthed those ingenious and previously unrecognised tools which Nature evolved  for dealing with conflicting evidence, for discounting bad clues, and for weighing up all the pros and cons.

The aim of this book is to open nature’s tool-bag and demonstrate her mental tools working one by one. The reader will end up equipped with an indispensable boy-scout’s penknife for the thinking mind.

         The book is aimed at two kinds of readers: those who feel they are educated – and those who feel they are not. The educated ones will discover that they have not learned Common Sense Thinking at school or university, and badly need to – while the uneducated ones will find that dropping out of education is no handicap if they now decide that they do want to think seriously and well. Anyone aged 14 or over could master it.

The approach is to work through numerous stories and examples, some serious, some light-hearted, all hopefully interesting and relevant. The book is really a series of detective stories in which we learn to work out rough odds on the various suspects (ideas) being guilty.

There are plenty of exercises for those who want to practice, with worked answers available free on the Internet.

Topics covered include:

  1. Different kinds of Thinking
  2. How scientists think and decide
  3. How to weight clues
  4. How to reckon the Odds
  5. Woolly thinking
  6. The Elephants in the room (Systematic Errors)
  7. Animal Intelligence
  8. Numbers and Errors in Thinking
  9. Why we don’t need Statistics any more.
  10. Common mistakes in Thinking and Deciding
  11. The extraordinary history of Thinking
  12. What is Science, and what is not?
  13. The meteoric ascent of mankind. What on Earth happened?

There are 16 chapters. Two hours devoted to each should equip the reader with the tools needed to go about tackling the hardest mental problems in life.

The complete Table of contents can be seen here:

https://mjdisney.org/wp-content/uploads/2021/07/TFOtoc-copy.pdf

The paperback book came out on Amazon early in 2020, has 600+ pages, is about 190 kw. in length and is priced at £14.50 with roughly equivalent prices in foreign markets. You can see much more by using the ‘Look Inside’ feature on Amazon. The Exercises, with answers, are on the web and elsewhere in this blog at

:https://mjdisney.org/wp-content/uploads/2020/11/TFOEXERCISES140918.pdf

Corrections as of June 2020 can be found at:

https://mjdisney.org/wp-content/uploads/2020/10/tfocorrsjun20.docx

Tags:, , , , , ,
Posted in My Books | 1 Comment »