Posts Tagged ‘hypothesis testing’

PARTICLE THEORISTS POISON COSMOLOGY

November 4, 2021

Despite three decades of effort and tens of millions of dollars spent on accelerators and their like, it looks as if Particle Physics is coming to a sad end. No new particles beyond those such as the Higgs Boson proposed 50 years ago, and in particular none of those Supersymmetric particles which theorists had hoped would explain that greatest of all scientific mysteries — Dark Matter. Of course there will now be cries for more money and even larger machines, after all the Large Hadron Collider at CERN in Geneva is a mere 27 kilometres in circumference. But wait! Perhaps there is something far more interesting and fundamental at work: Perhaps Particle theorists have misled themselves, and everybody else , through neglecting a philosophical principle at least a thousand years old called ‘Ockham’s Razor’ (OR), named after a mediaeval monk called Friar Ockham.

At the heart of the Scientific Method is the business of Hypothesis Testing, which is where OR comes in. It states “Always prefer the simplest hypothesis first” and that, I suggest, is where Particle Theorists went so horribly wrong. Their “Standard Model” — as they call it, is fiendishly complex — what with its Quarks, Gluons, ‘Asymptotic Freedom’ and so on and so on. How do we measure complexity in Science? By the number of ‘Free Parameters’ (FPs) needed to describe a theory. One way you can think of them is to say they are arbitrary numbers brought into a theory to force it to fit the experimental data. A ‘good theory’ doesn’t need many FPs because it fits the experimental world naturally(for instance Newton’s very successful Theory of Gravitation has only 2 FP s) The so called Standard Model of Particle Physics needs no less than 18 FP s which has always suggested that it is an ugly and unnatural construct. It should be no surprise then to find now that it actually looks to be wrong.

So why did theorists construct such an ugly model in the first place, mostly back in the 1960’s and 70’s? Probably because they didn’t understand just how fundamental OR is. And there’s some excuse for them — because the Philosophers of Science, the self-appointed arbiters of the Scientific Method, didn’t understand OR themselves. Even Einstein, who relied on it extensively, waffled about some plastic ‘God’.

As I see it Hypothesis Testing works like this. You have some data-points, with error bars of course, and you have your hypothesis which generates a smooth curve which you must try to fit through those points. If there are lot of points the Odds on your hypothetical curve fitting them all by chance must be small. So if it does so fit then the Odds are that the hypothesis is probably right. If it doesn’t fit then you can always complexify your hypothesis ,so twisting your hypothetical curve until it does fit. But you can see that’s not a very convincing way to proceed, because eventually you are always going to force a fit. In that case the Odds in favour of it being actually right vanish. And that, I would suggest is what happened to Particle Theory, starting half a century a century ago.

There’s nothing fundamentally wrong with modifying a theory to fit the facts, after all that’s how science progresses. But you have to be very frugal in doing so. Only introduce a new concept (FP) into your theory if it fits at the very least two more data points than its simpler predecessor. And that’s hard to do, but it won’t degrade the Odds on it being right. But if it only fits one more data point the Odds will generally degrade dramatically. And that’s what Particle Physicists were tempted to do; making names for themselves at the expense of undermining the Odds on their so called ‘Standard Model’ theory. And that’s why almost nobody believes in their theory anymore. It’s as if they’d undermined their currency by printing too many notes. It works for a while — then collapses!

I am not a Particle Physicist, thank God, I am an Astrophysicist. And what worries me is that those same Particle theorists have dragged their own dodgy practices into our subject, with predictably unhealthy consequences. Take “Dark Energy”, an entirely artificial concept dragged into Cosmology by a particle theorist called Ed Turner from the Fermi Lab (and the University of Chicago). Now astronomers are raising hundreds of millions of dollars to chase this fantasy around the cosmos when there’s no justification for doing so, none at all. It was a thoughtless quick-fix extra Free Parameter to fit the apparent acceleration of Cosmic Expansion inferred from Supernova measurements in 1998. Had its introduction explained TWO or more discrepancies between theory and observation we might have welcomed it in. But it didn’t. So it should never have been introduced in the first place. Never!

PS Actually the situation is far worse than I am implying because the bloody particle theorists who have undermined their own subject actually introduced two more unnecessary FPs into Cosmology before Dark Energy: ‘Inflation’ to cure Isotropy and nothing else, and ‘Dark Matter’ to fix the Cosmic chemical abundances. We need to throw them out too.

So where do we go from here? Cosmology should chuck out Dark Energy, Inflation and Dark Matter and start again without them. As for Particle Physics I suspect that they may have to go back 50 years and try to reconstruct a more parsimonious theory of particle interactions than the ‘Standard Model based on quarks and gluons. In his wonderful book ‘Constructing Quarks’ Andrew Pickering (Univ. Chicago Press 1981) suggested that that theory was a social construct anyway, the product of trendy acclamation, rather than sober assessment.

More generally all of us need to understand the process of Hypothesis Testing on which the modern world of ideas is entirely built. Because if that isn’t sound ,God help us all.

For much more on Ockham’s Razor see our post “Fuzzy Thinking and Ockham’s Razor’ under the ‘Thinking’ category here on our blog. For a detailed explanation of Ockham’s Razor and why it works go to the url:

https://mjdisney.org/wp-content/uploads/2021/11/AMSCI4-copy-1.pdf

But if you want to go into the whole business of Common Sense Thinking (CST) , of which Hypothesis Testing is only a part, try my book “Thinking for Ourselves” publ Amazon (2020) which is described in the ‘My Books ‘ Category on this site.

‘t

TFO&&&

August 21, 2021

UPDATES ON A LIVING BOOK

This post is so titled because it stands for “THINKING FOR OURSELVES-ADDITIONS” where “Thinking for Ourselves” refers to my book with that title originally published in 2020 and updated in 2021 (For details see elsewhere under ‘My Books’ Category or under Tags on ‘Thinking’.) But from now on I want the book to become live, so that it can be continually updated here on line. Here you will find Exercises with Answers, corrections, images, calculations, supporting data, more detailed and improved arguments, readers comments with my responses to what is intended to become what I call ‘A LIVING BOOK’.See at the bottom of this Post how to make such Comments.

All the additions are shown below, mostly under a Chapter number and page number in the paperback book, version 2021.

GENERAL

EXERCISES WITH ANSWERS can be found at;

at the following url:

https://mjdisney.org/wp-content/uploads/2020/10/tfoexans.pdf

AUTHOR’S MIXED FEELINGS ABOUT TFO (as of Aug 21)

         I finished the book 3 years ago with the surprising but triumphal discovery of Categorical Inference – which connects the whole scheme for Common Sense Thinking so naturally and necessarily with Animal Thinking and Evolution. And IF it’s right it could change the world.

         At that point I sometimes get struck with what  I believe they call ‘Imposter Syndrome’– how could little me have unearthed a powerful scheme entirely missed by giants such as Francis Bacon, Isaac Newton and Albert Einstein? It doesn’t seem likely does it?

         But then I look at some of its manifest achievements such as:

  • Explaining Humankind’s dramatic leap in mental capability around 1000 BC.
  • Its unique mechanism for balancing conflicting evidence, as illustrated with its success with Hidden Galaxies.
  • A first transparent and convincing explanation for Ockham’s Razor.
  • Its powerful mechanism (PAW) for dealing with Systematic Errors, which have kept us back so many times  for so long.
  • It’s perfect dovetailing into Animal Thinking and Darwinian Evolution.
  • The multiple new insights which spring from it – see this blog and my other book “History of the Brits’ [HOB ch.5]. For instance  it comes up with the keys to human Progress, what I call ‘The Seven Pillars of Wisdom’ .

So then I am reassured. But, but……Why little me again? All I can say to myself, and to potential readers is :

 “It was bloody minded doggedness more than anything else. I started out with the modest ambition to find out what I believed was already known  –  the Scientific Method, only to find to my surprise that it was not, but that it probably had something to do with Common Sense, but that hadn’t been defined either. So I asked myself a different question: ‘How could animals think?’ and thereafter progress became relatively rapid  because now I could entirely  ignore Philosophy, Mathematics and Religion.

So I didn’t have to be a genius, which I definitely am not. And one doesn’t have to be a genius to make a great discovery. Look at Darwin – he spent the first  30 years  of his idle life slaughtering wild creatures for fun. Basically he was an illiterate lout – but he stumbled upon the greatest scientific discovery of all because he happened to be in the right place at the right time – the Galapagos Islands  in 1838. But he was only there because his exasperated father had sent him out there as a punishment, saying “You wouldn’t even make a decent rat-catcher.” Indeed there’s little evidence of ‘genius’ in science more generally [ See Chapter 3 of TFO to see how great discoveries have been made in history] – so even if I’m not a genius , TFO  could still be right.”

CORRECTIONS.

As of 21/8/21 there are only 2 because I have just made two dozen corrections to the original paperback edition.They are

P 302: replace ‘Sherman’ with ‘Pershing’.

P 456, line 7: replace 13 with 23.

But the most important of those for purchasers of the older editions are at:

https://mjdisney.org/wp-content/uploads/2020/10/tfocorrsjun20.docx

ADDITIONS AND MODIFICATIONS BY CHAPTER

CHAPTER 1 (‘Can we learn to think better?’) p 15

CHAPTER 2 (‘Different kinds of Thinking’) p25

CHAPTER 3 (‘How do Scientists Think?) p46

CHAPTER 4 (‘Natural Thinking and Bayes’ Rule’) p95

There are several Posts on the fascinating subject of ‘Galaxies’ , including ‘Hidden Galaxies’, in the ‘Astronomy’ Category here, with many images.

CHAPTER 5 (‘The Detective’s Equation’) p132

CHAPTER 6 (‘Numbers and Thinking’) p154

CHAPTER 7 (‘Woolly Thinking and Ockham’s Razor’) p170

There are several posts here on ‘Big Bang Cosmology’ — which I use as a case study in dodgy thinking, under the ‘Astronomy’ Category’.

CHAPTER 8 (‘Common Sense’) p198

CHAPTER 9 (‘Error Analysis’) p236

CHAPTER 10 (‘Systematic Errors, The Elephants in the Room’) p268

CHAPTER 11 (‘Statistics – or Terror Analysis’) p294

Statisticians turned themselves from humble clerks into a dogmatic priesthood based on several misunderstandings, on their part. They need to be put firmly back on their stools. Having spent 30 years trying to teach Statistics at university, I gradually came to realise that the profession has got itself hopelessly lost in the No-man’s land between Induction and Deduction. Look what confusing advice they have given to the government over the Covid pandemic, They’re not scientists, they’re mostly priests, who hide behind higher mathematics when they are challenged. See Post “Statistics: exposed at last” under ‘Thinking’ Category.

CHAPTER 12 (‘Persuasion’) p342 t

CHAPTER 13 (‘Poor Thinking’) p357

CHAPTER 14 (‘The Extraordinary History of Thinking’) p407

CHAPTER 15 (‘The Peculiarities of Science’) p451

In Sect (15:12) ‘What about Mathematics’ I only gave some modest examples because I didn’t want to frighten off non-mathematical readers but on this site its maybe worth drawing attention to some more spectacular examples. For instance on pp 471-472 I then failed to recognise the full and dramatic implications of mathematics when applied to immigration: basically because immigrants arrive every year, while children arrive only a couple of times or so in a female’s life, immigration is no less than 160 times more significant than natural birthrate when it comes to population increase! Thus immigration into the UK at present is equivalent to 3 British mothers out of 4 having an extra child! If you don’t believe me, and I found it very difficult to believe it myself, you should consult the url:

https://mjdisney.org/wp-content/uploads/2020/10/immigmaths-copy.pdf

Then the modern world, including radio, broadcasting, television, Relativity, satellites, mobile phones, the internet…. were all implicit in a set of equations derived by two Brits in the 19th century, James Clerk Maxwell and Oliver Heaviside. You don’t have to understand the equations in detail but one can certainly admire a human artefact millions of times more momentous than either The Rosetta Stone or Tutenkamun’s Tomb. See:

https://mjdisney.org/wp-content/uploads/2021/04/MAXWELLSEQUATIONS.pdf

CHAPTER 16 (‘Consequences and the Ascent of Mankind’) p476

On p486 there is a very brief discussion of Time. If you want to see a deeper discussion of a profound topic see the Post “WHAT IS TIME?” under the Category ‘Thinking’. Those who want to look deeper into TIME can look at the Post ‘MAXWELL’S EQUATIONS’ (under ‘Thinking’ Category )which explains why Relativity has changed our view that Time is absolute; it’s not, according to physicists. Even so Time is still a great mystery; there seem to be several different kinds of time. all mistakenly labelled with the same four lettered word.

GLOSSARY p513

REFERENCES p526

APPENDICES pp 547 to 604

INDEX p612

COMMON SENSE & GOD

June 25, 2021

Common Sense Thinking needs some tool to discriminate between Truth and Falsehood, or more generally between sound hypotheses and unsound ones. ‘Hypothesis Testing’ ,as it is called, lives at the very heart of Science, Philosophy and Common Sense. As we now know it works by examining the various consequences C1 ,C2, …generated by that hypothesis to see whether they can be observed in practice. If they can be observed that improves our Odds O(H) on the hypothesis being true, if they cannot that reduces our Odds on it. But if the hypothesis generates no consequences we cannot test it , and so can say nothing about it one way or the other. That’s “Bayes’ Rule” which goes back at least as far as 1763 and probably much much further.

Take the hypothesis “God exists”. What consequences does it have? Once upon a time it was argued that the design of the natural world was so miraculous, perfect and improbable that it could only have been conceived by an Intelligent Creator. For instance how else to explain the spectacular plumage of the Rainbow Lorikeet ? [Click on the urls below to see their magnificence]

https://macaulaylibrary.org/asset/349487411/embed/800

https://macaulaylibrary.org/asset/199098091/embed/640

This was the very convincing “Argument by Design”, almost impossible to counter at the time. But in 1858 along came Darwin and Wallace who independently came up with the alternative hypothesis of “Evolution by Natural Selection”. As the Bishop of Worcester’s wife said of it: ” Dear me; let us hope it is not true. But if it is true , we must hope it doesn’t become widely known.”

The general point is that Inconsequential Hypotheses are hardly worth considering because there is no way of assessing their veracity, whereas Consequential Hypotheses are open to verification, at least in terms of their probability(Odds). Thus Evolution has subsequently been detected in, for instance, bacteria under stress , while I am not aware of any consequence for the existence of God which could be tested .

That’s not to say one can’t go on believing in a god — it’s just that the most consequential evidence on his/her hypothetical existence has an alternative, and partially verified explanation, even if it cannot be absolutely categorical.

Then there’s another important philosophical principle that can be brought to bear:Ockham’s Razor — “Always prefer the simpler hypothesis, because its more likely to be right’ [see my Post ‘Fuzzy Thinking & Common Sense’] .The problem with the God hypothesis is that there are so many inconsistent versions of it (4,000 known religions including 20 with a world-wide spread, according to Wikipedia).

Clever people have wasted a lot of their lives worrying about Inconsequential hypotheses — for instance the existence of Free Will [see Post. ‘Free Will and Common Sense] , or in the case of Mathematicians whether their subject was invented of discovered. It doesn’t matter. It’s Inconsequential.

LYING ABOUT HISTORY

May 7, 2021

It suits a lot of people’s private agendas to claim that the British Empire was an evil one. But was it? When I went to work in India I was confronted by a six foot, broad shouldered American in a sari who bellowed: ” You Briddish ought to be ashamed of what you did in India, cutting off the thumbs of all the weavers in Madras to protect your Lanka Shire cotton industry!” Naturally horrified I looked into the alleged crime.

I hope readers won’t be surprised to find it was nonsense, a canard put about by the Indian Congress party to win an election. But the point of this Post is to warn readers to be very careful before acting or voting on the basis of emotive historical narratives which could easily be lies, and to suggest a way to check them.

This is no small matter The Second World War was started by ‘Ludendorff’s Lie’ which Hitler and many other Germans chose to believe, while Scotland could vote to leave the United Kingdom because of a false historical narrative put about by The Scottish Nationalist Party.

As a Space scientist is wasn’t wise to take complex decisions on the basis of emotive tweets, but how were we to take them, winnowing the grain away from the chaff?

I have tried to apply the ‘Scientific Method of History to the hypothesis ‘Scotland would have fared better outside the United Kingdom’ and come up with odds of 250 to 1 against. see:

https://mjdisney.org/wp-content/uploads/2021/05/SCIMETHHIST.pdf

and have recently published ” HISTORY OF THE BRITS (From a scientists point of view)” 2020,Amazon, paperback £10, in which these ideas and techniques are discussed in a much wider context.The above url is extracted from it .

I would go far as to say that false history is very often a murder weapon far more deadly than shells or mines because it can stir up whole populations to set upon one another. One third of the German population is believed to have died during the religious Thirty Years War. At the present day Putin, Chi, Modi…… are all trying to use it to make their ‘countries great again’ with consequences which might be quite dreadful, even fatal to all of us. If you don’t believe me find out about ‘The Protocols of the Elders of Zion‘ , a historical falsehood which lead to tens of millions of deaths because decent people didn’t bother to check its veracity.

BIG BANG COSMOLOGY IS WRONG

February 20, 2021

If you wanted to know what to think of some fringe activity such as Spiritualism or Water Divining I doubt one would consult a professional first. After all you know that they must be committed. But what if you wanted to evaluate Big Bang Cosmology? Once again you can’t turn to the biased professionals, although they might argue that unless you are a professional you cannot know enough about the subject to take an informed position. But of course that is a dangerous stance to adopt, and the way in which priesthoods germinate, metastasize and sometimes come to dominate the world. They become immune to criticism because they will admit none but believers as critics. They become malignant, if not necessarily malign.

So what is the wise outsider to do? I would suggest they might consult those whose business it is to know much about the arcane subject- material in question without having to become paid exponents themselves. Cosmology for instance is in practice largely extra-galactic astronomy, so why not consult an extra-galactic astronomer who doesn’t claim to be a Cosmologist? Such an astronomer will know most of the technical arguments – without having to commit to them. That is where I stand with regard to Big Bang Cosmology, or BBC. My passion lies in Galaxies, the largest discrete objects in the Universe. But as they seem to be almost as old as the Cosmos, their origin must be entangled in the early evolution of the Universe itself, so I cannot ignore Cosmology, any more than Cosmology can ignore Galaxies which, so far as we know, comprise most of everything we can actually observe. And as visible galaxies exist in hundreds of thousands of millions, and can be observed in some detail nowadays, they should tell us more about Cosmology than vice-versa. And here is the rub: in BBC galaxies shouldn’t exist. As has been known for fifty years they would have been torn apart by radiation pressure before they could even form. So a desperate fix called CDM, standing for ‘Cold Dark Matter’ was adopted to try and repair the awful hole in the story. But despite many efforts to find out what it is, no one has been able to find any trace of CDM in half a century. Umm.

And there is another stark confrontation between galaxies and Cosmology. In an expanding Universe – the core assumption of BBC – distant galaxies should be totally invisible because of the ‘Tolman Effect’, a test for Expansion, which goes back to 1930. Then we didn’t possess the the telescopes to test it, but now, in the Hubble Space Telescope, we certainly do. And what do we find? That the observed Universe fails – and fails most dramatically – as you can see for yourself. Look at the figure:

The Hubble Ultra Deep Field, the deepest image of the Universe , taken with the Hubble Space Telescope, which I helped to design. All those tiny dots are actually high redshift galaxies a long long way away. If the Universe is really expanding we shouldn’t be able to see them. But……..

You can see it’s covered all over with a rash of tiny high-redshift galaxies – which simply shouldn’t be there, not if the Universe is expanding. If it was they ought to look no less than ten thousand times dimmer than they appear to be. Surely this is something BB Cosmologists ought to acknowledge? But they don’t. It’s been known since 1993 when we first fixed the telescope’s aberrated mirror, but ever since there has been a conspiracy of silence about the matter. As a designer of that and earlier cameras, I was staggered when I first saw the earliest deep Hubble images because I’d been assured by Cosmologists that Hubble would never see high-redshift galaxies. Yet there they were. There they are in their hundreds and thousands.

The only precedent I can think of occurred back in 1610 when Galileo pointed his little spyglass at Venus and found it to be a brilliant crescent pointing towards the Sun. The two-thousand-year-old Geocentric picture of the Cosmos was quite wrong, All the Planets, including the Earth, must be orbiting the Sun.

But what happened? Galileo was eventually seized by the Inquisition, forced to retract, and then imprisoned for life.

We don’t have an Inquisition any more but we do have Priests of a different kind: experts whose livelihoods, reputations and ambitions enforce adherence to a certain dogma. It’s not easy when you are an elderly, respected professor of Cosmology, with several books and hundreds of peer-reviewed papers behind you, to admit that you have been wasting yours, and everybody else’s time. And if the old won’t recant, why should the young, who still have their reputations and their livings to make? There is no Inquisition it is true but there are, in a highly competitive profession, appointment and tenure committees to please, journal-referees to propitiate. Brave myths to the contrary, academic success is based above all on allegiance to the Common Book of Prayer.

I know it will be hard for outsiders to believe in such conformity, I certainly wouldn’t have believed in it myself if I hadn’t experienced it at first hand, and to some extent colluded rather shamefully in it myself. Yes I went to conferences and politely pointed out the anomalies facing us in the sky. I even published papers in elite journals like ‘Nature’ demonstrating that real galaxies couldn’t possibly have formed in the CDM manner proclaimed by cosmological theorists. But when nobody responded, shouldn’t I have bellowed and trumpeted my doubts?

Honestly I should. But two things held me back; lack of self -confidence for one. Cosmology is a huge and complex subject mired in the hardest Mathematics and Physics – and perhaps I’d missed something – which the experts had not? Then again it wasn’t my real love. If I acquired a reputation as a madman I wouldn’t get the observing time on top telescopes I absolutely needed to do my Galaxy research. Many of us subscribe to popular myths, knowing them to be untrue. One well-known colleague told me that when he is applying for observing time he always alludes to CDM, which he knows to be diseased, because he’s found that if does not, he won’t get the time. And so CDM, a central dogma of BBC, continues alive, when it is so obviously wrong.

But enough of personal anguish and Sociology. How could the uncommitted thinker look dispassionately at the arguments for and against BBC and come to a balanced opinion?

There is a way – using Common Sense – if you know how it works – which most scientists, let alone other scholars, do not. All it will deliver is a provisional conclusion, with some kind of Odds on it attached. What I will do next is to exhibit two different attempts of mine to have a go at the BBC problem, so that readers can appreciate some of the philosophical subtleties involved.

The first, entitled “Doubts about Big Bang Cosmology” was published back in 2011, where my Odds against it being broadly right were only 4 to 1, disappointing, but hardly decisive. It is reasonably short yet contains the main arguments in a not too technical fashion I hope, so readers may care to see how those Odds were reached. You can find it at

https://mjdisney.org/wp-content/uploads/2021/02/Doubtsbigbang-copy.pdf

In cosmology itself nothing much changed dramatically over the next 4 years. But my understanding of Common Sense did when, in 2015, I stumbled upon the vitally important PAW or ‘Principal of Animal Wisdom’, indispensable to all thinkers who might otherwise be blown wildly off course by Systematic Errors. Now my Odds against BBC shot up dramatically to 128 to 1 against it being broadly right. Not only are they far more conclusive but they are , in my opinion , far more robust too because they rely on a whole network of interlocking and broadly concordant evidence. Without any need to repeat the cosmological arguments the new Inference Table, with its condemning Odds O(H|E) {i.e Odds on the Hypothesis H given all the evidence E} is briefly exhibited at

https://mjdisney.org/wp-content/uploads/2021/02/SCAMBBCTable-copy.docx

The conclusion I would draw from all this is that the Universe is trying to tell us something profound and interesting about itself, but we professionals, soaked in our preconceptions, and deafened by our Church choir, are unprepared to listen. After Galileo’s experience we should have anticipated, and some of us on board the Hubble did. But ….

Our susceptibility to misconceptions lies in our weak grasp of Common Sense today, and in particular our total ignorance of PAW, or The Principle of Animal Wisdom. Animals whose very survival depends on sound judgements, cannot afford to be taken in by misleading clues. So how do they discount them? That was the question I asked myself back in 2015. The answer is they cannot allow any single clue a predominating Weight – because that clue might be false, and fatal. They must rely on a network of weaker clues which reinforce one another. That is what I call PAW. And when I apply it to BBC the Odds against it shoot dramatically up. BBC can’t be right, it can’t. Something at least about it is deeply wrong, never mind the technical details. [To see more on the PAW go to Post ‘ANIMAL WISDOM & US’ in ‘Thinking’ Category].

If the PAW is so damned vital for animals then how did we ever lose sight of it? Because Priests preach Certainties – their influence, their power and their livelihoods all depend on proclaiming Certainties, whilst the PAW stands out firmly against them. And, to be fair, many of us prefer Certainties to uncomfortable uncertainty – which is all the natural world has to offer. So over the last few thousand years the PAW, which is grown-up, has become submerged by a childish and misbegotten craving for Certainty, which only priests, but not men of Common Sense, can deliver. As Voltaire put it: “Uncertainty is uncomfortable; Certainty is absurd.” See a talk on Youtube by me on this topic at

https://www.youtube.com/watch?v=KskJrJmfr34

What IS the universe trying to tell us ? It could be exciting.

CATEGORICAL INFERENCE

November 26, 2020

OTHERWISE KNOWN AS “COMMON SENSE THINKING

THE VERY VERY SIMPLE VERSION

As far as I can see Common Sense Thinking (CST henceforth) works like this: we all get ideas, they constantly bubble unasked to the surface of the mind; the real challenge is to decide which ones are sound [‘Hypothesis Testing’ it is called]. To determine that we look for evidence (clues) bearing on our idea or hypothesis H and place each clue in one of only 5 categories (This is the ‘Principle of Animal Wisdom’, or PAW for short):

TABLE (5:1) The Weights of Clues bearing on Idea H

Clue

Weight

Symbol

 

Strongly in favour of H

s

 

Weakly in favour of H

w

 

Neutral towards H

n

 

Weakly against H (underlined)

w 

 

Strongly against H (underlined)

s

 

We then combine (symbol ★ ) the Weights in obvious ways thus:

w★w = s

w★w = n

s★s = ss

s★w = w and so on.

And we finally decide to act on H only when the combined evidence reaches either sss [decide for H] or sss [decide against H]. This is a precautionary measure which saves us from making premature, possibly fatal decisions based on only two strong clues, one of which might be unsound.

SIMPLE EXAMPLE

A detective is having to decide whether to charge X with a crime [her hypothesis is ‘X is guilty’. Her thinking, based on the available evidence, might look like this:

TABLE (5:2) DETECTIVE’S THINKING

Clue

Her Weight

Accumulated Weight

Outcome

Motive

s

s

 

Opportunity

w

ws

 

Alibi

 w

s

 

Witness A

w

ws

 

Witness B

s

w

 

Witness C

w

s

 

Witness D

s

ss

 

Forensics

s

sss

Charges X

    

My scheme is nothing more than the systematic Association of an Idea H with different clues, combined with a simple precautionary mechanism for avoiding overhasty decisions. I suspect such CATEGORICAL INFERENCE (CI for short) is our main survival mechanism with roots that go back a billion years. You won’t find it in text-books on Inference or Logic; they appeal instead to notions such as Probability Theory, Bayes’ Theorem and Parsimony. The problem is that their authors disagree violently among themselves – so something must be seriously wrong. That’s why scientists ignore them and go on using Common Sense CI to progress.

Notice three important features of this scheme:

1) The more evidence the better. With a sufficiently long string of clues, even when they conflict, we can eventually reach a decision [sss or sss ] about H, one way or the other, provided (a major proviso) a record has been kept of the incoming clues, together with their Weights. For instance I was eventually able to bring my own tangled research project to a triumphant conclusion but only after using writing to compound 25 separate clues, some in stark conflict with the rest. This means the scheme can be used, but only by the literate, to handle highly complex tasks such as voyaging to the Moon.

2) The process is open-ended; there is always room to add new evidence to the tally whenever it is found. Thus it is Provisional in nature, and even after a decision to act has been taken there must be room for a change of mind – in other words to Adapt.

3) Rather than remember these unfamiliar symbols it turns out to be much easier to use betting Odds and replace “combine” (★) by the multiplication sign ×, ‘n’ by the number 1, s by 4, w by 2, underlined-w by ½, and underlined-s by ¼ . Then a decision in favour takes place when the Odds are 64 to 1 on or better, and against at Odds of 64 to 1 against or worse. In future that is what we do. But remember it is still Categorical Inference, no more and no less, a process innumerate animals could have used to survive in the wild. We have just changed the symbols

NB. This extract was taken from Chapter 5 of my book “History of the Brits” where it is later used to tackle some very thorny issues such as ‘Is America Britain’s friend or enemy?’, or ‘Would the Scots have been better off Independent’ and ‘Is mass immigration good or bad for Britain?’.

FUZZY THINKING AND OCKHAM’S RAZOR

November 24, 2020

Fuzzy thinking is far worse than fallacious thinking for whereas the latter may be spotted, or overturned by new evidence, the fuzzy variety may linger for millenia, causing endless harm, as we shall see.

There is a remedy against fuzzy thinking called “Ockham’s Razor (OR)” named after a mediaeval monk, though its roots stretch back into the classical world where it was labelled ‘lex parsimoniae‘ or ‘The Law of Parsimony’.

Ockham’s Razor states: “ALWAYS PREFER SIMPLE HYPOTHESES OVER COMPLEX ONES” which is easy enough to write down but damnably hard to justify. For instance both Newton and Einstein utterly relied upon it but both gave unsound reasons for doing so. Newton averred: “….for Nature is pleased with simplicity, and affects not the pomp of superfluous causes.” while Einstein waffled about God. He said, with regard to his Law of Gravitation “God would not have passed up the opportunity to make nature this simple.” [As it happens she had.]

SOME TRIUMPHS OF OCKHAM’S RAZOR:

1) Heliocentrism (A Sun-centred planetary system) was first advocated in modern times by Copernicus(1543) in De Revolutionibus. He didn’t have any new observations to justify his claim (the telescope wasn’t invented for another 30 years) but it was evidently much simpler than the traditional Earth -centred scheme, which needed twice as many arbitrary parameters to square it with the facts. [Confirmation only came in 1609 when Galileo with his spyglass spotted that Venus exhibited changing crescent- phases as it orbited the Sun.]

2) Newton recognised that the theory of Gravitation he proposed to explain the dynamics of the Solar System was ridiculous. He wrote: “That Gravity should be innate, inherent and essential to Matter so that one body should act upon another at a distance through a Vacuum, without the Mediation of anything else, by and which through their Action and Force may be conveyed from one to another, is to me so great an Absurdity that I believe that no Man who in philosophical matters has a competent Faculty of thinking can ever fall into it.” Nevertheless that one simple law explained so many things about the heavens and about he Earth that it was quickly and universally accepted…. because it was so parsimonious.

3) In his Origin of Species (1859) Darwin acknowledged that there were so many difficulties with his hypothesis of Evolution that he wrote in his conclusions: “That many and grave objections may be raised against the theory of descent with modification through natural selection, I do not deny. And I endeavour to give them full force.” And he did. Nevertheless many readers came to accept it rather than ascribing every peculiarity of Nature as due to a special intercession by God. Again because it was so much more parsimonious. And in the fullness of time ( a century) the various objections to Natural Selection melted away as Radioactivity and Continental Drift came to light.

4) It was Henri Poincare’ (1904) who first realised that the grave difficulties which then faced Physics could be resolved by accepting the Lorenz Transformations and modifying Newton’s Laws of Motion to agree with them. This is called ‘The Theory of Special Relativity’. However Einstein got the credit for it a year later by making a single outrageous assumption: “The speed of light is constant for all observers”, which was much less satisfactory from a philosophical point of view– but oh so much simpler algebraically. As in Newton’s case one outrageous assumption explained and predicted a thousand surprising observations: parsimony again.

5) And Parsimony isn’t just about science. Not at all. Take for example Military Intelligence. R.V.Jones who was head of Air Ministry Intelligence during the SWW, and who was responsible for the threats to the UK of Nazi bombing, radar, the V1 and the V2 missiles , later wrote a very fascinating book about his experiences ‘Most Secret War‘. In his summary at the end he calls Ockham’s Razor : “The Cardinal Principle of Military Intelligence.”

SOME FUZZY TRAGEDIES

1) The Four Elements was an idea promulgated by Aristotle around 300 BC in which all substances were supposed to be composed of a mixture of Earth, Water, Air and Fire. By adjusting this hypothetical mixture, and a deal of plausible sophistry, the old thinkers could explain everything – and therefore nothing. So long as it was widely believed, serious chemistry was unnecessary and therefore unpursued. This monster was a many headed gorgon bristling with free parameters. Whenever something didn’t fit you ascribed further properties (parameters) to your 4 imaginary elements and lo everything could be made to fit once again. Thus it couldn’t be overturned; because it was too fuzzy.

What we  would call Chemistry was ruled for over 2000 years by Aristotle’s hypothesis. Almost no material evidence supported his scheme but it appealed to religions such as Christianity and Islam looking for a comforting order to life. It wasn’t overturned so much as left behind by crisis, the firewood crisis which struck Britain in the 18th century after it had cut down most of its forests to build houses and ships. A new source of power had to be found and the mining of coal led to a desperate search for new materials and new contrivances such as pumps to stop the mines flooding. Experiments were necessary, and from experiments came evidence that made no sense within Aristotle’s fuzzy scheme. For instance burned in air some substances became heavier not lighter. And careful balance measurements initiated by Joseph Black in Glasgow University (1750, the English universities didn’t teach science then!) showed that substances combined in precise ratios to form compounds. Out of such observations the notions of atoms and molecules grew. These in turn gave rise to materials of great strength like steel, and to new compounds of great value such as artificial dyes. Crisis, experiment, discovery, understanding, wealth: The infinitely flexible, therefore unprogressive chemistry of Aristotle was simply left behind as unprofitable.

(B) The Four Humours was another fuzzy Greek hypothesis which held up progress for 2 millenia. Modelled on the Four Elements it imagined that health was determined by a balance of four liquids: choler, melancholer, phlegm and blood. Physicians who were learned in such jiggery-pokery dosed us, leeched us and charged us, shortening our lives as they impoverished our purses. Again the hypothesis was immune to criticism because it was infinitely adaptable. In place of bones it had an infinitude of free paramaters –  and what was more could earn good money. Again it couldn’t be displaced by evidence, being infinitely flexible. It was gradually superseded by more useful notions about physiology such as the germ theory of disease, a direct result of the invention of the microscope. But Greek Medicine held up real medicine for twenty centuries

(C) The ‘Argument by Design’ opined that all things wise and wonderful, all creatures great and small, were instances of The Creator’s wonderful powers of invention. After all no other cause could be imagined for the intricacies of Nature’s architecture, from the perfect spiral of a sea shell to the extravagance of a Rainbow Lorikeet’s plumage. Science at the ancient universities, even to the end of the nineteenth century, was solely aimed at uncovering such wonderful manifestations of the Almighty. Since nothing was outside His powers everything could be explained. There was no possibility of bringing Him down, since nothing was beyond Him, even burying fossils of inexplicable design merely to challenge our faith in Him. This ‘Argument by Design’ could have been rejected by nothing else but Parsimony, by the discovery of an alternative theory which was simpler, far simpler than a Great Designer in the Sky. And so it eventually was (see Darwin above) The trouble with Him was that he had an infinite number of free parameters (fudge factors).

RELIGION AND FUZZY THINKING

To my mind the greatest obstacle to progress in Western Society was Christianity. Forced upon the Roman Empire by Constantine on his deathbed (337 AD) in return for ‘absolution’ for his sins (he’d murdered his wife and son) it brought Thinking to an abrupt end for over a thousand years for, as Saint Augustine its early theologian wrote (~400 AD) : “There is another form of temptation, even more fraught with danger. This is the disease of curiosity ……It is this which drives us to try and discover the secrets of nature, those secrets which are beyond our understanding, which can avail us nothing and which man should not wish to learn.” Shortly afterwards Christians burned down the Great Library in Alexandria and executed its head by torture.

With no good evidence for its core belief in Jesus Christ as the Son of God Christianity (and other Abrahamic religions) at least offered a very desirable and necessary explanation for “the wonders of Nature” [see ‘The Argument by Design’ above]. But when Darwin and Parsimony had punctured that, thoughtful Christians, like the Bishop of Worcester’s wife, knew that Christianity’s days were numbered. As she put it : “Dear me, let us hope it is not true. But if it is true, let us hope it does not become widely known.”

BUT WHY WAS OCKHAM RIGHT?

Why should anyone accept Ockham’s Razor? That’s the central point. All the explanations I have examined are either unconvincing, or go off into stratospheric mathematics — which is the same thing. The problem is that anything as fundamental as Ockham’s Razor has to be grounded on a clear understanding of Common Sense — which was lacking. Put it another way: any claim to understand Common Sense Thinking must lead to a crystal clear explanation for OR. But don’t expect it to be simple or obvious — otherwise it would have popped out of the woodwork long since.

The secret, as always, turned out to be The Detective’s Equation (DE). Whenever you are trying to understand Hypothesis Testing, which is surely the aim, the DE can be used to calculate the Odds for or against the hypothesis under test, whenever the evidence and the assumptions are changed. So you can play around and find out what adds to one’s certainties and what subtracts, and out of such calculations Ockham’s Razor gradually emerged from the shadows. The best reason to believe any hypothesis is that it fits the existing evidence better than it has any right to do by chance. A simple hypothesis has little chance of fitting more complex data unless it is actually right, whereas a complex hypothesis deliberately contrived to fit it is hardly convincing. Much the best way to understand how things work is to look at a specific example, and we shall go through one in detail below while there is a whole chapter in my book Thinking for Ourselves about OR and its multifarious implications.

Because most scientists still don’t understand CST they don’t realize just how counterproductive it is to complexify their pet theories to fit new but inconvenient facts. Thus Big Bang Cosmology has been so seriously challenged by modern observations that cosmologists have been forced to fuzzify it with strange new parameters like Dark Energy, without realising that in doing so they’ve entirely undermined its credibility.

CONCLUSIONS

Once a hypothesis conflicts with the facts it cannot be left unmodified. But fixing it by introducing arbitrary modifications one by one to remove the separate discrepancies won’t work either because each such mod. will weaken the odds on the hypothesis overall. Only if one can find a mod. which offers to clear up several discrepancies at once should it be seriously considered. Of course that will be much harder — but nevertheless that is the challenge.

Applied to Big Bang Cosmology (BBC), Inflation (to fix isotropy), Cold Dark Matter (to fix galaxy-formation) and Dark Energy (to fix acceleration) shouldn’t be considered seriously because none was ambitious enough to fix more than one serious discrepancy. It was, and is, much healthier to admit that as it stands BBC is seriously at variance with the facts (observations). Some of it might be right ( e.g. expansion) and it is certainly hard to think of a plausible alternative. But it is much healthier for now to admit that BBC has failed. Trying to keep it alive, (Like Ancient Greek medicine) is the kind of fuzzy thinking which can hold up progress for generations, millennia perhaps. BBC has failed — we’ve got to start again.

Looking at the wider picture beyond Cosmology, we can see just how fundamental Ockham’s Razor must be to clear thinking of all kinds, from Astronomy to Military Intelligence. What was lacking was a transparent explanation of just how and why it works, but now the Detective’s Equation (i.e. Common Sense) has supplied that.

Ockham’s Razor must surely be one of the cornerstones of all serious systems of thought, including Common Sense .

My detailed explanation of why Ockham’s Razor works is given at:

https://mjdisney.org/wp-content/uploads/2020/10/scamsmv5.pdf

The best reference on the history and influence of Ockham’s Razor that I know of is Chapter 8 of Hugh Gauch Jr.’s volume “Scientific Method in Practice” CUP 2003. He goes so far as to say. “…. the scientific enterprise has never produced and never will produce a single conclusion without invoking parsimony. It is absolutely essential and pervasive.”

A readable but sceptical account of Big Bang Cosmology is In Search of the True Universe, by Martin Harwit, CUP 2013

P.S. Is there a quick way to check whether some hypothesis is unhealthily fuzzy? I believe there is: find out how many Free Parameters it has. BBC has 18. But so does The Standard Model of Particle Physics. That makes one wonder. That seems to be stuck in a cul-de-sac too: no exciting developments since the 1970’s . Umm. Do Quarks really exist? No one has actually captured one.[Constructing Quarks, by Andrew Pickering, Univ. Chicago Pr. 1984, makes for provocative reading].

THE METEORIC ASCENT OF HUMANKIND

November 18, 2020

Evolution is a painfully slow process. Modifications from generation to generation are generally imperceptible. How come then that humankind can launch telescopes into Space while our cousin the Chimpanzee, with whom we share 98% of our genes, is still struggling to crack nuts in the jungle? No wonder thoughtful people have invoked Divine Intervention, or the arrival of wisdom from elsewhere by interstellar spacecraft (Arthur C Clarke in the story/film “2001”) . If we are not to believe in miracles then we are faced with a fearsome puzzle: ‘How have we become so clever so quickly?

Had aliens come to the Earth a mere 10,000 years ago they would have had no good grounds for believing that humans would soon erect Santa Sofia, build Venice on piles in the Lagoon, devise the Hay-on-Wye Festival, organise the D-day landings, get to the Moon or launch the Hubble Space Telescope. In a blink of an eye in evolutionary terms, a marginal species has so come to dominate the Earth that we now worry that we will damage it irreparably. The zoologist Peter Medawar wrote: “For all their intelligence and dexterity. — qualities we have always attached great importance to — the higher primates (monkeys, apes and men ) have not been very successful. Human beings have a history of more than 500,000 years. Only during the last 5000 years or thereabouts have they begun to be, in a biological sense, a success.”

Then again the more we learn about other animals the harder it is to believe that we are much smarter than some of them. Jane Goodall found that we are by no means the only tool users; Cetaceans hunt collectively; birds navigate the globe precisely in ways we do not understand while crows can solve puzzles that defeat many humans. So it doesn’t look as if we are that much smarter than some other creatures….. and yet. Then again our advance has happened so recently, and spread so quickly. That sounds more like something cultural to me than organic Evolution. What trick have we learned in the last few thousand years or so that could boost our thinking capacity by a factor of something like a million? Finding it is not merely a colossal challenge, it may be the very clue we need to establish how we actually think. Explaining that factor of a million will be the acid test for any proposal which claims to understand Common Sense Thinking (CST).

After studying how scientists appear to think I found that we almost certainly use a method based on The Detective’s Equation. Now look hard at that Equation. Doesn’t it give you an idea?

The DETECTIVE’S EQUATION and its potential implications for the Miraculous Ascent of mankind, Can you work out the secret? It’s staring us in the face. Mind you it took me months to see it. But you have got an extra clue in the format of this script.

Yes it was WRITING! With it we were suddenly able to handle 10 or more clues instead of the 2 or 3 we could when we were illiterate. And in an EXPONENTIAL process like the Detective’s Equation that can make a difference of order millions. If you don’t believe me study the following Table of Odds derived from that Equation:

THINKERNAverage Weight (Av.)(Av.)NDECISIVENESS ODDS
Our Cat346464:1 Decisive in favour
Me 3464Decisive
Me & pen104MillionsVery Decisive
Me & pen101.560Decisive with confused Evidence
Team & pens15230,000Very Decisive with confusing evidence
Team & pens 151.525,000Ditto but more confusing
Research Community251.2100Decisive even with Very confused evidence
THE DECISIVENESS TABLE
Decisiveness values for different numbers of clues with different average Weights per clue. If the Av’s were less than 1 the Odds would be equally decisive but against; for instance if N were 10 but the average weight were 1/4, the Odds would be millions to one against the hypothesis H.

Recall that [‘Principle of Animal Wisdom’ or PAW] individual Weights can only take the values 4, 2, I, 1/2 or 1/4 so their averages in a particular investigation could take any value between 4.0 and 0.25. Where the evidence is confused or conflicting that average will tend towards zero as the various clues cancel one another out. Even so, with enough clues (N), we could still come to a decisive view regarding H, whatever H might be.

So I am suggesting that the Miraculous Ascent of Humankind can be ascribed entirely to our development of writing.

To back up that suggestion note:

(a) It fits almost perfectly into the chronology. The earliest phonetic script appears on the sarcophagus of the King of Biblos, a port in Phoenicia just North of present day Beirut and dated about 1200 BC. According to Herodotus, about 800 BC Cadmus took this script to the Greeks who modified it considerably, to include vowels (previously ‘understood’) and reversed it to write left to right. About 600 AD, via Etruria, it reached the Latinas, fore-runners of the Romans; and so we were off. Previous non-phonetic scripts were probably not flexible enough for sophisticated thinking, as well as being confined to a tiny priestly cast because so many glyphs were needed, thousands instead of tens.

Phonetic scripts, Phoenician in the centre, Greek first Left, Latin second; Persian first Right, Arabic second. The great similarities suggest that phonetic script was invented only once, probably around 1500 BC somewhere near Biblos in what is now the Lebanon. Courtesy Wikpedia, in which there are many fine articles on the history of scripts and languages,

(b) To demonstrate to yourself just how vital writing is to thinking, try to do a puzzle such as a Sudoku, a Crossword or a Codeword, without writing anything down. It can’t be done. As Einstein put it: “My pen and I are smarter than I am.”

(c) Daniel Boorstin the American scholar wrote: “I write to discover what I think.” Me too.

(d) Large or complex projects are absolutely dependant on documentation. It may be a pain in the arse but it is indispensable. For instance I was a member of the Hubble Space Telescope project for 35 years. So many teams were involved, so many individuals passed in and out of each team, so many subsystems relied upon one another, so many modifications were and had to be made that, without precise and regularly updated records, the whole thing would have been impossible. For instance Wide Field Camera One was vital, and depended on its CCD detectors. But when the first batch got ruined they couldn’t be replaced because the highly skilled technician at Texas Instruments who had ‘thinned’ them had gone off to have a baby and no one could either repeat her feat or find her. Had there been relevant documentation……. Likewise the effectiveness of your treatment by the NHS will be entirely dependent on the medical records that are meant to follow you around. It’s not bureaucracy, it is the indispensable accumulation of relevant information.

(e) If writing is all that makes us so smart we can no longer suppose that we are smarter than many other illiterate animals, be they chimpanzees or pelicans [see below]. Perhaps we should leave more room for them on this planet?

(f) There are so many fascinating and sometimes profound connections between Writing and Thinking, that one could write a whole book about them: in fact I have [see Thinking for Ourselves under Category My Books]. But let me mention just one last one here. The human need for Certainty gave rise to both Deduction (Logic ) on the one hand and Dogma (religion) on the other. But with Common Sense Thinking, allied with Writing, we no longer need either because, at least in principle we could reach sufficiently high Odds for or against any hypothesis to act on it with safety.

Being short this has to be a fairly superficial post. But you can follow up the rationality for the Detective’s Equation and Weights at:

https://mjdisney.org/wp-content/uploads/2020/10/scamsmv5.pdf

or in my aforementioned book Thinking for Ourselves (see ‘my books’ Category here). I also have a couple of essays about really smart animals who have taught me much including ‘Our Jack‘ about the young Jackdaw who shared my life for a short while at:

https://mjdisney.org/wp-content/uploads/2020/11/OUR-JACK-1.pdf

and ‘Browning and the Cockatoos‘ about a troop of Cockatoos which outsmarted the smartest guy in Canberra, which is at:

https://mjdisney.org/wp-content/uploads/2020/11/Browningcocks-copy-2.pdf

ARGUING DISPASSIONATELY

October 29, 2020

The world is full of bad arguments, the resentments they cause, and the messes they leave behind. I have recently discovered a far better way to argue, which I want to share.

Serious Thinking amounts to having an argument with oneself — looking at the evidence, weighting the various clues, then coming to a measured conclusion — if the combined Odds look good enough. There’s no need to become angry with oneself in the process. So why do we sometimes get angry with someone else who disagrees with us about Brexit say or Immigration?

I am a scientist who has spent the past 20 years trying to find out exactly how successful scientists think. And now I know. It turns out that they use “Categorical Inference (CI)” which I will describe shortly. The point is that if Categorical Inference is the way to think successfully it should also be the way to argue successfully , where ‘successful’ doesn’t mean ‘winning’ but arriving at the correct conclusion.

I suspect that we sometimes get angry with our opponents in a conventional argument because we imagine that they are trying to cheat us by using illegitimate tactics. That may sometimes be the the case but most often it is because we cannot see how they have arrived at their conclusions, just as they cannot see how we could possibly have arrived at ours. In other words the conventional process of argumentation is insufficiently transparent.

But that is only part of the problem. A second bone of contention is the Weighting of the different pieces of evidence (Clues). At present one side can pick a certain clue and then weight it so heavily as to claim victory, whatever the other side might have to say. That cannot be either productive, or right. Finally there has to be a sensible way of putting all the clues, with their chosen Weights, together so as to arrive at their Combined Odds one way or the other. All these things Categorical Inference does, and has been doing for millions of years, for CI is nothing more or less than Common Sense (CS) — the main survival mechanism of all us creatures on Earth. It is just that we humans have latterly allowed Culture, Religion and Baducation to overwhelm it.

Let me give one dramatic, and ultimately tragic example: ‘Ludendorff’s Lie’. General von Ludendorff was the brilliant but unstable commander of the Kaiser’s armies in the First World War. In August 1918 those armies were comprehensively defeated in front of Amiens by the combined French and British Commonwealth forces, and recoiled in irreversible retreat towards Berlin. Ludendorff panicked , rang up his prime minister and demanded that the government conclude an armistice at once, before Germany was occupied. But after the Armistice he claimed that his brave armies hadn’t been defeated at all, but had been ‘stabbed in the back’ by the civil government. A lot of angry Germans, including Corporal Hitler, believed him, and so the war had to be fought all over again in 1939, with tragic consequences for everyone, including Germany.

Now the point here is that a single clue — which happened to be false — carried enough Weight to plunge an entire continent into war. But there is a lot of misleading evidence out there in the world, not all of it deliberately false. In any productive argument there has to be a mechanism for curbing its influence, and in Categorical Inference there is; I call it ‘The Principle of Animal Wisdom‘ (PAW for short). Without PAW our species would never have survived.

If I am right in suggesting that Categorical Inference is an extremely ancient mechanism which evolved many millions of years ago among our animal forbears, then it must be pretty straightforward and indeed it is. In fact it was so bloody simple that I missed it altogether until I’d finished my Thinking book and had to go back and add it in retrospectively (Appendix 9). So let us look at a short outline of CI which can be found at:

https://mjdisney.org/wp-content/uploads/2020/10/catinfvshort.pdf

If we don’t know how to argue dispassionately, either we will hold a passionate argument — seldom fruitful — or we will avoid the argument altogether, which could be even worse. Thus finding out how to argue dispassionately was an intensely liberating experience for me. Now I am prepared to discuss tendentious matters which I would have shied away from before. Let’s look at an example.

One of my family, who was being taught history at his school in Hackney, passionately claimed that “The Brits should be utterly ashamed of their empire”. I wasn’t so sure so I decided to put the evidence together using Categorical Inference and here is what turned out: an Inference Table which you can examine here:

https://mjdisney.org/wp-content/uploads/2020/10/empshame.pdf

In this context it doesn’t matter whether you agree or disagree with the conclusion. But you can see there is at least a transparent procedure for carrying out an argument about such a tendentious matter. You can examine all my chosen clues, the Weights I have attached to them, and the Odds for or against, building up in the final column. The vital PAW enters in preventing me from attaching a Weight of more than 4 in favour any clue , or of less than 1/4 against.

These rules for dispassionate arguing are no more and no less than the rules for wise thinking (Common Sense) laid out in black and white. Subconsciously perhaps, great scientists have followed them because they know that in the natural world evidence frequently conflicts, whilst even the strongest appearing clues may later prove to be unsound. For instance the evidence used to dismiss Evolution, namely that the Earth was far too young, turned out, once radioactivity was discovered, to be spectacularly wrong.

If we can’t all learn to argue dispassionately, then when is mankind ever going to move on?

There is a more detailed discussion of CI at:

https://mjdisney.org/wp-content/uploads/2020/10/scamsmv5.pdf

but if you really want to delve into thinking and arguing, along with their entangled history, then you might like to look at my book Thinking For Ourselves which is intended to be accessible to everyone . It is described elsewhere on this site under the ‘My Books’ Category,.

STATISTICS: EXPOSED AT LAST

October 28, 2020

Why can’t all those professors of medical statistics give governments sound, or at least consistent advice about the Covid pandemic? I am sorry to say it is because Statistics is deeply flawed in its very foundations.

I taught Statistics at university for 30 years, at first with zeal, then with growing puzzlement, finally with disillusion. Towards the end I couldn’t bring myself to teach the students “Hypothesis Testing” — the central ambition of the whole enterprise.

Collecting data is fine: the more the merrier. Analysing that data in search of useful information is essential. But turning information into a wise recommendation for action turns out to be fiendishly difficult. Why? Because the real world is far more complicated than the artificial world of card-playing, from which Probability Theory evolved long ago. And at its heart Statistics is the application of Probability Theory to real situations — like outbreaks of Corona Virus.

The problem is this: in a card game there are 52 cards so that all possible combinations of cards can be imagined — and calculated. But in the real world the combinations are infinite and so incalculable. Faced with this absolute road-block professional statisticians try to fudge their way round it by “making approximations” that is to say by pretending that arcane mathematical results drawn from card-play still apply approximately to a world of awkward germs, and bloody awkward people.

But mostly they don’t, We scientists know that there are such things in the real world as Systematic Errors, that is to say misconceptions which no amount calculation, or approximation, can ever surmount. Take one example: earthquake waves travel through the Earth arguing that it must be rigid. The great guru of geophysics at Cambridge University, Harold Jeffreys ,used it to maintain that therefore Continental Drift must be impossible — holding back the subject for 50 years. But he was making a Systematic error in assuming that because rock was rigid on a timescale of seconds (waves) it must likewise be rigid on a timescale of millions of years. Had he gone for a walk on a beach in say Pembrokeshire, and seen the dramatic folding of the rocks, he would have realised he was talking nonsense.

Ironically, in his case, one ghastly mistake led to another. Sir Harold, as he became, morphed alas into an even greater guru — on the subject of The Scientific Method, and founder of the school of “Objective Bayesian Statistics” — which is highly fashionable in academic circles today. But wrong, as Henri Poincare’ argued in the nineteenth century.

Once one knows what to look for (but only then) it’s not difficult to spot the flaws in the all-too-many (to be healthy,) text-books of Statistics, . For instance:

a) They pretend that Systematic Errors do not exist.

b) They use mathematical notions such as “The Normal Distribution”, and misapply them to the real world, justifying what they are doing by appealing to the ‘Central Limit Theorem’ — which most appear not to understand.

c) They disagree violently among themselves, and divide into many schools — which explains those all-too-numerous textbooks on the subject

d) They hardly ever come up with sound insights which couldn’t have been reached anyway using plain Common Sense (e.g. Smoking and Lung Cancer).

WHAT SHOULD WE DO ABOUT IT?

It’s all very well criticising Statistics, but what are we to do about it in the present crisis? I suggest:

1) We should listen to Statistician’s advice, but grant it only the same degree of respect we would accord to Economist’s predictions. Neither are remotely scientists.

2) We should disregard all academic titles like Professor or Doctor because they have become meaningless nowadays. Shameless grade inflation in British academe is a scam for demanding unearned respect from the public and unearned rents from the young and vulnerable. No one fails a doctorate nowadays whilst you can now become a professor by filling in a form and have your colleagues (who all want to become ‘Professors’ too of course) countersign it.

3) We should all take a crash course in Common Sense Thinking so that we could do that very needful Hypothesis Testing for ourselves, but soundly.

All these matters are covered in considerable detail in Thinking For Ourselves (described elsewhere on this site. It supplies many worked examples and some exercises with answers.) Practically anyone literate should be able to understand it while technical types will benefit from learning why they don’t need to learn Statistics.

Meanwhile there are two addenda attached to this post. A more technical survey with references, on the weaknesses of Statistics at:

https://mjdisney.org/wp-content/uploads/2020/10/statsdead.pdf

And a shortish extract from my book Thinking for Ourselves explaining why we have all, me especially, struggled so long with this tendentious and difficult subject, at :

https://mjdisney.org/wp-content/uploads/2020/10/apolstat.pdf