Posts Tagged ‘detectives equation’


November 24, 2020

Fuzzy thinking is far worse than fallacious thinking for whereas the latter may be spotted, or overturned by new evidence, the fuzzy variety may linger for millenia, causing endless harm, as we shall see.

There is a remedy against fuzzy thinking called “Ockham’s Razor (OR)” named after a mediaeval monk, though its roots stretch back into the classical world where it was labelled ‘lex parsimoniae‘ or ‘The Law of Parsimony’.

Ockham’s Razor states: “ALWAYS PREFER SIMPLE HYPOTHESES OVER COMPLEX ONES” which is easy enough to write down but damnably hard to justify. For instance both Newton and Einstein utterly relied upon it but both gave unsound reasons for doing so. Newton averred: “….for Nature is pleased with simplicity, and affects not the pomp of superfluous causes.” while Einstein waffled about God. He said, with regard to his Law of Gravitation “God would not have passed up the opportunity to make nature this simple.” [As it happens she had.]


1) Heliocentrism (A Sun-centred planetary system) was first advocated in modern times by Copernicus(1543) in De Revolutionibus. He didn’t have any new observations to justify his claim (the telescope wasn’t invented for another 30 years) but it was evidently much simpler than the traditional Earth -centred scheme, which needed twice as many arbitrary parameters to square it with the facts. [Confirmation only came in 1609 when Galileo with his spyglass spotted that Venus exhibited changing crescent- phases as it orbited the Sun.]

2) Newton recognised that the theory of Gravitation he proposed to explain the dynamics of the Solar System was ridiculous. He wrote: “That Gravity should be innate, inherent and essential to Matter so that one body should act upon another at a distance through a Vacuum, without the Mediation of anything else, by and which through their Action and Force may be conveyed from one to another, is to me so great an Absurdity that I believe that no Man who in philosophical matters has a competent Faculty of thinking can ever fall into it.” Nevertheless that one simple law explained so many things about the heavens and about he Earth that it was quickly and universally accepted…. because it was so parsimonious.

3) In his Origin of Species (1859) Darwin acknowledged that there were so many difficulties with his hypothesis of Evolution that he wrote in his conclusions: “That many and grave objections may be raised against the theory of descent with modification through natural selection, I do not deny. And I endeavour to give them full force.” And he did. Nevertheless many readers came to accept it rather than ascribing every peculiarity of Nature as due to a special intercession by God. Again because it was so much more parsimonious. And in the fullness of time ( a century) the various objections to Natural Selection melted away as Radioactivity and Continental Drift came to light.

4) It was Henri Poincare’ (1904) who first realised that the grave difficulties which then faced Physics could be resolved by accepting the Lorenz Transformations and modifying Newton’s Laws of Motion to agree with them. This is called ‘The Theory of Special Relativity’. However Einstein got the credit for it a year later by making a single outrageous assumption: “The speed of light is constant for all observers”, which was much less satisfactory from a philosophical point of view– but oh so much simpler algebraically. As in Newton’s case one outrageous assumption explained and predicted a thousand surprising observations: parsimony again.

5) And Parsimony isn’t just about science. Not at all. Take for example Military Intelligence. R.V.Jones who was head of Air Ministry Intelligence during the SWW, and who was responsible for the threats to the UK of Nazi bombing, radar, the V1 and the V2 missiles , later wrote a very fascinating book about his experiences ‘Most Secret War‘. In his summary at the end he calls Ockham’s Razor : “The Cardinal Principle of Military Intelligence.”


1) The Four Elements was an idea promulgated by Aristotle around 300 BC in which all substances were supposed to be composed of a mixture of Earth, Water, Air and Fire. By adjusting this hypothetical mixture, and a deal of plausible sophistry, the old thinkers could explain everything – and therefore nothing. So long as it was widely believed, serious chemistry was unnecessary and therefore unpursued. This monster was a many headed gorgon bristling with free parameters. Whenever something didn’t fit you ascribed further properties (parameters) to your 4 imaginary elements and lo everything could be made to fit once again. Thus it couldn’t be overturned; because it was too fuzzy.

What we  would call Chemistry was ruled for over 2000 years by Aristotle’s hypothesis. Almost no material evidence supported his scheme but it appealed to religions such as Christianity and Islam looking for a comforting order to life. It wasn’t overturned so much as left behind by crisis, the firewood crisis which struck Britain in the 18th century after it had cut down most of its forests to build houses and ships. A new source of power had to be found and the mining of coal led to a desperate search for new materials and new contrivances such as pumps to stop the mines flooding. Experiments were necessary, and from experiments came evidence that made no sense within Aristotle’s fuzzy scheme. For instance burned in air some substances became heavier not lighter. And careful balance measurements initiated by Joseph Black in Glasgow University (1750, the English universities didn’t teach science then!) showed that substances combined in precise ratios to form compounds. Out of such observations the notions of atoms and molecules grew. These in turn gave rise to materials of great strength like steel, and to new compounds of great value such as artificial dyes. Crisis, experiment, discovery, understanding, wealth: The infinitely flexible, therefore unprogressive chemistry of Aristotle was simply left behind as unprofitable.

(B) The Four Humours was another fuzzy Greek hypothesis which held up progress for 2 millenia. Modelled on the Four Elements it imagined that health was determined by a balance of four liquids: choler, melancholer, phlegm and blood. Physicians who were learned in such jiggery-pokery dosed us, leeched us and charged us, shortening our lives as they impoverished our purses. Again the hypothesis was immune to criticism because it was infinitely adaptable. In place of bones it had an infinitude of free paramaters –  and what was more could earn good money. Again it couldn’t be displaced by evidence, being infinitely flexible. It was gradually superseded by more useful notions about physiology such as the germ theory of disease, a direct result of the invention of the microscope. But Greek Medicine held up real medicine for twenty centuries

(C) The ‘Argument by Design’ opined that all things wise and wonderful, all creatures great and small, were instances of The Creator’s wonderful powers of invention. After all no other cause could be imagined for the intricacies of Nature’s architecture, from the perfect spiral of a sea shell to the extravagance of a Rainbow Lorikeet’s plumage. Science at the ancient universities, even to the end of the nineteenth century, was solely aimed at uncovering such wonderful manifestations of the Almighty. Since nothing was outside His powers everything could be explained. There was no possibility of bringing Him down, since nothing was beyond Him, even burying fossils of inexplicable design merely to challenge our faith in Him. This ‘Argument by Design’ could have been rejected by nothing else but Parsimony, by the discovery of an alternative theory which was simpler, far simpler than a Great Designer in the Sky. And so it eventually was (see Darwin above) The trouble with Him was that he had an infinite number of free parameters (fudge factors).


To my mind the greatest obstacle to progress in Western Society was Christianity. Forced upon the Roman Empire by Constantine on his deathbed (337 AD) in return for ‘absolution’ for his sins (he’d murdered his wife and son) it brought Thinking to an abrupt end for over a thousand years for, as Saint Augustine its early theologian wrote (~400 AD) : “There is another form of temptation, even more fraught with danger. This is the disease of curiosity ……It is this which drives us to try and discover the secrets of nature, those secrets which are beyond our understanding, which can avail us nothing and which man should not wish to learn.” Shortly afterwards Christians burned down the Great Library in Alexandria and executed its head by torture.

With no good evidence for its core belief in Jesus Christ as the Son of God Christianity (and other Abrahamic religions) at least offered a very desirable and necessary explanation for “the wonders of Nature” [see ‘The Argument by Design’ above]. But when Darwin and Parsimony had punctured that, thoughtful Christians, like the Bishop of Worcester’s wife, knew that Christianity’s days were numbered. As she put it : “Dear me, let us hope it is not true. But if it is true, let us hope it does not become widely known.”


Why should anyone accept Ockham’s Razor? That’s the central point. All the explanations I have examined are either unconvincing, or go off into stratospheric mathematics — which is the same thing. The problem is that anything as fundamental as Ockham’s Razor has to be grounded on a clear understanding of Common Sense — which was lacking. Put it another way: any claim to understand Common Sense Thinking must lead to a crystal clear explanation for OR. But don’t expect it to be simple or obvious — otherwise it would have popped out of the woodwork long since.

The secret, as always, turned out to be The Detective’s Equation (DE). Whenever you are trying to understand Hypothesis Testing, which is surely the aim, the DE can be used to calculate the Odds for or against the hypothesis under test, whenever the evidence and the assumptions are changed. So you can play around and find out what adds to one’s certainties and what subtracts, and out of such calculations Ockham’s Razor gradually emerged from the shadows. The best reason to believe any hypothesis is that it fits the existing evidence better than it has any right to do by chance. A simple hypothesis has little chance of fitting more complex data unless it is actually right, whereas a complex hypothesis deliberately contrived to fit it is hardly convincing. Much the best way to understand how things work is to look at a specific example, and we shall go through one in detail below while there is a whole chapter in my book Thinking for Ourselves about OR and its multifarious implications.

Because most scientists still don’t understand CST they don’t realize just how counterproductive it is to complexify their pet theories to fit new but inconvenient facts. Thus Big Bang Cosmology has been so seriously challenged by modern observations that cosmologists have been forced to fuzzify it with strange new parameters like Dark Energy, without realising that in doing so they’ve entirely undermined its credibility.


Once a hypothesis conflicts with the facts it cannot be left unmodified. But fixing it by introducing arbitrary modifications one by one to remove the separate discrepancies won’t work either because each such mod. will weaken the odds on the hypothesis overall. Only if one can find a mod. which offers to clear up several discrepancies at once should it be seriously considered. Of course that will be much harder — but nevertheless that is the challenge.

Applied to Big Bang Cosmology (BBC), Inflation (to fix isotropy), Cold Dark Matter (to fix galaxy-formation) and Dark Energy (to fix acceleration) shouldn’t be considered seriously because none was ambitious enough to fix more than one serious discrepancy. It was, and is, much healthier to admit that as it stands BBC is seriously at variance with the facts (observations). Some of it might be right ( e.g. expansion) and it is certainly hard to think of a plausible alternative. But it is much healthier for now to admit that BBC has failed. Trying to keep it alive, (Like Ancient Greek medicine) is the kind of fuzzy thinking which can hold up progress for generations, millennia perhaps. BBC has failed — we’ve got to start again.

Looking at the wider picture beyond Cosmology, we can see just how fundamental Ockham’s Razor must be to clear thinking of all kinds, from Astronomy to Military Intelligence. What was lacking was a transparent explanation of just how and why it works, but now the Detective’s Equation (i.e. Common Sense) has supplied that.

Ockham’s Razor must surely be one of the cornerstones of all serious systems of thought, including Common Sense .

My detailed explanation of why Ockham’s Razor works is given at:

The best reference on the history and influence of Ockham’s Razor that I know of is Chapter 8 of Hugh Gauch Jr.’s volume “Scientific Method in Practice” CUP 2003. He goes so far as to say. “…. the scientific enterprise has never produced and never will produce a single conclusion without invoking parsimony. It is absolutely essential and pervasive.”

A readable but sceptical account of Big Bang Cosmology is In Search of the True Universe, by Martin Harwit, CUP 2013

P.S. Is there a quick way to check whether some hypothesis is unhealthily fuzzy? I believe there is: find out how many Free Parameters it has. BBC has 18. But so does The Standard Model of Particle Physics. That makes one wonder. That seems to be stuck in a cul-de-sac too: no exciting developments since the 1970’s . Umm. Do Quarks really exist? No one has actually captured one.[Constructing Quarks, by Andrew Pickering, Univ. Chicago Pr. 1984, makes for provocative reading].


November 18, 2020

Evolution is a painfully slow process. Modifications from generation to generation are generally imperceptible. How come then that humankind can launch telescopes into Space while our cousin the Chimpanzee, with whom we share 98% of our genes, is still struggling to crack nuts in the jungle? No wonder thoughtful people have invoked Divine Intervention, or the arrival of wisdom from elsewhere by interstellar spacecraft (Arthur C Clarke in the story/film “2001”) . If we are not to believe in miracles then we are faced with a fearsome puzzle: ‘How have we become so clever so quickly?

Had aliens come to the Earth a mere 10,000 years ago they would have had no good grounds for believing that humans would soon erect Santa Sofia, build Venice on piles in the Lagoon, devise the Hay-on-Wye Festival, organise the D-day landings, get to the Moon or launch the Hubble Space Telescope. In a blink of an eye in evolutionary terms, a marginal species has so come to dominate the Earth that we now worry that we will damage it irreparably. The zoologist Peter Medawar wrote: “For all their intelligence and dexterity. — qualities we have always attached great importance to — the higher primates (monkeys, apes and men ) have not been very successful. Human beings have a history of more than 500,000 years. Only during the last 5000 years or thereabouts have they begun to be, in a biological sense, a success.”

Then again the more we learn about other animals the harder it is to believe that we are much smarter than some of them. Jane Goodall found that we are by no means the only tool users; Cetaceans hunt collectively; birds navigate the globe precisely in ways we do not understand while crows can solve puzzles that defeat many humans. So it doesn’t look as if we are that much smarter than some other creatures….. and yet. Then again our advance has happened so recently, and spread so quickly. That sounds more like something cultural to me than organic Evolution. What trick have we learned in the last few thousand years or so that could boost our thinking capacity by a factor of something like a million? Finding it is not merely a colossal challenge, it may be the very clue we need to establish how we actually think. Explaining that factor of a million will be the acid test for any proposal which claims to understand Common Sense Thinking (CST).

After studying how scientists appear to think I found that we almost certainly use a method based on The Detective’s Equation. Now look hard at that Equation. Doesn’t it give you an idea?

The DETECTIVE’S EQUATION and its potential implications for the Miraculous Ascent of mankind, Can you work out the secret? It’s staring us in the face. Mind you it took me months to see it. But you have got an extra clue in the format of this script.

Yes it was WRITING! With it we were suddenly able to handle 10 or more clues instead of the 2 or 3 we could when we were illiterate. And in an EXPONENTIAL process like the Detective’s Equation that can make a difference of order millions. If you don’t believe me study the following Table of Odds derived from that Equation:

THINKERNo.of Clues N Average Weight (Av.) (Av.)N DECISIVENESS ODDS
Our Cat34 64 64:1 Decisive in favour
Me 34 64 Decisive
Me & pen104 Millions Very Decisive
Me & pen101.5 60 Decisive with confused Evidence
Team & pens152 30,000 Very Decisive with confusing evidence
Team & pens 151.525,000 Ditto but more confusing
Research Community251.2 100 Decisive even with Very confused evidence
Decisiveness values for different numbers of clues with different average Weights per clue. If the Av’s were less than 1 the Odds would be equally decisive but against; for instance if N were 10 but the average weight were 1/4, the Odds would be millions to one against the hypothesis H.

Recall that [‘Principle of Animal Wisdom’ or PAW] individual Weights can only take the values 4, 2, I, 1/2 or 1/4 so their averages in a particular investigation could take any value between 4.0 and 0.25. Where the evidence is confused or conflicting that average will tend towards zero as the various clues cancel one another out. Even so, with enough clues (N), we could still come to a decisive view regarding H, whatever H might be.

So I am suggesting that the Miraculous Ascent of Humankind can be ascribed entirely to our development of writing.

To back up that suggestion note:

(a) It fits almost perfectly into the chronology. The earliest phonetic script appears on the sarcophagus of the King of Biblos, a port in Phoenicia just North of present day Beirut and dated about 1200 BC. According to Herodotus, about 800 BC Cadmus took this script to the Greeks who modified it considerably, to include vowels (previously ‘understood’) and reversed it to write left to right. About 600 AD, via Etruria, it reached the Latinas, fore-runners of the Romans; and so we were off. Previous non-phonetic scripts were probably not flexible enough for sophisticated thinking, as well as being confined to a tiny priestly cast because so many glyphs were needed, thousands instead of tens.

Phonetic scripts, Phoenician in the centre, Greek first Left, Latin second; Persian first Right, Arabic second. The great similarities suggest that phonetic script was invented only once, probably around 1500 BC somewhere near Biblos in what is now the Lebanon. Courtesy Wikpedia, in which there are many fine articles on the history of scripts and languages,

(b) To demonstrate to yourself just how vital writing is to thinking, try to do a puzzle such as a Sudoku, a Crossword or a Codeword, without writing anything down. It can’t be done. As Einstein put it: “My pen and I are smarter than I am.”

(c) Daniel Boorstin the American scholar wrote: “I write to discover what I think.” Me too.

(d) Large or complex projects are absolutely dependant on documentation. It may be a pain in the arse but it is indispensable. For instance I was a member of the Hubble Space Telescope project for 35 years. So many teams were involved, so many individuals passed in and out of each team, so many subsystems relied upon one another, so many modifications were and had to be made that, without precise and regularly updated records, the whole thing would have been impossible. For instance Wide Field Camera One was vital, and depended on its CCD detectors. But when the first batch got ruined they couldn’t be replaced because the highly skilled technician at Texas Instruments who had ‘thinned’ them had gone off to have a baby and no one could either repeat her feat or find her. Had there been relevant documentation……. Likewise the effectiveness of your treatment by the NHS will be entirely dependent on the medical records that are meant to follow you around. It’s not bureaucracy, it is the indispensable accumulation of relevant information.

(e) If writing is all that makes us so smart we can no longer suppose that we are smarter than many other illiterate animals, be they chimpanzees or pelicans [see below]. Perhaps we should leave more room for them on this planet?

(f) There are so many fascinating and sometimes profound connections between Writing and Thinking, that one could write a whole book about them: in fact I have [see Thinking for Ourselves under Category My Books]. But let me mention just one last one here. The human need for Certainty gave rise to both Deduction (Logic ) on the one hand and Dogma (religion) on the other. But with Common Sense Thinking, allied with Writing, we no longer need either because, at least in principle we could reach sufficiently high Odds for or against any hypothesis to act on it with safety.

Being short this has to be a fairly superficial post. But you can follow up the rationality for the Detective’s Equation and Weights at:

or in my aforementioned book Thinking for Ourselves (see ‘my books’ Category here). I also have a couple of essays about really smart animals who have taught me much including ‘Our Jack‘ about the young Jackdaw who shared my life for a short while at:

and ‘Browning and the Cockatoos‘ about a troop of Cockatoos which outsmarted the smartest guy in Canberra, which is at:


October 23, 2020

Common Sense is our chief survival mechanism. It has seen us through a billion years of Evolution. Without it any organism would quickly go under. Watch any wild animal, even birds in your garden. They are constantly having to weigh the Odds of Opportunity versus Risk — and when they get it wrong they die.

Making decisions wisely will dictate the course of our lives: how we make a living, who to marry, where to live, how to raise our kids…… But how do we decide? Almost entirely by using Common Sense. But how does it work? They certainly don’t teach us at school or university. You might say: “But they don’t need to, we inherit Common Sense with our genes”. True enough, but today we live in such a changed world from our hairy ancestors that to apply Common Sense Thinking (CST) to it we need to understand exactly how it works. That means we have to learn concepts such as Bayes’ Gambling Rule, Categorical Inference and the Principle of Animal Wisdom (PAW)….., as anyone over the age of 14 could.

There is no room here to explain how CST works but if we look at some few of its manifold implications that might encourage readers to dig deeper:

(1) Philosophers don’t understand serious thinking — they never have. For instance the Ancient Greeks thought it was based on Deduction which, as one can demonstrate, it cannot be.

(2) If CST is an inherited survival mechanism, and we share 98 percent of our genes with chimpanzees, how come we can launch telescopes into Space while they are still struggling to crack nuts in the jungle?

(3) It turns out that, because of an invention made about three thousand years ago near Byblos in Asia Minor, humans have multiplied their capacity to use Common Sense a million-fold .

(4) If that is so, does that mean that some other animals might be, genetically speaking, just as smart as us? Very probably yes. For instance Sulphur Crested Cockatoos have solved the Population Control Problem, which seems to be alas, entirely beyond us.

(5) All serious thinking turns out to be provisional. One’s conclusions can always be overturned by new evidence. Thus Certainty is unattainable in the real world! This has profound moral, philosophical and historical consequences. All civilisation and progress must rest on provisionality, and thus tolerance.

(6) It is possible to demonstrate, using CST, that even small amounts of dishonesty can fatally handicap any agent involved in a competitive situation, be it an individual in society, a firm in business, or a nation at war. ‘Nice guys come in last’ is complete nonsense. In the long run ‘Honest chaps will come in first’.

(7) The secret of Science’s success is not just Logic, Mathematics, Experiment or superior evidence; it is CST.

(8) Not all subjects are amenable to the scientific approach enabled by CST. For instance Economics and Psychology can never become sciences and are mostly hocus-pocus capable, like witch-doctory, of inflicting considerable harm. For instance the case for Free Trade, beloved of Economists, is , as one can show using CST, arrant nonsense. Indeed half-baked economics is the cause of much misery, including mass employment.

Those who would like to get to the bottom of Common Sense Thinking could read my paperback ‘Thinking For Ourselves’ [Amazon, 2020, 605pp, £14:50. There are exercises with answers on this site as well as stuff about the book in the ‘my books’ category] .Meanwhile there is more about the History of Thinking which you can download from:

More recently (2022) I have published a very much abridged version of the big book (600 pages) above called Common Sense Thinking, which is only 60 odd pages long and which anyone 14 or older should be able to read and understand in a matter of days (because I’ve left all the tricky bits out). See a complete description of it under the My Books Category in this blog.


October 21, 2020

Science works, so we suppose, because it is more evidence-based, more logical and more objective than other subjects. So much so that nowadays we are all urged to argue in an ‘evidence-based’ manner.

The extraordinary truth though is that nobody knows what ‘The Scientific Method’ is, or how it actually works.

Back in 1997 when my own scientific project sank into a quagmire of ‘Conflicting Evidence’ I felt that the only way to rescue it would be to track down The Scientific Method and apply it to my problem. Easier said than done. Little did I realise that I was embarked on a quest that would last 20 years and range across 25 centuries. The gurus of the business, the Philosophers of Science and the Statisticians, turned out to have little grasp of real Science and were embroiled in squabbles of their own making, having to do with the colour of angels. Indeed they’d so befuddled the subject that no proper scientist would go anywhere near it. Instead scientists carried on, as they always had done, using Common Sense. Unfortunately, from my point of view, none of them seemed willing, or able, to explain just how Common Sense Thinking worked. So I set sail to find out, starting from the suspicion that Common Sense must be a survival mechanism largely inherited from our animal forbears.

Finally (2018) I cracked it . If it seems insane, not to say downright immodest, to claim that one is the first to understand how Common Sense Thinking, and The Scientific Method (much the same), work, I entirely agree. But sometimes the truth dawns first not on the brilliant, but on the first to ask the right question — in this case “How do animals Think?”, because of course they do, otherwise they wouldn’t still be here.

Einstein averred that ; “Science is no more than a refinement of everyday thinking” but admitted “The physicist cannot proceed without a much more difficult problem (than physics), the problem of analysing the nature of everyday thinking.”

If Science stems from Common Sense, and Common Sense is born with us as a vital part of our inherited survival strategy, then why do we need to understand it? Because it must be adapted to a modern world vastly different from the ancient one in which it evolved. To think straight nowadays, and make best use of wonderful modern tools such as search engines and the Internet, we need to become thoroughly familiar with Bayes’ Rule, Ockham’s Razor, the Principle of Animal Wisdom (PAW), Gambling Theory and The Detective’s Equation — almost none of which form part of a contemporary education, even though they could all be picked up by 14-year-olds. Indeed much of modern education actively undermines our capacity to think straight.

If you are interested in thinking as well as it is possible for us humans to do you might want to look at my book Thinking for Ourselves [ Amazon paperback, 2020, 605 pages, £14.50]. The book is described in more detail elsewhere on this site while there is a short and I hope readable resume’ to be read at: