Posts Tagged ‘hypothesis testing’

ARGUING DISPASSIONATELY

October 29, 2020

The world is full of bad arguments, the resentments they cause, and the messes they leave behind. I have recently discovered a far better way to argue, which I want to share.

Serious Thinking amounts to having an argument with oneself — looking at the evidence, weighting the various clues, then coming to a measured conclusion — if the combined Odds look good enough. There’s no need to become angry with oneself in the process. So why do we sometimes get angry with someone else who disagrees with us about Brexit say or Immigration?

I am a scientist who has spent the past 20 years trying to find out exactly how successful scientists think. And now I know. It turns out that they use “Categorical Inference (CI)” which I will describe shortly. The point is that if Categorical Inference is the way to think successfully it should also be the way to argue successfully , where ‘successful’ doesn’t mean ‘winning’ but arriving at the correct conclusion.

I suspect that we sometimes get angry with our opponents in a conventional argument because we imagine that they are trying to cheat us by using illegitimate tactics. That may sometimes be the the case but most often it is because we cannot see how they have arrived at their conclusions, just as they cannot see how we could possibly have arrived at ours. In other words the conventional process of argumentation is insufficiently transparent.

But that is only part of the problem. A second bone of contention is the Weighting of the different pieces of evidence (Clues). At present one side can pick a certain clue and then weight it so heavily as to claim victory, whatever the other side might have to say. That cannot be either productive, or right. Finally there has to be a sensible way of putting all the clues, with their chosen Weights, together so as to arrive at their Combined Odds one way or the other. All these things Categorical Inference does, and has been doing for millions of years, for CI is nothing more or less than Common Sense (CS) — the main survival mechanism of all us creatures on Earth. It is just that we humans have latterly allowed Culture, Religion and Baducation to overwhelm it.

Let me give one dramatic, and ultimately tragic example: ‘Ludendorff’s Lie’. General von Ludendorff was the brilliant but unstable commander of the Kaiser’s armies in the First World War. In August 1918 those armies were comprehensively defeated in front of Amiens by the combined French and British Commonwealth forces, and recoiled in irreversible retreat towards Berlin. Ludendorff panicked , rang up his prime minister and demanded that the government conclude an armistice at once, before Germany was occupied. But after the Armistice he claimed that his brave armies hadn’t been defeated at all, but had been ‘stabbed in the back’ by the civil government. A lot of angry Germans, including Corporal Hitler, believed him, and so the war had to be fought all over again in 1939, with tragic consequences for everyone, including Germany.

Now the point here is that a single clue — which happened to be false — carried enough Weight to plunge an entire continent into war. But there is a lot of misleading evidence out there in the world, not all of it deliberately false. In any productive argument there has to be a mechanism for curbing its influence, and in Categorical Inference there is; I call it ‘The Principle of Animal Wisdom‘ (PAW for short). Without PAW our species would never have survived.

If I am right in suggesting that Categorical Inference is an extremely ancient mechanism which evolved many millions of years ago among our animal forbears, then it must be pretty straightforward and indeed it is. In fact it was so bloody simple that I missed it altogether until I’d finished my Thinking book and had to go back and add it in retrospectively (Appendix 9). So let us look at a short outline of CI which can be found at:

https://mjdisney.org/wp-content/uploads/2020/10/catinfvshort.pdf

If we don’t know how to argue dispassionately, either we will hold a passionate argument — seldom fruitful — or we will avoid the argument altogether, which could be even worse. Thus finding out how to argue dispassionately was an intensely liberating experience for me. Now I am prepared to discuss tendentious matters which I would have shied away from before. Let’s look at an example.

One of my family, who was being taught history at his school in Hackney, passionately claimed that “The Brits should be utterly ashamed of their empire”. I wasn’t so sure so I decided to put the evidence together using Categorical Inference and here is what turned out: an Inference Table which you can examine here:

https://mjdisney.org/wp-content/uploads/2020/10/empshame.pdf

In this context it doesn’t matter whether you agree or disagree with the conclusion. But you can see there is at least a transparent procedure for carrying out an argument about such a tendentious matter. You can examine all my chosen clues, the Weights I have attached to them, and the Odds for or against, building up in the final column. The vital PAW enters in preventing me from attaching a Weight of more than 4 in favour any clue , or of less than 1/4 against.

These rules for dispassionate arguing are no more and no less than the rules for wise thinking (Common Sense) laid out in black and white. Subconsciously perhaps, great scientists have followed them because they know that in the natural world evidence frequently conflicts, whilst even the strongest appearing clues may later prove to be unsound. For instance the evidence used to dismiss Evolution, namely that the Earth was far too young, turned out, once radioactivity was discovered, to be spectacularly wrong.

If we can’t all learn to argue dispassionately, then when is mankind ever going to move on?

There is a more detailed discussion of CI at:

https://mjdisney.org/wp-content/uploads/2020/10/scamsmv5.pdf

but if you really want to delve into thinking and arguing, along with their entangled history, then you might like to look at my book Thinking For Ourselves which is intended to be accessible to everyone . It is described elsewhere on this site under the ‘My Books’ Category,.

STATISTICS: EXPOSED AT LAST

October 28, 2020

Why can’t all those professors of medical statistics give governments sound, or at least consistent advice about the Covid pandemic? I am sorry to say it is because Statistics is deeply flawed in its very foundations.

I taught Statistics at university for 30 years, at first with zeal, then with growing puzzlement, finally with disillusion. Towards the end I couldn’t bring myself to teach the students “Hypothesis Testing” — the central ambition of the whole enterprise.

Collecting data is fine: the more the merrier. Analysing that data in search of useful information is essential. But turning information into a wise recommendation for action turns out to be fiendishly difficult. Why? Because the real world is far more complicated than the artificial world of card-playing, from which Probability Theory evolved long ago. And at its heart Statistics is the application of Probability Theory to real situations — like outbreaks of Corona Virus.

The problem is this: in a card game there are 52 cards so that all possible combinations of cards can be imagined — and calculated. But in the real world the combinations are infinite and so incalculable. Faced with this absolute road-block professional statisticians try to fudge their way round it by “making approximations” that is to say by pretending that arcane mathematical results drawn from card-play still apply approximately to a world of awkward germs, and bloody awkward people.

But mostly they don’t, We scientists know that there are such things in the real world as Systematic Errors, that is to say misconceptions which no amount calculation, or approximation, can ever surmount. Take one example: earthquake waves travel through the Earth arguing that it must be rigid. The great guru of geophysics at Cambridge University, Harold Jeffreys ,used it to maintain that therefore Continental Drift must be impossible — holding back the subject for 50 years. But he was making a Systematic error in assuming that because rock was rigid on a timescale of seconds (waves) it must likewise be rigid on a timescale of millions of years. Had he gone for a walk on a beach in say Pembrokeshire, and seen the dramatic folding of the rocks, he would have realised he was talking nonsense.

Ironically, in his case, one ghastly mistake led to another. Sir Harold, as he became, morphed alas into an even greater guru — on the subject of The Scientific Method, and founder of the school of “Objective Bayesian Statistics” — which is highly fashionable in academic circles today. But wrong, as Henri Poincare’ argued in the nineteenth century.

Once one knows what to look for (but only then) it’s not difficult to spot the flaws in the all-too-many (to be healthy,) text-books of Statistics, . For instance:

a) They pretend that Systematic Errors do not exist.

b) They use mathematical notions such as “The Normal Distribution”, and misapply them to the real world, justifying what they are doing by appealing to the ‘Central Limit Theorem’ — which most appear not to understand.

c) They disagree violently among themselves, and divide into many schools — which explains those all-too-numerous textbooks on the subject

d) They hardly ever come up with sound insights which couldn’t have been reached anyway using plain Common Sense (e.g. Smoking and Lung Cancer).

WHAT SHOULD WE DO ABOUT IT?

It’s all very well criticising Statistics, but what are we to do about it in the present crisis? I suggest:

1) We should listen to Statistician’s advice, but grant it only the same degree of respect we would accord to Economist’s predictions. Neither are remotely scientists.

2) We should disregard all academic titles like Professor or Doctor because they have become meaningless nowadays. Shameless grade inflation in British academe is a scam for demanding unearned respect from the public and unearned rents from the young and vulnerable. No one fails a doctorate nowadays whilst you can now become a professor by filling in a form and have your colleagues (who all want to become ‘Professors’ too of course) countersign it.

3) We should all take a crash course in Common Sense Thinking so that we could do that very needful Hypothesis Testing for ourselves, but soundly.[I have just produced a very short book (60 pages) entitled Common Sense Thinking whose details are in the ‘my books’ Category elsewhere in this blog.]

All these matters are covered in considerable detail in Thinking For Ourselves (described elsewhere on this site. It supplies many worked examples and some exercises with answers.) Practically anyone literate should be able to understand it while technical types will benefit from learning why they don’t need to learn Statistics.

Meanwhile there are two addenda attached to this post. A more technical survey with references, on the weaknesses of Statistics at:

https://mjdisney.org/wp-content/uploads/2020/10/statsdead.pdf

And a shortish extract from my book Thinking for Ourselves explaining why we have all, me especially, struggled so long with this tendentious and difficult subject, at :

https://mjdisney.org/wp-content/uploads/2020/10/apolstat.pdf

THE FOLLY OF FREE TRADE

October 23, 2020

         If there is one thing that nearly all economists believe, and preach, it is the benefit of Free Trade. As a result all Britain’s great industries have either closed down, or are in the process: coal, steel, ship-building, cotton mills in Lancashire, woollen mills in Yorkshire, cars, motor-cycles, bicycles, trucks, clocks and pottery in the Midlands, white goods, aircraft, computers, electronics, shoes …….going, going, gone. But it isn’t just Britain. Youth unemployment in France is 25%, 40% in Italy and Spain. And look at America: its great manufacturing centres such as Pittsburgh, Detroit, Cleveland….. are now part of that broken rust belt which rose in despair and voted for Trump. What have we all done to ourselves? I will now argue that what the academic economists proclaim is so good for us is actually a deadly poison.

An imported commodity may be dramatically cheaper at the point of retail sale than its domestically produced equivalent. Unfortunately though imports can also have large Sunken Costs arising from losses in domestic employment, investment and profits. And none of us can afford to ignore such hidden costs because we will all have to stump up for them in the end in the form of extra taxes to pay for unemployment benefits, retraining and relocating workers,  lost capital and wasted infrastructure (factories, roads, schools, shops, hospitals….). And that says nothing of the misery involved in breaking up communities, families and friends. All that should be obvious; but not apparently to our Economist friends.

What needs to be made, commodity by commodity, is a calculation of the benefits of  a particular Free Trade set against the Sunken Costs which we will have to be borne by the wider community as a whole (i.e. the importing nation). That shouldn’t be too difficult – and it isn’t. I won’t bore you with the algebra but it is all in the attached article.

Let’s take just one dramatic example: a motor car imported into Britain; it doesn’t matter where from. According to my calculation it will have to be 64 per cent cheaper at the point of retail; sale  than its domestic equivalent to be a bargain.  Sixty Four Percent ! Most of the foreign cars on Britain’s (Frances’s, America’s…..) roads are thus an absolute disaster for the importing country as a whole because the Sunken Costs far exceed the benefits. Ditto for many other countries and other commodities (though bananas will still be welcome in Britain). The more sophisticated a country is in social terms the less it can afford to indulge in Free Trade because its sunken social costs (mostly investments in people ) are so high – by definition. Free Trade makes far more sense for unsophisticated countries because their people-investments are (equally by definition) so much lower. [China for instance barely has a social  welfare system so, by the same argument, it benefits from a wide variety of free trades.]

      I couldn’t believe this calculation when I first made it in 2016. But it has been checked by several other people with far more commercial background than I. It is  right. But please check it out because it is so important for you and your family.

     So why do Economists still preach the nonsense they do about Free Trade? I’m sorry to say that it’s chiefly because Economists appear to be too simple-minded to recognize the fallacies underlying their own profession. Unfortunately the harm they have done already is almost incalculable.

N.B. My argument is NOT Economics, merely accountancy. The distinction is that Economists have to make assumptions about how humans will react. I have not.

           THE FALLACY THAT ECONOMICS IS A SCIENCE.

         The essential skill of any kind of science is hypothesis-testing.  In my book [ ‘Thinking For Ourselves‘, Amazon paperback, 2020] I demonstrate how such testing works using Common Sense Thinking. But It will only work if the number of possible hypotheses (to explain the evidence) is finite, and indeed very limited in number. Thus dream-interpretation can never become a science because the number of possible explanations (hypotheses) for any dream is unlimited. If there were an infinite number of possible hypotheses  then the initial Odds on any one of them being right would have to be infinitely small, and no amount of subsequent evidence can make something infinitely small finite – that is the obvious logic. Philosophers call this “The Principle of Limited Variety” (PLV for short). The Greeks, the Romans and the biblical Jews were all big on dream-interpretation, but now that we understand the PLV we have (except for psychologists) given the dodgy practice up.

         So what about Economics – can that be a science? For Economics to claim that it is, or could become a science, it must demonstrate that the Principle of Limited Variety applies to it. But how could it do that? Take the recent financial crash of 2007/8. Practically nobody foresaw it, but dozens of books and thousands of learned papers have been written about it since, pointing to different culprits which include: greedy bankers, toxic mortgages, opaque financial instruments, over-leveraging, vast international imbalances (China saving versus US borrowing), auditors in cahoots with the companies that paid them, Fanny Mae and Fanny Mac (you don’t want to know), the scuppering of the 1944 conference on international banking at Bretton Woods, Nixon refusing to back the US dollar in the aftermath of the Vietnam War (1971), over-saving, poor wealth distribution, flash trading, inadequately financed pension funds going in search of unrealistic returns, poor or non-existent supervision of the system by financial supervisors, the Euro, hubris following the collapse of Communism, a naïve belief in ‘perfect markets’, the inappropriate use of ‘The Normal Distribution’ by financial ‘Quants’, insurers ignoring the possibility of correlated market movements, extremely foolish advice given by the actuarial profession, dishonesty on the part of politicians willing to buy votes by offering unaffordable utopias and raising government debts, house owners foolishly believing they were rich because house prices were rising…..and so on and so on. When I read and try to understand the various hypotheses, they all carry a degree of plausibility to me. Moreover they can interact with one another in a whole variety of plausible and dramatic ways leading to an almost infinite number of compounded hypotheses – completely abrogating the Principle of Limited Variety.

Thus it must be true that Economics is not, and never can become, a science!

There is another way to look at the matter. Imagine that Economics   is   a science capable of generating reliable predictions. Suppose that it predicts that farmers will make more money from selling beef than selling milk. Then smart farmers will switch from dairying to beef production. Through scarcity the price of milk will rise; through oversupply the price of beef will fall. The very prediction of the allegedly sound Economic theory has proved to be self-defeating (‘reflexive’ in the jargon). And it seems to me that any ‘science of human behaviour’ would be self-defeating in the same way.

Thus everybody needs to understand that Economics is a church built on quick-sand. However much one might wish it otherwise, nothing can ever be done to rescue that situation. This argument is so simple that one has to wonder why Economists themselves do not understand it. Perhaps they don’t want to.

J.K. Galbraith, the historian of Economics was right when he wrote: “Economics was invented to make Astrology look respectable”.

The good news is that although Free Trade is a paralysing disease it is not  malignant. We could cut it out tomorrow if we wanted to and return to ruddy health. But to do that we first have to convince ourselves that it is bloody unhealthy. So check out the full argument at:

https://mjdisney.org/wp-content/uploads/2020/10/freetrade.pdf

HOW COMMON SENSE WORKS

October 23, 2020

Common Sense is our chief survival mechanism. It has seen us through a billion years of Evolution. Without it any organism would quickly go under. Watch any wild animal, even birds in your garden. They are constantly having to weigh the Odds of Opportunity versus Risk — and when they get it wrong they die.

Making decisions wisely will dictate the course of our lives: how we make a living, who to marry, where to live, how to raise our kids…… But how do we decide? Almost entirely by using Common Sense. But how does it work? They certainly don’t teach us at school or university. You might say: “But they don’t need to, we inherit Common Sense with our genes”. True enough, but today we live in such a changed world from our hairy ancestors that to apply Common Sense Thinking (CST) to it we need to understand exactly how it works. That means we have to learn concepts such as Bayes’ Gambling Rule, Categorical Inference and the Principle of Animal Wisdom (PAW)….., as anyone over the age of 14 could.

There is no room here to explain how CST works but if we look at some few of its manifold implications that might encourage readers to dig deeper:

(1) Philosophers don’t understand serious thinking — they never have. For instance the Ancient Greeks thought it was based on Deduction which, as one can demonstrate, it cannot be.

(2) If CST is an inherited survival mechanism, and we share 98 percent of our genes with chimpanzees, how come we can launch telescopes into Space while they are still struggling to crack nuts in the jungle?

(3) It turns out that, because of an invention made about three thousand years ago near Byblos in Asia Minor, humans have multiplied their capacity to use Common Sense a million-fold .

(4) If that is so, does that mean that some other animals might be, genetically speaking, just as smart as us? Very probably yes. For instance Sulphur Crested Cockatoos have solved the Population Control Problem, which seems to be alas, entirely beyond us.

(5) All serious thinking turns out to be provisional. One’s conclusions can always be overturned by new evidence. Thus Certainty is unattainable in the real world! This has profound moral, philosophical and historical consequences. All civilisation and progress must rest on provisionality, and thus tolerance.

(6) It is possible to demonstrate, using CST, that even small amounts of dishonesty can fatally handicap any agent involved in a competitive situation, be it an individual in society, a firm in business, or a nation at war. ‘Nice guys come in last’ is complete nonsense. In the long run ‘Honest chaps will come in first’.

(7) The secret of Science’s success is not just Logic, Mathematics, Experiment or superior evidence; it is CST.

(8) Not all subjects are amenable to the scientific approach enabled by CST. For instance Economics and Psychology can never become sciences and are mostly hocus-pocus capable, like witch-doctory, of inflicting considerable harm. For instance the case for Free Trade, beloved of Economists, is , as one can show using CST, arrant nonsense. Indeed half-baked economics is the cause of much misery, including mass employment.

Those who would like to get to the bottom of Common Sense Thinking could read my paperback ‘Thinking For Ourselves’ [Amazon, 2020, 605pp, £14:50. There are exercises with answers on this site as well as stuff about the book in the ‘my books’ category] .Meanwhile there is more about the History of Thinking which you can download from:

https://mjdisney.org/wp-content/uploads/2020/10/histthink20.pdf

More recently (2022) I have published a very much abridged version of the big book (600 pages) above called Common Sense Thinking, which is only 60 odd pages long and which anyone 14 or older should be able to read and understand in a matter of days (because I’ve left all the tricky bits out). See a complete description of it under the My Books Category in this blog.

WHAT IS THE SCIENTIFIC METHOD?

October 21, 2020

Science works, so we suppose, because it is more evidence-based, more logical and more objective than other subjects. So much so that nowadays we are all urged to argue in an ‘evidence-based’ manner.

The extraordinary truth though is that nobody knows what ‘The Scientific Method’ is, or how it actually works.

Back in 1997 when my own scientific project sank into a quagmire of ‘Conflicting Evidence’ I felt that the only way to rescue it would be to track down The Scientific Method and apply it to my problem. Easier said than done. Little did I realise that I was embarked on a quest that would last 20 years and range across 25 centuries. The gurus of the business, the Philosophers of Science and the Statisticians, turned out to have little grasp of real Science and were embroiled in squabbles of their own making, having to do with the colour of angels. Indeed they’d so befuddled the subject that no proper scientist would go anywhere near it. Instead scientists carried on, as they always had done, using Common Sense. Unfortunately, from my point of view, none of them seemed willing, or able, to explain just how Common Sense Thinking worked. So I set sail to find out, starting from the suspicion that Common Sense must be a survival mechanism largely inherited from our animal forbears.

Finally (2018) I cracked it . If it seems insane, not to say downright immodest, to claim that one is the first to understand how Common Sense Thinking, and The Scientific Method (much the same), work, I entirely agree. But sometimes the truth dawns first not on the brilliant, but on the first to ask the right question — in this case “How do animals Think?”, because of course they do, otherwise they wouldn’t still be here.

Einstein averred that ; “Science is no more than a refinement of everyday thinking” but admitted “The physicist cannot proceed without a much more difficult problem (than physics), the problem of analysing the nature of everyday thinking.”

If Science stems from Common Sense, and Common Sense is born with us as a vital part of our inherited survival strategy, then why do we need to understand it? Because it must be adapted to a modern world vastly different from the ancient one in which it evolved. To think straight nowadays, and make best use of wonderful modern tools such as search engines and the Internet, we need to become thoroughly familiar with Bayes’ Rule, Ockham’s Razor, the Principle of Animal Wisdom (PAW), Gambling Theory and The Detective’s Equation — almost none of which form part of a contemporary education, even though they could all be picked up by 14-year-olds. Indeed much of modern education actively undermines our capacity to think straight.

If you are interested in thinking as well as it is possible for us humans to do you might want to look at my book Thinking for Ourselves [ Amazon paperback, 2020, 605 pages, £14.50]. The book is described in more detail elsewhere on this site while there is a short and I hope readable resume’ to be read at:

https://mjdisney.org/wp-content/uploads/2020/10/scamsmv5.pdf

WRIGHT BROTHERS, FRAUDS?

October 17, 2020

Ever since a tiny warbler landed on our ship in mid Atlantic I have been fascinated by animal flight in general. It took me me ten years to work out how our would-be “Christopher Columbus” had done it. What a revelation … what a delightful surprise. By then I had become an expert on low speed aerodynamics, and a glider pilot who soared with birds all over the globe.

In 2009, the centenary of Bleriot’s first flight across the English Channel, I dusted down my simple aerodynamics and applied it to his famous exploit. Everything worked out perfectly: range, speed, endurance, power, fuel….. So while such matters were fresh in mind I thought “Why not apply the same analysis to the Wright Brothers’ famous first powered flight alleged to have taken place at Kitty Hawk in 1903?”

The famous photo allegedly of the Wright Brothers first powered flight near Kittyhawk in 1903. But there are many suspicious circumstances surrounding it. For instance it wasn’t released until FIVE YEARS AFTERWARDS. The only eyewitnesses said they had to pull it up a hill beforehand, so it was probably just another glider. It was allegedly “destroyed by a gust of wind immediately afterwards” so no one qualified ever inspected it. It’s got no undercarriage. Only a madman would try to make the first powered flight in a high wind. The claimed telegram announcing the triumph has never been traced. And most telling no replica with the same feeble engine-power has been able to repeat it. In subsequent years the Wrights did fly, but only with the aid of a powerful catapult. That’s not powered flight: taking off is the critical part

This time nothing worked out. As a long term Wright admirer who had even visited their workshop in Ohio I assumed I’d made some terrible mistake. So I looked deeply into the case and bit by bit their whole story fell apart. I reckon the Odds are about a hundred thousand to one that the famous Kittyhawk flight was total fabrication. You can follow up the investigation for yourself by clicking on:

https://mjdisney.org/wp-content/uploads/2021/09/WRIGHTFRAUDS2-copy.pdf

while there is a primer on simple aerodynamics at:

https://mjdisney.org/wp-content/uploads/2021/07/AEROTASTER2-copy.pdf

while there is my novel Pterodactyl’s Blood which covers many of these matters and is discussed elsewhere on this site under ‘My Books’ Category