This week’s Nature features a call to arms from George Ellis and Joe Silk, entitled Scientific method: Defend the integrity of physics. I’m very glad to see well-known physicists highlighting the serious problem for the credibility of science raised by the string theory multiverse and the associated ongoing campaign to justify the failures of string theory by attacking the scientific method. Acknowledging evidence that an idea you cherished doesn’t work is at the core of what science is and physics now has a major problem with prominent theorists refusing to abide by this principle. Ellis and Silk do a great job of identifying and characterizing an important challenge the scientific community is facing.
The issue is however complicated, and while the Nature piece carefully and clearly addresses some of the complexities, there are places where things get over-simplified. In particular, the introduction frames the issue as whether a theory being “sufficiently elegant and explanatory” allows it to not need experimental testing. The problem with the string theory multiverse though is not this, since such a theory is the antithesis of “elegant and explanatory”. There’s just about nothing in science as inelegant as the various attempts (e.g. the KKLT mechanism) to make string theory fit with known physics, and “the multiverse did it” is no more an actual explanation of anything than “a big omnipotent turtle did it”.
Trying to cut through the complexities, Ellis and Silk write:
In our view, the issue boils down to clarifying one question: what potential observational or experimental evidence is there that would persuade you that the theory is wrong and lead you to abandoning it? If there is none, it is not a scientific theory.
This is at the heart of the matter, but there are subtleties. A common recent move among some prominent string theorists has been to argue that string theory is falsifiable: it is based on quantum mechanics, so if experiments falsify quantum mechanics, they falsify string theory. This just makes clear that the question of falsifiability can be slippery. Philosophers of science are experts at the intricacies of such questions and Ellis and Silk are right to call for help from them.
They also make the interesting call for the convening of a conference to address these issues. How such a thing would work and how it might be helpful seem well worth thinking about. As for one of their other recommendations though:
In the meantime, journal editors and publishers could assign speculative work to other research categories — such as mathematical rather than physical cosmology — according to its potential testability.
I’m leery of the impulse among physicists to solve their problem of how to deal with bad physics by calling it mathematics. Yes, there is good mathematics that has come out of untestable ideas about string theory, but no, this doesn’t include the string landscape/multiverse cop-out, which physicists need to face up to themselves.
For the specific arguments from Sean Carroll and Richard Dawid that Ellis and Silk address, I’ve written about them elsewhere, see for instance here, where I discussed in some detail Dawid’s arguments.
Update: Sabine Hossenfelder has commentary on this here.
Update: Taking the opposite side of the argument in January’s Smithsonian magazine is by colleague Brian Greene, with an article entitled Is String Theory About to Unravel?. As you might expect, Brian’s answer is “No”, and he gives a good account of the point of view Ellis and Silk are warning against. He mentions the possibility of encouraging news for string theory from the next LHC run, but says that “I now hold only modest hope that the theory will confront data during my lifetime.”
Update: Sean Carroll responds to the criticism from Ellis and Silk with a tweet characterizing them as belonging to the “falsifiability police”:
My real problem with the falsifiability police is: we don’t get to demand ahead of time what kind of theory correctly describes the world.
Update: Gordon Kane joins the fight in a comment at Nature, claiming that, before the LHC, string theory predicted a gluino mass of 1.5 TeV.
The literature contains clear and easily understood predictions published before LHC from compactified string theories that gluinos, for example, should have been too heavy to find in Run 1 but will be found in Run 2 (gluino mass of about 1.5 TeV).
As far as I can tell, this is utter nonsense, with Kane publicly claiming string theory predictions of a gluino mass of around 600 GeV (see page 22 of this) back in 2011, then moving the “prediction” up as Run 1 data falsified his earlier predictions. Kane at least makes falsifiable predictions, the problem with him only comes when they get falsified…
Update: Chad Orzel has his take here.
Update: Adam Frank has an essay on this here.
I quite like the idea to rename it, this is an important issue when it comes to funding. It would make it abundantly clear what this is an what it isn’t. This really is my biggest frustration when it comes to high energy theory, cosmology and quantum gravity today, that in my book much of it simply isn’t physics. Jeez, have you recently looked at the hep-th arxiv? Most of it is in the best case mathematical physics, in the worst case philosophy, or maybe art. I kinda wish it was at least science fiction, then it would be more interesting.
If you think that the mathematicians or philosophers don’t want it either, then call it something else, but please don’t let them block money and jobs that funding agencies mistakenly believe go into theoretical physics. (Yes, that’s right, what I actually mean is give me a job and boot some string theorists.)
Needless to say, I have made myself very unpopular among my colleagues with this opinion 😉
I don’t want anybody who really wants to work on string theory to stop – if it makes them happy, please. But don’t sell it as something it is not. And I say this as somebody who has a paper in physics.hist-ph, so don’t get me wrong, I do think philosophy is relevant and we’d even be better off paying more attention to what the philosophers are trying to tell us – sometimes at least.
As always, thanks for the link and for your discussion. I commented there.
Bee,
One of the odd sociological things I’ve noticed about the arxiv hep-th category is that it increasingly has large numbers of papers about quantum gravity/inflation/cosmology. What’s odd is that I would have thought that the “Quantum Cosmology” in gr-qc was precisely intended to provide the right home for this topic. One interpretation is that people coming out of a string theory background don’t want to be associated with those coming out a GR background…
I make it a practice to always refer to “work on strings” instead of the far, far more common label. One of the greatest disservices to the scientific community was the wide use of Theory label. The word Theory should be reserved for only be used for the best science has to offer: Newtonian mechanics, special relativity, general relativity, quantum mechanics, and the standard model are all physics theories. They each are logically coherent and make many predictions. I think we may have to replace some of these theories with better theories, but any upgrade must be coherent and have predictions.
Same number of characters, one more space: “work on strings” is my small way to defend the integrity of physics, and science in general.
Peter: Not sure about this. Annoyingly, almost all of these topics are cross-linked now. It’s annoying because more often than not I have the same paper 3 times in my feed. At this point I myself am so confused which arxiv category to best post in, that I normally do the same – after somebody once asked me why I post in qg-qc “which nobody reads”.
In any case, I have a longer comment on the Nature piece at Starts With A Bang. Thinking that maybe it’s of interest for some of your readers, the link is here:
https://medium.com/starts-with-a-bang/does-the-scientific-method-need-revision-d7514e2598f3
gr-qc I meant, sorry.
Bee,
Thanks for the link, I’ll add that to the text of the posting.
I am looking at arXiv postings at the arXiv site, so the cross-posted ones come later. I do mean that what’s odd is that articles on inflation, string cosmology, etc. are getting posted with hep-th as the primary category, cross-posted to gr-qc, not the reverse, which is what one would have naively expected.
“A common recent move among some prominent string theorists has been to argue that string theory is falsifiable: it is based on quantum mechanics, so if experiments falsify quantum mechanics, they falsify string theory”.
This is not how science works. In real life there must be a reason to take a theory or model seriously, in particular non-trivial (good enough) confirmation by experiment. Science is about inferencing in the Bayesian way. The 1930 theme was that some statements cannot be verified whereas they can (in principle) be falsified. E.g. a statement like “there are no pink elefants” is not confirmed by never seeing one, it can be falsified by seeing one. That sort of logic.
In normal physics a theory is never deemed credible merely because it is potentially falsifiable. That may be a necessary quality of a theory, but it obviously has nothing to do with making the theory credible. A credible theory is a theory that WORKS. Why don’t these “theorists” understand this? We could discuss what is meant with “work”, but to pretend that everyting is fine as long as the possibility of falsifiability cannot be proved to be identically zero is a joke. I have never seen this type of argument in real physics such as solid state physics and theory, or in relativity theory, or in quantum mechanics. Why was quantum mechanics with the probability interpretation accepted? Because it worked. Why was special relativity accepted? Because it worked. Why does physics work with theories that are evidently not totally correct – but only approximate or effective? Because these theories work. In real physics nobody presents a theory without showing that it has at least some correspondance with experiment or observation. I do not understand why so many seem to believe that a mere theoretical possibility of falsification should make a theory credible to a reasonable extent. Falsifiability is merely a negative criterion: a model that cannot possibly be falsified cannot be scientific. Turning it into a positive criterion makes no sense.
String theory does not work. That is all. People come with “theoretical falsifiability” only because they have nothing else.
@Bee: I guess that the original source of one of the images at your commentary is AbstruseGoose. Sorry for putting it here, I am not social enough to leave message there.
In his “The Road to Reality” , Penrose argues that supersymmetry has an unfalsifiable(un-Popperian) character because you can always argue that the superpartners can be detected with higher energies, then he asks:
“Does the ‘un-Popperian’ character of such models (like supersymmetry) make them unacceptable as scientific theories? I think that such a stringent Popperian judgement would be definitely too harsh. For an intriguing example, recall Dirac’s argument that the mere existence of a single magnetic monople somewhere in the cosmos could provide an explanation for the fact that each particle in the universe has an electric charge that is an integral multiple of some fixed value (as is indeed observed). The theory which asserts that such a monopole exists somewhere is distinctly un-Popperian. That theory could be established by the discovery of such a particle, but it appears not to be refutable, as Popper’s criterion would require; for, if the theory is wrong, no matter how long experimenters search in vain, their inability to find a monopole would not disprove the theory! Yet the theory is certainly a scientific one, well worthy of serious consideration.”
R.Penrose, The Road To reality, ch:34,
I have added “(like supersymmetry)” to the text because Penrose has mentioned it as an example of un-Popperian idea a few lines earlier,
As you see, being falsifiable is not necessary for the basic assumptions of a scientific theory (like existence of a magnetic monopole somewhere in universe), but it is necessary for the predictions of such theory (like the fact that electric charge is an integral multiple of some fixed value). In other words, we may have a theory based on some unfalsifiable assumptions, but the theory can be scientific if it makes falsifiable predictions. And it can become “credible” if the predictions are verified.
A more complete version of what martin said, can be:”a model that its predictions cannot possibly be falsified cannot be scientific. Turning it into a positive criterion makes no sense.”
In the string theory case, it seems that neither the basic assumptions (like replacing particles with tiny strings) nor the predictions (if any!) are falsifiable and it makes the situation very difficult.
Maybe these things can take a page from mathematics and be described as conjectures instead of theories. IE string conjecture, multiverse conjecture, inflation conjecture, quantum gravity conjecture. Maybe the burdens on conjectures are less stringent. Along the road to theories from conjectures might lie testable hypotheses.
Hi Peter,
I see. I just don’t notice this in my reader – I have hep-th, hep-ph and gr-qc in the same feed and (that includes all the cross-links), so I often don’t even know which category a paper is posted in.
Anyway, I think the main reason people post papers in one arxiv and not in another is not the name of the arxiv but the people who they know read it. Another issue with the arxiv categories is of course all the AdS/CFT stuff, much of which goes under high energy physics, though there really isn’t much high energy physics in it.
A simple way to avoid these issues would be to allow people to tag their papers with certain keywords.
Martin,
Thanks for this. I vaguely recalled that I had seen this elsewhere before, but couldn’t recall where. I’ll update the reference.
Brian Greene writes well as always, but I cannot help feeling that he would have been better off switching fields as advised in the autumn of 1984. I cannot say that the unification of General Relativity and Quantum Field Theory was high on my own list of priorities, though I was aware of the messianic zeal of those, like him, who latched on to the new band wagon in the succeeding months. My problem was, and is, that there is no point in trying to unify anything with Quantum Field Theory until we actually have one.
I feel that theoretical physics is not unique in science in having been infected with non-science of a variety of forms. I daren’t quote any examples because I’d be off topic, but the causes are not restricted to this area – people trying to publish as much as possible, the relentless push of money, and the way the media will hype very preliminary research reports that don’t need to be right, just exciting.
I don’t know what the answer is, but it might help if people from different disciplines realised how widespread these problems are.
Penrose’s monopole example is I think only unfalsifiable if the existence of a single monopole produces “… the fact that each particle in the universe has an electric charge that is an integral multiple of some fixed value…” but also has no other consequences that are not more easily falsified by observation than the detection of the single monopole. Also, the theory would need to specify that there was in fact only one monopole, or some other practically undetectable number of them.
I think that the utilitarian character of science (predictions of the results of experiments and the background on which engineering can be developed), physics in particular, to which indeed Popperian argument applies, is only one side of the coin. The second, and not less important is to tell a story about the world. Such story might be full of dragons, fairies and multiverses -nothing wrong with it. Whats might be wrong would be to say that your story is the only one, but I do not think even hard core string theorists say that nowadays.
Jerzy,
The predictions of the results of experiments is not simply part of the utilitarian character of science, because the experiments themselves generally lack a utilitarian motivation. Rather, they are designed and performed specifically to test theories. There may or may not be a utilitarian interest in those theories’ predictions, and in any case that interest is largely beside the point.
People have been telling a stories about the world for millennia. What matters about science is its specific, focused and serious efforts to find out whether or not those stories are false, or by elimination might be true. The only reason we can seriously contemplate in science that there might only be one true story about the world is because of the objective elimination of candidates through such a process of elimination, leaving just a few standing. Of course we can never know for sure that we have found that one true story, even when “a few” becomes one.
For these reasons the testability of theories is of critical importance. Sean Carroll deplores demanding “ahead of time what kind of theory correctly describes the world.” Nevertheless we have no alternative but to require—ahead of time—that scientific theories be testable. This may very well be a non-trivial constraint on what the world can be like, but we’ve always known that this is implicit in the effort to do science. A world that is sufficiently capricious—a world in which reproducible results can never be sustained—would not be a world in which science can be done. It would be strictly the domain of gods, demons, and fairies, or something even more chaotic. Indeed, for most of human history the world appeared to most people to be more or less just this.
(PS: The use of the word “story” in this context is problematic, but I’ll have to leave that alone for now.)
I really had a big laugh with Gordon Kane. I wonder if anybody still takes him seriously, it’s so ridiculous.
The discussion mixes up two arguments.
(1) The string idea does not work.
(2) The string idea is not falsifiable.
Argument 1 is tested by checking with experiment. For example, we can check whether supersymmetry exists. Discussing argument 2 is not smart. The argument would imply that the whole string approach is nonsense. Therefore all string workers get angry when falsifiability is discussed; they know that they are not doing nonsense. A fair and honorable discussion should only focus on argument 1; the rest is gratuitous hostility and leads to hostile reactions.
Stating that an idea is wrong is different from stating that it is nonsense. Also Ellis and Silk mix up the two arguments, and thus elicit a lot of hostile reactions. As expected.
This situation leads to my personal point of view on unification research:
(a) The unified theory we are all so desperately looking for has not yet been found because feedback, criticism, responses from reviewers and responses from funding agencies always slip from argument 1 to argument 2, thus discouraging anybody attempting to work on the issue.
(b) The unified theory will only be found by a researcher with no need for funding and who works in hiding. (Like for Fermat’s last theorem.)
Let us create a climate free of hostility – a climate that allows loners to work on their own. Then unification will appear.
I am currently reading Hermann Weyl. Since its a real book and not a PDF, I can’t find the exact quote and am only paraphrasing from memory, but he says something to the effect: Philosophers wish to think a thought and immediately get the truth, but science takes a very long time and philosophers are just impatient.
Increasingly, as I become familiar with this whole debate, Weyl’s sentiment seems a wise attitude to take.
Thanks,
Don
Chris W.,
I probably should write “utilitarian”, by which I mean “to be able to develop a technology i.e., to produce artifacts (not only material, but also, say, medical procedures) that behave according to the designer’s plans”. As for the story, I precisely said what I meant, and therefore I do not agree with you. In my view, the only requirement is that the story is consistent, in particular consistent with other stories that we consider “correct”
I finally got around to reading the Nature article, and came across this line:
This is clearly taking things too far in the opposite direction. The theoretical discovery of color confinement and asymptotic freedom clearly increased belief in the quark model, and there was even a Nobel Prize given out in connection with it.
But that’s because this theoretical discovery improved the alignment of theory and experiment, which is the test that theories need to pass.
I think the work of Gerald Holton is helpful in this context of what is science and what not (Thematic Origins of Scientific Thought). It shows how important mental dispositions are.
As an experimentalist, having physicists making claims that are, in principle, not falsifiable does not appear bad. You can think of it like this:
I have just thought of something and I think it explains some physical phenomena. If it never gets published (arXiv) no one will know about it. It might have a flaw that others can point out. It may be useful in peripheral ways that help others connect ideas of their own. It may, of course, be of no benefit to anyone, but what harm has come?
In my field, the reverse tends to be true. People do not publish enough of their mistakes to let others know not to try a specific method that is doomed to fail. Science is becoming more about not doing, but deciding what not to do. I think we all have to get used to that.
“It may, of course, be of no benefit to anyone, but what harm has come?”
You’d be right if funding was unlimited. Since it isn’t, there has to be fights over whether or not various lines of thought are crap.
It makes perfect sense to describe modern string theory as a form of mathematics. Mathematicians may not view their work as speculative* but that’s really what it is; conjuring rules to describe an imaginary system, and then deducing the consequences of these rules. String theorists are only different in that they sometimes eschew the levels of rigor employed by mathematicians.
At any rate, this whole debacle is just what is guaranteed to occur when experiment, which is the driving force behind physics and all science, atrophies due to lack of reasonable targets. Abandoning string theory won’t save HEP; the only thing that can do that is some actual experimental results.
*Some mathematicians view their work as an investigation on equal footing with say, chemistry: much as the chemist investigates the properties of an object found in reality, so too does the mathematician. Different algebras are as real to them as different polymers, but this is a ludicrous position. Mathematicians just don’t hope necessarily that their work has actual relevance to reality.
I think that virtually no mathematician would say that “string theory is mathematics” and that people claiming this are not mathematicians (Ellis and Silk certainly aren’t). What mathematicians do is work to create new mathematics. Applying known mathematics doesn’t make you a mathematician any more than my using the laptop I’m typing into makes me a computer scientist. “String theory” now covers a wide variety of different things, some of which try to use mathematics that is not well-understood, and can very well lead to new mathematics. Some of that activity certainly can be seen as mathematics research (a lot of Witten’s work falls into this area for instance). Other parts of string theory, for instance those trying to put together complicated models of “string phenomenology” or “string cosmology”, are just using well-known mathematical techniques and are in no sense mathematics research. They’re straightforwardly physics, sometimes using mathematics that just doesn’t happen to be part of the usual physics curriculum. The failure to connect successfully to reality doesn’t make this work mathematics, it makes it failed physics.
The pure mathematician’s view on string theorists seems to be roughly: “They believe in a lot of conjectures, some of which turn out to be very insightful, but their confidence in these beliefs is difficult to comprehend, as is their obsessive focus in certain specific classes of examples.” Genuine and interesting new mathematics, such as ‘monstrous moonshine’, has been inspired by developments in theoretical physics, even if we don’t trust what the physicists claim as ‘theorems’. It’s reminiscent of the story of the Italian school of algebraic geometry in the early 20th century: some people working in this area became increasingly cavalier with standards of rigour, and for a long time ‘got away with it’, in that they obtained a number of important results that were ultimately proved correct by more rigorous standards, but eventually the contradictions started to pile up and nobody had any confidence any more about which results were correct.
“It is based on quantum mechanics, so if experiments falsify quantum mechanics, they falsify string theory. This just makes clear that the question of falsifiability can be slippery.”
No, excuse me, this is not some deep smart logical paradox, don’t let these people lower everybody’s standards. It’s a failure to grasp ridiculously basic logical concepts to a degree horrifying for someone who gets public funds (directly, instead of as a pupil or a patient). “Send someone to fetch a child of five.”
I recently retired after 40 years in astrophysics, during which time I succeeded in making of myself something of a pariah, although all I wanted to do was practice physics, physically. I am South African, partially educated at the University of Cape Town, George Ellis’s academic home. At this stage of my life I can say what I like without jeopardising my meagre pension. And what I say is this: thank heavens for George Ellis, Peter Woit, the late Geoff Burbidge, and those few others who had the courage of their convictions and stood up to the corruption of science. My swansong, and indeed also my magnum opus, is my third book, Stephen Hawking Smoked My Socks, a treatment of the influence of belief in the formulation of our opinions, scientific or otherwise. In it, I acknowledge the courage of Ellis, Burbidge, and you, Peter. I salute you, Sir.
A point of view which is quite interesting was published here by Johannes Koelman which states that string theory, M-Theory and other quantum gravity models can claim a degree of verification by post-diction. M-Theory in particular reproduces the Standard Model of particle Physics (many of them actually) and General Relativity. By predicting the same low energy behavior as those models theories gain their verification.
What is your take on such an argument?
Personally while I find this line of thought interesting I am not totally convinced. Postdiction of known physics only means a model is plausible. What needs verification are the unique features of a hypothesis. What needs verification are the predictions that go beyond known physics.
Let us hope this kind of thinking does not gain traction. Who would they have make that call? What “wonderful group” would get to decide that this model which deals with physical phenomena is physics and this other model over there isn’t?
“Who would they have make that call? What “wonderful group” would get to decide that this model which deals with physical phenomena is physics and this other model over there isn’t?”
Nobody can stop anybody from claiming that some idea is physics. But people do make decisions about what research will be funded, who will be hired, and what papers will be published in reputable journals.
It is how the world works.
@Douglas Sweetser – I like this “work on strings.” The overuse of “theory” was perhaps lifted from the follies of the humanities of recent decades.
As for “predicting quantum mechanics,” this is as absurd as “predicting the Standard Model.” In both cases, the theories in question pre-existed “work on strings” — as did supersymmetry, higher dimensions, and inflation — and depend in no way on “work on strings.”
What amazes me is how long this desperation has lasted and how many who might have done some science have been driven from the field by the “work on strings” monomania.