Kragh's approach is historical throughout, with a focus on people rather than abstract ideas, on their relationships with one another and on where their ideas came from and how they changed over time. Evaluation of the strengths and weaknesses of theories is mostly done through presentation of the contrasting views of proponents and opponents; and where Kragh does present his own perspective it is often indirect, based on measures of influence such as citation metrics.
The opening chapter presents three separate case studies from the early history of science. Descartes attempted "a rational explanation of all natural phenomena" through the kinematics of continuous media vortexes. The Jesuit Roger Boscovich had an early "theory of everything" based on point particles, which had "an air of unreality when confronted with specific problems from the physical world". And Friedrich Schelling's Naturphilosophie is considered as a representative of Romantic natural philosophy.
The Victorian vortex theory of matter, whose most prominent proponent was William Thompson (Lord Kelvin), imagined atoms as closed vortices in an underlying continuous medium, though "there never was a single, concisely defined theory of vortex atoms". The theory's big attraction was its monist simplicity. (Kragh digresses here to Karl Pearson's alternative monist theory of ether and matter, which didn't involve vortices.) Attempts to explain gravity failed, as did attempts to apply it to chemistry, but the theory continued to be attractive even after it was abandoned — it "ought to be true even if it is not" (Michelson) — remaining popular with religious believers and spiritualists unhappy about the materialism associated with traditional atomic theories.
The discovery of the electron and the success of Maxwell's electromagnetic theory tempted some to make these the fundamental building block of all physics, deriving mass and/or mechanics. Though Lorenz had pointed out the incompatibility with quantum ideas early, Gustav Mie attempted to integrate relativity into the electromagnetic world view, with a field theory of matter. These ideas were particularly popular in Germany and influenced mathematicians and theoreticians such as Hilbert, whose program envisaged reducing physics to pure mathematics, and Weyl. In this context Kragh also discusses Einstein's rationalism.
Based on relativity and making much of constants such as the number of electrons in the universe, Milne and Eddington undertook "grand and enormously ambitious projects aimed at a full reconstruction of physics modelled on abstract mathematics". There were some empirical results from these projects, but "the agreement was not all that important to Eddington, who was unwilling to let a conflict between a beautiful theory and empirical data ruin the theory". Support for such rationalist cosmologies was largely restricted to the United Kingdom and a high profile debate in Nature in 1937 pitted proponents against critics such as Herbert Dingle.
Looking at debates over whether cosmology was scientific or not, Kragh focuses on the steady state theory, originally proposed by Hoyle and Bondi and Gold. This took as a fundamental postulate "the perfect cosmological principle", that the large-scale features of the universe do not vary with either space or time.
"Arguments that appeal to metascientific principles such as unity, simplicity, and coherence are by their very nature ambiguous. What some scientists saw as methodological virtues of the steady-state theory, others saw as deficiencies and reasons to distrust it."Some of the criticisms were just as abstract and philosophical, however, including attempts at purely logical refutation. Being easy to falsify was a methodological strength of the steady-state theory, but actually being falsified was not so good for it. Hoyle and Narlikar continued with variants, however, notably oscillatory steady-state theories.
Heisenberg's S-matrix theory emphasized observable states before and after interactions. In the "bootstrap programme", Geoffrey Chew applied this approach to hadrons and the strong force, making all hadrons equally fundamental (an idea which influenced Gell-Mann). He refused to generalise this too far, but he saw the potential application to other forces and even to space and time, and turned to more philosophical justifications for the bootstrap when its empirical failure became clear.
These traditions are pretty much defunct as research programmes. In the second half of Higher Speculations Kragh turns to strands of speculative thinking that still, to a greater or lesser degree, drive research.
Kragh focuses on a few of the many ideas that have involved varying constants of nature. Following on from the ideas of Eddington — and using the "large number hypothesis", where very large dimensionless constants that are roughly the same must be related in a fundamental way — Dirac constructed a cosmology with a time-varying gravitational constant. In the 1960s a number of physicists considered the possibility of a varying fine structure constant, an idea which has seen renewed interest following potential testing by quasar observations. And between 1999 and 2003 there was a burst of work on cosmologies with varying speeds of light (VSL), though "interest in the idea has since declined".
Cyclical theories of the universe can be traced back to Origen and through the 19th century, relativistic cosmologies of the 1920s, when many difficulties "were not enough to kill an idea ... which a minority of cosmologists found too attractive to be wrong", and to Sakharov and other Russian physicists. The most recent major revival was in 2001 in the "ekpyrotic" model of Paul Steinhardt and Neil Turok, which was cyclic without singularities and drew on string theory. Citations show that this theory remains alive, but that it has never been nearly as popular as standard inflation cosmology. This and some other cyclical theories will potentially be testable in the near future, using gravitational wave detectors.
Some kind of an anthropic principle can perhaps be found as far back as Lucretius, but the modern idea derives from Robert Dicke, who took Dirac's large number hypothesis in a different direction, and Brandon Carter, who coined the name "anthropic principle" and popularised it. Kragh traces the history of this principle, both "strong" (the universe must be such as to support the existence of observers) and "weak" (properties of human observers have to be taken into account) and considers the claims of successful predictions (though these are often "postdictions" only claimed as such much later).
Again Kragh goes back to the Greeks with his background, but multiverse ideas really begin with Everett's "many worlds" interpretation of quantum mechanics. (And "multiverse" itself appears to have first been used in a scientific paper title in 1998.) Though "ideas of many universes were well known about 1980, the majority of physicists and cosmologists tended to consider them heterodox, weird, and speculative". The idea's "increasing popularity" has been driven by links to anthropic thinking, to string theory, and to speculation about a theory of everything (here Kragh looks at Tegmark's "Mathematical Universe Hypothesis"). This has brought criticism: Is multiverse cosmology science? Can we have inherently unobservable worlds? What do we do with infinities in the interpretation of probabilities? And multiverse proponents disagree on whether to reject Popperian falsificationism or whether their theories are compatible with it.
Kragh's account of string theory goes back to Kaluza-Klein theory, the revolution sparked by the 1984 Schwarz-Green paper, and Witten's 1995 introduction of M-theory, as well as covering the current state of research. He also surveys criticisms of string theory, where "never before in the history of science has [sic] so many resources been devoted to a purely theoretical resource programme over such an extended period of time", and considers an alternative in loop quantum gravity.
A final chapter glances at SETI and astrobiology but focuses on speculation about the far future and the possibility of the indefinite survival of intelligent life, by figures such as Dyson and Tipler. This is an appropriate way to end the book, since ideas of cosmobiology and physical eschatology are perhaps the most speculative of all those considered, venturing close to religious belief and science fiction.
A short epilogue touches on broad connections between speculative theories and "theories of everything", religion, and the possibility of an "epistemic shift" and changes in the definition of science. Otherwise there is little attempt to fit the material in Higher Speculations into larger frameworks.
The case-study structure means that the discussion of many general topics is scattered. Themes which recur repeatedly but which don't receive an integrated treatment include the influence of falsificationism, the epistemological status of cosmology, the correlation of theories with nationality, and the persistence of abandoned scientific ideas among religious believers and spiritualists. The emphasis on the perspectives of participants mostly prevents any repetition here being entirely wasteful, however.
Kragh makes no real attempt to generalise about the role of speculative thought in physics. Alongside the two parts of Higher Speculations, covering failed, dead-end speculative ventures and live but controversial ones, it would have been interesting to have looked at examples of ideas which were similarly inspired or influenced but which are no longer controversial because they have been integrated into accepted science.
Kragh assumes familiarity with basic cosmology and fundamental physics, and with their broad history, but only at the level of the better popular science presentations. He doesn't shy away from equations or relevant details, but doesn't assume knowledge of tensor analysis or quantum field theory or anything like that. He also provides some background in his exposition of the history. Along with philosophers and historians of science, researchers within the area and curious outsiders should also appreciate Higher Speculations, though its depth and detail — it is not itself popular science — will deter a mass audience.
The case-study structure also has its advantages here: the chapters are largely self-contained and can be read individually by those with specific interests. Anyone just after a thirty-page introduction to anthropic science and its history, for example, could do a lot worse than read Kragh's chapter on that.