Should research always have clear objectives?

The biblical story of Saul finding his kingdom by chance while searching for his father’s lost donkeys offers an important lesson for scientists. Instead of defining our research objectives narrowly, we must open our minds to completely different and more exciting discoveries that may be lurking at the periphery of our field of view.

Arno Penzias and Robert Wilson made such an unexpected discovery in 1965, when their attempts to reduce the noise in their state-of-the-art radio antenna led them to discern the cosmic microwave background. They noticed a noise floor, which turned out to be the radiation left over from the Big Bang. This watershed discovery, which fundamentally changed our view of the universe, was made at Bell Labs – not at a premier research university.

Assembling new data is essential to scientific progress. Data play the important role of guiding scientists toward new discoveries and solutions, as well as to new puzzles that need to be solved, thereby keeping the scientific process honest and dynamic. Extended periods without new data facilitate – indeed, foster – the unrestrained growth of speculative-theory bubbles.

Even failures to explain puzzling data are essential to the scientific process, with the challenges that data pose encouraging creative individuals to develop new ways of thinking about physical reality. Over extended periods of time – decades or longer – a data-driven culture without programmatic reins offers such extensive benefits that profit-oriented businesses often choose to support it.

Most famously, Bell Labs recognized the virtues of such a culture in the 1930s-1970s, assembling a collection of creative physicists to whom it gave unadulterated freedom. This ethos produced some of the twentieth century’s most important discoveries in science and technology, including the foundations of radio astronomy in 1932 and the invention of the transistor in 1947.

Bell Labs scientists also developed information theory in 1948, solar cells in 1954, lasers in 1958, the first communications satellites in 1962, charged-coupled devices in 1969, and fiber-optic networks in 1976. Without patience and foresight, Bell Labs could not have reaped the tremendous long-term benefits of these breakthroughs.

In science, as in any complicated and creative endeavour, uniform opinions and approaches will always prove sterile. The coexistence of disparate ideas cultivates competition and progress.

Of course, it is difficult to know which exploratory path will bear fruit, and there is no shortage of novel scientific ideas that were proven wrong. Failure should thus be accepted as a natural ingredient in a culture of innovation.

The fact is that high-risk research, just like high-risk capital investments in the business world, has the potential to be more profitable than safer approaches. If even one non-mainstream idea bears fruit, it could transform our view of reality and justify all of those heterodox hypotheses that do not.

Albert Einstein, for example, did not foresee the importance of his innovative theory of gravity to the development of precise GPS navigation systems. Similarly, Christopher Columbus, funded by the Spanish crown, sailed westward to find a new trade route to the East Indies, but discovered the “New World” instead. His financial backers clearly benefited from his unexpected discovery, as he claimed parts of America for the Spanish Empire.

The lesson is clear: while it is important to justify flagship scientific missions by what we expect to find, funding should be based mainly on their potential to lead to unexpected discoveries.

This is not to say that agenda-driven projects do not also lead to important breakthroughs. The recent discovery of the Higgs boson was the culmination of a programmatic experimental effort to confirm a theoretical idea, proposed in the 1960s, that lies at the foundation of the standard model of particle physics. Though the discovery was anticipated, the future advances that it will enable are unforeseeable.

It would not be prudent for agencies to allocate all of their funding to high-risk research. But they should allocate a small fraction – say, 20% – of their resources to research that is not tied to specific goals. Such a funding scheme is essential for promoting breakthroughs in the long run, because it encourages researchers to take on risky projects with fundamentally unpredictable outcomes but potentially high gains. Most important, it would give individuals the freedom to respond to unexpected insights as they arise, rather than compelling them to follow a prescribed agenda.

This approach demands an understanding that progress will not be steady over time, because discoveries rest on extensive preparatory work. It is thus inappropriate to measure success based on the contemporaneous level of allocated resources. Lost resources (time and money) should never be a concern in a culture that is not tied to a specific programmatic agenda, because an unexpected discovery could be far more valuable in the long term than these lost resources.

The advice Saul received from Samuel, who crowned him King after their chance meeting, is appropriate: “As for the donkeys you lost three days ago, do not worry about them…”

Published in collaboration with Project Syndicate

Author: Abraham Loeb, Professor of Science and Chair of the Astronomy Department at Harvard University, is Director of Harvard’s Institute for Theory and Computation.

Image: Physicist Urs Duerig looks at a prototype of an IBM NanoFrazor 3D nano printing tool at a laboratory of IBM Research in Rueschlikon, near Zurich, April 23, 2014. REUTERS/Arnd Wiegmann