Depends on what “it” we are trying to solve. If metascience is to go the distance, it needs a larger ambition than improving how we produce scientific knowledge.
Sitting near the front row at AEI’s A Conversation with Michael Kratsios, I expected grievances. I expected rhetorical flourishes. I expected obfuscation. I expected highfalutin political theater laden with references to a new golden age.
Instead, I found myself in what felt like an unusually stimulating graduate seminar, with a guest speaker offering crisp answers to Professor Tony Mills‘s insightful questions about his recent publication, perhaps one of the most consequential critiques of U.S. science policy in recent memory from a sitting administration.
I was transported back to when I was Michael Kratsios’s age. He is now the director of the White House Office of Science and Technology Policy. I was a freshly minted PhD with a hot take on institutional innovation and technological change: a more productive science does not necessarily mean richer economic or social rewards.
“The way we operate the machine matters as much as how many coins you’re putting in the machine” – Michael Kratsios
I thought many of the issues Kratsios raised would resonate with Don Kash and Dan Sarewitz, the two science-policy voices who live rent-free in my head.
Don was concerned with the increasingly complex organizational and institutional arrangements required to go from knowing to doing. Japan’s success in overtaking the U.S. in critical technology industries, for example, was as much about coordinating production, organizational learning, and industrial policy as it was about advancing technology.
Kratsios is also concerned about the separation between explicit and tacit knowledge and the physical distance between science, engineering, and development. He cites Elon Musk’s approach of keeping engineering design and production in the same building in Texas rather than offshoring manufacturing to Taiwan. He points out that it’s not just the people with STEM PhDs that matter. I suspect Don would agree with that assertion.
Dan was concerned about the “public” part of “public science”: “The problem with the input-output model is that it can’t tell us very much about what actually matters: how the size, organization, and productivity of the R&D enterprise itself relates to the achievement of the societal outcomes that we desire and expect,” he testified in a 2010 House Science Committee hearing.
Kratsios is concerned about the enterprise as well: “I don’t know about causation, but let’s … do some correlation stuff here. Like, you know, we continue to be attempting to do the same thing with just [a] variable amount of money as an input, hoping that the outputs [are] going to change. In reality, it hasn’t. So, I think … the way we operate the machine matters as much as how many coins you’re putting in the machine.”
“Technological stagnation …. in the world of atoms in an era where … we had this massive revolution in the world of bits.” – Michael Kratsios
Kratsios, however, seemed more narrowly focused on one particular aspect of the machine, scientific productivity. This distinction became clearer when Tony asked whether he agreed that “science itself was slowing down.”
The two voices in my head would care about scientific productivity. However, their concerns extended to other critical components of the scientific enterprise. In his work with Barry Bozeman, Dan argued that public purposes needed to help shape the direction of research from the outset. Otherwise, even commercially successful innovations could result in public-value failures. Similarly, Don, in his work with Bob Rycroft, argued that we needed adaptive public policies. Climate setting, surveying, coordinating, and gap-filling needed to be part of an integrated response.
Kratsios connected the technological stagnation problem Peter Thiel wrote about to the productivity problem in science. He agreed that we shouldn’t evaluate the health of the scientific enterprise by the size of the R&D budget or the number of STEM PhDs. Yet Kratsios seemed most eager to launch a government-wide metascience initiative to improve the efficiency of the knowledge-production machine, with scientific breakthroughs as the desired payoff.
Is that not a version of the linear model he so eagerly wants to leave behind? Perhaps not entirely. His appreciation of tacit knowledge and proximity between research and production suggests otherwise. But why, then, concentrate so much of the reform effort on the organization of scientific discovery?
“A Roadmap That All of Us Were to Drive” – Bob Galvin, former CEO, Motorola
This tension between Kratsios’s systemic diagnosis and his comparatively narrow reform proposals is what perplexed me most in the Science, A New Golden Age report when I first read it. I felt that by jumping from 1945 to 2025, it missed the critical learning and course corrections in the middle years. The more I listened to proposed solutions, focused research organizations, golden tickets, I-PhDs, and metascience units everywhere, the more convinced I became that we needed to recover some of the institutional learning about technological innovation and governance that occurred during the 1970s and 1980s.
One episode from that period, which I examined in depth two decades later for a dissertation chapter, kept coming back to me: the development of Motorola’s Technology Roadmaps.
In the mid-1970s, Motorola struggled to keep pace with its competitors despite considerable scientific and technological capabilities. Its chairman, Bob Galvin, came to recognize that the problem wasn’t simply what the company knew, but how it organized what it knew, what it anticipated, and what it committed to doing about it.
During a visit to the Motorola labs in 1976, Galvin joined a meeting between senior executives and laboratory managers and expressed his frustration with Motorola’s technology governance. He declared that the company needed “a roadmap that all of us were to drive.”
Galvin drew a simple chart and announced that he would return in six months for a roadmap review. A few weeks later, John Mitchell, Motorola’s chief operating officer, asked whether he really wanted it. Yes, Galvin replied. Mitchell told him the proposed approach was incomplete. “Fine, do what makes sense,” Galvin responded.
Mitchell and Bill Weisz, the CEO, recognized an opportunity to formalize what they had been attempting for years: bringing the technological and business sides of Motorola into a common planning process. They turned to Martin Cooper, who was then leading the development of Motorola’s cellular telephone. What began as a simple executive request evolved, through nearly five years of experimentation and refinement, into a company-wide process for improving technology governance. The roadmap also included a formal governance system and six core principles, including joint commitment, periodic reassessment, and institutionalized dissent.
The roadmap was not a static prediction of future technological developments. It became a shared framework for connecting technological possibilities, business strategies, and resource commitments. Periodic reviews let the company revise its course, while minority reports kept alternative technological pathways alive. Its value lay as much in the organizational learning it enabled as in the knowledge it generated.
“The Societal Outcomes That We Desire and Expect” — Dan Sarewitz
Compared with the federal government, Motorola faced a relatively straightforward question of purpose: how to improve technological competitiveness and regain market share. Yet even with that common objective, it needed more than better forecasting or more efficient R&D management. It needed an institutional process that could integrate explicit and tacit knowledge, align organizational commitments, and adapt to changing technological and market conditions. In the words of Don and Bob Rycroft, Motorola needed to shift its technology governance process from being static, analytic, and isolated to being dynamic, synthetic, and integrated.
I have little quarrel with experimenting to improve reproducibility, reduce administrative burdens, encourage risk-taking, or allocate research funding more effectively. But even if these experiments succeed on their own terms, how will we know whether the collection of reforms adds up to an innovation system capable of delivering desirable societal outcomes?
Kratsios clearly recognizes that scientific discoveries don’t automatically become useful technologies. Yet his proposed reforms seem to place extraordinary weight on accelerating discovery, while giving comparatively less attention to how the broader innovation system should be governed to achieve public purposes, “the societal outcomes that we desire and expect,” as Dan said in his 2010 testimony.
And what should those public purposes be? In a subsequent interview with Tyler Cowen, Kratsios reaffirmed the same broad objectives Bush outlined in the opening section of his report: “Is the research ultimately resulting in a better life for Americans, a healthier life for Americans, and protecting our national security?”
These are worthy aspirations. But what does a better, healthier, or more secure life mean to Americans today? Who decides which outcomes should take priority, and how should we reconcile competing objectives? What measures would indicate whether we are making progress? And how do we link those measures to the inputs, outputs, and institutional interventions that metascience seeks to optimize?
Most importantly, how do we keep revisiting these choices as science, technology, and society evolve?
If we are willing to experiment with how scientific research is funded and conducted, shouldn’t we also experiment with how we define its purposes, connect it to technological innovation, and evaluate its contribution to society?
If Metascience is the answer, what is the question?
A week later, as my graduate-student self returned to the no-longer-50-something me, Don, Dan, and Galvin were still occupying my thoughts. So was the question of whether Kratsios had framed the innovation problem sufficiently.
There is much to get excited about in Kratsios’s diagnosis of the problem, particularly his willingness to experiment with how we fund, incentivize, evaluate, and administer public science. But what is “the roadmap all of us are to drive”? The challenge isn’t simply to optimize the scientific funding machine. It is to build the capacity to steer an evolving innovation ecosystem toward public purposes that themselves must remain open to reconsideration as science, technology, and society change.
Metascience units seem like promising experiments, just as the roadmap was when Galvin first pitched it to his top managers. However, he was sufficiently aware of the complexity of Motorola’s problem to let his managers challenge and substantially transform his original proposal.
What Motorola realized it needed was an approach that was distributed yet integrated, open yet committed, and adaptable yet principled. My hope, therefore, is that Kratsios extends his experimentation beyond individual reforms to the institutional arrangements that connect scientific research, technological innovation, and public purposes. The danger is settling prematurely on interventions simply because they are easier to deploy and scale.
If Kratsios looked beyond government in the innovation governance challenge, he would find a wide array of established and emerging approaches across the academic, non-government, and private sectors that address different parts of the innovation puzzle. Strategic foresight and roadmapping help institutions anticipate technological trajectories and link possibilities to decisions. Participatory technology assessment and public-value mapping help reveal societal priorities, conflicting values, and consequences that technical experts may overlook. Anticipatory governance and responsible innovation offer frameworks for connecting assessment, reflection, engagement, and institutional adaptation.
The point is not to adopt all these approaches everywhere. It is to investigate how they can work together as part of an adaptive system of innovation governance that can learn and self-correct in real time as scientific, technological, and social frontiers change. Our experience building the ECAST network shows how participatory technology assessment can be socialized, tested, evaluated, adapted, and gradually integrated into public decision-making, even without an established institutional home.
However, none of these tools, in isolation, will make much sense or be as effective, transformative, durable, and sustainable if they are not framed as a challenge to connect scientific productivity with outcomes, including economic, social, cultural, and institutional elements. The challenge cannot be solved within the boundaries of individual projects, programs, or agencies, even though much of the experimentation must begin there. If metascience is to go the distance, it needs a larger ambition than improving how we produce scientific knowledge.
What the moment calls for is a science of science policy outcomes that improves our understanding of how research investments, institutions, and interventions contribute to desirable societal changes. But knowing is not enough. We also need an experimental system of innovation governance that connects knowledge to agenda setting, resource commitments, policy decisions, and continuous adaptation. The two must develop together.
I walked into that room expecting political theater and found myself in a graduate seminar. I walked out encouraged by the willingness to question old assumptions, yet wondering whether we were still asking too small a question.
Kratsios wants to improve the machine. Galvin showed us how to build a roadmap we could all drive. Don reminded us that the terrain keeps changing. Dan insisted that the destination matters. Perhaps the next golden age of science needs all four.