Diverging Goals of Science

If you ask a scientist why their research should be funded, you’ll usually get a story about the grand innovations their field is working towards. A cell biologist may find new ways to harness the immune system against cancers, a geologist’s discoveries may enable better prediction of dangerous seismic events, and a quantum physicist may pave the way for improvements in quantum computing. (And given the current political climate, I should probably add that these are genuinely great reasons and should more than justify funding scientific research.)

But if you ask a scientist why they actually do their research, the answers are often different. Responses vary from person to person, but my favorite answer that I see repeated again and again is some iteration of “because science is really cool.” Understanding how our immune system can reliably identify threats is fascinating! Understanding the materials which make up our planet is incredibly interesting! Understanding what the hell is going on with quantum physics is awe-inspiring! 

Plenty of people enjoy thinking about scientific discoveries and dreaming about their implications, but it’s hard to imagine why anyone would sacrifice endless hours at relatively low pay just to discover the details of how the world works unless that person absolutely loves discovery. Not the far-off implications of their discoveries, but the knowledge for knowledge’s own sake. 

Usually these two goals– bettering humanity and seeking understanding– have been harmoniously aligned. And correspondingly, we have adopted scientific methodologies as canon that allow us to simultaneously pursue these dual goals. One such methodological tool is the metric of “predictive power.” It is hard to identify what constitutes understanding, especially the partial understanding we incrementally improve through scientific exploration. But we can give ourselves operationalized goalposts through the metric of predictive power. Predictive power allows us to meaningfully compare scientific theories. Our current quantum-informed atomic model is an improvement on the Bohr model because we can now better predict atomic behavior. 

Predictive power serves the goal of bettering humanity by centering the practical results of scientific theories: the ability to predict how the world will behave. If we can predict when the immune system will target cancerous cells, we can develop cures. If we can predict volcanic eruptions, we can plan around them. If we can predict quantum behavior, we can build technologies reliant on it. 

Predictive power also serves the goal of seeking understanding, albeit indirectly. Generally, if we want to improve our predictive power, we need to meaningfully understand something new about the system. In order to better predict the spread of illness, we needed to actually understand germ theory. We never could have achieved the level of predictive power we currently have in epidemiology without that understanding. And so we can justify our search for deep understanding by pointing to predictive power. Without understanding, our predictive power will remain limited. So in order to improve predictive power, we must seek understanding. 

But crucially, improvements in predictive power do not necessarily equate to improvements in understanding. Even historically, major scientific discoveries can be critiqued as improving predictive power without actually improving understanding. After Newton’s famous characterization of gravity, Leibniz lamented the conflation of understanding with mere naming and mathematical description: 

How do bodies fall? If left to their own Gravitation (so termed) they fall perpendicularly to the earth. If impelled sideways, they obey the measure of that impulsion; but fall to the earth notwithstanding. They describe an arch in their fall proportionate to the power with which they are impelled– but it will be an arch. 

Then how does this happen? We mistake, I believe, the effect for the cause! 

Leibniz does not deny the equations which describe gravitation, but he denies that putting a term to those equations actually amounts to meaningful understanding. Newton’s theory of gravity may be correct and predictive, but it offers no explanation as to the origin or mechanisms of the phenomenon. 

If the formalization of gravity had not permitted us to describe celestial bodies with the same equations as earthly ones, I’d be sympathetic to Leibniz’ point. Most of the meat of the theory of gravity, at least in terms of pushing forward understanding, is in the realization that the same force can apply to every single object, regardless of size or shape or location in the cosmos.

But Leibniz does force us to disentangle those points. To simply state that objects behave according to a given equation does not make us understand why. Understanding only really comes when we can either answer or– more commonly– simplify a why. Gravitation simplifies and unifies a whole host of those questions: “why doesn’t the moon fall to earth?” and “why do the tides behave as they do?” and “why do earthly objects fall?” are all transformed into the singular question “why is there gravity?” That form of understanding– the unification and simplification of disparate questions– is a powerful one. Most forms of scientific understanding can be framed as similar projects of unification. But if a concept does not simplify or answer a scientific question, then no matter how powerfully we can predict the world with that concept, we have not gained any real understanding. 

Take, for example, the study of neural networks (and, by extension, more complex architectures like LLMs which structurally rely on neural nets). We love to call these “black boxes,” but really they’re about as transparent as a box can be: we know every weight, every learning parameter, every training rule. If you had enough time, you could sit at home with paper and pencil, work out the gradient descent, and calculate every update. We actually have amazing predictive power over these things. It’s just that instead of a unified F = ma, we have a completely unsimplified explanation. I know the exact math to do to calculate how a given neural net would respond to any input. It’s the exact math that the neural net itself does! I can do every single calculation of every single node and predict the output pretty much perfectly.

Despite that perfect predictability, we still call this problem unsolved. This is partly practical– perfect predictability is useless if I can only apply it to one input at a time out of infinite possibilities. But I think this also reflects our notion of understanding. Even if the practical concerns disappeared, I still think we would consider neural nets to be “black boxes.” We can know all the equations that govern a neural net without actually understanding what they’re doing. Just as the equation of gravity does not tell us the real cause of gravity, the complete mathematical picture of a neural net does not tell us why neural nets respond as they do or how AI’s emergent behavior comes about. Not in a manner comprehensible to a human mind anyway. 

Therein lies the tension: we may be fast approaching a moment where the limits of what human minds can comprehend no longer dictate the limits of our practical capabilities. Our two scientific goals– understanding the world and bettering humanity– are approaching a fundamental divergence. We are used to brief divides. Sometimes theory outpaces practice and sometimes, with luck, we stumble into practice beyond our theory. But this new divide is already a much greater chasm, and the speed of AI advancement only promises to enlarge it.

For the first time in human history, we are creating prediction machines with the potential to race well beyond our human understanding. 

Even prior to the recent whirlwind of LLM advances, these prediction machines have impacted research. Diagnostic tests used to be limited by our understanding of which features were diagnostic. And, while sometimes we could identify a few features by correlation alone, the best diagnostic tests were built on a mechanistic understanding of why differences would arise between sick and healthy groups. Now, we don’t even need to know which features matter as long as a prediction machine can predict correctly. Understanding be damned; there are human lives at stake. If the collection of unsimplified equations that constitute a neural network can correctly classify moles as benign or cancerous, who cares whether or not our human minds can express the rule they’re using or not? 

If we were just talking about “old school” machine learning, I would probably pause here and make the case that we should still invest in understanding for the sake of bettering humanity. Sure, image classifiers can sometimes beat out pathologists, but how much better could medicine be if we actually understood why? Then we could build even better tests, even better classifiers, and, ultimately, even better cures. Simple machine learning can help us diagnose, treat, and innovate a bit beyond our intellectual foundations, but it is still limited by our scientific understanding. Improving our understanding would still work towards the goal of bettering humanity. 

In fact, that is the kind of case I used to make all the time. I used to be (and still somewhat am) frustrated by the “distraction” of using machine learning to chase small gains in predictive power without improving understanding. Sure, I thought it was good to use technology to optimize applications, but I was worried that failing to sufficiently invest in the pursuit of deep understanding would hold us back from the kind of major developments that transform lives. 

Those arguments hold for the time being (especially in fields which require actual experiments to be performed), but the past few years have convinced me that they may not hold for much longer. It seems increasingly plausible that current AI models will soon become capable of transformative developments if they are not already at that level. The current focus is on math and perhaps computer science as these are the most accessible to LLMs, but there aren’t very deep lasting reasons that it ought to stop there. 

With OpenAI’s recent deluge of mathematical proofs, the research community still claims the authority to judge the value of this AI-generated knowledge, and OpenAI still sticks to the line that AI will be a tool for research rather than a replacement. But with the incredible pace that AI can work at, it’s not hard to imagine a world in which AI can build upon its own work faster than we can check it. I am not a mathematician, so I hesitate to make strong claims on the importance of this particular release, but my impression is that mathematicians themselves certainly see the transformative power of this ‘tool’ (so termed). 

If these kinds of AI advances can outpace us, they may render human understanding unnecessary for progress. Furthermore, I don’t see a reason that an AI-driven research regime would be bounded by the same limitations as human understanding: we may need to simplify and unify questions, but that is not necessary to achieve better predictive power. If that is the metric we choose, AI may not only render our understanding unnecessary for progress, but instead turn it into a hindrance. If we only act on discoveries that have been understood and accepted by people, we will at best slow down life-saving progress and, at worst, stymie entire fields of work which deal with unsimplifiable systems. Medicine has already accepted some of these compromises, and other fields of research may soon be called on to join it.

In my own field of neuroscience, I was recently at an event in which the topic of research in the world of machine learning came up. Someone raised the question at the back of many scientists' minds– how do we go about science in a world where “predictive power” is increasingly divorced from understanding? Like many fields, we’ve already been long lamenting the steady stream of “throw a decoder at it” papers, in which a complex dataset is fed into a prediction machine and the resulting opaque but successful predictive power is published as a result. But this was the first time I’d seen the question raised without placing blame on those papers. Instead, we asked what to do if those papers actually became the correct way to achieve some of the practical goals of neuroscience. If we take seriously the possibility that we can best get predictive power using opaque AI tools which interpret decoders that are not human-readable, we’re left with the tough question of deciding how understanding fits in the picture, if it even fits in at all. 

Personally, my answer is that I care intrinsically about understanding. I am in neuroscience because I want to understand how neurons underlie complex thoughts. I spend my days in lab because the brain is an insane structure capable of incredible things and I want to discover how and why it works. Even if that is not the most pragmatic path, it is the one I want to go down.

That answer was reasonably popular among the neuroscientists at that event. People are motivated to do science because they want to understand the world. But that is not the only perspective. It’s all well and good to value learning for learning’s sake individually, but science receives popular support and government funding in exchange for the promise of producing practical advancements. Currently, we bundle the support for understanding with the support for advancements– we treat the mission to understand neural computation as the same mission that will cure Alzheimer’s and treat mental illness.

I have no intention of arguing that we should jettison pragmatism: we don’t need to. Rather, I argue that we should adapt to our changing intellectual landscape by disentangling the goal of understanding from the goal of practical betterment of life. I believe that both goals are valuable and each goal ought to be its own independent priority.

It seems obvious to me that we should better our lives. This is effectively tautological under most moral systems: doing good is good. If the best way to do good is to use well-tested but opaque means of attaining predictive power, we should still do it.

I also believe that pushing forward human understanding is good, independent of possible applications. Even if we reach a world where we’ve entirely alleviated suffering with amazing advancements in medicine, I would still want to understand why those amazing advancements work. We are curious creatures with a unique capacity to work together to satisfy that curiosity better than any individual could alone. We’ve built up structures such that the average undergraduate understands the world better than prominent scientists did mere centuries ago. High school physics covers discoveries that took generations upon generations of thinkers to figure out. We all have the privilege of standing on the shoulders of giants– who in turn stand on shoulders on shoulders on shoulders. I, for one, would like to keep on climbing. 

Different people may disagree on the exact balance of how much we should value and, by extension, materially support each of these two goals. I am not intending to argue for any particular weighting of these priorities right now: I am merely trying to disentangle the goal of understanding from the goal of application in the hopes of facilitating clearer conversation. I openly value the goal of understanding very strongly, but I don’t expect that to be universal. What I do expect is that we will need to talk about these two goals as increasingly separate priorities as AI capabilities improve. Each is a priority in its own right and as it makes less and less sense to bundle them together, we will need to decide how to allocate our support to the independent projects of bettering humanity and of understanding the world. 

We’ve already accepted the principle that we can pursue an activity for its own sake in other domains. We’ve long eliminated the need for excellence in athleticism that used to be justified by the frequency of hand-to-hand combat or the plethora of manual jobs that no longer exist. Yet we still take pride in athleticism, both in athletes and in ourselves. It feels good to be fast and strong. Photography eliminated the pragmatic need for skilled realism in portraying the world, yet we still value the skill and those who cultivate it gain value and meaning through doing so. 

It may be time to explicitly extend this courtesy to the domain of understanding. Many institutions which have traditionally bundled understanding and application are beginning to fray at the edges. I have discussed the example of academic research, but that is not the only domain which is showing signs of this strain. A college degree used to bundle the value of learning with the value of a career payoff. But when the content taught in college courses diverges from the rapidly AI-driven changes in the real world, students question the value of the degree as anything but an empty credential. We can lament students cheating with AI all we want, but from a pragmatic perspective, they’re just responding appropriately to the incentives at hand. If Claude or ChatGPT can get their degree, it can probably do their subsequent job, too. Yet there still seems to be something distasteful about skipping that learning entirely– we fear that without sufficient education, we will have an uncritical and uninformed population. An uneducated world is an inherently undemocratic and anti-humanist place. Even if AI can do each pragmatic task better than a person could.

I don’t run because I need the skill. We have trains and planes and cars aplenty. I don’t practice calligraphy to produce texts. We have printing presses. I don’t, as a native English speaker, learn a second language because I realistically need it to communicate. I do all these things because they have intrinsic value to my life. People love to learn and improve themselves. The goals we can achieve are not less meaningful because they’re not practically necessary. Rather, we derive meaning itself from having those goals. 

We consider it perfectly justified to construct infrastructure that enables these kinds of goals. Good cities build parks and running paths. We subsidize free or low-cost museums to showcase art and libraries to read books. We offer K-12 education to better future generations– and we gladly include PE classes and music lessons even if these are unlikely to train children for pragmatic goals. In our own personal lives, most of us are also willing to chase these goals. We pay for education, for gym memberships, for art, and for the expertise of those who develop new ideas in those domains. I would gladly include efforts towards improving our collective understanding of the world within that category. 

AI-driven research can and should be used to save lives, even if that process is less satisfying than traditional human research. But the pursuit of human understanding is valuable in its own right as it helps make those lives worth living.