I scroll through most of the posts in my Twitter feed, but one recent post caught my attention and has occupied my thoughts for days now.  The tweet, which recounted a conversation between a scientist and their parent, conveyed an intriguing point: The emergence of different findings from COVID-19 research seems abnormal to some because the scientific process usually progresses to a later stage before findings are disseminated to the general public.  For me, this post prompted a question that tapped into my interest in the philosophy of science: How is it possible for different findings to emerge from science over time?  Considering this question can shine a light into a process that may seem like a black box and help us make sense of scientific study in its earlier stages.

To understand how different findings over time are possible, we have to start with what the late Stephen Jay Gould called the “basic mode of reasoning in empirical science”: induction.  Scientists use induction when they draw general conclusions from isolated observations made under particular conditions.  These general conclusions are used to predict what will happen in situations where the particular conditions appear to be the same.  If you think about this carefully, you’ll quickly realize we all reason this way in life; it’s how we experience and anticipate the world around us.  For a simple example, let’s say you purchase a fancy, new, one-cup coffee maker that you’ve never used before.  During the first seven days of use, you observe that pressing the brew button results in a cup of goodness (dark roast, if you’re like me) within minutes.  Understandably, as you press the button on day eight, you do so with the expectation that you’ll soon have a cup of coffee to enjoy.  This expectation stems from your general conclusion that hitting the button always results in brewed coffee within minutes, which was formed from isolated observations of this occurring in the past.  This type of reasoning is premised on the assumption, or belief, that there is uniformity in the material world.

Now, what happens when your expectation fails and the coffee cup doesn’t fill with brewed coffee in minutes?  In these moments of relative distress, you’re not likely to question the uniformity of nature.  Why? Because you recognize that the issue isn’t with nature, the issue is the particular conditions have changed in some way, even if you aren’t sure how.  In this case, let’s say that you accidentally hit a different setting that only brews based on a pre-set timer.  As failed predictions occur and your awareness of possible conditions increases, the general conclusion that guides future expectations gets more nuanced and precise.  Now your more refined, general conclusion is that hitting the brew button will always result in brewed coffee within minutes when the timed brew setting is not enabled.  It is important to note that this refined conclusion is different from the initial version.

This same idea is true for the scientific process.  When a scientific prediction fails to match reality, there are four possible explanations: 

1. The specific observations used to formulate the general conclusion were flawed (the “data problem”).
2. The general conclusion doesn’t accurately reflect the specific observations that were made (the “logic problem”).
3. The particular conditions are different (“the environmental problem”)
4. Nature is no longer uniform (“the existential problem”).

The scientific community uses the practices of peer-review and reproducible experiments to mitigate the data and logic problems and few, if any, scientists ever question nature’s uniformity.  However, similar to our coffee analogy, the environmental problem can still cause predictions to fail despite quality data and logic.  When this occurs, increased knowledge of possible environmental conditions is used to refine the general conclusion regarding the phenomenon being studied.  By necessity, this process of scientific refinement yields general conclusions over time that are different from each other.  Now, multiply this dynamic by numerous researchers and labs and the likelihood of different findings due to different environmental conditions grows quickly.

There are at least two important things to note about the occurrence of different findings over time.  First, it seems reasonable to think that their frequency is likely to be greater for new areas of scientific study.  When a new area of study begins, there is relatively little awareness of the possible environmental conditions that may exist.  The early stages of studying a phenomenon are likely to see more rapid increase in awareness and a more rapid refinement of general conclusions, which will generate more findings that are different.  In the later stages, after a longer period of study, this process of scientific refinement is likely to occur less rapidly and lead to fewer differences.  Second, and most importantly, the environmental problem isn’t just a limitation of science.  It’s a human limitation.  Ultimately, the environmental problem is unavoidable in science and in daily human reasoning because humans are not omniscient.  Sometimes we forget that scientists are indeed humans.  If scientists specifically, and humanity in general, are both limited by the environmental problem, what distinguishes the scientific method from average human reasoning?  One of the major differences is the general, communal desire within science to eliminate the data and logic problems discussed earlier.  While this doesn’t mitigate all issues involving flawed data or logic, it does help to ensure that general conclusions about the material world are as close to reality as possible.

By Jonathan Hanes

The opinions expressed above belong only to the author and do not represent the University of Wisconsin-Milwaukee.