Drivers of research may need to be tweaked to ensure better contribution to society
John Ioannidis is professor of medicine, health research and policy, and statistics at Stanford University where he holds the C.F. Rehnborg chair in disease prevention and directs the Stanford Prevention Research Center (SPRC). Prior to moving to Stanford in 2010, he was professor at the University of Ioannina School of Medicine in Greece.
He is best known for his work on biases that pervade research and on how to improve research methods and practices. He is the co-founder and co-director of the Meta-Research Innovation Center at Stanford (METRICS) which focuses on advancing research excellence and avoiding the common situation where most of the published research are false. In this interview with EuroScientist he reflects on possible new drivers for research to help scientists do research for the common good in the future.
Do research drivers need to be revised to enable research to be more effective?
Clearly, there is plenty of room for improvement. Most systems focus on productivity and generation of ‘significant’ findings.
Publications become tokens that buy grants and both of them buy faculty promotions. Less sophisticated universities and research institutions are happy even with just generating more papers, regardless of their impact. This ends up generating tons of low quality waste. More sophisticated universities also focus on claiming widely publicised discoveries, but most of them are not really as important as touted and often they are totally false and impossible to replicate.
I think that productivity and discovery-seeking is not necessarily a bad driver of research per se. But it needs to be coupled with emphasis on high-quality and rigorous methods.
Other aspects that are currently undervalued and need to become more important as drivers include replication/reproducibility, a sharing culture, and focusing on the translational potential of research–at least for applied research–as I have outlined in a Viewpoint article in JAMA in 2014.
As part of the research reorganisation, do you believe that policies related to grant allocation need to be revised too? If so, how?
I always marvel about how grant allocation strategies were developed. They are different across countries, funding agencies and institutions, but their common denominator is that they have no scientific evidence to support how they work.
We don’t really know what the best way is to review science. Peer review is important and I trust is useful, but it is unlikely that we are exploiting its full potential. Some empirical studies suggest that most current peer review practices allow for large components of randomness, may allow major errors to pass through, and are negative towards out-of-the-box, truly innovative ideas.
This does not mean that I would immediately replace the way things are done with other processes simply because they seem better. Science needs to be scientifically studied. This applies also to policies of grant allocation. This should be done ideally with rigorous, comparative, experimental studies.
Should the drivers of research be made better aligned with serving society’s needs? If so, do you believe that we need to reward those who do? If so, how?
In principle, absolutely, research needs to serve society’s needs. However, the devil is in the details.
Society is best served when scientific research is well-funded. And when the best and brightest are attracted to work in science as well as when research is done with rigorous methods that safeguard getting the most credible results possible. It is also best served when conflicts of interest are minimised.
Research that directly addresses a problem of major importance for society should have priority when it comes to investigations that have clear deliverables. However, there is a large share of research where there are no clear deliverables and it is impossible to tell ahead of time where, how, when, and if societal benefit will be reaped. This type of blue skies research is also important.
Therefore, society is best served in principle when we do the best research we can do and in the most rigorous way. And we need to reward those who perform such research, regardless of whether they can immediately develop deliverables or not. How to best do this should be the focus of scientific studies rather than philosophical, administrative or political agendas.
What kind of new social contract do we need between scientists and citizens?
Scientists have a contract to seek the truth, no matter what. Given my medical background, I would also add that scientists also have an imperative to do good and do no harm – broadly defined beyond just health. Beyond that, scientists should have no primordial obligation by default to other social structures, be it governments, political parties, social lobbies, industry, corporations, NGOs, or other.
Scientists can help citizens by providing accurate information in making the best decisions and avoid being misled. The ideal goal would be to end up in a society where all citizens are also scientists to some extent–that is, they have a fair, balanced view of how much we know; and, more importantly, how much we don’t know.
Do you anticipate potential issues in giving a greater say to society’s representatives in grant allocation policies or other form of participation from people outside research concerning future research directions?
I am very much in favour of involving society’s representatives in grant allocation practices, especially for grants that do have deliverables and aim to have a practical application that will affect people’s lives. However, we need more empirical evidence to understand how to do this and how to avoid the introduction of even more biases. For example, in a study we did in PCORI, we found that involvement of patient stakeholders resulted in changing extensively which grants were selected for funding. Half of the grants prioritised by scientists were discarded and replaced by grants selected by patients plus scientists.
But this does not guarantee that the outcomes will necessarily be better or that society’s representatives cannot be biased. Indeed, industry-related conflicts may well manipulate the views of patients and citizens and society preferences; perhaps even more easily than they can manipulate well-trained scientists.
Featured image credit: John Ioannidis
Go back to the Special Issue: Common Good Economy
- All good things come to an end - 30 March, 2018
- Ivo Verbeek: cutting the middle man in language editing - 21 March, 2018
- Podcast: How open science could benefit from blockchain - 31 January, 2018