Evaluation moves in mysterious ways

With the development of web-based technologies, the new generation of scientists–often referred to as digital natives–will not be evaluated in the same way as previous generation scientists, known as the digital migrants.

This creates a generational divide.

It also could create some potential tensions. Some, among experienced science practitioners, may fear that the advent of new evaluation methods may jeopardise their chance of further career development. They may be concerned that new evaluation methods would not be as prone to recognise their merit. They do not recognise next generation metrics, such as altmetrics, because they are not fully validated yet. They may also look at evolving forms of peer-review, such as open peer-review, suspiciously, due to its lack of confidentiality. It is thus understandable that some would display quite a strong level of resistance to change.

These tensions reflect, to a certain extent, the existing divide between the digital natives and the digital migrants.

By contrast, in these times of rapid technological change, the digital natives themselves may feel that there is lack of adequate evaluation system to assess them. It is all very well for them to focus on doing good science. But they also need to devote some of their time ensuring that the community knows about their research and recognises their achievements. Technology is changing the way scientific results are being communicated, and most importantly, shared and accessed by the scientific community and everyone else. Therefore, it is important to recognise good science communication too. Until this is achieved, it may therefore be difficult for these newcomers to identify by which of the new metrics they will be judged; be it for career progression or for their next grant award.

Measuring what constitutes good science remains partly limited by the capabilities of the metrics used. Further consensus has yet to be achieved in the community on how best to evaluate research. Measuring good science communication may be trickier even, due to the lack of maturity of science communication 2.0.

Meanwhile, to identify good work from talented scientists, regardless of whether they are digital native or migrants, review panels have one more option. Having taken into account all available measurement methods—however technologically sophisticated—they often still resort to the old-fashioned metrics that is common sense.

Featured image credit: CC BY-NC-ND 2.0 by Gwen Vanhee

Go back to the Special Issue: Research evaluation

Sabine Louët

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.