Reproducibility

Many research results are difficult – sometimes impossible – to reproduce. Science advances by trial and error: failing to reproduce a result is not always a failure, but part of a process that could be ultimately self-correcting.

Indeed, when we plan a scientific experiment, we accept a (relatively small) risk of wrongly concluding that an effect exists. Each stage of the scientific method is crucial to ensuring that this risk is fully controlled. In practice, however, this process can be affected by a number of problems. These can include methodological shortcomings, lack of power in experiments, too much flexibility in statistical analyses, selective publication of results, and so on.

Research has been carried out to assess the reproducibility of results published in the scientific literature, for example in the fields of psychology and cancer biology. It has shown that difficulties in reproducing the results of experiments published in journals considered prestigious are frequent.

The notion of reproducibility cuts across different facets of trustworthy research. We can thus cite :

• the reproducibility of methods, which indicates that study procedures and data are described with sufficient precision to be exactly repeatable.
• the reproducibility of results, which means that identical results are actually obtained when a new study is carried out independently, with procedures identical to those of the initial study.
• the reproducibility of inferences, which goes even further, and takes into account the interpretation of results. It relates to the fact that identical conclusions can be drawn from a re-analysis of the original study and from a new experiment.

All aspects of reproducibility can be greatly improved by an open science approach, and by numerous tools such as those presented below.

Florian Naudet