"What I cannot create, I do not understand." – R. Feynman
Jonas Kubilius

Reproducibility encouraged by publishers: finally!

So, today Nature Publishing Group says:

Too many biologists still do not receive adequate training in statistics and other quantitative aspects of their subject. Mentoring of young scientists on matters of rigor and transparency is inconsistent at best. In academia, the ever-increasing pressures to publish and obtain the next level of funding provide little incentive to pursue and publish studies that contradict or confirm previously published results. Those who would put effort into documenting the validity or irreproducibility of a published piece of work have little prospect of seeing their efforts valued by journals and funders; meanwhile, funding and efforts are wasted on false assumptions. Raising standards | Nature Neuroscience

And so they are now going to ask to report explicitly:

  • Sample size and justification that it is sufficient for the expected effects (e.g., power analysis)
  • Justification why any participants were not reported
  • Randomization procedure
  • If investigator was blind to group assignments
  • How many times an experiment has been already replicated in the lab and if it ever failed
  • Statistical method details:
    • name of test or exact procedure
    • does it meet assumptions (e.g.,  no normality violations?)
    • one-sided or two-sided tests
    • similar variance between groups?
    • multiple comparisons
    • accounting for small sample sizes
    • means/medians
    • error bars
    • p values
  • Step-by-step protocols
  • Open sourcing materials, data, and protocols (hello, figshare)
  • A number of field-specific details such informed consents, reagents etc.

I am excited that journals are starting to recognize the importance of reproducibility just when I’m debating multiple (crazy) hours I put in producing reproducible research and receive no thanks. Importantly, publishers are slowly realizing that they, together with funding agencies, are the power that drives these good practices to be adopted by the academic community by explicitly requiring to report them. I’m not a good friend of publishers, but today NPG once again positively surprises me.