Will your software be able to pass the ten-year reproducibility challenge? This article presents the findings of 38 scientists who took the challenge and attempted to reproduce results from previously published computational papers.
| Resource information | Details |
|---|---|
| Article title | Challenge to scientists: does your ten-year-old code still run? |
| Author | Jeffrey M. Perkel |
| Focus | Scientific Software Reproducibility |
Challenge to scientists: does your ten-year-old code still run? presents interesting findings from the challenge, originally conceived in 2019, and taken the following year by a total of 38 participants. The challenge dared scientists to reproduce results from computational papers they published ten years previously, or earlier. The purpose was to find techniques to produce code and results that can be reproduced a decade later.
While you may not experience the problems that were encountered by the participants, such as retrieving software from a floppy disc or other obsolete technology, or having your hand-written notes destroyed in a fire, it would still be wise to consider what would be necessary to allow you to reproduce your results in ten years. While there are many current technologies and practices that aid in reproducibility, there are also many new challenges, such as "rapidly evolving API's and reliance on third-party libraries" and fragmented code due to use of computational notebooks. One common thread suggested from participants to encourage reproducibility is to increase documentation. Additional useful ideas can be found in the included reproducibility checklist.
