How to SNIFF out Good Scientific Software
New PLOS Computational Biology paper: "Ten Quick Tips to SNIFF Out Sustainable and Secure Scientific Software"
For over a decade, the Ten Simple Rules and Quick Tips series at PLOS Computational Biology has told computational biologists how to build software the way software developers would.1 Hardly any of that guidance speaks to the researcher, the one deciding whether a tool someone else wrote is safe to build an analysis on.
This week I published a paper in PLOS Computational Biology with my long-time colleague VP Nagraj and a few other colleagues here at the UVA School of Data Science: Neal Magee, Tom Stewart, and Karsten Siller.
Nagraj VP, Siller KH, Stewart T, Magee N, Turner SD (2026) Ten quick tips to SNIFF out sustainable and secure scientific software. PLOS Computational Biology 22(7): e1014510. https://doi.org/10.1371/journal.pcbi.1014510.
This paper differs from many that came before it by taking the user’s side of that issue described above: how as a user of scientific software to you sniff out whether the software you’re looking at is sustainable, secure, well-maintained, etc. We organize the advice around 5 features that spell SNIFF: source, network, interaction, fit, and fragility.2
Source and network are about provenance: who maintains the tool and why, and whether it sits inside a governed community like Bioconductor, Galaxy, or nf-core, or whether it stands alone. Interaction and fit are practical checks: how responsive the maintainers are, how engaged the user base is, whether the tool matches your problem, and whether its license permits what you intend to do. Fragility is the dependency stack, and whether the thing needs root or a disabled security setting to run.
We wrote SNIFF as a set of guiding principles. Our paper guides potential users through signals that together suggest the tool will still run and still be supported a year from now. A tool can be well cited but thin on documentation, or cleanly packaged and kept alive by one overworked person who stops answering issues the month they change jobs. Sustained development, more than citation count, tracks with whether a bioinformatics tool produces accurate results. We know this.
Throughout writing this paper I questioned whether a scholarly article was the best outlet for advice like this. I think most of our tips will hold up over time, but the last tip will probably be the one that ages the fastest: scrutinize unreviewed AI-generated code. A single author, enormous one-off commits, and inconsistent style used to be a marker of a rushed project, or something thrown up on GitHub at a reviewer’s request without much intention of maintenance. Now these can also describe something vibe-coded into existence over a weekend. Those surface signals don’t cleanly separate those two anymore, so we point to positive evidence instead: an automated test suite, continuous integration on every change, and pull requests merged by someone other than the author. Tools like git-ai are beginning to annotate AI authorship at the commit level, which could replace some of the guesswork.
The trust question is getting harder because AI has made it cheap to publish something that looks maintained. It’ll have a clean README, a few tidy commits, and maybe even a container that builds. SNIFF is a framework to evaluate whether the code you’re evaluating has the community and infrastructure behind it that’ll give you more confidence that the tool and its developers are playing the long game.
Go read the paper:
Nagraj VP, Siller KH, Stewart T, Magee N, Turner SD (2026) Ten quick tips to SNIFF out sustainable and secure scientific software. PLOS Computational Biology 22(7): e1014510. https://doi.org/10.1371/journal.pcbi.1014510.
Just to name a few. We cited more in the paper (not listed here):
Galiwango R, Whalen CJ, Kebirungi G et al. Ten simple rules for building and maintaining sustainable high-performance computing infrastructure for research in resource-limited settings. PLOS Computational Biology 2025;21(9):e1013481. https://doi.org/10.1371/journal.pcbi.1013481.
Gallagher K, Creswell R, Lambert B et al. Ten simple rules for training scientists to make better software. PLOS Computational Biology 2024;20(9):e1012410. https://doi.org/10.1371/journal.pcbi.1012410.
Hunter-Zinck H, De Siqueira AF, Vásquez VN et al. Ten simple rules on writing clean and reliable open-source scientific software. PLoS Comput Biol 2021;17(11):e1009481. https://doi.org/10.1371/journal.pcbi.1009481.
Lannelongue L, Grealey J, Bateman A et al. Ten simple rules to make your computing more environmentally sustainable. PLOS Computational Biology 2021;17(9):e1009324. https://doi.org/10.1371/journal.pcbi.1009324.
List M, Ebert P, Albrecht F. Ten Simple Rules for Developing Usable Software in Computational Biology. PLOS Computational Biology 2017;13(1):e1005265. https://doi.org/10.1371/journal.pcbi.1005265.
Nüst D, Sochat V, Marwick B et al. Ten simple rules for writing Dockerfiles for reproducible data science. PLOS Computational Biology 2020;16(11):e1008316. https://doi.org/10.1371/journal.pcbi.1008316.
Taschuk M, Wilson G. Ten simple rules for making research software more robust. PLoS Comput Biol 2017;13(4):e1005412. https://doi.org/10.1371/journal.pcbi.1005412.
Wilson G. Twelve quick tips for software design. PLOS Computational Biology 2022;18(2):e1009809. https://doi.org/10.1371/journal.pcbi.1009809.