A Claude skill for pre-submission peer review
Open-source skill to use Claude + Consensus for pre-submission mock peer review backed by citations to published, peer-reviewed literature.
I’ve used QED Science and the Nature Research Assistant to review manuscripts I’m writing before I submit. They’re fine.
There’s an argument to be made for openly developing SKILL.md files or prompts for AI-assisted peer review, so others can take, use, and modify to fit their needs. I’m sure all QED/Nature/whatever are doing is wrapping a frontier model with a detailed SKILL.md and/or a lightweight harness, with connectors to PubMed, bioRxiv, etc.
A few weeks ago I wrote a Claude skill to do just that. It leans on Consensus, and the Consensus Claude MCP connector, so that all the resources it cites during a mock peer review come from actual published and peer-reviewed literature.
Here’s the skill: github.com/stephenturner/skill-peer-review-assistant.
The peer review assistant takes a file manuscript and produces a structured peer review report grounded in live literature searches through Consensus. It runs several targeted queries: checking whether the paper’s central claims hold up against the broader literature, finding recent high-impact papers absent from the reference list, and assessing whether the methods the authors used have been superseded. The output is a Word document with named sections covering background accuracy, missing citations, methods assessment, major and minor concerns, a clear recommendation, and a full audit log of every search query and result count. It only cites what Consensus actually returned in that session, flags any searches that failed or hit plan-tier caps, and distinguishes between concerns that threaten the paper’s conclusions and those that don’t.
In the workshop I taught a couple of weeks ago I demonstrated using this skill on an actual manuscript I had in prep. It’s toward the end of the video here.
For demonstration purposes here, I ran the skill on a paper that I already published.
Nagraj VP, Benefield AE, Williams D, & Turner SD. (2024). PLANES: Plausibility Analysis of Epidemiological Signals. PLoS ONE 20.3 (2025): e0320442. DOI: 10.1371/journal.pone.0320442.
I wrote about the paper here.
Here’s the peer review I got back.
The Claude skill I wrote leaves a lot to be desired. I like how Nature Review Assistant goes through the entire manuscript and calls out problems and unsubstantiated claims as inline comments in a Word docx. But that’s the point — if we develop these skills and harnesses as a community in the open, we can all work to make these better and customize them to particular fields or journals.
In addition to the benefits of open-source / community development, I also have the benefit of consolidated control over my data. If you’re paying for Claude or you’re on an enterprise plan, they’re not training on your inputs or outputs.1 Keeping the ecosystem contained means I have fewer third party companies’ TOS I need to keep track of.
Personally, I’m not too concerned here anyway. The whole point of writing a manuscript is to eventually publish said manuscript.