Another six things this week. Stretching to find a common theme here: each of them touch on knowledge in some way or another:

1. Anthropic and AE Studio’s GRAM, and why removing knowledge from weights isn’t the same as removing risk.

2. PhD admissions down, and what funding uncertainty does to a cohort.

3. A Brown economics class where the take-home midterm averaged near 100% and the in-person final averaged below 50%.

4. Watching long form reading erode (keep reading 😉)

5. AI labs becoming biotech firms.

6. on the “Bio Mythos” moment, and building biosecurity assurance.

And a note, with a very special thanks to my paid subscribers (remember, every post here is free and open, regardless of whether you pay): These kinds of recap posts take a lot of time and energy to write. And exceptionally so while avoiding AI for discovery, scanning, summarizing, triage. I also have a major, very major proposal deadline at the end of this month. While I have a few other essays I’ll be publishing soon, you won’t see another recap like this for a while. Follow some of the other Newsletters I recommend to keep you caught up.

Subscribe now

1. An off switch*

Anthropic and AE Studio published a method this week for giving a model a removable compartment per category of dual use knowledge.

• Blog post: https://alignment.anthropic.com/2026/modular-pretraining/

• Synopsis: https://alignment.anthropic.com/2026/modular-pretraining/

• Paper: https://ae.studio/research/modular-pretraining/modular-pretraining-enables-access-control.pdf

• Code: https://github.com/agencyenterprise/modular-pretraining

They call it GRAM, for Gradient-Routed Auxiliary Modules: add extra neurons to every layer, route the gradient updates from virology or cybersecurity or nuclear physics text into that category’s module during pretraining, then delete the module at inference when you don’t want the capability. One training run yields a model you can reconfigure many ways across domains instead of training many filtered models.

They compared with post-hoc unlearning. When they tried to fine tune a removed capability back in, the unlearning baseline (MaxEnt) recovered almost fully, while GRAM and plain data filtering held. Unlearning after the fact suppresses more than it removes.

* = The asterisks: everything is measured in next token loss and not actual downstream performance, the dual use data was a sliver of the mix, nothing has touched a production Claude model, and some capabilities may be

so entangled with general knowledge that no method can separate them cleanly.

Really cool work here but important to pair this with a well-designed uplift study, and better yet, measuring correlates of uplift. Lower next token loss on virology tokens isn’t the quantity that most people (policymakers, biosecurity folks, general public) care about. Uplift is: can someone holding the ablated model do the dangerous thing any less well than someone with a search engine? So model editing and unlearning work needs paired human uplift studies, same models, capability toggled on and off, measured against a real baseline. Removal that looks clean in loss space but buys no drop in uplift might look like safety but isn’t.

2. Planting an orchard

PhD admissions this fall dropped 15% from last year across 55 Association of American Universities members, per the AAU Data Exchange. Those schools confer about half the country’s research doctorates. This all stacks on last years numbers: for the 42 schools that reported fall 2025, new enrollments were already down. I.e., Two straight years of contraction at the institutions that produce most of the nation’s new scientists.

Some are far worse than the average. MIT expects nearly 20% fewer new graduate students, about 500 people, citing federal awards down 20% and the new endowment tax. Caltech is cutting new graduate admissions 40% across the board for fall 2026, and its graduate dean was explicit that the driver is a lack of funding certainty, not any specific cut. UW’s astronomy chair took zero new doctoral students this year, the first time since 2016.

I look at this with research dean hat on. You need students to drive research. The cohort you don’t admit this year doesn’t come back when the budget stabilizes.

Subscribe now

3. “We cannot choose to become idiots”

Roberto Serrano, who has taught welfare economics at Brown for nearly 20 years, gave a take home midterm for the first time this spring, partly because students were uneasy in a classroom after the December shooting on campus. The class, normally around 30, had 86. The midterm averaged 96% against a historical 65-to-80 range, on an exam he’d made harder. He ran the questions through ChatGPT, got answers matching his students’ work down to the same overcomplicated proof strategy, and told the class he suspected widespread AI use. He moved the final in-person.

18 people dropped, 9 skipped the final altogether, and of those who sat it the average was 48%, the lowest he’s recorded, against a prior floor of 65. 19 failed.

A 96% to 48% collapse between take home and proctored is strong circumstantial evidence the take home scores weren’t the students’ own work. It isn’t proof about any individual. Brown’s academic code committee wants separate complaints per student with exam copies. AI detectors would throw false positives and negatives in bulk. The committee’s own AI report, out the same week, recommends de-emphasizing punishment. His own summary was blunter:

4. The end of reading, again

The Atlantic’s August cover essay, by Rose Horowitch, argues America has gone postliterate: not illiterate, but losing the higher order comprehension sustained reading builds. Fewer than half of adults read a book of any kind in 2022. Reading for pleasure on a given day fell from 28% in 2004 to 16 in 2023, and gambling has passed reading as a leisure activity. Text now thrives inside a shrinking minority, about 20% of adults accounting for more than 80% of books read.

What are you reading now? Last week I finished re-reading Andy Weir’s Project Hail Mary (the encore was just as good). I just started reading Stewart Brand’s Maintenance of Everything, while listening to David Sedaris’s newest collection of essays, The Land and Its People (read by him).

5. A biotech startup now?

5a. Claude Science and drug discovery

at Scaling Biotech on Anthropic’s early-July drug-discovery announcements: Claude Science (which I test-drove here) and an internal effort to find candidates for rare and orphan diseases. His read is that it backfires commercially. Tokens are expensive, open weight models are closing the gap, and

the whole point of it is to make you use Anthropic’s models

instead of self hosting, so it has to stay pricey. Worse, an in-house drug discovery team reads to pharma customers as a future competitor trained on their data, however carefully the rare disease problem is chosen.

5b. Nature Biotech commentary

Related, Amelia Palermo’s Nature Biotechnology comment provides some structural context.

In April, Anthropic acquired Coefficient Bio for about $400 million (all stock): roughly 8 months old, <10 computational biologists, no clinical assets. Against a $380B valuation that’s about 0.1% dilution. In other words it’s a standard tech exit rather than a typical pharma one, with a value Palermo calls

a purchase of foundation model talent and code base.

She sees a new class of buyer, frontier labs acquiring biology platforms for models, data, and people instead of drugs, and predicts several such deals by 2030.

5c. Job ads

I just took a look at some of the job ads, which point to how serious these companies seem. Anthropic isn’t only reselling Claude to pharma. Looks like they’re building a wet lab, hiring bench scientists and the people to run it:

• Research Associate, Biology

• Research Operations Lead, Biology

• Research Scientist, Life Sciences (Experimental Biology)

• Biological Safety Research Scientist

OpenAI’s public bio hiring leans the a different direction, toward safety, policy, and red teaming. Which is near and dear to my heart. I can’t wait to see what this team does here.

• Biosafety Red Teaming Specialist

• Model Policy, Chemical & Biological Risk

• Product Manager, Bio Safety

• Product Policy, Biosecurity Policy Manager

• Researcher, Frontier Biological and Chemical Risks

Most of these listings are in the $300k-400k range, plus equity.

Share

6. Before the Bio Mythos moment

Governments tend to react to AI risk only after a surprise, and Claude Mythos was the cyber case: its ability to find and exploit weaknesses in critical infrastructure software pushed cyber capability to the top of the agenda, and the Fable takedown (reportedly 90 minutes for Anthropic to bar foreign access before it pulled the model worldwide) showed how little assurance infrastructure exists to make those calls deliberately. The controls have since been lifted, though Commerce keeps the right to reimpose them. The argument is that a “Bio Mythos” moment, a model crossing into dangerous biological capability, would trigger the same improvisation in a domain where the danger is harder to measure and, in their words,

self-replicating, offense-dominant, and can have enormous societal ramifications.

Their fix is to build the assurance infrastructure before that moment: independent evaluators with real model access and a liability safe harbor, standardized capability tests like their BioTIER, and a default against exposing advanced bio capabilities to unverified users.

Their Bio Capabilities Index shows the trend rising even as newer models add refusals. This is the missing half of the GRAM story in #1 above: you can build an off switch, but someone has to decide when to flip it, and that call needs more than a back channel phone call (maybe more well designed uplift studies, and better yet, measuring correlates of uplift). For my fellow Virginia readers, the state’s 2026 legislation already directed its technology commission to study an independent verification framework.

Subscribe now

I am submitting this comment in opposition to the proposed revisions to 2 CFR Part 200.

I write in my personal capacity as a research-active faculty member, not on behalf of my institution. I am an associate professor at a leading school of data science at an R1 university, where I also hold a leadership role in research administration. I have held multi-year federal awards from agencies including DoW, IARPA, DARPA, NIH, NIJ, and others, and I develop and review federal proposals as part of my regular work. My comments below describe concrete operational harms I expect from specific provisions.

I share the broad concern that converting decades-old guidance into binding regulation across hundreds of sections in a single action, with one government-wide effective date, leaves grantees little ability to assess and respond to each change. Several specific provisions warrant withdrawal.

Sec 200.205 (political review of awards). Requiring politically appointed officials to review awards and forbidding them from deferring to peer reviewers removes the quality control that protects the public’s investment. Merit review by qualified scientists is the mechanism that screens proposals for technical soundness and feasibility. Undefined criteria such as “anti-American values” add unpredictability that no investigator can plan around and invite inconsistent application.

Sec 200.340 (discretionary termination). A standard allowing termination mid-project for shifting “national interest” reasons, with only a brief written rationale and no appeal, converts every multi-year award into an at-will arrangement. I cannot responsibly hire a postdoc on a 4-year award, or commit a doctoral student’s dissertation to it, if the award can be canceled at any time for reasons unrelated to performance. The cost falls on staff, trainees, and in some cases study participants.

Sec 200.432, sec 200.454, and sec 200.461 (conferences, subscriptions, publication costs). These have been routine allowable costs for decades. Making journal subscriptions categorically unallowable cuts researchers off from the literature their projects require. Requiring agency pre-approval for each conference removes the venues where findings are presented and collaborations form. Most directly, disallowing publication costs contradicts the 2022 OSTP public-access requirement, which obligates these same researchers to make federally funded articles publicly available. A rule cannot require open-access publication and disallow the cost of achieving it.

Sec 200.220 (foreign collaboration). Fields including pathogen genomics and biosecurity depend on international data and partnerships, because infectious disease surveillance does not stop at a border. A broad prohibition on collaboration with covered foreign entities, rather than targeted restrictions tied to verified risk, would weaken the work that protects public health and national security while competitor nations face no equivalent constraint.

For over 15 years I’ve worked in defense of the American people, applying genomics to biosecurity and national security problems under federal contract from the Department of War and other federal agencies. That work made this crystal clear to me, that our nation’s security and its prosperity rest on the same foundation: a research enterprise free to follow the evidence, recruit the best talent wherever it is found, and publish openly. The provisions above weaken that foundation and would slow the discoveries that protect public health and keep the United States and its allies and partners ahead of its competitors and adversaries. I share the goal of responsible stewardship of federal funds, but that comes from merit review and stable, transparent commitments, not from political control over which science gets done.

I urge OMB to withdraw this rule. At a minimum, I request that the provisions identified above not be finalized.

Respectfully,

Stephen Turner, Ph.D.

Associate Professor of Data Science

If you haven’t been following what’s happening here, (former NIH program official for 2 decades) has a few recent posts laying out what this all means. You can leave a comment as well, anonymous if you wish. As I’m writing this over 140,000 comments have been submitted.

• The OMB Rule That Makes the Funding Cuts Legal

• My Public Comment on Proposed OMB Rule: Regulation for Federal Financial Assistance

• A Rule Nobody Voted On Could Cut Federal Funding to Your Community

• This new OMB Rule Is Bigger Than Science. Much Bigger.

• What We Need to do NEXT: OMB’s Proposed Federal Financial Assistance Rule (OMB-2026-0034)

• Summary of Key Changes in OMB’s Proposed Federal Financial Assistance Rule

Subscribe now

Wen Bin Goh, W., Polster, A., Wong, L. et al. Rethinking bioinformatics expertise in the era of artificial intelligence. npj Digit. Med. 9, 398 (2026). DOI: 10.1038/s41746-026-02777-1. https://www.nature.com/articles/s41746-026-02777-1.

My friend Tommy Tang, director of bioinformatics at AstraZeneca, wrote positively on his newsletter about the paper and what it means for bioinformaticians. Go read his post if you want a contrasting perspective to what you’re about to read here.

It’s a fine paper, and I agree with much of it. I think the paper misses something though. “The profession survives” and “your job survives” are kind of different things. AI can move analysts up the value chain but the upper tier was always small, and I don’t think AI is necessarily going to expand it.

When I ran a bioinformatics core here at UVA many years ago, and later when I led large teams across several companies in the national security sector, the org chart had two broad tiers, and the compensation surveys (Radford and the like) drew a similar line. It’s been a minute, so I can’t remember the exact terminology, but we had two named labor categories. There were bioinformatics analysts, often recent grads or master’s degree holders, writing and debugging code, running analyses, reporting up. And there were bioinformatics scientists, usually PhDs, who spent more time on design, interpretation, and client communication, and far less time in the terminal.

The two overlapped heavily. An analyst is a scientist, and a scientist still does analysis. And there was no caste system on my team. In fact I only remembered who was classified as analyst and who as scientist when we put proposals and cost volumes together for federal agencies.

But the scientist band paid substantially more, partly because there were fewer of those roles. One senior scientist could direct several analysts.

This paper’s argument seems to assume that when AI absorbs the analyst work, those analysts move up into the scientist tier and become the experts directing the AI. That tier was always smaller. The pyramid does not widen at the top because you automated the bottom. If AI now writes the code, debugs it, and runs the first-pass analysis that 10 analysts used to handle, you don’t suddenly need 10 more principal scientists. You need about the same number of senior people you already had, each overseeing more AIs and fewer juniors.

I think the paper skips another problem too. The analyst job was where people learned to become scientists. You misspecified enough DESeq2 design formulae, and after several discussions with me or other senior scientists on the team, you’d develop a feel for when an output is wrong or some contrast doesn’t make sense. Automate the apprenticeship1 and you cut off the path that produced the experts the paper calls irreplaceable.

Will demand expand enough to take up the slack? That I don’t know. Cheaper bioinformatics could mean more experiments get the full treatment and more projects become feasible. Even then, the composition shifts toward the senior tier and the entry rungs thin out.

The field being essential and the field employing as many people are different claims. “AI cannot judge biological meaning, so bioinformaticians remain necessary” tells you the profession survives. It says nothing about how many it employs, or who they are.

⁂

Now that I’m back in the front of a classroom, my job is clearer than it was years ago. The apprenticeship that used to happen on the job, the small corrections that taught an analyst when a result smelled wrong or whatever, these are the things industry isn’t going to pay for. If that kind of apprenticeship happens anywhere now, it’s got to happen here, deliberately, before a student enters the workforce.

So that changes what I teach. I used to teach programming to biologists in my Biological Data Science with R course (bdsr.stephenturner.us). This fall I’m flipping that around and I’ll be teaching genomics fundamentals to data scientists. I’ll be spending less time (or possibly none at all?) on the syntax that the models already write, and much more time on the things it can’t judge for itself, like whether a batch effect is passing for biology or whether a contrast even makes sense or what about the study population is causing that weird off-diagonal Q-Q plot of p-values from a GWAS.2

The students who learn that are (I hope!) the ones who end up in the small senior tier, the experts behind the AI rather than the people it replaced. There as few of those senior-level seats as there always were, and AI isn’t expanding them. What I can try to do now is make sure the people I send out are the ones who get them.

Subscribe now

In any case, here are the five things that interested me the most this week.

1. OpenAI builds a genomics benchmark

2. RAND asks whether LLM agents can drive biological tools

3. David Brooks on who thrives once intelligence gets cheap

4. NIH’s 235-word screen, and a list to check your abstract against

5. QED scores preprints and names the top 1%

Subscribe now

1. OpenAI grades its own homework

OpenAI’s Jeremiah Li and Andrew Ho posted GeneBench-Pro on bioRxiv, an expanded version of GeneBench. OpenAI also wrote a blog post about it.

Li, J. H. & Ho, A. J. GeneBench-Pro: Evaluating Multistage Statistical Reasoning in Genomics, Quantitative Biology, and Translational Biomedicine. 2026.06.29.735386 bioRxiv Preprint at https://doi.org/10.64898/2026.06.29.735386 (2026).

It’s 129 problems across 10 domains with a genomics core. Each one hands an agent a short bit of context and a target quantity to estimate, then makes it work through a series of dependent decision points. These decision points are the kind of inferential forks where one plausible wrong turn can reshape everything that comes downstream. The point is to test whether a model can run a realistic multi-stage analysis end to end and not just answer multiple choice questions (though some multiple choice questions can be challenging!).

The scores: GPT-5.6 Sol Pro tops out at 31.5%, GPT-5.6 Sol at 28.7%, GPT-5.5 at 12%, and the best non-GPT model, Claude Opus 4.8, at 16%. OpenAI wrote a benchmark, ran its own models on it, and reported that they win. Worth saying plainly IMHO. In fairness, they released some problems publicly and gave many others to Artificial Analysis for independent scoring, which is more transparency than most self-graded benchmarks bother with.

There’s lot’s of buzz about GLM 5.2 lately, so I was a little surprised to see it rank so low compared to other models here.

Failure modes are interesting. The models often do most of the workflow correctly, then show what the authors call “a consistent gap between noticing and acting”: they flag a diagnostic signal but don’t carry its implication over to the decision it should change, so they pick the wrong estimator or stay on a plausible but wrong path.

That matches what I saw in my Claude Science test drive the same day. It collected the data and ran the analysis without much help, and a quick read through its code turned up several spots where it had assumed something I wouldn’t have.

The timing was crowded. This dropped alongside Claude Science and the Fable 5 redeployment. If you use Fable 5, note the cost change: it’s included for up to half your weekly usage through July 7, then it runs on metered usage credits after that.

Fable 5 will be included for up to 50% of weekly usage limits through July 7, after which it will be available via usage credits.

2. AIxBio: Picking the tool is the easy part

RAND’s Center on AI, Security, and Technology (CAST) tested 7 frontier LLM agents on two questions: can they pick the right computational biology tool for a task, and can they operate one (report, PDF).

Lee, J. et al. Can LLM Agents Select and Engage with Biological Tools? An Initial Biosecurity Assessment. https://www.rand.org/pubs/research_reports/RRA4741-1.html (2026).

The case studies used EVEscape, which predicts immune escape, on influenza hemagglutinin and Lassa GPC, and ESM3, a protein design model, on GFP and influenza HA.

Asked to name the right tool for a stated function, the models did well, many around 80%. Put the same question inside a realistic biological workflow and accuracy fell across the board. Operating the tools was mixed and depended on the specific protein, and when the agents failed it was usually because they mishandled data or fumbled an autonomous step, not because they lacked the knowledge. Feeding an agent more biological detail didn’t reliably help, and on the HA-targeted ESM3 task more information sometimes made things worse, which undercuts the tidy “expertise plus AI equals uplift” story.

And the closed-weight models refused a lot, especially on viral proteins and obvious dual-use requests, through either the model declining or a content filter catching it. Agents can already handle the front end of this work, that could lower the expertise barrier for a non-expert, and it deserves closer testing. This is an initial assessment on toy tasks, and they say so, so I’d strongly resist reading it as evidence that agents can build a functional biological weapon.

Subscribe now

3. Intelligence is cheap, volition isn’t

David Brooks, writing in The Atlantic, starts with the observation that AI hasn’t handed anyone a 15-hour workweek.

He cites ActivTrak data showing that when workers adopted AI their email and chat time more than doubled, their business software use rose 94%, and their focused, uninterrupted work fell 9%, plus a Berkeley Haas finding that people started pulling previously-outsourced tasks back in-house because AI made them easy.

My favorite line in the essay:

He sorts people into three groups. Productive Passengers, low appetite for hard thinking, get more done and get hollowed out. Reluctant Optimizers mean to resist and get pulled in anyway. Mental Marathoners push back and use AI to widen their range rather than shrink it. Behind the hollowing-out worry he stacks a pile of studies: an MIT Media Lab result where brain connectivity dropped by half during ChatGPT-assisted tasks, a colonoscopy study where physicians’ adenoma detection fell after the AI was taken away, and a Wharton experiment where people accepted a deliberately-wrong model’s answers 80% of the time.

Some good practical advice at the end. Ask for hints instead of answers, write your own take on a blank page before you open the chat, and “ask for thinkers, not thinking,” meaning have the model summarize who has already worked on your problem so you can go read them. I started my AI Dry July this week, so I’ll report back on whether a month of added friction does anything measurable for my gamma waves.

4. A list of 235 words to avoid

Max Kozlov at Nature reports that hundreds of NIH grant applications are stuck in a screen that runs after peer review. An algorithm checks each application’s title, abstract, and summary sections against a list of disfavored terms, 235 of them as of February, including “gender,” “climate change,” “racism,” and “fossil fuel.” A hit flags the application, and the program officer is then told to renegotiate the language or drop the project.

This screen runs after two rounds of peer review and after the program officers and the institute director have already judged the work fundable. Then it enters a phase called Status 19, where NIH leaders and an HHS counselor can weigh in, with internal feedback that Nature obtained questioning whether a given study is worth funding because it looks “likely to end up in a Congressional waste report.” Most applications clear in two weeks, but a tenth of the renewals that reached this phase this fiscal year have sat for more than seven weeks, some indefinitely. NIH told Nature it keeps no banned-word list and doesn’t base funding on specific words; the internal documents suggest flagged projects simply draw far more scrutiny.

I pulled the 235 terms out of the article into a searchable page, with the CSV on GitHub, so you can run your own grant proposal against it. You can also search/page through the flagged terms below.

5. The 1%, scored

QED Science ran its QED Score, an anonymized AI metric for a manuscript’s originality and validity, across 57k bioRxiv preprints from May 2025 to April 2026 and named the top slice The 1%. Their whitepaper reports three validation studies: an AUC of 0.867 separating expert-labeled “Limited” papers from the rest on a 925-paper set, a Spearman ρ of 0.63 between a preprint’s score and the SJR of the journal it eventually landed in (scored with models whose knowledge cutoffs predated the corpus, to rule out contamination), and a blinded head-to-head where experts sided with the QED-favored paper in 75% of 60 decisive judgments.

I’ve been building my own peer-review skill in the open on the theory that most of these products are a frontier model wrapped in a good SKILL.md plus some connectors, and I’d rather that scaffolding be something the rest of us can see and fork.

Other open tabs

A few other tabs I have open that I’ll get to here soon.

• created a refusal benchmark for biosecurity risk assessment (result, paper, blog). I wrote about refusal theater here a few weeks ago.

• The Hitchhiker's Guide to Agentic AI: From Foundations to Systems

• Autodata: An agentic data scientist to create high quality synthetic data

• Paper mill cancer studies get double the number of citations as genuine papers

• Lab-created ‘SpudCell’ marks ‘stunning’ step toward building life from scratch

• This Cell Feeds, Grows and Reproduces. And It’s Manmade.

• posit::glimpse() Newsletter – July 2026

• Redeploying Claude Fable 5 \ Anthropic

• Am I an AI Luddite? - by Archie Hall - Archie’s Substack

Subscribe now

I took it for a test drive to do a literature review, and for another little project that involved data collection and analysis that I’ve been putting off for well over a year. It went off and worked on its own for a couple hours without intervention. While it didn’t one-shot everything I had in mind, it’s definitely a step change in how I’d have previously approached this little project.

Subscribe now

What Anthropic says about Claude Science

Screenshots in this section are from Anthropic’s blog post.

I’ve watched enough “AI for science” launches turn out to be a chat window with a biology system prompt. I tried to read into the blog post to see what parts that go past that. There are a few.

Provenance. When the app makes a figure, it attaches the code that produced it, the environment it ran in, a plain-language account of what it did, and the full message history that led there. This makes it sound like you can open a plot from months ago, see exactly how it was made, then ask it to drop the gridlines or switch an axis to log scale and watch it edit its own code. I.e., docs and chat logs attached to the artifact itself. Nothing revolutionary here, but some nice cohesion that my stack currently lacks.

Reviewer agent. While an analysis runs, a separate agent checks the outputs and flags bad citations, numbers it can’t trace to a source, and figures that don’t match the code that generated them, correcting as it goes. I.e., one agent producing and another checking, aimed at the part of comp bio where mistakes are hardest to catch and most expensive to ship.

It runs where your data already lives. Looks like you can install it on your laptop, a lab Linux box, or an HPC login node, and it writes and submits jobs over SSH. The Python and R kernels persist across the session.

Claude Science is pre-configured for genomics, single-cell, proteomics, and cheminformatics, connecting to 60+ scientific databases out of the box.

Subscribe now

Trying it out for myself

Screenshots in this section are my own.

I wanted to get a closer look at this before I start my AI Dry July. I downloaded and signed in. It’ll ask you which connectors you want to use, and which skills you want to enable.

It suggested a few things it could do for me. It suggested I let it run a literature search on AI and biosecurity, focusing on dual-use and human uplift. I also prompted it with a question of my own that would involve collecting data and doing some analysis.

Results below, but first off, I’ll note that it started chewing through my session limits very quickly. There’s a handy update to the Usage section that shows you which sessions are using prose/tools tokens the most. I burned through my 5 hour limit with two prompts, and ate about another $50 in extra usage. Which is actually incredibly cheap relative to the time it would have taken me to do some of what it did here for me.

Literature review: AI x Biosecurity

When I logged in Claude Science suggested I let it run a literature search on AI and biosecurity, focusing on dual-use and human uplift. I said sure, and it wrote a prompt for me.

Conduct a comprehensive meta-analysis of literature at the intersection of AI and biosecurity to identify key themes, gaps, and implications for research in this emerging area.

It started writing some code to search through OpenAlex, then ran that code, made some plots, and gave me back some spreadsheets of papers to look through, as well as a report (not really a meta-analysis) summarizing the state of the field, recent trends, geographical analysis, etc.

Some results and artifacts:

Ai Biosecurity Corpus

242KB ∙ XLSX file

Download

Download

IUCN red list analysis by taxonomy

When writing this paper that was published at Nature Reviews Biodiversity, In an earlier draft, I wrote something to the effect of species on the IUCN red list can be “downgraded” to less threatened categories (e.g., critically endangered to endangered, endangered to vulnerable). An astute reviewer picked up on this and called me out on it — if I’m going to say such a thing, I’d need to back that up with some actual data or analysis.

I never had the time to do this analysis, so I just deleted the line and submitted the paper without any analysis. But I always thought it would have been interesting. Are particular taxa “downgraded” more than other taxa? How does this differ over time and geography? How might those downgradings correlate or anti-correlate with loss of genetic diversity, as we discussed in the paper?

I gave it some rough guidance:

I’d like to pull some data from the IUCN API and do some summary analysis on trends in categorization by taxonomy class, preferably using R

And set it to work.

It started poking around the IUCN API using R, inside an R notebook running in my browser that I could interact.

It then presented me with an analysis plan for my approval.

After this it continued for about 2 hours1 until it got all the data and finished the first round of the analysis, complete with a few hundred lines of R code in an ipynb that I could go in and run in the browser. I’m showing a few of the results of the analysis below, along with parts of the narrative.

Snapshot: who is most threatened now

Across all data-sufficient species, 31.8% are threatened (VU+EN+CR). The threat is concentrated in plants, lichens, and several invertebrate groups, not the charismatic vertebrates:

• Cycads (Cycadopsida) are the most threatened large class on Earth — 69% of assessed species. Lichens (60%), mosses (60%), ferns/clubmosses (~45%), and flowering plants (43–45%) follow.

• The best-assessed vertebrates sit much lower: birds 11%, ray-finned fishes 15% — and because birds/mammals are essentially completely assessed, those figures are population-level truths, while most plant/invertebrate percentages are “of what’s been assessed so far.”

• Data deficiency is its own signal: sea cucumbers (66% DD), cephalopods (56%), and earthworms (54%) are groups where status is largely unknown.

Temporal: are categories getting worse?

Comparing each species to itself (earliest vs. latest global assessment, 36,492 taxa):

• Sharks and rays (Chondrichthyes) show the clearest deterioration — 30.5% uplisted vs. 8.1% downlisted (net +139 species), and the largest within-cohort risk increase (+0.35). Corals and cycads also show genuine net worsening.

• Important caveat: several classes (amphibians, dicots, mammals) show net “improvement” in raw numbers, but this is almost certainly not real recovery — raw category changes mix genuine status change with non-genuine reclassification (taxonomic revisions, improved knowledge, criteria changes). IUCN’s official Red List Index, which isolates genuine change, shows net deterioration for the comprehensively-assessed groups. The robust genuine signal here is the sharks-and-rays decline.

What I showed above is only a few lines of the analysis and results I’ve gotten. I’m reserving the rest of the story for a paper I’d love to eventually write once I get time to come back to this to meticulously verify everything it did for me here. I also need to step in and steer the analysis a bit. Taking a quick glance through some of the code it produced I can tell it made some choices here that I probably wouldn’t have made.

Subscribe now

What does Claude Science mean for science?

Claude Science is landing at an interesting time, just after several recent high profile AI co-scientist papers published in Nature, and with several federal funders’ signals pointing the same direction. And, interestingly, on the same day that OpenAI introduces GeneBench-Pro.

ARPA-H’s IGoR program, which we proposed to, wants an AI-driven, interoperable research system with standardized protocols and a marketplace of validated labs, cloud labs included, that execute experiments and feed results back into models, with a stated goal of producing validated knowledge much faster than we manage now. Just last week DARPA’s Biological Technology Office put out a request for information on advancing autonomous science for biological applications, asking the field where the real bottlenecks are. The request is for input, not money, but it’s clear where DARPA is headed. The agencies funding the next decade of biology are betting on autonomy.

I don’t read any of this as the scientist getting written out. I think the thing that’ll work will keep a live, fleshy, real-life person deciding which question is worth asking and whether a result that looks interesting actually is. My short test drive made that concrete. Claude Scientist automated collecting the data and running the analysis, and then in just a brief glance through the narrative and code, I caught the spots where it had assumed something I wouldn’t have.

Claude Science and other tools that will surely follow it raise the ceiling on what a single scientist can attempt. A grad student who can fold a protein and kick off a deep database/literature search before lunch is probably working on harder problems than the one who couldn’t. That’s the bet I hope we can hold funders and toolmakers to: build systems that make researchers more capable, not less necessary,2 and keep the researcher in the catbird seat where ultimate judgment and decisionmaking happens.

Subscribe now

There’s an argument to be made for openly developing SKILL.md files or prompts for AI-assisted peer review, so others can take, use, and modify to fit their needs. I’m sure all QED/Nature/whatever are doing is wrapping a frontier model with a detailed SKILL.md and/or a lightweight harness, with connectors to PubMed, bioRxiv, etc.

A few weeks ago I wrote a Claude skill to do just that. It leans on Consensus, and the Consensus Claude MCP connector, so that all the resources it cites during a mock peer review come from actual published and peer-reviewed literature.

Here’s the skill: github.com/stephenturner/skill-peer-review-assistant.

The peer review assistant takes a file manuscript and produces a structured peer review report grounded in live literature searches through Consensus. It runs several targeted queries: checking whether the paper’s central claims hold up against the broader literature, finding recent high-impact papers absent from the reference list, and assessing whether the methods the authors used have been superseded. The output is a Word document with named sections covering background accuracy, missing citations, methods assessment, major and minor concerns, a clear recommendation, and a full audit log of every search query and result count. It only cites what Consensus actually returned in that session, flags any searches that failed or hit plan-tier caps, and distinguishes between concerns that threaten the paper’s conclusions and those that don’t.

Claude Skill: Mock Peer Review

In the workshop I taught a couple of weeks ago I demonstrated using this skill on an actual manuscript I had in prep. It’s toward the end of the video here.

For demonstration purposes here, I ran the skill on a paper that I already published.

I wrote about the paper here.

Here’s the peer review I got back.

PLANES peer review

60.7KB ∙ PDF file

Download

This review was prepared with assistance from Consensus (consensus.app), which was used to search peer-reviewed literature for claim verification, citation completeness, and methods assessment. All search results are documented in the Audit Log.

Download

The Claude skill I wrote leaves a lot to be desired. I like how Nature Review Assistant goes through the entire manuscript and calls out problems and unsubstantiated claims as inline comments in a Word docx. But that’s the point — if we develop these skills and harnesses as a community in the open, we can all work to make these better and customize them to particular fields or journals.

In addition to the benefits of open-source / community development, I also have the benefit of consolidated control over my data. If you’re paying for Claude or you’re on an enterprise plan, they’re not training on your inputs or outputs.1 Keeping the ecosystem contained means I have fewer third party companies’ TOS I need to keep track of.

Subscribe now

A May 2025 executive order prohibits federal funding for dangerous gain of function (DGoF) research, defined through the 7 DURC criteria written into the 2012 dual use policy: enhancing harm, defeating immunity, conferring drug resistance, increasing transmissibility, altering host range, raising host susceptibility, or reconstituting an eradicated agent. The order also calls for a way to govern such work outside federal funding. Which research the term covers has become a funding and legal question, i.e., not just semantics.

The authors began with 20,099 papers, filtered them with dictionaries and named-entity extraction, then read titles and abstracts by hand. Over 73% concern gain of function mutations in human disease, cancer, neurobiology, and similar areas, with no microbe in sight. Among the 86 experimental papers in the presumptive DGoF set, most investigate antimicrobial resistance in fungi and bacteria, work that stays inside the original organism. Fifteen fulfilled the DURC criteria.

On the flip side, most research that does meet the DGoF definition never calls itself gain of function. The team documents 62 papers that move a sequence of concern into a heterologous microbe and give it a new pathogenic function, and only 12 use “GoF” anywhere in the title or abstract. Screening on the keyword would miss most of the research oversight is meant to catch.

Trained reviewers also disagree on the edge cases. Three biodefense professionals, each with more than 15 years of experience, evaluated 22 viral papers. They agreed on 9 as DGoF and 6 as not, and split on the remaining 7.

Godbold’s group offers what they have been building for over 20 years: a function-based catalog of sequences of concern, paired with the DURC criteria, so a reviewer can ask what a gene does to a host rather than whether a paper used a particular phrase. For oversight that has to draw clear and defensible lines, that is a steadier handle than the words authors pick for their own abstracts.

Subscribe now

In the workshop I talk about using Consensus,1 Consensus + Zotero, and topped it off with some really fun demos using Consensus + Claude via MCP.

Subscribe now

If you’re at UVA, make sure to keep an eye on what and are writing over at for announcements about future events like this and much more on what’s happening in AI at the University of Virginia.

AI Exchange @ UVA

AI @ UVA is a bi-weekly publication covering cutting-edge applications of Artificial Intelligence.

By Varun Korisapati

1. Cal Newport tells the AI labs to cut the doom trolling

2. SecureBio puts its AI biology scores on one dashboard

3. RAND’s pitch for a cryptographic paper trail on AI-designed biology

4. The anti-scaling law and AI’s crowding problem in drug discovery

5. Local models are finally good enough for real work

6. Heng Li’s minibwa replaces BWA-MEM

Subscribe now

1. Cal Newport wants the doom trolling to stop

I have read Cal Newport for years, going back to Deep Work, which I recommend to anyone who says they can’t find time to think. So I came to his new NYT op-ed already sympathetic: Dear A.I. Companies: The Doom Trolling Needs to Stop. His target is what he calls doom trolling: labs cataloging the catastrophes their models might cause while insisting they are powerless to stop building them. He points at Anthropic’s “When AI builds itself” report and at Sam Altman’s posting images of the Death Star before announcing GPT-5.

Cal’s argument: either the labs believe the risk is real, in which case the only defensible move is to stop and lobby everyone else to stop, or they do not, in which case the doom talk is marketing disguised as conscience.

I liked his analogy:

Imagine if the Ford Motor Company put out a report saying that it feared its popular F-150 trucks might soon start bursting into flames, but that there was nothing the company could do about it because automotive technology was too inevitable and important to slow down. You’re probably struggling to picture this scenario because no reasonable consumer product company would ever act like this.

No consumer-products company talks this way, and he thinks AI should not get a pass for it. (He also swings at the Fable and Mythos export-control mess, reading the administration’s move as calling Anthropic’s bluff. More on that below.)

I part ways a bit. Treating a frontier model like an F-150 assumes we already agree on what the product is for and how it fails, which is the thing still in dispute. I write more about this in the biosecurity tag here.

2. SecureBio puts its AI biology scores on one dashboard

SecureBio launched a public dashboard collecting its model evaluations on biosecurity-relevant tasks, from the Virology Capabilities Test to newer agentic benchmarks like ABC-Bench. It spans 9 companies and more than a dozen metrics across three years, rolled into a “Bio Capabilities Index” built on the same method as Epoch’s capabilities index. TLDR: frontier models now clear human-expert baselines on these tests and keep climbing.

3. RAND wants a cryptographic paper trail for AI-designed biology

A new RAND CAST report, Verifiable Audit Trails for AI-Enabled Biological Design Tools (Berke, Kilian, Griffin, Atanda, Vazquez) proposes a cross-tool, tamper-evident audit trail for AI-enabled biological design tools: hardware-bound cryptographic signing, hash chaining, and append-only transparency logs so a synthesis provider or regulator can verify how a sequence was produced. The building blocks are mature in adjacent fields, and the authors argue that the technical side is the easy part.

The report is admitting the real obstacles are not technical. Nobody is set up to govern shared infrastructure, the burden falls unevenly across the research community, and institutions lack the capacity to review audit records even if they had them.

Subscribe now

4. The anti-scaling law, and why AI might crowd drug development

Liang Chang’s essay was one of the the bests thing I read this week: The Anti-Scaling Law in Biology, and Why AI Could Make Crowding Worse Before Making Drug Development Better.

He starts from two real wins: Revolution Medicines’ pan-RAS inhibitor daraxonrasib, which moved median overall survival in treated metastatic pancreatic cancer from 6.7 to 13.2 months (HR 0.40), and Lilly’s VERVE-102 base editor, which dropped PCSK9 by up to 88% and LDL by up to 62% from a single infusion. Then he pushes back on the tech reflex that says the next hundred are queued up behind them.

His claim is an anti-scaling law. In software, proving one case lets you copy it a hundred times. In biology the first proof of concept usually consumes the single best target, because you never picked it at random, and the second is harder than the first.

AI raises the floor on execution (binder design, optimization, trial ops) without moving the ceiling on the step that decides whether a drug works, which is choosing the right target for the right disease. The odds a drug entering trials reaches approval sat near 10% in 2014 and about 8% by 2020, straight through the genomics and ML era. His fear is that cheaper engineering plus training-data gravity funnels everyone onto the same validated targets, so you get 300-plus CD19 programs instead of new biology.

5. Local models grew up

Vicki Boykis recently keynoted our Applied Machine Learning Conference here in Charlottesville. I loved her talk. You can watch it here.

In her most recent blog post, Running local models is good now, she makes the case that local models crossed a line in the last few months. On a 2022 M2 Mac with 64GB, she is now doing agentic coding locally with Gemma 4 at roughly 75% of frontier speed and accuracy, refactoring a notebook into modules and writing tests, all inside a Docker sandbox. Her test for whether a model is good enough is whether she still double-checks it against an API model, and she does that a lot less now.

The HN thread is a useful counterweight to the enthusiasm.

I don’t know about good, I use a lot of local models and they’re still pretty painful to run locally

You have dense models (qwen 27b, gemma 31b) who are pretty smart, but pretty slow

You have MoE models (gemma 26b, qwen 35b, north mini code 30b) who are pretty fast, but make a lot of mistakes

You need a lot of memory to run these well, quantization makes tool calling weaker, so most run at 4 bit quants and are wondering why it kinda sucks and that’s because you’ve essentially lobotomized the model (I recommend unsloth quants, i recommend 6bit for MoEs and 5bit for dense)

So you need a lot of compute to make the pre-fill fast, you need bandwidth to make the decode fast, you need a lot of memory to hold everything - lot of ifs

On top of that, your laptop becomes a loud hot churning machine, it’s uncomfortable to work with.

So are they good? not really. Do they work? yes

One commenter daily-drives Qwen3.6-27B and prefers it to the frontier models for his workflow precisely because he wants the model to type while he does the thinking, though another warns you should not generalize from Sonnet to a lab’s actual flagship. The recurring objections are cost (”some of us have a budget”) and reliability: small context windows, and local models slipping into hallucinated tool-call JSON, which is where agentic loops break. Vicki is describing a real shift for personalized, low-recency dev work, with the production bar still a step out.

6. minibwa retires BWA-MEM

Heng Li and Nils Homer released minibwa, and Li is direct about what it is: the full replacement for BWA-MEM.

Read the preprint. Instead of another drop-in clone that preserves identical output, minibwa makes breaking changes, splicing BWA-MEM’s variable-length seeding onto minimap2’s chaining and SIMD alignment, with ropebwt3’s SMEM search and aggressive memory prefetching. The result is about 4x BWA-MEM and over 2x BWA-MEM2 at comparable accuracy, under 20GB RAM, with native long-read and directional bisulfite support that BWA-MEM never had.

On accuracy, the comparison is closer than the speed numbers suggest. On simulated short reads BWA-MEM is slightly more accurate, and the difference comes entirely from centromeric and acrocentric regions where minibwa deliberately stops trying, since you cannot place those reads correctly anyway.

One footnote I enjoyed: the Rust rewrite, minibwa-rs, was mostly written by coding agents, which tells you something about where bioinformatics tooling is heading.

Subscribe now

In other news…

The Fable 5 and Mythos 5 shutdown: the Commerce Department issued an export-control directive citing national security, and Anthropic pulled both models for every customer to comply. Rounding up a few articles on the topic.

• Anthropic’s own statement frames it as a misunderstanding over a narrow, non-universal jailbreak (essentially asking the model to “fix this code”), and notes access to all other models is unaffected.

• The HN discussion of that statement runs the predictable split over whether this is a safety call or a political one.

• Katie Moussouris at Luta Security argues the export controls harm US cyber defense, since the flagged behavior is the same find-fix-test loop defenders run every day, and points to the freefable.org open letter.

• Transformer’s Shakeel Hashim reads it as an AI licensing regime arriving through the back door, arbitrary and post-hoc, built on export law never designed for this.

• Axios reports the shutdown came down to personality clashes and a communication breakdown with the administration, set off by Amazon’s Andy Jassy calling Treasury Secretary Bessent.

• The Atlantic argues the episode is how America loses the AI race.

• The Washington Post covers how Anthropic lost the White House’s trust, and then its flagship product.

• The NYT’s opinion page calls it the start of a new kind of conflict.

• The Superintelligence newsletter asks when a model becomes too powerful to stay public.

Subscribe now

And lastly, the ACM AI Letters Vol. 1 No. 2 (June 2026) is now available online.1 I want to read all of the articles in the new issue.

1. Balancing Comfort and Growth: A Dual-Mode Theory of Human–AI Development

2. Autonomy or Guidance: What Users Want from AI versus Human Advisors

3. AI Ethics Must Speak the Language of Business

4. Identifying, Evaluating, and Mitigating Risks of AI Thought Partnerships

5. Rethinking Scientific Practice in the Age of Artificial Intelligence

6. AI Collaboration and the Decentering of Human Creativity

7. The Harder Charge: Teaching Technical AI Literacy across Majors

Share

As with Dario’s previous (longer) essays this one’s a good read. Given what happened two days after this essay was published, I had to pause and read this section twice.

It is time to go beyond transparency to more serious and binding regulation of AI. I believe the best analogy, at least at the current stage of the exponential, is to cars, airplanes, or drugs—powerful technologies essential to the modern economy, but capable of killing large numbers of people if designed or operated poorly. I therefore believe we should model AI regulation on agencies like the Federal Aviation Administration (FAA). Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety. I am grateful to see the Trump administration’s Executive Order move incrementally towards a greater role for government in AI, though Anthropic’s proposal recommends even further action. Our proposal includes the following elements:

• Models above a threshold of compute should undergo mandatory testing by a qualified third party for their level of risk in four specific areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D that could accelerate these other risks.

• The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks. This power must be scoped to the above four specific risks and there must be protective measures against political favoritism or arbitrary decisions.

Subscribe now

Now I use Positron instead of RStudio, and Raycast (free version) instead of Alfred.1

With a few keystrokes (CMD-space to activate Raycast, type “pos” to search through projects in Positron) I can launch a project in Positron. Here’s what it looks like.

Here’s how to set that up. The gist is outlined in the Positron docs here.

Subscribe now

Positron + project manager

The first step is to install the project manager extension in Positron. Once you do that, the project manager appears as a stack of folders in the Activity Bar. When clicked, you gain access to a clickable, organized listing of your projects. Here’s mine, showing my favorite folders (with a redaction).

From here, in Positron, you can just click on a folder and it’ll open a new Positron window running in that project. This is essentially the same thing as double-clicking the .Rproj file in RStudio.

Subscribe now

Raycast

Next, install Raycast and the Visual Studio Project Manager Extension (yes, VS Code). In the settings, make a few changes, annotated below. Add “pos” as the alias, and in the options on the right, set “Positron” as your “VS Code”. I know that looks weird, but this is how it’s done.

Now, when you activate Raycast, you can type “pos” and you’ll bring up the Positron project manager.

From here you can select which project you want to open. Hit Enter and it’ll open Positron running in that folder/project.

The workflow is described here in the Positron docs.

Subscribe now

Share

Here are my open browser tabs I have open that I hope to catch up on soon.

Subscribe now

Blogs/newsletters/etc

1. Dario Amodei — Policy on the AI Exponential

2. Paving the way for agents in biology \ Anthropic1

3. NIH RFI on limiting the number of grants per PI

4. Claude Fable 5 and Claude Mythos 5 \ Anthropic2

5. Eight Predictions for the Future of Higher Education

6. Matt Lubin: Five Things: June 7, 2026

7. Is AI More Expensive Than the Employees It's Replacing?

8. The Anti-Scaling Law in Biology, and Why AI Could Make Crowding Worse Before Making Drug Development Better

9. Sasha Gusev: Thoughts on AI in academia

10. How LLMs Actually Work | 0xkato

11. A cell is not a spreadsheet- why “Virtual Cells” are still mostly hype

12. Anthropic: When AI builds itself

13. Can A.I. Produce Writing That We Actually Want to Read?

14. Is a compute crunch coming?

15. Built to benefit everyone: our plan | OpenAI

16. Bots are scraping open data — how should researchers respond?

17. Why Are Cells Small? - Niko McCarty

18. How to build a cancer vaccine, and whether they will work this time

Papers

1. The total eclipse of bioinformatics: From disruption to convention, and a gentle warning

2. Dual-use artificial intelligence and biology: upstream risk-benefit reviews

3. Molecular de-extinction looks to the past to find the molecules of the future

4. AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

5. Pleiotropic shared heritability quantifies the shared genetic variance of common diseases

6. Ten simple rules for teaching data science

7. Depth normalization for single-cell genomics count data and Lior’s explainer

Subscribe now

2

I haven’t had a chance to do anything with Fable yet, mostly because I work in AIxBio, and Bio is off limits. And because I’m a biologist, Fable refuses to talk to me (“Who am I?” leads to safety flags and demotion of the rest of the conversation to Opus). Precautionary principal is probably the right move here given the benchmarks, and I think managed access will likely be the way these models are released from here out.

I’m not an activist, and I don’t use this space for advocacy. As a rule, I keep my opinions about research politics and policy off the site (and anywhere else online in general). I’m breaking that rule today, because this reaches every part of the scientific research enterprise and whatever role the United States still hopes to play in it.

The Office of Management and Budget has proposed rewriting 2 CFR Part 200, the rule that governs how every federal grant gets spent. A Science editorial lays out what that would mean for research: every funding decision routed through political review, multiyear grants terminable with no due process, and case-by-case approval for any award that spends a dollar outside the US, which would effectively end most international collaboration. Steve Usdin of BioCentury, quoted in the editorial, called the drug industry’s silence “complicity in the destruction of US science.” The Science editorial ends with a rallying call:

, who spent >20 years as an NIH program officer, writes in an essay making the point that this is much larger than science. 2 CFR Part 200 is the universal framework for federal grants, so the same political override and discretionary termination provisions land on Medicaid, Title I schools, highway funds, and tribal health programs.

The comment period closes July 13. Ginexi’s how-to guide is a great place to start, and mechanics are important - since a thousand identical form letters count as one comment, so write your own, in plain language, and cite the specific provisions that would hurt your work. If you hold, have ever held, or in the future want to hold a federal grant, this is ten minutes well spent.

File a comment on OMB-2026-0034

Now, onto the five things that interested me this week. Most of this week circles one question, whether we can measure what AI models actually do and trust the measurement. OpenAI decides the answer is yes and is shipping a life-sciences model to vetted partners.

1. Open models stay (only!) about four months behind the frontier.

2. Where evals actually sit in the biorisk evidence stack.

3. DARPA and NSF want universities for AI Forge (bring an IP-sharing agreement).

4. Two papers demonstrate AIxBio refusals tell you almost nothing.

5. OpenAI hands GPT-Rosalind to biodefense partners.

1. Four months off the pace

Epoch AI put a number on this open/frontier model gap: since January, the best open-weight models have trailed the closed frontier by about 4 months, or 8 points on Epoch’s Capabilities Index, roughly the distance from GPT-5 to GPT-5.5.

The “four months” figure uses a generous definition of catching up: an open model counts if it beats the old frontier in at least 5% of bootstrap samples. Epoch also notes that closed labs sit on their most capable models for safety or commercial reasons, so four months probably understates it.

As frontier models move to token/usage-based pricing, the open option is real and usable, they’re not terribly behind on some benchmarks. Whether those benchmark points translate into the work you actually do is a separate question, and section 4 suggests the answer is messier than a single index implies.

Subscribe now

2. Best evidence per buck: AIxBio evals

at answers a recurring jab at AI biorisk benchmarks (they don’t measure what we care about, so why bother) by placing evals in a four-tier hierarchy of evidence.

SecureBio

The Role of Evals in the Biorisk Evidence Hierarchy

We frequently hear questions along the lines of “what is the theory of change for benchmarks?“ or “evals aren’t measuring what we actually care about, so why the effort?“ Tl;dr: evals provide better evidence than just arguing from first principles, they are much cheaper and repeatable than wet lab uplift studies, and can help us to rule out capabilities…

Read more

a month ago · 10 likes · 2 comments · SecureBio and Jasper Götting

Tier 1 is arguing from first principles, which got the field started but convinces no skeptic. Tier 2 is evals, where we are now. Tier 3 is real-world uplift studies, the gold standard and the expensive one. Tier 4 is an actual incident, which nobody wants as a data point.

evals provide the best cheap, comparable and repeatable evidence we can get

I really liked the section Götting calls the rocky terrain between tiers 2 and 3, so much that I’m quoting the whole segment in full below. A well-powered uplift study costs millions and goes stale the moment the next model ships. Jasper proposes a bridge: if you collect rich data on how participants used the model during an uplift study, you can map their failures and successes back onto benchmark tasks, building what he calls correlates of uplift. That gives you a cheap proxy you can re-run on every new model instead of re-running the whole study. A small number of expensive studies can underpin a much larger cheap monitoring effort.

Assume you ran your large uplift study, found some effect, and two months later, the new generation of frontier models is released. Can you make any sensible claim about the impact on real-world risk of these models (besides “probably as good as the last gen“) without re-running everything? Maybe! You carefully approach the edge of the cost cliff you valiantly scaled and peer down to the vast plain of evals again.

If rich, qualitative data on LLM usage were collected during the uplift study, you should in principle be able to map the failures and AI-assisted successes of your study participants onto benchmarks measuring a corresponding capability. If model A failed to guide participants around a common methodological pitfall, and model B didn’t, is the same pattern detectable in a benchmark task about this method? Is the delta in overall uplift-effectiveness between models or model generations the same as the delta in the relevant benchmark results? Did you find interesting model differences and behaviors that don’t yet map onto a benchmark, but such a benchmark could be made? Did you encounter missing refusals to questions that should be refused in hindsight? All these can let you draw connections between evals and real-world uplift studies.

Finding these connections and correlations between uplift studies and in silico evals (“correlates of uplift”) is extremely important as it provides you with an OOM cheaper estimator for the underlying model capability that you can tweak and run over and over again for every newly released model; and, crucially, also for pre-release testing and risk assessment.

Jasper is blunt about the failure modes (specification gaming, underelicitation, sandbagging, weak construct validity), but the point stands: in biorisk, a false negative costs vastly more than a false positive, so waiting for tier 3 certainty has a price that critics rarely put on the books.

See also:

1. Hong, S. Z. et al. Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology. Preprint at https://doi.org/10.48550/arXiv.2602.16703 (2026).

2. Zhang, C. B. C. et al. LLM Novice Uplift on Dual-Use, In Silico Biology Tasks. Preprint at https://doi.org/10.48550/arXiv.2602.23329 (2026).

3. Paskov, P. et al. RCTs & Human Uplift Studies: Methodological Challenges and Practical Solutions for Frontier AI Evaluation. Preprint at https://doi.org/10.48550/arXiv.2603.11001 (2026).

3. Forging ahead with AI Forge

DARPA and the NSF, with CAISI at NIST, announced AI Forge, a program to fund university research on 3 problems: AI interpretability, AI control, and adversarial robustness. The structure is a forum of universities, frontier AI companies, and government, running what DARPA calls Project Ventures, fast one-year university efforts from $750K to $3M or more, several per year. The Critical AI Challenges report alignts in part with this week’s other items, heavy on whether you can evaluate, interpret, and trust model behavior. One of its challenges is evaluating AI systems for scientific discovery once they outrun human expertise.

The RFI (DARPA-SN-26-80) asking universities to describe their capabilities is due June 22 at 5 PM ET. Two requirements are worth noting before spending too much time on a response (which must come from your VPR/Provost, not your individual lab). Key personnel are limited to US citizens and permanent residents, and the IP generated is meant to be shared across the forum, preferably under the MIT License.

This RFI is targeting U.S. universities. Further, AI Forge Project Venture designated key personnel (e.g., Principal Investigators [PIs], Co-PIs, and key research personnel) are anticipated to be limited to U.S. citizens or permanent residents.

Please note that Intellectual Property (IP) generated from Project Ventures is intended to be shared among AI Forge forum participants, preferably under liberal open-source licensing (e.g., the MIT License). Universities unable or unwilling to agree to a shared IP framework should not respond to this RFI.

Subscribe now

4. Refusal theater

Two preprints landed this week on we evaluate model safety in biology: we count how often a model refuses, and almost never check whether the refusal means anything. Both are preprints, and one is a hackathon project rather than peer-reviewed work, so weight them accordingly.

BioRefusalAudit, by Caleb DeLeeuw, built over a single weekend on a consumer GPU, asks whether a refusal is structurally sound or melts away under small changes. The results across five small models are not reassuring. Gemma 2 2B never refused across 75 prompts; it just hedged. Gemma 4 refused 65 of 75 prompts with chat-template formatting and 0 of 75 without it, and both Gemma models dropped to zero refusals once you capped output at 80 tokens, the kind of cap production systems use for cost and latency. Qwen and Phi-3 swung the other way, flagging 83 to 87% of benign biology as hazardous. The psilocybin probe was interesting: some models refused to discuss cultivating a legal-to-possess, biologically non-toxic substance more often than genuinely hazardous biology, which suggests refusal is tracking legal status and cultural taboo rather than actual CBRN risk.

RefusalBench, from Applied Scientific Intelligence, is the heavier study: 141 prompts in matched triples that hold the task constant and vary only biological risk tier, run across 19 frontier models. On identical prompts, strict refusal rates ran from 0.1% to 94.6%. Jurisdiction didn’t predict refusal, but provider did, with Anthropic’s API stack predicting refusal at an odds ratio around 21. The authors are careful that this is an access-path effect, not a statement about model weights: 99.8% of Anthropic’s refusals carried the same canned policy reason code, consistent with template-based filtering rather than case-by-case reasoning.

The TLDR I took away from this is that refusal rate misranks safety.

Grok 4.20 (lol, seriously?) had the best tier discrimination in the panel while ranking only seventh by refusal rate, and Claude Opus 4.7’s discrimination dropped 65% from the prior version, driven entirely by refusing more legitimate research prompts (benign-tier false positives jumped from 33% to 77%) with no gain in catching dual-use ones. 9 of 18 frontier models showed a hedge-but-help pattern at the dual-use tier.

Put together, a 4B model on a laptop and the frontier lineup all tell the same story in that refusal rate is close to noise as a safety signal. Thinking of wiring an LLM into a protein design pipeline? A more conservative model isn’t a safer one if it terminates your legitimate workflow at step 2, and the RefusalBench authors found the most-refusing models were often the worst at telling benign from dangerous. That a vendor selling agentic biology tooling ran this study is worth noting, but the matched-triple design and the should-refuse control set make it hard to write off.

All the data and prompts are available at github.com/AppliedScientific/refusalbench.

5. Rosalind plays defense

OpenAI launched Rosalind Biodefense, sponsoring access to GPT-Rosalind, its life-sciences reasoning model, for vetted developers building biodefense tools, and extending trusted access to select US government and allied partners. The launch cohort includes Fourth Eon and SecureBio on the screening and detection side, with LLNL, Johns Hopkins APL, and CEPI on the government side. The pitch is “defensive acceleration,” putting frontier capability in defenders’ hands first.

A vetted-access model is a sensible way to deploy a tool whose offensive and defensive uses are the same capability pointed in opposite directions, which is exactly why OpenAI gates it similar to what Anthropic did with Mythos. It is also a way to expand deployment of a model the company itself treats as High Capability in biology (the threshold it first crossed with ChatGPT agent last July) while keeping the optics clean (take “meaningfully advantage defenders” as a goal not a demonstrated result, yet!).

This dropped the same week as the OpenAI Foundation’s resilience announcement (more below) and a new cyber-focused executive order, so the bio, cyber, and safety messaging is moving together. And SecureBio appears here as a detection partner, in section 2 as the source of the evals argument, and again in the community roundup below. For a field this consequential, it runs on a small but mighty number of organizations doing important and exciting work!

Subscribe now

In other news…

(Section drafted/organized with Claude Opus 4.8)

• Frontiers’ open special issue, Artificial Intelligence and the Future of Biosecurity (edited by Bin Hu of Los Alamos, Dov Greenbaum of Yale, and Michelle Holko at Berkeley), collects perspectives and frameworks on the dual-use problem of AI-enabled biology. Worth a scan rather than a read-through:

  • Glycol vapor pathogen disinfection: a defensive technology for emerging biothreats

  • Automated Laboratory Security Tiers

  • Aligning innovation and security in AI-enabled biotechnology

  • Toward relational biosecurity

  • Know your scientist: KYC as biosecurity infrastructure

  • Protein design, generative AI and biological security

  • Methods for safely sharing dual-use genetic data

  • Without safeguards, AI-Biology integration risks accelerating future pandemics

• The June GCBR organization update rounds up Active Site, the Asia Centre for Health Security, IBBIS, and SecureBio.

• at always has a great weekly recap, so much so that I model the recap here off of his weekly “Five Things” series. His Five Things: May 31, 2026 post covers Illinois SB 315, ESMFold2, papal encyclical, Opus 4.8, OpenAI’s Rosalind Biodefense.

• A new executive order on AI innovation and security sets up a “covered frontier model” designation, a Treasury-run vulnerability clearinghouse, and a voluntary window of up to 30 days of early government access to qualifying models before wider release.

• The OpenAI Foundation detailed an AI Resilience program across four areas (bio, cyber, model safety, and effects on young people), saying it is finalizing more than $130M in grants and intends to commit over $1B in the next year.

• openRxiv (Richard Sever and Tracy Teal) sketched where bioRxiv and medRxiv are headed on trust signals as AI makes fake-but-plausible research cheaper: verified data deposits, funder metadata (now on roughly 80% of papers), and the hard problem of identity verification, since ORCID disambiguates authors but doesn’t confirm they are who they claim.

• NeurIPS desk-rejected 178 position-paper submissions (18.4%) for being substantially AI-written, using the Pangram detector, with another 123 asked to prove human authorship.

Subscribe now

I’ve developed a similar habitual relationship with AI. I reach for it thoughtlessly when I see an essay that’s a little longer than I want to read,2 or when I don’t feel like writing that cover letter for a manuscript,3 or when I want to avoid the mental strain of thinking hard about the architecture of whatever code I’m working on.4

AI deskilling is real. A couple weeks ago I was writing specific aims for a proposal with a few brilliant colleagues here in SDS, and I felt the constant urge to open up a Claude window. Just to write aims! In another setting, while in a coaching session with one of my CSTE Data Science Team Training teams last week, I opened Positron to demonstrate some fairly basic tidyr rectangling I’ve done hundreds of times, and couldn’t easily remember how to use the pivot_*() functions. It felt like coming back to a foreign language after a few years without using it.5 AI deskilling is real, and if you use these tools with any regularity you know it deep down.

Next month I’m doing an AI Dry July.6 I invite you to join me.

I’m going to resist using generative AI as much as possible for the whole month. With alcohol in Sober October the rule is easy —7 if it contains ethanol, I’m not having it. It’s a little more difficult with AI. You can’t do a web search or read a product without AI overviews. But generally, I’m not going to reach for ChatGPT or Claude for reading, writing, or for most coding problems.8 I’m teaching a new class this fall (Genomics Foundations for Data Scientists). I’m going to ask my students to struggle through some of the readings and assignments,9 so it’s only fair if I struggle through creating those assignments and course materials. I’m exhausted from reading AI slop in everything from academic literature to the New York Times to many of the blogs and newsletters I personally recommend, so it’s about time to stop compulsively reaching for AI to polish my own writing.10

I don’t really care that my mental arithmetic is terrible because of calculators, or that my navigation skills aren’t what they were back in the pre-GPS days of the 90s (the best decade). I want to avoid the habitual slide into offloading some part of nearly every cognitive task to the chatbot. At least for a month, to re-center myself and take stock of where the deskilling is creeping in.

I know I’ve written very positively about AI in this newsletter, in everything from writing code to peer review. And I’ll continue to do so. I’m neither an AI evangelical nor an AI doomer, and I’ll highlight problems where I see them. Even in 2026 it’s possible (gasp!) to have nuanced views. And I’m starting to think habitual and compulsive AI use is probably not that great for me over the long run.

We’ll see how this works out. I’ll write something at the end of the next month on how it went and what I took away from it.

Subscribe now

I’ve taken a couple of these. They’re short. If it matters to you, when you complete the course you can get a certificate that you can wire up to your LinkedIn profile, which link out to the verified cert (example).

Full list as of this week below.

Claude 101: Learn how to use Claude for everyday work tasks, understand core features, and explore resources for more advanced learning on other topics.

Claude Code 101: Learn how to use Claude Code effectively in your daily development workflow.

Introduction to Claude Cowork: Learn to work alongside Claude on your real files and projects. This hands-on course covers the Cowork task loop, plugins and skills, file and research workflows, and how to steer multi-step work responsibly, so you’re productive in your first week.

Claude Code in Action: Integrate Claude Code into your development workflow.

AI Fluency: Framework & Foundations: Learn to collaborate with AI systems effectively, efficiently, ethically, and safely.

Building with the Claude API: This comprehensive course covers the full spectrum of working with Anthropic models using the Claude API.

Introduction to Model Context Protocol: Learn to build Model Context Protocol servers and clients from scratch using Python. Master MCP’s three core primitives (tools, resources, and prompts) to connect Claude with external services.

AI Fluency for educators: This course empowers faculty, instructional designers, and educational leaders to apply AI Fluency into their own teaching practice and institutional strategy.

AI Fluency for students: This course empowers students to develop AI Fluency skills that enhance learning, career planning, and academic success through responsible AI collaboration.

Model Context Protocol: Advanced Topics: Discover advanced Model Context Protocol implementation patterns including sampling, notifications, file system access, and transport mechanisms for production MCP server development.

Claude with Amazon Bedrock: As part of an accreditation program created for AWS, Anthropic launched a first-of-its-kind training for AWS employees. Here’s the full course so you can follow along.

Claude with Google Cloud’s Vertex AI: This comprehensive course covers the full spectrum of working with Anthropic models through Google Cloud’s Vertex AI.

Teaching AI Fluency: This course empowers academic faculty, instructional designers, and others to teach and assess AI Fluency in instructor-led settings.

AI Fluency for nonprofits: This course empowers nonprofit professionals to develop AI fluency in order to increase organizational impact and efficiency while staying true to their mission and values.

Introduction to agent skills: Learn how to build, configure, and share Skills in Claude Code, reusable markdown instructions that Claude automatically applies to the right tasks at the right time. This course takes you from creating your first Skill to distributing them across teams and troubleshooting common issues.

Introduction to subagents: Learn how to use and create sub-agents in Claude Code to manage context, delegate tasks, and build specialized workflows that keep your main conversation clean and focused.

AI Capabilities and Limitations: An introductory course about how AI works.

Subscribe now

Subscribe now

1. Choosing to stay human, and what 370,000 college essays say about that

Two pieces from different authors converging on a similar point, and one I’ll be writing about next week as well.

Ethan Mollick’s essay Choosing to Stay Human isn’t necessarily arguing against using AI or even AI for writing, but focuses on what to hand over. He’s fine offloading phone numbers, arithmetic, cursive. He’s not fine reflexively offloading writing, because writing is one of those tasks where the doing is the point. He cites his own BCG consultant study and a pair of Wharton-affiliated education experiments where small differences in classroom AI deployment produced opposite outcomes. In Turkey, students with plain ChatGPT did worse on tests than students without it. In Taipei, students with an AI tutor that gave personalized problem sequences scored 0.15 SD higher, roughly six to nine months of additional schooling.

Related is companion piece is Rebecca Winthrop’s NYT op-ed on a Georgetown research program led by Adam Green that has been tracking college application essays before and after ChatGPT. In a study of 370,000+ personal statements, post-ChatGPT essays got rated as more creative by human judges while offering fewer novel ideas. A separate study found human-written essays contained up to 8x more new ideas than AI-assisted ones. The homogenizing effect was largest on students furthest from the mean, including neurodivergent students and racial and linguistic minorities. I.e., the kids whose voices are most distinctive are the ones most flattened by the assist.

Stay tuned next week. I’m writing a very short piece on the topic.

2. The despair of the professor

The natural sequel is Jay Caspian Kang’s New Yorker piece, “The Despair of the Professor in the Age of A.I.” collecting testimonials from faculty members about what AI has done to their teaching. Hard to read. But nothing in it is surprising (I’d guess most academics have heard versions of these stories from colleagues).

The piece opens with Jane Sloan Peters, a religious studies professor at Mount Saint Vincent, describing a course she’d taught for years called “Letters from Prison.” Before AI, students struggled to land on themes and arrived somewhere personal through that struggle. Last year, every one of her students turned in something polished and empty.

From Beth Ritter-Conn at Belmont, on her honors students:

The tipping point was last year when I had Honors students—Honors students!—using A.I. to write reflection journals. Literally the only task there is “tell me what you are thinking inside your own head.” There is no right or wrong answer. It’s just, Give me your thoughts on this thing. And I had students who outsourced that task to the robots.

From Susanna Boxall, a philosophy lecturer at Chico State, on online classes:

Now, online classes are a simulacrum of education: the students pretend to learn, and I have to pretend that I am teaching them something.

From Neal Hebert at Grambling State, who teaches theater and now assigns plays too obscure for ChatGPT to know about:

I’ve stopped being a collaborator in these intro courses and started being a plagiarism cop, and I do resent that a bit.

And from Jeremiah Croster, who teaches English at Houston City College:

The get-a-degree approach was already winning even before A.I., but now that it’s here, the education part is starting to feel like something someone will write about in a history book. Or maybe A.I. will do it.

Not every voice in the piece is despairing. Auyon Siddiq at UCLA Anderson made his stats exam fully AI-permitted and reports the average is still 75%, because students who don’t understand the material aren’t saved by the tool. Daniel Silver at Toronto Scarborough redesigned his sociology assignments as multi-agent simulations and says the best final projects showed more creativity than what he used to get. So it’s not uniformly grim, but the dominant note is loss.

Subscribe now

3. ESMFold2 and the protein biology “world model”

Biohub (formerly known as EvolutionaryScale, now merged with the CZ Biohub network) announced ESMFold2 and a new ESM Cambrian (ESMC) family of protein language models this week, alongside an atlas of 6.8 billion protein sequences with 1.1 billion predicted structures. The paper is long and I haven’t read it, so I’ll be careful here.

What Alex Rives says in his thread, in his words: ESMFold2 is state of the art on protein interactions, especially antibody-antigen complexes. They designed and experimentally validated miniprotein binders and single-chain antibodies against five therapeutic targets with good hit rates including a PD-L1 minibinder that restores T-cell signaling at functional IC50 comparable to atezolizumab. They describe this as evidence that the model has materialized “a world model of protein biology.”

What is a “world model” anyway? Seems like “world model” is doing a lot of marketing work in AI right now, with no agreed-on technical meaning. Whether calling it a “world model” adds anything beyond “the representations are structured in biologically meaningful ways” is a fair question, right?

Either way, a quick look at their press release benchmarks and binder design results all look exciting. I’m looking forward to seeing what holds up under peer review.

4. Pope Leo on AI

Jill Lepore’s New Yorker piece on Pope Leo XIV’s new encyclical Magnifica Humanitas provides a nice overview for those of us without time to read the 40k+ word papal document on AI. Tech leaders have spent 20 years borrowing the rhetoric of mission, salvation, and disruption from religion, and the first American Pope, who took his name in reference to Leo XIII’s 1891 Rerum Novarum on industrial labor, has decided to reclaim the territory. From Lepore:

Little in the encyclical is surprising; its force lies in its being said all at once.

I haven’t read the whole encyclical yet so I’m not going to summarize it. It’s interesting that (and this is partly Lepore’s read but also visible in a quick skim of the text itself) part of the encyclical is about language and writing. There’s a whole subsection on “an ecology of communication.” The Pope and Ethan Mollick and Rebecca Winthrop in #1 above are, in different vocabularies, worried about the same thing, that we are losing the conditions under which humans develop their own thoughts.

Our first task is neither to demonize nor idolize technological tools, but to utilize them on the basis of a fundamental principle, namely that truth is a common good and not the property of those with power or influence. We must therefore promote an ecology of communication.

Just pausing to say here that I find it amusing and puzzling that the company I keep who’d normally be first in line to criticize the Church, its anti-scientific doctrine, and its history of abuses, are now lining up behind the Pope as soon as he says something critical of AI. Cat’s post on the topic stuck with me.

5. NIH’s reinterpretation of “foreign component”

Jeffrey Brainard at Science reports that NIH grant managers are privately telling investigators they need pre-approval for any co-authorship with a scholar affiliated with a foreign institution, even when all the work was done in the United States. NASA is reportedly doing something similar for collaborations with researchers in China. Neither agency has issued public guidance.

The mechanism is that NIH has long required approval for grants with a “foreign component,” historically defined as significant scientific work performed outside the U.S. Now NIH is suggesting that mere coauthorship with someone at a foreign institution now counts as a foreign component, regardless of where the work happened. NIH is asking grantees to remove papers with foreign coauthors from annual progress reports if no foreign component was originally approved. An NIGMS email seen by Science asks institutions to promise that flagged U.S. authors won’t collaborate with the relevant foreign coauthors in the future.

This creates pressure to remove foreign-affiliated coauthors from papers before submission, which violates standard publication ethics. Foreign-affiliated coauthors include visiting students and postdocs temporarily in the U.S., collaborators who donated reagents but didn’t run experiments, and U.S.-trained scientists who moved abroad after the work was done. The rule is seems to be administered through private emails rather than formal guidance which means it can shift without anyone being able to point to the rule that changed.

Subscribe now

AI Exchange @ UVA Substack

OpenScholar Synthesizes Scientific Literature and Reduces Hallucitations

This is a guest post from Stephen Turner. Dr. Turner is an Associate Professor of Data Science and Assistant Dean for Research in the University of Virginia School of Data Science. He writes regularly about AI, data science, biomedical research, and biosecurity in his newsletter…

Read more

2 months ago · 7 likes · 4 comments · Stephen D. Turner

A recent Nature paper from UW and Allen AI introduces OpenScholar, a retrieval-augmented language model that outperforms GPT-4o1 on scientific literature tasks despite being a fraction of its size.

Asai, et al. (2026). Synthesizing scientific literature with retrieval-augmented language models. Nature. https://doi.org/10.1038/s41586-025-10072-4

OpenScholar addresses a problem with using LLMs for research: citation hallucination.2 When asked to cite recent literature across fields like computer science and biomedicine, GPT-4o3 fabricates citations 78-90% of the time. The references look plausible, but the papers simply don’t exist.

Subscribe now

What the paper showed

OpenScholar tries to ground responses in a data store of 45 million open-access papers with 236 million passage embeddings. Instead of generating citations from parametric memory, it retrieves relevant passages, synthesizes them, and iteratively refines its output through a self-feedback loop. The model drafts a response, critiques it, retrieves additional context if gaps are identified, and revises. This cycle continues until the output meets quality thresholds for factuality and citation accuracy.

On ScholarQABench (a new benchmark the team developed with nearly 3,000 expert-written queries), OpenScholar-8B outperformed GPT-4o by 6.1% in correctness and beat PaperQA2 by 5.5%.

When the team ran human evaluations with 16 PhD-level experts across computer science, physics, and biomedicine, experts preferred OpenScholar’s responses over human-written answers 51% of the time (70% when using GPT-4o as the backbone).

Open source

Everything is released: models, training data, data store, retrieval indices, benchmark, and a public demo. The demo has already attracted 30,000+ users generating 90,000 queries.

• Paper: https://www.nature.com/articles/s41586-025-10072-4

• Training data: https://huggingface.co/datasets/OpenSciLM/OS_Train_Data

• Benchmarking: https://github.com/AkariAsai/ScholarQABench/tree/main/data

• Queries: https://huggingface.co/datasets/allenai/openscilm_queries

• OpenScholar code: https://github.com/AkariAsai/OpenScholar

• Code to run the evals: https://github.com/AkariAsai/ScholarQABench

• Public demo: https://openscholar.allen.ai

Subscribe now

Limitations

A notable limitation: the benchmarking predates current frontier models, and only compares OpenScholar to now-deprecated ancient models. recently remarked on a different paper to this effect:

That is: Benchmarks age fast. If a weaker model is close, a better model will often clear the bar later, and we learn little from the gap.

GPT-5.5 with extended thinking, Claude Opus 4.7, and commercial tools with deep research capabilities (like Gemini Deep Research or Perplexity Pro’s updated pipelines) remain untested. Whether domain-specialized RAG maintains its edge against these newer systems is an open question.

See also

See also my previous post on Consensus, and wiring up your own Zotero library to it, to be able to ask and answer questions about your own literature collection, find gaps, and fill those gaps in.

Come to my workshop

On June 17 2026 at noon I’ll be teaching a workshop for UVA’s AI Research Initiative AI upskilling series: Smarter Literature Reviews with AI-Powered Tools. You can register for the workshop here. Grab your spot now!

Subscribe now

1. ’s case for shaping AI-for-biology before it shapes us

2. RAND and Helena on AIxBio mitigations

3. on the state of AI in genomics

4. Nature’s AI scientists week, and the editorial pushback

5. 45 expert scientists review the reviewers

Subscribe now

1. Pannu’s three-part case for shaping AI-for-biology

(Johns Hopkins, Center for Health Security) published a three-part series last week on shaping AI progress for biology and biosecurity.

Part 1 sets up the series. AI will compress decades of biological research into years, but cures won’t arrive by default, and the same systems that enable them can lower the barrier to weaponizing pathogens. We need proactive policy on both sides.

Jassi Pannu

Part 1: Shaping AI progress for biology and biosecurity

This essay is Part 1 in a series on why we should leverage AI to advance biomedicine, while simultaneously building our resilience to biological risks. If you have ideas for how to shape AI progress for biology and biosecurity, submit to IFP’s The Launch Sequence…

Read more

2 months ago · 9 likes · Jassi Pannu

Part 2 is where it gets interesting. Pannu lays out what she calls autonomous biological discovery: AI systems that automate every step of the research cycle, including managing the cycle itself, with orgs like Isomorphic Labs, FutureHouse, Ginkgo Bioworks and others entering the fray.

Evolution selects for reproductive success and gets stuck in fitness valleys. AI-driven design doesn’t have that constraint. Whether that’s a feature or a terrifying bug depends on what you’re designing.

Jassi Pannu

Part 2: Where is AI for biology headed?

This essay is Part 2 in a series on why we should leverage AI to advance biomedicine, while simultaneously building our resilience to biological risks. If you have ideas for how to shape AI progress for biology and biosecurity, submit to IFP’s The Launch Sequence…

Read more

2 months ago · 6 likes · 2 comments · Jassi Pannu

Part 3 centers on smallpox eradication: 171 years between Jenner’s cowpox demonstration in 1796 and Henderson’s 1967 campaign, and only 10 of those years were spent actually eradicating. Her conclusion is that the bottleneck wasn’t tech, it was coordination and political will, so even if AI drives the marginal cost of biology research to zero, we shouldn’t expect cures to deploy themselves.

Jassi Pannu

Part 3: Where AI will fall short for solving disease, and what to do about it

This essay is Part 3 in a series on why we should leverage AI to advance biomedicine, while simultaneously building our resilience to biological risks. If you have ideas for how to shape AI progress for biology and biosecurity, submit to IFP’s The Launch Sequence…

Read more

2 months ago · 6 likes · 3 comments · Jassi Pannu

2. RAND and Helena on AIxBio mitigations

RAND and Helena released the conference proceedings (full PDF here) from a January 2026 workshop on AI-enabled biological threats. 22 participants from frontier labs, biotech, biosecurity, and academia, working over two days in DC under Chatham House rules, with three threat scenarios: a millenarian nonstate group releasing a novel influenza A, an agroterrorism scenario targeting US wheat with an engineered fungal pathogen, and a state-sponsored insider attack on a semiconductor plant using a biofilm-forming bacterium. Scenarios were deliberately compressed and the document withholds specifics for infohazard reasons.

The pandemic group prioritized pathogen-agnostic physical defenses (high-quality PPE, indoor air quality with filtration and UV), a voluntary credentialing system called “BioTrust” modeled on ORCID, and AI “guardian models” for intent monitoring. The agroterrorism group went after holistic biosurveillance, information-sharing modeled on the Kansas Intelligence Fusion Center, and synthetic DNA screening for agricultural pathogens (which gets less attention than human-pathogen screening, and the participants thought that was the most well-scoped problem of the bunch). The critical infrastructure group went hardest on LLM-side interventions: investing in safeguards to better infer intent from prompt patterns, information-sharing between LLM companies via something like the Frontier Model Forum, federated cross-platform behavior analysis, and (this one is interesting) training LLMs to de-escalate malicious intent by adapting techniques from suicide prevention hotlines.

The participants (which included Twist Bioscience, Anthropic, Microsoft, SecureBio, Los Alamos, and others) kept running into the same wall, which the report names explicitly:

A central theme was that technical feasibility and political backing together determine a mitigation’s success.

I.e., most of these will fail without sustained funding and political will, and almost none of them have either. The other recurring caveat was attribution. AI-enabled biological incidents may remain unattributed indefinitely, which weakens deterrence and complicates response authority.

Subscribe now

3. Seven points on AI in genomics

ran the University of Chicago’s annual genetics, genomics, and systems biology symposium last week, and turned the speaker lineup into seven points on the current state of AI in genomics.

Ran’s Substack

Seven points on the current state of AI in genomics

Last Friday, the Committee on Genetics, Genomics & Systems Biology hosted its annual symposium at the University of Chicago, this year on the theme of AI in Genomics. We brought together six speakers whose work spans much of the interesting territory in the field right now…

Read more

2 months ago · 23 likes · 2 comments · Ran Blekhman

The whole post is worth reading (<10 minutes). I’m just going to highlight a few.

First, the scaling laws may not hold for DNA. Evo 2 is 40 billion parameters trained on 9 trillion nucleotides spanning every domain of life. But Vishniakov et al. (2025) compared seven genomic foundation models against randomly initialized baselines of matched architecture across 52 tasks. The random baselines often matched or beat the pretrained models. Tang et al. (2025) found that raw one-hot encoded sequences are competitive with learned DNA-LM representations on regulatory genomics tasks. As Alex Lu put it at the symposium, DNA isn’t natural language: low signal-to-noise, vast repetitive tracts, no obvious word or sentence analogs, and sparse functional elements that interact combinatorially across long distances.

Second, Arjun Krishnan’s rule of thumb on benchmarks:

The best model is usually the one that is consistently number 2 in benchmarks across the literature.

Whoever publishes a model also designs the benchmark, and the benchmark almost always flatters the model. A model that’s consistently competitive but rarely first-place is more likely to be genuinely strong than one that wins on the benchmark its own authors built. I’m stealing this.

Third, toward the end, is a succinct rule for trainees using AI tools:

This is the cleanest articulation of the trainee-and-AI problem I’ve seen. I’ve written about this before, highlighting work from a new colleague and co-author, Arjun Krishnan:

The friction of doing it the slow way is often the friction of actually learning, the “productive struggle” I’ve written about here before. An AI tool that produces output you can’t evaluate is just a black box you’re forced to trust. I’d extend that beyond trainees, frankly.

4. Nature’s AI scientists week, and the editorial pushback

On Tuesday, Nature published three full-length papers on AI scientists, an editorial that hedges, and a comment piece that pushes back. All on the same day.

1. Ghareeb, A. E. et al. A multi-agent system for automating scientific discovery. Nature 1–3 (2026) doi:10.1038/s41586-026-10652-y.

2. Aygün, E. et al. An AI system to help scientists write expert-level empirical software. Nature 1–3 (2026) doi:10.1038/s41586-026-10658-6.

3. Gottweis, J. et al. Accelerating scientific discovery with Co-Scientist. Nature 1–3 (2026) doi:10.1038/s41586-026-10644-y.

4. Messeri, L. & Crockett, M. J. The uncritical adoption of AI in science is alarming — we urgently need guard rails. Nature 653, 675–676 (2026).

5. Why AI cannot do good science without humans. Nature 653, 650–650 (2026).

The three papers: Google DeepMind’s ERA, an LLM-plus-tree-search system that discovered 40 novel single-cell analysis methods that outperformed the top human methods on a public leaderboard, and 14 COVID hospitalization forecasting models that beat the CDC ensemble. Google’s Co-Scientist, a multi-agent system built on Gemini that helped identify in vitro–validated drug repurposing candidates for acute myeloid leukemia and (in a now-famous demo) recovered an antibiotic-resistance hypothesis that a Imperial College team had spent a decade developing but hadn’t yet published, in days. And FutureHouse’s Robin, which autonomously proposed enhancing RPE phagocytosis as a strategy for dry AMD, identified ripasudil (a clinically used ROCK inhibitor never previously proposed for AMD) as a candidate, validated it in vitro, then proposed an RNA-seq follow-up that fingered ABCA1 as a possible novel target. All hypotheses, experimental directions, data analyses, and main-text figures in the Robin paper were produced by Robin.

These are real results. With that throat-clearing out of the way—

Then there’s the editorial, “Why AI cannot do good science without humans”, which is mostly anodyne until the closing paragraph:

Again: not every grant proposal need include an AI project.

Once more: not every grant proposal need include an AI project.

I read this as the editorial board, deliberately, on the day they published three papers about AI scientists, telling reviewers and program officers not to use “no AI angle” as a reason to triage a proposal.

The comment piece is more pointed. Lisa Messeri (Yale anthropology) and M. J. Crockett (Princeton psychology) wrote “Uncritical use of AI in science needs reality check”. Some empirical claims: Hao et al. (2026) analyzed 41.3 million papers across biology, medicine, chemistry, physics, materials science, and geology and concluded that AI adoption seems to “induce authors to converge on the same solutions to known problems rather than create new ones.” I wrote about this earlier this year:

Kusumegi et al. (2025) looked at 264,125 papers and found that in LLM-assisted papers, good writing stopped being a useful heuristic for scientific quality. Organization Science audited 6,957 submissions from 2021 to 2026 and found LLM-assisted papers had poorer scientific quality by acceptance rate. The closing argument is about deskilling: cleaning raw data, reading and summarizing the literature, the entry-level grunt work that AI is now offered as a solution for, is also how scientists develop the tacit knowledge needed to supervise AI-assisted workflows. If trainees don’t develop those skills, who oversees the AI?

Three papers showcasing autonomous discovery, an editorial gently telling reviewers not to fetishize AI angles in proposals, and a comment piece arguing that the productivity gains may be hollowing out the next generation. Read them together. Or, if you don’t have time, listen to Nature’s podcast.

5. 45 expert scientists review the reviewers

One more, and this connects to a paper I co-authored. A preprint went up at arXiv:2605.20668 titled “On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists.” Big study. 45 domain scientists annotated reviews of Nature-family papers, comparing official human reviewers against three frontier LLM agents. Headline results:

• On aggregate review-item quality, all three AI reviewers exceed the lowest-rated human, and GPT-5.2 exceeds the top-rated human.

• AI reviewers raise more significant items but with lower correctness.

• Replacing one human reviewer with one AI reviewer minimally erodes panel diversity, because human reviewers themselves surface largely disjoint sets of criticisms.

• AI reviewers can augment but not replace a human panel.

• Current frontier AI reviewers in an agentic framework provide genuine value on the rigor- and code-heavy aspects of peer review, while systematically failing on the field-context aspects.

On that last point: it’s the same argument that Agnieszka Swiatecka-Urban, Arjun Krishnan, and I argued for in our preprint earlier this year.

Our claim was that AI is best deployed as a rubric enforcer for the systematic, criterion-checkable parts of review (consistency between scores and comments, statistical reporting, completeness of evaluation, internal consistency of reviewer reasoning) while humans retain authority on the parts that depend on argumentative-world knowledge (novelty, feasibility, recognizing creative leaps, judging whether an ambitious proposal might fail spectacularly or succeed brilliantly). The arXiv paper, working with a completely different methodology and 45 domain scientists doing item-level annotation of real Nature-family reviews, lands in the same place. AI is strong on rigor and code, weak on field context.

Subscribe now

My friend and colleague VP Nagraj published a new R package called ctrlvee that makes this a lot easier.

• CRAN: https://cran.r-project.org/package=ctrlvee

• GitHub: https://github.com/vpnagraj/ctrlvee

It does one thing. Put your cursor anywhere in an R script in Positron or RStudio, call the add-in, provide a URL, and a few milliseconds later you’ll have all the code from that page in your editor, separated by chunk boundaries (along with some metadata and a note to check the license!).

The package README provides a demonstration using the “Data Validation and QA” chapter of my Data Science Team Training book (dstt.stephenturner.us).

1. Install the package: install.packages("ctrlvee")

2. Run the add-in. In Positron you’ll open the command palette, search for Run RStudio Addin, then extract external R code and insert inline. You’ll get a modal asking you for a URL.

3. Paste one in. E.g., https://dstt.stephenturner.us/validation.html

4. The R code from the website appears in your editor 🚀

Here’s a demo.

Subscribe now

Here’s what the extracted/inserted code looks like, from this source.

# -----------------------------------------------------------------
# Chunks fetched by ctrlvee from: https://dstt.stephenturner.us/validation.html
# Strategy: Rendered HTML page
# Date: 2026-05-16 05:14:44
# Chunks: 8
# NOTE: Check the source license before reusing this code.
# -----------------------------------------------------------------

flu <- data.frame(
    week = c(1, 2, 3, 4, 4),
    county = c("Fairfax", "Arlington", NA, "Loudoun", "Loudoun"),
    disease = c("Flu", "Flu", "Flu", "Flu", "Flu"),
    cases = c(23, 41, 18, -5, 12),
    rate = c(2.1, 3.8, 1.6, NA, 1.1)
)

flu

# ---- chunk boundary ----

if (any(flu$cases < 0, na.rm = TRUE)) {
    stop("Negative case counts detected. Inspect raw data before proceeding.")
}

# ---- chunk boundary ----

stopifnot(
    "Negative case counts" = all(flu$cases >= 0, na.rm = TRUE),
    "Missing county values" = !anyNA(flu$county),
    "Duplicate records" = !anyDuplicated(flu[, c("week", "county")])
)

# ---- chunk boundary ----

install.packages("pointblank")

# ---- chunk boundary ----

library(pointblank)

agent <- create_agent(tbl = flu, label = "Weekly flu surveillance") |>
    col_vals_gte(
        columns = cases,
        value = 0,
        label = "Case counts must be non-negative"
    ) |>
    col_vals_not_null(
        columns = c(week, county),
        label = "Week and county cannot be missing"
    ) |>
    rows_distinct(
        columns = c(week, county),
        label = "No duplicate week/county records"
    ) |>
    interrogate()

agent

# ---- chunk boundary ----

create_agent(tbl = flu, label = "Weekly flu surveillance — extended") |>
    col_is_numeric(
        columns = c(cases, rate),
        label = "Case count and rate must be numeric"
    ) |>
    col_vals_in_set(
        columns = disease,
        set = c("Flu", "COVID-19", "RSV"),
        label = "Disease must be from the approved list"
    ) |>
    col_vals_between(
        columns = week,
        left = 1,
        right = 52,
        label = "Week must be between 1 and 52"
    ) |>
    col_vals_gte(
        columns = rate,
        value = 0,
        na_pass = TRUE,
        label = "Rate must be non-negative (NAs allowed)"
    ) |>
    interrogate()

# ---- chunk boundary ----

if (!all_passed(agent)) {
    stop("Data validation failed. Review the agent report before proceeding.")
}

# ---- chunk boundary ----

library(readr)
library(pointblank)

flu <- read_csv("data/flu-2024.csv")

# Validate immediately after reading
agent <- create_agent(tbl = flu, label = "flu-2024 validation") |>
    col_vals_gte(columns = cases, value = 0, label = "No negative counts") |>
    col_vals_not_null(columns = c(week, county), label = "No missing keys") |>
    rows_distinct(columns = c(week, county), label = "No duplicate records") |>
    interrogate()

if (!all_passed(agent)) {
    stop("Validation failed — see agent report above.")
}

Subscribe now

1. The GitHub ship is sinking, the lifeboats leak

2. Vibe-coded into a corner

3. The Lancet sues the AI that cites The Lancet

4. Hallucinated citations, now with a denominator

5. arXiv takes the banhammer out for AI-written work

Subscribe now

1. The GitHub ship is sinking, the lifeboats leak

David Bushell wrote a vivid eulogy for GitHub (“GitHub used to be cool and now it’s a lame slop graveyard”), and a few days later GitLab’s new CEO Bill Staples published “GitLab Act 2”, which to me sounds like a flailing pivot toward agentic everything.

Bushell’s case against GitHub is part vibes but contains an uptime chart that does look bad after the Microsoft acquisition.

When I posted this chart on Bluesky earlier this week, I had a few folks pointing to the deluge of AI coding agents and automated pull requests as the reason. But if you look closely at the chart, this service degradation long predated AI coding agents and ChatGPT itself.

But: Git is not GitHub. He recommends Codeberg (running Forgejo) as the safe alternative, with self-hosted Forgejo as the power-user option. He also recommended GitLab, then added an edit:

oh dear, they’ve contracted the rot

The Staples letter is what he means. GitLab is reorganizing into roughly 60 smaller R&D teams, flattening management, exiting up to 30% of its country footprint, and (oh my): “rewiring internal processes with AI agents, automating the reviews, approvals, and handoffs to speed us up.” The strategic thesis is that “software will be built by machines, directed by people.”

The blog post calls git “designed for human-rate commits” and pitches a “generational rebuild of the underlying infrastructure to handle agent-rate work as the default.”

I’m not an agent power user, and maybe I’ll retract my thoughts on this in a few months. But, if your agents are opening so many merge requests that git itself is the bottleneck, the answer is probably fewer agents.

It’ll be interesting to see if this translates into actual migrations. Pretty much all of computational biology / bioinformatics lives on GitHub: software packages, snakemake/nf-core workflows, Bioconductor packages, lab repos, course materials. Don’t forget about other infrastructure as well: GitHub Pages (e.g. pkgdown pages for R packages), CI w/ GitHub Actions, GitHub container registry, etc. Moving the social graph (issues, PRs, stars, discoverability) is enormous work, and I don’t know if Codeberg could absorb the load. I can’t imagine what a realistic and practical exit plan would look like.

2. Vibe-coded into a corner

I’m not usually one to give air to the anti-AI hype. Most of what I read here is boring and irritating. However, Shubham’s “I’m going back to writing code by hand” is good. He spent ~30 weekends and >200 commits building k10s, a GPU-aware Kubernetes TUI, entirely through Claude. Then he sat down and read model.go for the first time. It was 1690 lines. He’s archiving it and starting over in Rust.

AI builds features, not architecture; every prompt landed cleanly, the cumulative effect was a god object. Vibe-coding “made everything feel cheap” so scope crept from a niche GPU tool to a generic k9s clone. My favorite line:

like “em-dash” is to ai writing, “god-object” is to ai coding

His proposed remedy is putting architectural invariants into CLAUDE.md or AGENTS.md so the model sees them on every invocation.

See also the HN thread. One comment stuck out: “Can’t you just ask AI to break up large files into smaller ones and also explain how the code works so you can understand it?” If the AI got you into the god-object hole, asking the AI to refactor its way out is probably not the move.

I was at a week-long innovation lab around AI and drug discovery this weekend. A thought in the back of my head all week was something I keep coming back to in my thinking in computational biology: the bottleneck in agentic science is evaluation, not generation. You can prompt your way to a working pipeline in a weekend. You can’t prompt your way to knowing it’s right.

3. The Lancet sues the AI that cites The Lancet

Elsevier has joined a class-action lawsuit against Meta alleging Meta reproduced copyrighted works to train Llama. As Nature notes, this is the first AI copyright suit from major publishing houses. Elsevier publishes Cell and The Lancet, so a fair chunk of the biomedical literature is now formally a plaintiff against a frontier lab.

The lawsuit alleges Meta used Common Crawl and also that Meta downloaded and torrented works from LibGen and Sci-Hub. Meta’s defense is the usual one: training is transformative use. US judges in two 2025 rulings reportedly distinguished between training (often transformative) and acquisition (the act of downloading copyrighted material), which is where torrenting from Sci-Hub is going to be a problem for Meta regardless of how fair use resolves.

Irony is dead.

Elsevier sells access to research funded by taxpayers and donated by authors who weren’t paid for the manuscript and often paid Elsevier APCs to make it open. Meta scraped some of that content, allegedly via Sci-Hub, which exists precisely because Elsevier’s access model is widely considered indefensible. Now Elsevier wants damages on behalf of authors who didn’t get paid the first time around either. The most coherent position is probably that Meta should have paid for licensed access and that Elsevier should not be the entity collecting the check, but that isn’t on offer.

If this case establishes that scraping paywalled academic content is infringement, the practical effect on open-science-trained models could be larger than the effect on Llama. Llama already exists. A future model that wants to train on biomedical literature now has a clearer legal target on its back.

Subscribe now

4. Hallucinated citations, now with a denominator

A new Lancet correspondence by Topaz et al. audited 2.5 million biomedical papers for fabricated references, and the numbers are bad. The accompanying Comment by Bauchner and Rivara argues that any published paper with a fabricated reference should be retracted, which is a reasonable position that approximately no one will implement.

Topaz and colleagues pulled >100 million references from >2 million papers in PubMed Central’s Open Access subset between Jan 2023 and Feb 2026. They kept the 97 million (77%) with a PMID, verified those against PubMed, Crossref, OpenAlex, and Google Scholar, used Claude to filter reference errors (misformatted but real) from genuine fabrications, and validated precision at 91% with three independent reviewers. They found 4,046 fabricated references across 2,810 papers.

The fabrication rate increased more than 12 times, from approximately four per 10,000 papers in 2023, to 51.3 per 10,000 papers in the fourth quarter of 2025

The inflection point is mid-2024, which is when LLM-assisted papers would start clearing typical 100-200 day submission lags. One paper on ureteroileal anastomotic techniques had 18 of 30 verified references fabricated, each tailored to the surgical topic and attributed to real urologists. The authors also flag a paper-mill pattern: the same two-author pair appearing across 11 papers in a single surgical journal in 2025.

Read this along with #3 above. If Elsevier wins the case against Meta, the legal incentives push toward licensed-only training data. The Lancet audit suggests the academic literature itself is already meaningfully polluted by models trained on it. I think this will get worse before it gets better.

5. arXiv takes the banhammer out for AI-written work

Yesterday, Thomas G. Dietterich (arXiv moderator for cs.LG) announced that arXiv will implement 1-year ban for papers containing incontrovertible evidence of unchecked LLM-generated errors, such as hallucinated references or results. Here’s the full text of the thread:

Attention arXiv authors: Our Code of Conduct states that by signing your name as an author of a paper, each author takes full responsibility for all its contents, irrespective of how the contents were generated.

If generative AI tools generate inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content, and that output is included in scientific works, it is the responsibility of the author(s).

We have recently clarified our penalties for this. If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper.

The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue.

Examples of incontrovertible evidence: hallucinated references, meta-comments from the LLM (”here is a 200 word summary; would you like me to make any changes?”; “the data in this table is illustrative, fill it in with the real numbers from your experiments”).

I’m about as split on this one as the comments on the OP are. On one hand, GOOD. Hallucinated citations and unchecked AI-authored content is polluting the literature everywhere. I think this borderlines on scientific misconduct, and it destroys trust in the entire scientific enterprise.

Oh the other hand there are lots of problems with detection and enforcement. AI detection software is a cat and mouse game that doesn’t work reliably. And on the enforcement side — what happens when you’re a co-author on a paper with a hallucinated citation? Say I tell my grad student to work on the methods section and they get an undergrad to write a small section that they worked on, and said undergrad carelessly inserts a fabricated citation. If this paper makes its way onto arXiv, does that infraction go all the way up the chain with the banhammer coming down on all co-authors? Have you ever submitted a paper written with a bunch of co-authors and meticulously looked up every reference or checked that every parameter setting in the detailed methods section is actually a valid parameter for whatever tool was being used? I have.

I think a middle ground solution with a big red banner or warning flag on the abstract page or search results noting that the paper likely contains unverified AI-generated content.

Again, the detection piece is difficult, and authors should be given a chance to respond before getting the scarlet letter. I think the 1-year ban is severe, and if applied to all co-authors, could be extremely damaging to one’s career just because a middle author responsible for page 98 of a large supplemental info section was careless and wasn’t carefully scrutinized. I also worry that such a ban could be inconsistently applied since there’s a lot of subjectivity and guesswork involved with this endeavor.

Subscribe now