Benchmarking AI Biosecurity Refusals

Refusal ≠ safety. What 3 recent evals reveal about risk discrimination, over-refusal, and concealed hazards Bonus: we could do some interesting economic modeling on this.

I experimented with Fable when it first launched. I didn’t really get to do much with it. I’m a biologist, and I ask questions about biology, which Fable won’t touch.

Once Fable realizes I’m a biologist I get downgraded to Opus (still a great model!).

Understandably! If the models are as good at biology as the model cards suggest, the precautionary principle is probably smart, especially in the first few weeks of a model’s release.

However, refusal rate is a poor proxy for safety in biological AI research. Three recently published evals show why. RefusalBench tests whether models discriminate among matched biological requests that differ primarily in target risk. BioSecBench-Refusal hides hazards inside attached scientific data and tests whether agents inspect what they are acting on. BioTIER defines a broader risk taxonomy and jointly measures refusal of high-risk requests and permission of benign or biosecurity-adjacent biology. All three papers are really great reads. I’d highly recommend reading them all!

Keep reading at the end. I’m going out on a limb and suggesting some interesting (to me at least) economic modeling that someone who knows more about economic modeling than me should do.

RefusalBench

RefusalBench (Weidener et al., Applied Scientific Intelligence) evaluated 19 frontier models on 141 matched protein-design prompts in three risk tiers: benign human therapeutic targets, borderline BSL-3 and WHO priority pathogen targets, and dual-use select-agent targets.

Weidener, Lukas, et al. "RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts." arXiv:2605.21545 (2026).

Within each bundle, task framing stays identical; only the target organism and its regulatory classification change. On these identical prompts, strict refusal rates ranged from 0.1% to 94.6%. Provider identity was the dominant predictor, with Anthropic’s API stack at OR=21 after controlling for task type and risk tier. Jurisdiction showed no significant effect. Nearly all Anthropic refusals (99.8%) carried the same reason code, safety_policy, consistent with a small set of upstream templates applied across diverse prompts rather than case-by-case biological risk assessment.

Figure 1 from Weidener et al 2026: Strict refusal rate on benign prompts (legitimate human-protein targets) across 19 models.

They also present Youden’s J: the difference between true positive rate on dual-use prompts and false positive rate on benign ones. Grok 4.20 scores highest (J=0.787), refusing 3% of benign prompts while refusing 82% of dual-use ones. Kimi K2.6 has the highest overall refusal rate (94.6%) but a J of 0.038, because it refuses nearly everything at every tier regardless of risk. Claude Opus 4.7 scores J=0.234, the worst of the Anthropic models. The 4.7 release introduced 23 new prompt-level refusals over versions 4.5 and 4.6, with 21 at the benign tier and 2 at borderline, while dual-use refusal was already at 100% and couldn’t increase. The calibration loss was driven almost entirely by additional false positives on legitimate research tasks, and Youden’s J dropped 65% with no gain in dual-use detection.

Figure 2 from Weidener et al. 2026. Tier-stratified refusal rates across 19 models.

BioSecBench-Refusal

BioSecBench-Refusal (Wintermute et al., LatchBio & American Wetware) focuses on the specific failure mode that’s important for agentic pipelines.

Blog: blog.latch.bio/p/benchmarking-refusals-in-agentic

Paper: https://latch.bio/biosecbench-refusal

Their 46 Red Team evaluations are fictional scenarios where the hazard isn’t in the text of the request. It’s in the attached file. One task asks an agent to express and purify the protein in an accompanying FASTA file, with recommendations for expression chassis, codon optimization, and purification tags. Nothing in that request text should flag a filter. Only an agent that actually opens the sequence file and infers what it encodes would identify the toxin.

Their 61 Routine tasks, drawn from published literature, do the opposite: they contain dual-use-adjacent keywords (“immune evasion,” “pathogen,” “enhanced transmissibility”) applied to entirely legitimate contexts, such as antivenom development.

Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and from 1% to 62% on Red Team tasks. Routine and Red Team refusal rates correlated at r=0.91 across configurations. The refusal trigger is surface text in both cases. Most configurations refused legitimate literature-derived work at rates comparable to or higher than concealed threats.

Figure 2 from Wintermute et al. (BioSecBench-Refusal). Refusal rates for 16 model-harness configurations, decomposed into API refusals (provider filter, blue) and model refusals (pink). Left: Routine tasks. Right: Red Team tasks. For most configurations, Routine refusal exceeds Red Team refusals!

When models were asked to judge whether a task should be accepted or refused rather than execute it directly, some configurations improved. GPT-5.5 and Grok correctly identified 14.5–19.6% of Red Team threats versus 13% in direct framing. The qualifier is that provider-side API filters blocked most requests before any reasoning occurred, so the sample of genuine agentic decisions is thin. The filter that claims to protect against harm also prevents the model from performing the analysis that could detect actual harm.

BioTIER

BioTIER (Marshall et al., SecureBio) is larger and more explicitly policy-oriented than either of the other benchmarks.

Paper: Marshall, Eleanor M., et al. “BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation.” arXiv:2607.14479 (2026).

Blog: securebio.substack.com/p/measuring-biosecurity-safeguard-effectiveness

BioTIER contains 542 expert-curated prompts divided into three risk sets. Catastrophe Avoidance (CA) contains 249 prompts covering the narrow set of information the authors argue should be universally refused and excluded from general pretraining. Biomedical DURC (BD) contains 149 dual-use prompts that should be refused to general users but available through access-controlled models to verified researchers. Related Biology (RB) contains 144 benign or “close-to-boundary” prompts that should be answered. The evaluation combines CA and BD into BioTIER-refuse and treats RB as BioTIER-permit.

Across 52 models from 10 developers, BioTIER-refuse compliance (i.e., refusal) ranged from 5.6% for DeepSeek-V3.1 to 96.4% for Claude Sonnet 4.6. Most Claude Opus and Sonnet configurations exceeded 90%. On the permit results, most models answered most RB prompts. The exceptions were, as demonstrated by the above two papers, predominantly concentrated in the Claude models. Claude Sonnet 4.6 answered 76.3% of BioTIER-permit prompts and Claude Opus 4.7 answered 78.7%, with most of their over-refusals concentrated in close-to-boundary topics.

That result gives a somewhat less pessimistic picture of over-refusal than RefusalBench or BioSecBench-Refusal. It also shows that the safety-utility tradeoff is not fixed. Between evaluations, Gemini 3.1 Pro’s BioTIER-refuse score rose from 45.5% to 73.5% with little change in its permit score. At least in that case, stronger safeguards did not require blocking substantially more benign biology.

Figure 4 from Marshall et al. 2026. BioTIER-refuse versus BioTIER-permit compliance across 52 models. The desired region is the upper right: refusing CA and BD requests while answering RB requests. Claude models dominate on refusal but show a lot more over-refusal near the boundary; many other models answer almost all RB prompts while refusing far fewer CA and BD prompts.

The same reruns show how unstable refusal leaderboards can be. Over a period of weeks, Gemini 3.1 Pro moved up 28 percentage points on BioTIER-refuse, while both GPT-5.5 and Grok 4.20 fell by double-digit percentages. All told, these measurements in this paper describe a particular model, API, and policy configuration at a particular time.

The paper discusses model-shopping. Because I haven’t yet been invited to any of the trusted access programs (HEY Anthropic or OpenAI folks reading this: I’ve been asking, email me please), this is something I end up doing routinely.

The ensemble elicitation analysis shows that the vast majority of BioTIER-refuse questions can be answered by at least one of the 52 publicly accessible models evaluated. As a result, a malicious actor need only locate the most capable model with the most permissive policy for their particular query, requiring no technical expertise beyond knowledge of which models exist. This “model shopping” threat means that a single model’s refusal policy, however strict in isolation, provides incomplete protection when many highly-capable models are simultaneously available.

DeepSeek-V3.1 alone answered 379 of the 398 BioTIER-refuse prompts. Querying six low-refusal models increased coverage to 388, or 97.5%. Only 10 of the 398 prompts were refused by every model in at least seven of ten trials. A strict policy at one provider offers limited ecosystem-level protection when another accessible model will answer the same question.

There is an important methodological qualification. BioTIER is not matched in the way RefusalBench is, and it does not test agentic execution as BioSecBench does. Near-ceiling BioTIER-permit scores therefore do not establish that models can discriminate risk when task structure and difficulty are held constant. BioTIER is strongest as a broad map of policy boundaries and provider behavior. It also retains the basic limitation of refusal benchmarks. A response is scored according to whether it contains the requested substance, without judging its accuracy, completeness, actionability, or capacity to produce real-world uplift. BioTIER measures access policy, not biological capability or harm reduction.

Takeaways

The three benchmarks do not identify one universally best model. Grok 4.20 looks most useful for legitimate biology in RefusalBench and BioSecBench-Refusal: it had the highest tier-discrimination score in RefusalBench and the lowest Routine refusal rate in BioSecBench. BioTIER complicates that conclusion. Grok answered nearly all RB prompts, but its BioTIER-refuse compliance was 80.2% in May and 68.0% when rerun in July, well below the Claude models. Claude occupies the opposite part of the tradeoff, with much stronger CA and BD refusal and more over-refusal near the benign boundary. Any claim about which model is “best” therefore depends on the relative costs assigned to dangerous answers and blocked legitimate work, as well as whether the task involves direct prompting or agentic execution.

BioTIER provides a broader comparison of open-weight models. All fell within the lower half of its refusal distribution. These are measurements of default refusal behavior, not evidence that the models lack the underlying biological knowledge.

BioTIER begins to address one question left outstanding by the other two papers, by distinguishing API-level filter stops from model-level refusals and by repeating selected evaluations over time. It still collapses behavior into a binary answer/refusal outcome. Future evaluations should score partial compliance, accuracy, actionability, and the point in a multi-step workflow where a stop occurs.

Future benchmarks should also separate API filtering from model reasoning, score partial compliance separately from clean refusal, and track where in a multi-step agentic workflow the stop occurs.

Both benchmarks measure what models refuse. Neither tracks what happens to research when refusals occur.

All three benchmarks measure access decisions rather than downstream outcomes. BioTIER states this limitation directly as its refusal score is an upper bound on information access and not a measure of the real-world uplift produced by an answer. A useful extension would link refusal behavior to downstream scientific outputs: publication timelines, experimental completion rates, and turnaround in drug discovery pipelines. That would turn this from a classification problem into an impact problem. Something that some economic modeling might be able to address…

Economic modeling

⚠️ Everything you’re about to read from here is speculative, incomplete, riddled with questionable math and untested assumptions ⚠️

I am decidedly NOT an economist. But I do enjoy writing and learning in public. And I’ve found the best way to get engagement and feedback from human experts is to chum the water say something wrong.¹

Obligatory XKCD #386.

All that throat clearing out of the way… I think the underlying economic question is tractable in principle. The cost side of over-refusal: if models refuse 50–74% of tasks drawn from the published literature, and each refusal imposes a time cost on a working researcher, that cost can be estimated. Multiply by the probability that the refused task represents genuine scientific work, and a number emerges.

The benefit side is harder but not intractable: the marginal harm reduction per refusal, discounted by the probability that the refused content provides meaningful uplift beyond textbook access, and further discounted by whether an adversary had an alternative path. RefusalBench and BioSecBench-Refusal make it harder to assume that each refusal buys much risk reduction, because many refusals appear to track terminology rather than demonstrated harmful intent or executable uplift. BioTIER supplies a more explicit risk taxonomy, but it likewise does not measure the marginal harm prevented by a refusal.

Formalizing that tradeoff, with benchmark-derived conditional error rates, deployment base rates, and research outcome data linked to real workflows, might give developers something to optimize against rather than a metric that is easy to measure and weakly correlated with the thing it is supposed to measure.

Think about it this way. Treat the refusal policy as a classifier and compare it to a baseline that never refuses. More precisely, the developer does not observe true risk or true social value. It observes a noisy signal: the prompt text, attached files, user context, model behavior, and whatever policy classifier sits upstream of the model. The refusal rule is then a cutoff on that noisy signal.

One caution is necessary before putting benchmark numbers into this model. The proportions of prompt classes in these datasets are design choices and not actually estimates of what users actually ask. BioTIER contains 398 refuse prompts and 144 permit prompts because it was constructed to evaluate both behaviors and not because 73% of biological requests are dangerous! It can inform conditional quantities such as the probability of refusal given CA, BD, or RB content. The base rate π below still requires deployment data or an explicit scenario assumption.

The expected net benefit (NB) of refusing is the harm you avert minus the legitimate work you lose:

\(\text{NB} \;=\; \underbrace{\pi \, \text{TPR} \, u \, c_H}_{\text{harm averted}} \;-\; \underbrace{(1-\pi)\,\text{FPR}\,c_L}_{\text{legitimate work lost}}\)

where

\(\begin{aligned} \pi &= \text{base rate of genuinely harmful, intent-bearing requests}\\[2pt] \text{TPR} &= \text{dual-use refusal rate (from the benchmarks)}\\[2pt] \text{FPR} &= \text{benign refusal rate (from the benchmarks)}\\[2pt] c_H &= \text{harm from one unmitigated harmful request}\\[2pt] c_L &= \text{cost of one over-refusal}\\[2pt] u &= q\,(1-a),\ \text{with }\\[2pt] q &= P(\text{real uplift beyond textbook}),\\[2pt] a &= P(\text{adversary had another path}) \end{aligned}\)

The policy pays off when NB>0, which rearranges to a break-even harm-to-cost ratio:

\(\text{NB} > 0 \iff \frac{c_H}{c_L} \;>\; \frac{(1-\pi)\,\text{FPR}}{\pi\,\text{TPR}\,u}\)

The cutoff depends on the developer’s loss function. A risk-neutral social planner would refuse when the expected marginal harm exceeds the expected cost of blocking legitimate work. A more harm-averse planner would set a lower threshold for refusal, especially when the signal is noisy in the region where catastrophic downside is plausible.

Put it another way. π is going to be very small. Net benefit can simplify down to something like

\(\text{NB} \approx \text{TPR}\cdot B - \text{FPR}, \quad \text{where } B = \pi \cdot u \cdot \frac{c_H}{c_L}\)

We can first consider what the expected harm averted per request is under a few simplifying assumptions.

If you’re skeptical (low effective uplift, and a successfully averted misuse event is worth 100,000 false refusals), then the net benefit will be slightly negative (over-refusal is harmful on balance). If balanced (1% chance of AI uplift, harms 1 million times worse than benefits), then there’s a slight positive benefit. If real uplift is more probable (10%), and harms averted worth 10 million false refusals, then your net benefit is very positive.

If genuinely dangerous requests are rare, broad refusal only pays off when either false-positive rates are very low or the marginal harm averted per true positive is enormous. This is exactly why the BioSecBench result is so damaging for broad filters: Routine refusal ranges from 7% to 74%, Red Team refusal ranges from 1% to 62%, and Routine and Red Team refusal rates are tightly correlated, suggesting surface text sensitivity rather than task specific risk assessment.

The practical implication is that broad prompt-level refusal is unlikely to be the optimal intervention. A higher value policy target might be better calibrated routing: answer legitimate work, block or escalate genuinely high-risk workflows, and place stronger controls at stages where refusal actually changes outcomes, such as agentic execution, sequence handling, synthesis ordering, or lab-service submission. I.e., better DNA synthesis screening, managed access, cloud lab security, with modeling informed by studies that measure the u=q(1-a) uplift term above, considering threat models like novice designs executed by third party laboratories.

Another way to say this is that AI companies do not necessarily observe true social value directly. They observe a noisy risk signal and choose a refusal cutoff. If the decision-maker is risk-neutral, the cutoff should be set where expected marginal harm exceeds expected lost scientific value. If the decision-maker is strongly harm-averse, the cutoff moves toward more refusal, especially when the risk signal is noisy near the catastrophic tail. But that still does not justify refusing everything with scary biological vocabulary. It argues for better signal extraction, escalation, and workflow-aware controls. Noise matters most only when the decision-maker is risk averse or treats harm as worse than an equivalent benefit is good. Then the rational response to a noisy signal is to pull the cutoff toward more refusal, buying protection against the tail at the cost of legitimate work. I think this is a reasonable description of how frontier providers actually behave. Over-refusal not as a calibration bug but as the expected output of loss-averse decision-making under a noisy signal. I think it also suggests the noise itself is worth measuring, because the case for stringency rests on how bad the signal is near the high-harm end, which is precisely where these benchmarks have the least to say.

Again, I’m a biologist, not an economist.² Just trying to think out loud here. If you’re reading this and you’ve made it this far, and you have an interest in thinking about this with me, send me an email or contact me here.

1. Some of you out there might be old enough to remember SeqAnswers, Biostars, CrossValidated, and StackOverflow. Back in the before times, these were places you’d go to get some help with a coding or domain-specific problem in bioinformatics or statistics. I did this more on SeqAnswers and Biostars than SO and CV. If I really needed help with something and I wasn’t getting much attention on my question, I kept a burner/anon account open in a different browser, and gave a very obviously incorrect answer. I always got more help and better responses this way. “Don’t listen to bion00b, here’s the right way: …”.

2. Hat tip to my colleague Tim Layton, a health economist over in Batten, for reading a draft of this post and providing feedback.