I don’t like mandatory AI disclosures. And it’s awkward because I require them from my students. If you’ve been reading here lately you also know that I’m trying to drastically reduce how much writing I delegate to AI. Writing helps me think. I lose something when I turn that over to Claude.
But I want to steelman the case against disclosure requirements, including my own.
Harlan Krumholz, EIC of the Journal of the American College of Cardiology and Professor of Medicine at Yale, just published an editorial explaining why JACC doesn’t require AI disclosures. Authors remain responsible for what they publish. AI used to conduct research still belongs in the methods. I agree. A journal can collect a disclosure from authors but that doesn’t help them get any better at checking whether their conclusions are supported by their results. An editor and reviewer still has to check the evidence and references. Humans can mis-specify a model parameter or cite the wrong paper in support of a flimsy argument at just as high or higher rate than AI.
Enforcement is impossible I think. Sure, you could run a manuscript through Pangram or whatever, but a detector can’t reconstruct the author’s process. It can’t establish whether someone asked an LLM for an argument, then wrote every sentence themselves. Authors who follow the rules have to explain themselves, but someone willing to be dishonest can leave the box unchecked or disclose nothing.
Disclosures cast suspicion all around. Disclosing AI assistance will make a reviewer suspicious before they’ve even read the paper or proposal. Saying you didn’t use AI at all will almost certainly invite suspicion that you’re lying. This recent paper shows that readers rated creative writing less favorably when they believed AI had helped write it. This wasn’t a study of peer review but I think it gives us reason to worry that disclosure changes judgments independently of quality.1
I also suspect these policies will age badly. A spellcheck disclosure in a paper from the 1990s would look silly today. AI can do much more than spellcheck, but consider for a moment that the boundary can start there. Start somewhere all reasonable people would agree on. Finding and fixing a typo hardly warrants a declaration. Correcting grammar seems fine too, right? Now what about shortening a sentence or improving its clarity? I constantly find myself at or near word count limits, and it seems quick and easy to get an LLM’s help. Here’s where it starts to get harder to classify. What about tightening up an argument? Reorganizing paragraphs changes the argument’s presentation. Keep moving forward: Suggesting the argument, or generating the whole discussion section (or worse, the results section!) gets into uses I think most of us would reject in most settings. We can disagree about where to stop without pretending everyone agrees on the same disclosure threshold. I expect most writing will eventually involve some AI assistance, making a yes/no declaration even less informative.
I still have reservations about abandoning AI disclosures in my classroom. As I wrote recently, emphasizing an argument made my a colleague here in the psychology department at UVA: students write to develop abilities they haven’t mastered yet. I ask them to disclose AI use because I want to understand what they practiced and how they’re using AI to (hopefully) enhance their learning. I have a really great group of students this year, and I assume honesty and good intent. But the AI disclosure statement itself establishes neither. And after having read many of these already this semester, even an accurate account doesn’t really tell me whether a student learned anything. I haven’t dropped the requirement but I’m increasingly unsure how much it really tells me.
My relationship with AI is changing
Personal essay, weekend edition. Starting with a familiar quote often attributed to English economist John Maynard Keynes:
AI in the workplace vs. AI in education
There’s a funny thing that happens to me from time to time. I randomly stumble across an essay or a paper that resonates with me, sometimes one that I can’t stop thinking about for days. Then I come to find out the author is a colleague here at my own University of Virginia in the building next door who I never knew of.
Pretty much everyone I know uses AI for writing regularly, to varying degrees. What I find kind of ironic is that some of the same people will dismiss someone else’s work (a paper they’re reviewing, a grant proposal, etc.) with “whatever this is all obviously ChatGPT.” Not uncommon I suppose — our explanations of our own use seem much more charitable than our assumptions about anyone else’s.