Personal essay, weekend edition. Starting with a familiar quote often attributed to English economist John Maynard Keynes:1
When the facts change, I change my mind. What do you do, sir?
I’ve argued here that refusing a useful tool on principle is stubbornness. And after a few weeks of trying to go without AI at all, I still feel that way for the most part. But my relationship with AI is changing, especially around writing and scientific inquiry.
I’m drowning in slop
My inboxes (personal and otherwise) are filled with slop. My social feeds are filled with slop. Many of the papers I read are slop. My Hacker News links are slop. Many of the blogs I read are becoming increasingly filled with slop.2 People who I know and deeply respect are churning out AI slop at a pace I find simultaneously dizzying and disappointing (I’m not innocent here: there’s some slop on some older posts here I’m embarrassed about, but I’m keeping them here unedited as a reminder). It’s not just writing. I can’t go a day without seeing a slide deck that looks exactly like this. IYKYK.
I’m not the only one
David Dewhurst is a program manager at DARPA. He described watching capable people he respects lose their creativity in real time as they present AI-generated content as original work. They point to length and coverage as achievements. I get some long comprehensive document and I can’t find the sender’s judgment anywhere in it. If I wanted Claude’s analysis or opinion on this or that, I’m very well capable of asking Claude for Claude’s analysis or opinion on this or that. Don’t be a meat proxy.
I feel the same way when I’m reading the scientific literature. Anshul Kundaje argues that papers have stopped working as measures of productivity, expertise, or accomplishment.
The argument is obviously correct if you think about it for a half a minute. A publication on someone’s CV tells me way less than it used to about what they contributed or understand. But as a newly returned academic, I can tell you Anshul is also correct that it’ll take (too much) time before we figure out new ways to assess impact and expertise, while we desperately try to hold onto our old ways, bean counting glam journal publication and H-indices and such. We still need ways to document research impact and contribution, but these days a publication doesn’t really tell us much about what its authors contributed or even understand. Earlier this week the Washington Post describes a full professor at University of Chicago having published more than 200 papers and 14 books this year alone. Case in point.
TMLR editor Nihar Shah contacted authors of 10 submissions that were already going to be desk rejected. One of them withdrew their submission. One said they were unavailable, and one didn’t show up to the scheduled meeting. Of the remainder he met with, three authors couldn’t answer basic questions about their papers, and three more struggled with technical details. The authors of the one remaining paper were able to answer basic questions, but Shah identified major errors in a central claim. Now this small sample can’t tell you how common the problem is or how much AI caused it, but it says something. Shah also noted that this took >20 hours so it’s obviously not a scalable solution.
Related, Giorgio Gilestro wrote an essay that’s been making the rounds this week, calling where this ends the Weimar of Knowledge. The marginal cost of producing content is essentially zero, but the cost of attention is still high. The worry is that overwhelmed readers will rely even more on institutional prestige to decide what to read or whose work you should pay attention to. Obviously making it harder for good researchers outside well-known institutions to ever get read.
Tom Dietterich, arXiv editor in chief, shared a few thoughts on this problem. Ultimately I think the scale of the problem makes oral interviews for preprints unworkable even though I think this could really help cut down on the slop we see flooding academic publications. I’ve written about AI for peer review, and I do think there’s a real place for it, but I’m also not bullish on an AI-written, AI-reviewed journal without human authors, for papers which will surely never be read by an actual human being. What are we even doing here.
Then there are proposals. I can’t personally speak to the degree to which NIH study sections or NSF review panels are dealing with wholly AI-generated proposals. I’m serving on a NSF panel later this year so I’ll get a sense of things soon. But I’d be willing to wager the number is far higher than any of us wish.3
I’m reading less because of it
I’ve noticed a change in myself, and I don’t like it. I’m losing interest in reading.
If you’ve spent any time using AI yourself you can smell the Claudish a mile away, no Pangram needed.4 Maybe these things will get better at writing eventually, but that’s kind of beside the point. Even flawless prose wouldn’t make me less reluctant to spend an hour reading something that its author spent less than 10 minutes “writing.”
Part of the reason I read the things I read is to understand how another person thinks. To understand their own internal thought process, whether that’s about a genetics paper, how to run a research lab and mentor students in the AI age, the optimal cover crop for Virginia winters, or traffic-calming approaches for improved bike and pedestrian safety. These days I wonder how much of that person is actually present.
I still use AI, just more judiciously
With some AI sparkle ✨ thing built into everything these days the persistent temptation to use AI for literally every part of thinking and writing is hard to resist. My AI Dry July experiment which I eventually aborted, gave me reasons to write more myself. I could better recall my arguments when I wrote 100% of them, I could better understand how the sections fit together because I fussed around with the story arc for hours, and I found revision to be much easier and less frustrating. I enjoyed it more, too. People much smarter than me on the topic have written volumes about writing and thinking, so I’ll just confirm here: every time I delegated the writing I missed out on much of the thinking that I would have done while trying to think through or explain an idea.
I think this should make you feel concern about your professional value. If you outsource all your proposals or lectures or manuscripts or whatever creative work you do to a competent AI, you’re choosing to let atrophy the judgment and creativity your employer pays you to do. You become less capable while appearing more productive. If you think people around you won’t notice, you’re very wrong.
I still want AI’s help. Paul Bloom describes models catching errors that human reviewers miss. I recently published a preprint on AI and biosecurity, and while I actually think AI disclosures are actually a net negative in most cases, I did write a lengthy disclosure describing in detail how I’m using a multi-agent workflow to check my work against the papers I cited, including numbers and stats and my descriptions of their findings. This was useful, until it wasn’t: the biosecurity subject matter quickly led both models I used to refuse further fact-checking.
I haven’t settled on some kind of policy for every part of my work. I expect to keep using these tools and finding them useful. They’re great for anything I do involving code. Lately I’ve been using Claude Code extensively to make little throwaway web apps to demonstrate a concept to my students far better than I could do with a janky PowerPoint animation. And I recently submitted a large proposal to develop a closed-loop AI scientist connected to a wet lab for autonomous biomedical research. But since that failed experiment in July I’m really trying to hand over less of my writing and thinking to the machines, and the majority of what I consume these days gives me more reason to take that commitment seriously as much as I can. I want to remain capable of doing the work I put my name on.
Although the attribution is questionable.
Substack, which hosts this blog you’re reading now, and the vast majority of things I like to read, recently made a controversial decision to integrate Pangram’s AI detection into the platform itself. If you’re reading on the Substack web reader or iOS app you can click a button to check this. Screenshot below. I think Substack made a correct strategic move here. I love that blogs are making a comeback. But with more and more slop polluting the blogosphere, Substack is making a bet (one that I think will pay off) that most people, in fact, don’t want to read slop. Much less pay for the privilege of doing so. It’s really interesting, and telling, to see the reaction to this. My new favorite genre of essays are those vehemently and dogmatically anti-AI everything, which are, in fact, written mostly or entirely by AI. I’m hoping that writers here, now knowing that slop will be exposed, will be incentivized to not produce so much slop. And sure, you can turn off the AI detection feature, but that’s a glaring admission that the whole post is, in fact, slop.
Free research-on-research project idea. Go to NIH RePORTER ExPORTER, and download CSV files of all the abstracts for funded grants in the last 10 years. Do the same for all the JSON files you can download from NSF. Pick a subset of a few thousand, and run them through Pangram’s API. Compare pre-2023 to post-2024. Now here’s the part that won’t make you any friends. Do the same analysis, stratified by institution, department, or dare I say, even by PI. Pangram isn’t perfect but it’s surprisingly accurate by way of personal experience.
I’ve even tried writing my own Claudish to English translator for the writing I shouldn’t be doing anyway, but it only goes so far.
I’ve come to the same conclusion about AI and my creativity and understanding. I see terrible examples of colleagues shamelessly saying “I put your figure into AI and it said this”. I was pretty insulted and embarrassed for them.
Great take on AI in writing, particularly academic works. I recently downloaded data from NIH RePORT on K99 and F32 postdoctoral fellowships to look at success rates by institute since 2018. I'll have to dive back in to run an experiment on AI text detection across the years of data I already have!