AI Dry July: Aborted

Reflections on trying to go a month without AI. I made it two weeks (sort of). 5.2k words, 23 minutes reading time.

Back in June I wrote about going the entire month of July without using AI. I wrote about why I’m doing it and what I hoped to achieve.

I failed.

I made it about halfway through the month before I had to call it off. It started off with some personal / non-work exceptions here and there. Then some exceptions crept into my day job. Then the opportunity costs of not using AI for some of the things I had going on this month became too high.

This is a reflection on the attempt. The good, the bad, where I cheated, and where I’m at now. At >5k words it’s the longest thing I’ve ever written on this newsletter by a large margin.

1. Positives

2. Negatives

3. Exceptions

4. Conclusions

1. Positives

Better recall and mental maps

An admission (I’m not alone): In the nearly 4 years ChatGPT and the like have been on the scene I’ve taken my fair share of turns writing large chunks of text for proposals, pitch decks for companies I’ve been involved with, responses to reviewers, and so on. And every time, without exception, I can’t really recall what I wrote or easily explain the reasoning behind it. This tracks with that MIT study showing something similar. If I had to answer questions about something I wrote with heavy assistance from AI I would have struggled. Or even just going back to revise largely AI-generated text - good editors have a knack for revising other people’s writing, but I hate doing it. When I’m revising my own writing, I remember how this paragraph in the approach connected to that section in the significance, and how it all ties together on the aims page. I have a mental map of where things are, because I did the hard work to consider the scaffolding and move things around during an edit.

When working with large chunks of AI-generated writing it feels like revising someone else’s writing. With latest gen frontier models (Opus 4.8, GPT-5.6) the writing is probably really good, but I don’t start the process with a map of the throughlines connecting parts of the narrative. The same thing is true with code. Writing code is harder. Reading some else’s code (even if it’s good, perhaps especially if it’s good) can be much harder.

An immediate benefit I got out of my few weeks of not using AI for writing was having a much tighter connection to the text I write. For sure, it takes much longer to get words on the page, but revisions are much easier, and I feel much more confident in follow-up discussion than if I had used AI extensively in ideation and creation.

The same is true with reading. Sure, an AI can summarize a paper for me, pulling out key ideas and such. I wrote about this here recently.

FOCUS Prompt for Summarizing Academic Papers

But my understanding and later recall of the paper, the motivations behind the experiments, the design (and its flaws), the execution (and flaws there too), are nowhere near where they’d be if I’d have read the paper (or at least the intro, discussion, and figures). Most importantly, I develop a sense of where the gaps are (and thus where the opportunities are) much more easily when I’m reading a paper than when I’m relying on an AI summary. All things I knew before starting this experiment, but the self-experimentation reinforced my priors.

Less stress about AI tells, and enjoying writing again

Will I go back to never using AI while writing? No. It’s a helpful tool, and I’ll use it when it serves me, albeit a little more judiciously. However, when I’m using AI for writing I end up spending far more time than I want removing the AI-isms. So much so that I wrote a Claude skill to remove these patterns from the writing I don’t really want to do anyway (cover letters for manuscripts, for example - the abstract should suffice, don’t require me to write a cover letter!).

De-slop the text you shouldn't be writing anyway

It works well but it isn’t perfect. Is going back to remove the em-dashes and the “not x, not Y, but Z” and the like a good use of my time? Is this the thing I worked so hard for for the past few decades, all the training, all the effort crafting my voice, all the writing I do here to try to become a better writer? Do I want to spend my time doing this at all? No. Not at all. Maybe (surely?) the AIs will get better at not using these bland and tired constructions, but I imagine something else will take the place of the emdashes and tricolon abuse and whatnot.

These past few weeks I was much happier spending more time thinking of how to phrase something or fleshing out an idea, while spending zero time managing AI and correcting all the tells. Was the writing I did myself unassisted better than writing with an AI assist followed by extensive editing? I don’t know. I’d be willing to wager it might not be (test your own judgment, you’ll be surprised). But I certainly enjoyed the experience of writing a lot more, and stayed in the flow for longer, without the sidebar of all my other recent Claude chats begging to distract me (see below on bugs, bikes, engines, plumbing).

Reduced token anxiety

Earlier this year I wrote about token anxiety and how session limits can really upset my work life balance. And I’m not in the camp of using the “good enough” faster/cheaper models when a better one exists. The cost of a hallucination or mistake that Kimi makes that Opus doesn’t is far more expensive than the tokens I burn on Opus. I’m maxing Opus 4.8 with high or extra high thinking enabled most of the time.

The Four-Hour Session Treadmill

When not using AI, I didn’t have this session limit anxiety, and that freed up so much mental bandwidth to focus on a problem on my terms, on my timeline, rather than when my session limit restarted.

Attention defragmentation

The past few weeks I had a better handle on my attention.

With no chat window open there was no sidebar of half-finished conversations begging for my attention, no checking whether the thing I asked for thirty seconds ago had finished. I wrote in longer stretches with deeper flow (being summer and not being as meeting-packed around here helps). A morning that used to splinter into a bunch of prompt-and-wait cycles, these past few weeks became a more cohesive single deep flow session.

The dopamine hit from one-shotting a working prototype is real, and I defend it below. I think that same hit fragments a writing session. Every prompt is a small slot machine, and the pull to keep feeding it competes with the slower payoff of working a on paragraph or whatever until it feels right. A couple weeks of abstinence (or at least moderation) made me notice how often I’d reach for AI without ever deciding to.

Learning about new features in tools I use

I still like writing code. I don’t get to do it much in my job any more, but there’s still something fun about solving a little puzzle, like figuring out a video game with very cryptic goals. It’s not as valuable of a skill as it used to be, but I still enjoy it.

Much ink has been spilled on the risks and benefits of writing code with AI. I’ll just note that I was able to discover or rediscover newer features of tools and libraries that the LLMs don’t really know about yet. Take dplyr 1.2.0, for instance, released in February. There are some really nice expansions of the filter() function, as well as some nice additions like recode_values(). Same thing with Quarto. Quarto 1.9 has some nice goodies like PDF accessibility features and support for Typst books (which I used for the PDF version of my DSTT book).

Data Science Team Training

2. Negatives

Spending more time on worse teaching materials

I’ve been building course materials for a new genomics foundation course I’ll be teaching this fall to our data science students. As I mentioned in the original post, I’m going to ask my students to struggle through some of the readings and assignments, so it’s only fair if I struggle through creating those assignments and course materials.

However, one of the things I’ve found AI really helpful for is for building one-off web apps to demonstrate a concept. I’ve found vibe coding a throwaway interactive web app can take <5 minutes and can be far superior to janky diagrams I can build in PowerPoint that take much more time. For example, back in May I vibe coded this little interactive genetic drift simulator in pure HTML+JavaScript deployed to a GitHub pages site. I suppose I could have made an exception for this kind of thing, but I wanted to avoid that dopamine hit you get from one-shotting something like this, afraid of starting a slippery slope to abandon the whole AI Dry July plan (which, I eventually did). I’ll probably go back to some of my materials I created over the last few weeks and vibe up a few more tools like this one.

Screenshot from an interactive genetic drift simulator I vibed up in Codex.

Spending more time on inferior literature discovery

Another thing I missed was the kind of research that you can do with specialized tools like Consensus (like I talked about here) or even with the generic “Deep Research” tools in Claude/ChatGPT. On a few literature searches I ended up spending more time on this and surfaced far fewer papers than if I had used some AI help here. I suppose I could have made an exception for this one too, but it’s nearly impossible to separate the “find me papers around this topic” from “summarize all the papers you found on this topic.” Most of these tools do both at once.

AI-powered Literature Review & Synthesis

Zotero + Consensus AI

I put off several projects that involve writing code

I don’t get to write a lot of code these days, but still I had a few small projects in mind that I wanted to do this month. I ended up just not starting some of these projects at all. One involved tinkering around with an nf-core workflow for CRISPR screen analysis, and another involved getting some data from the IUCN Red List API and doing something with it (I started doing this with Claude Science right at the end of June, but put the project down these last few weeks). In both cases, I had a pretty clear picture of what I want to do, but the activation energy was just high enough in both to make me put them off. I haven’t written Nextflow code in a while, and I didn’t feel like reading the API docs for the IUCN project. I could have spent time on both, but (1) since I don’t need to write much Nextflow I don’t care that I’m deskilling there, and (2) the LLMs are great at reading API docs and helping me construct the correct call, such that doing this myself would have been more time-consuming with little/no benefit to me.

Opportunity costs: Everything took longer, and some things I wanted to do didn’t happen at all

More generally, the bigger cost to me over the experiment was time, and the things I didn’t do. There were a few RFPs with short turnaround times that slipped. I worked a few evenings and a weekend on a side hustle project that probably would have taken me an hour if I wasn’t doing this to myself.

Writing/coding/doing/etc without AI is fine when my calendar is open, like it is over the summer with no courses, far fewer meetings, fewer demands on my time. It’s a different call when I have every 15 minutes of every day scheduled this week and it’s Thursday night and I have a midnight deadline.

I don’t want to dress up doing things without AI as something that’s always virtuous. Struggling through a complicated topic in a proposal or manuscript or code base is an investment. Struggling through administrative chores or boilerplate I’ll never think about again is just slower and an inefficient waste of my time (and my time requires more water and energy usage than a Claude query). Without doubt I lost hours to friction that taught me nothing, nor sharpened any skill I care to retain.

Where you go when you’re stuck now

While working up some examples for a Bioconductor lab I want to incorporate I ran into some trouble. When I hit a wall this month I noticed where I used to turn to for help. I previously turned to SeqAnswers or Biostars. Stack Overflow was the reflex for a decade, and it’s a shell of what it once was. The good answers are old and the people who wrote them have scattered. It’s not 2018 any more. The fallbacks have eroded, and AI is part of why. It handed me a faster tool but it drained the places I used to go when I needed help working something out. The cost of abstaining is higher now.

3. Exceptions

I had to make a few exceptions along the way. AI meeting notes I can’t live without. With some of the other items, could I have done these without genAI? Sure, but it would have taken me much longer, and at great expense. The things here aren’t things I care about deskilling in, because I don’t really have much skill in anyway. If anything, I’ve learned an incredible amount from asking follow-up questions, and attempting to DIY things I never would have without AI help.

But one exception leads to another, leads to another, and so on.

Meeting notes

I have Zoom AI companion automatically enabled on all of my meetings, even when they’re in person. Unapologetically. It’s a godsend. I can better engage in a conversation if I’m not scribbling down notes. And I can’t write by hand that fast (and typing notes on a computer is distracting to me and my counterparts - I could be writing an essay like the one you’re reading now, for all you’d know). And I rely on these notes all the time.

So I didn’t turn this feature off, and I’m glad I didn’t. Writing legible notes fast by hand with a pen isn’t a skill I’m worried about deskilling in. Could I have asked my EA to come to all my meetings and take notes by hand? Sure, but man what an expensive waste of their time this would have been, to absolutely no one’s benefit.

Generating alt text

I’m working on slides for a genomics course I’ll be teaching in the fall. We’ll need to meet the federal digital accessibility compliance deadline next Spring, so I’m going ahead and taking care of this now. A big component here is adding alt-text for screen readers to the images. I don’t know what PowerPoint is using behind the scenes to do this, but it’s pretty good. I manually examine everyone before approving, and I haven’t caught one yet that isn’t perfect, or at least much better than what I could write with limited time and competing priorities.

I wrote here a while back about how to do this in Quarto with Claude Code if you’re using Quarto for slides or other materials.

Adding Alt Text in Quarto with Claude Code

Fixing broken stuff

I used AI liberally for mechanical/maintenance issues. I needed to troubleshoot my son’s mountain bike. I was chasing gears for hours trying to troubleshoot the derailleur indexing when the culprit was actually a bent hanger. My lawnmower self-propulsion stopped working, and I used ChatGPT to help me narrow down whether it was a transmission or worn gears. I also used Chat/Claude to help me with a plumbing issue, and an issue with the rod bearings on my Kia’s engine causing excessive oil consumption. Sure I guess I could have spent $100 at a bike shop, $200 at some small engine repair shop, $500 for a plumber, and $1,000+ at the chop shop, but some AI assistance and DIY elbow grease saved me thousands of dollars and many hours of hauling broken gear around to all these places.

I recently had a conversation with a fellow Software Carpentry instructor about AI in the SWC curriculum. The discussion touched on the possibility of using AI as a learning assistant. I’m torn on this one. Using AI for these things really helped me learn more about bike components, small engine repair, etc. But I think the difference here is that there was a physical barrier. Claude could tell me to how to adjust limit screws and cable tension until things lined up, but couldn’t do it for me. I physically had to put my phone down and do something with my hands in the real world. Contrast that to coding - it’ll just do it for you, or at minimum be the helpful assistant, asking you, “would you like me to implement and test this for you?”

Identifying flora, fauna, and pests

The next class of exception fell into insect, plant, and plant pathogen identification. I found a few really interesting moths in my garden. I wanted to confirm that indeed my hedges are covered in poison ivy. And I figured out that a palm plant in my office is infested with spider mites (and asked for treatment options). I asked what could be done about these damn lanternflies (nothing). Sure, there are other apps out there that let you do this, but don’t they all or mostly use genAI behind the scenes? And there are probably local Slack or Discord servers with channels dedicated to this kind of thing, but I bet most real people on those servers are using AI before responding.

Visualizing ideas for home projects

I have a mud room / foyer between my back door and the entrance to the rest of the house. The floors needed stripping, sanding, and painting. Badly. We did a two-color diamond pattern, but before committing I tried a few color combinations by taking a photo of the room (top left) and telling Nano Banana what I had in mind, supplying candidate colors from the Behr paint catalog. This helped me visualize what the room might look like before starting the project (bottom row). We ended up going with black and white (top right).

Top row: real photos, before and after. Bottom row: AI-generated mockups from Gemini / Nano Banana supplying Behr paint catalog numbers.

It helped me outdoors also. Last year we ripped out a section of the lawn and planted a bunch of natives. Long term we’d love to rip out the entire lawn and make the entire backyard a meadow filled with natives, maybe even get a few honeybee hives. I asked chat for some ideas about what this might look like after going over my preferences, and while the plants shown aren’t necessarily central Virginia native species, it helped me visualize what a no-grass meadow in place of a lawn might look like.

More than anything, this exercise convinced me that I need to pay a real landscape architect to come over, in real life, and help me with design. Which, I’m doing. This will be a labor of love for years to come, and I know enough to know when I need a professional. Also, there’s something not quite right about using an algorithm to design a more inviting backyard for me to touch grass, literally.

Left: my current backyard. Right: replacing the lawn with a meadow of natives, courtesy of ChatGPT. Conclusion: I need to hire a landscape architect.

Day job exception #1: proposal review

This exception that really started the slippery slope at work.

I have a very tight deadline for a major proposal that we’re leading here at UVA in partnership with several companies and university centers/departments. While I tried to (and ultimately was happy that I did) write most of what I contributed to the pre-proposal without relying on AI, using AI as a reviewer was absolutely necessary.

The solicitation itself suggested to applicants that the sponsor would be reviewing proposals with AI. At least that’s what the Proposal Evaluation → Proposal Review Process → Handling sensitive information section seems to suggest. I have very reliable intelligence to suggest that other federal funders are also doing this.

As such, not using AI during prep for continuous review as the role of an evaluator in the sponsor’s office wouldn’t be some honorable thing to brag about. It would just be stubborn and foolish. And disrespectful to my colleagues - we’re all spending hundreds of ours collectively on this proposal, and we should do whatever it takes to maximize the chances this gets funded. This isn’t a typical NIH/NSF proposal. It’s a behemoth with tons of compliance checks, any one of which could get your proposal that cost thousands of hours collectively, dismissed without review as being nonconforming.

Day job exception #2: Fact checking me on an AI/Biosecurity paper I’m writing

Back in the GPT-4o / Opus 3 days, or even now with most open-weight models, it was the human’s job to fact check the AI. Now I’m getting AI to fact check me.

For a few weeks now I’ve been writing a mini review paper on AI and Biosecurity, summarizing a lot of the truly great work coming out of RAND, NTI, SecureBio, Active Site, CLTR, UK AISI, US CAISI, IGSC, Microsoft, NIST, IBBIS, MIT, Scale AI, Berkeley, National Academies, NSCEB, Johns Hopkins, several of the frontier labs, and many others. The paper is expanding on many of the topics I’ve covered here.

It started off as another newsletter post here, but became so long and extensive that decided to turn this into a paper I’ll try to publish soon. And I did this without using AI beyond some citation network analysis and literature discovery.

I read all these papers I’m citing in the review. Every one of them. And I wrote the review without using AI. However, with a few hundred papers, preprints, whitepapers, model cards, benchmarking websites, conference talks, and so on, and because I’m citing numbers and such in tables, I have a bit of anxiety about publishing this, even in preprint form, for fear that I’ve misrepresented a benchmarking claim or study design (at best), or misstated a result or numeric value (at worst). This is a very small community, and many authors are colleagues.

So I set up a simple but effective fact-checking agent with Claude Cowork. I exported the Zotero library along with a BiBTeX file of all the references I used in my manuscript. The BiBTeX file pointed to the location of each citation’s PDF file on disk. I then got Claude to read every sentence, and where there’s a number or claim backed by a citation, to read the actual PDF of the thing I’m citing and ensure that I didn’t misrepresent the claim, and to return a table of claims and numbers cited, together with a judgment (verified, questionable, misrepresentation, inaccurate).

This thing burned hundreds of dollars in tokens reading hundreds of PDFs, making notes, and fact-checking my own writing, all using Opus 4.8-extra. But worth it to avoid even a single, simple human slip.

If you’ve made it this far, thanks for reading. And if you’re an expert in the AIxBio / Biosecurity world (especially if you’re at one of the organizations and likely one of the authors I’ve cited in this manuscript), and you want to review it and offer comments on this manuscript even before I publish it as a preprint, please email me (address here). Also, if you’re interested in being a peer-reviewer if/when I eventually try to get it published in a journal, let me know that also.

Other day job exceptions

As noted in the Negatives section, the opportunity costs were extraordinarily high, too high for me to take a month of complete abstinence. A few examples:

Fable

Anthropic redeployed Fable in the beginning of July, giving everyone on a subscription plan a week to use it before it dropped back to API rates only. I had a sort of work project, sort of side project thing in mind — something that would comb through the grants.gov and sam.gov postings using their respective APIs, then customize alerts to me based on my interest with some AI thing reading through the solicitation details. Fable one-shotted this thing, perfectly, installable with uv, unit tests with pytest. I still have work to do to add some features I’d like to add before releasing, but it’s a great start, one that I wouldn’t have started at all without AI.

AI Upskilling Series at UVA

The UVA AI Upskilling series is still going strong over the summer. I’m attending several of these. Hard to avoid using AI at an AI upskilling workshop.

Stress testing my course assignments / assessments

In designing my upcoming fall course, I just wanted to see how well Claude could complete the assignments.

Answer: perfectly, instantly, and cheaply even with Sonnet 5. For both code and paper discussion assignments.

See also my previous lamentation on this topic, and toward the bottom of this post on why code isn’t going to be front and center in this course.

4. Conclusions

So where does this leave me? Not as an abstainer.

There used to be a time when intelligent people could make a defensible argument that these tools aren’t useful, they make too many mistakes, or that they’re just worthless next-token predictors. That point has passed, and these arguments are no longer opinions. They’re just factually wrong, incorrect statements. Driven by identity more than evidence.

The rule I’m keeping from my AI (semi-) Dry July fortnight is about which productive struggles are worth keeping. Some amount of friction is useful. At least looking at my code while debugging helps me consider how the system fits together, the way fixing one broken derailleur means I can fix the next. Other frictions teach me nothing, and I usually don’t feel diminished by handing it off. The category error I’d drifted into was treating every cognitive task like something I could offload, outsourcing the parts that were actually mine to learn and defend.

So the policy, for me (which will evolve, no doubt). Prose I’ll have to defend, proposals I’ll write, papers in my own field, lectures I’ll deliver, and some aspects of software architecture: these are all are mine to do (or at least mine to start off with). One-off demos, first-pass literature scans, editing, and genuinely disposable work I don’t care about can and will go to the machine. And as I’ve mentioned in previous posts, I have never and will never use AI speak for me: if I’m expressing an opinion, or a sentence ever has an “I” in it, it’s written by me, with zero LLMs speaking for me.

Teaching materials will probably land on a 90/10 split in favor of not using AI. Not because AI can’t make good teaching materials, but because I’ll be asking students to struggle through the material so it’s only fair that I struggle through design, and also for the reasons outlined in Positive #1 (mental maps and recall).

I don’t know what higher education will look like in 5 years, but abstinence-only education has never worked for anything, and I don’t think that you should only engage in data science if you’re in a long-term committed relationship with a card-carrying Data Scientist™. When, how, and to what degree to thoughtfully incorporate AI into my classroom is something I don’t have a perfect answer to yet.

These few weeks of self-experimentation didn’t change my mind so much as sharpen what I already knew. AI does result in deskilling. Some things I’m okay deskilling in, and others I’m not. I’m going to be more intentional with what I hand off to the LLMs, and I’m no longer handing over wholesale the work I’d miss if I forgot how to do it, or anything I’ll ever need to understand, defend, revise, and own.

⁂

Coda: caffeine

Over 15 years ago I made a terrible life decision. I tried giving up caffeine. I had developed a deep physiological dependency, getting severe headaches without my morning megadose. I traveled and I went out camping and backpacking more than I do now, and carrying coffee making supplies wasn’t worth the weight and space and hassle it required. I thought I could rid myself of the addition and live more freely without the drug. It took me months to completely cut caffeine, and even months after having zero caffeine the craving never went away. I always felt like a part of my brain wasn’t firing on all cylinders, and the craving for caffeine was still preoccupying.

I started adding caffeine back into my life on a very limited basis, once per week or less, and found the drug absolutely supercharged my productivity, mood, and focus (unsurprisingly). Once a week turned into twice per week, every day, and very soon back to multiple cups daily. The point is, even long after the drug had metabolized out of my system I always felt like I was missing a limb, like some part of my cognition had the brakes engaged, a governor module I couldn’t remove. A physical addiction in the literal sense.

Michael Pollan describes addiction in his 2021 book, This Is Your Mind on Plants. In an earlier interview with NPR he notes:

“I think the word ‘addiction’ has a lot of moral baggage attached to it,” he says. “As [Johns Hopkins researcher] Roland Griffiths told me, if you have a steady supply of something, you can afford it and it’s not interfering with your life, there’s nothing wrong with being addicted.”

Is access to generative AI like access to my morning coffee? I have a steady supply. I can afford it (at least for now, as long as my employer keeps footing the bill). Is it interfering with my life? This experiment didn’t give me a straight answer to that one.