Keep a Diary

You should keep a diary.

Not the kind you write before bed about your feelings. A research diary. A thinking diary. A diary of what you learned, what you decided, and why.

If you’re working with LLMs — and increasingly, who isn’t — this is not optional advice. It’s load-bearing infrastructure.

The Problem You Already Have

You’re in the middle of a project. You had a conversation with an LLM last Tuesday where you worked through a tricky design decision. You considered three approaches, rejected two for specific reasons, and chose the third. Today you can’t remember which approach you chose or why you rejected the others. The LLM certainly doesn’t remember — that context is gone.

So you have the same conversation again. You re-derive the same conclusions. You might even make a different decision this time, not because you have new information but because you forgot the old information. The LLM helps you think it through again, confident and fluent, as if for the first time. Because for it, it is.

This is the sawtooth pattern. You build up reasoning, lose it, rebuild it, lose it again. Each cycle wastes the work of the previous one. Doyle identified this problem in 1979: “discarding derivation records may be condemning ourselves to continually rederiving information.” He was talking about AI systems, but he was also talking about you.

The Solution Is Older Than Computers

Scientists have always kept lab notebooks. The practice isn’t about memory — scientists have good memories. It’s about justification. When you write down what you observed, what you concluded, and why, the reasoning becomes inspectable. You can return to it months later and evaluate whether the conclusion still holds given what you know now. You can share it with a collaborator who wasn’t in the room. You can catch your own mistakes.

The lab notebook is external memory — knowledge that lives outside your head, carries its justifications, and lets you trace how you got from observation to conclusion. Scientists didn’t need LLMs to realize this was essential. They needed it because human memory is lossy, human reasoning is fallible, and important work requires a record that survives both.

What Changes with LLMs

Two things change.

First, the throughput of thinking goes up dramatically. An afternoon with an LLM can cover ground that would have taken weeks of solo work. You explore more options, analyze more data, consider more angles. The volume of reasoning that needs to be captured per unit of time is much higher than in a traditional lab notebook.

Second, the LLM is a second thinker with zero memory. Every session starts from scratch unless you provide context. The diary is how you provide that context. Without it, you’re re-briefing a brilliant but amnesic collaborator every morning. With it, you hand them the notebook and say “read this, then let’s continue.”

What Goes in the Diary

An entry is a unit of thinking. Not a transcript of a conversation — that’s too raw. Not a polished paper — that’s too refined. An entry captures:

• What you learned. An experiment result. A finding from reading code. Something a reviewer pointed out.
• What you decided. Which approach you chose and why. Which alternatives you rejected and why.
• What surprised you. The unexpected finding. The assumption that turned out to be wrong. The connection you didn’t see coming.
• What you don’t know yet. The open question. The experiment you need to run. The thing that would change your mind.

Each entry is dated. Each entry stands alone — you can read it cold and understand what it says without reading every entry before it. Each entry is specific enough that it’s useful six months later when you’ve forgotten the context.

The Practice

I’ve written 296 entries over four months using a simple CLI tool (entry create "title"). Each entry is a markdown file in a date-organized directory. The LLM can read them, search them, reference them. When I start a new session, the LLM can read the last few entries and pick up where we left off.

Some entries are small — a paragraph capturing a single insight. Some are large — a full analysis of an experiment with data tables and implications. The important thing is not the size but the habit. If you learned something today that you’d need to re-derive tomorrow, write it down.

The entries compound. A single entry is a note. A hundred entries are a knowledge base. When I needed to write a paper about confabulation in LLMs, the entries from two months of experiments were the raw material — the observations, the results, the failed hypotheses, the corrections. The paper was assembled from entries, not invented from scratch.

The Diary as Infrastructure

Here’s what I didn’t expect: the diary changes how you think, not just what you remember.

When you know you’re going to write an entry, you think more carefully. You distinguish between “I feel like this is true” and “here’s the evidence.” You notice when you’re making an assumption. You catch yourself hand-waving. The act of writing for a future reader — even if that reader is yourself in two weeks — forces clarity.

And when you compose entries into larger artifacts — papers, blog posts, presentations — you’re not starting from a blank page. You’re selecting and arranging material that’s already been thought through. The heavy lifting happened in the entries. The composition is assembly, not invention.

You don’t sit down to write a paper. You sit down with four months of entries and realize the paper is already there, distributed across dozens of dated observations. You just have to find the shape.

Start Today

You don’t need special tools. A directory of markdown files organized by date works. I use a CLI (entry create "title") that creates a file at entries/YYYY/MM/DD/title-slug.md with a date header. Anything that creates a dated text file will do.

Write one entry today about whatever you’re working on. What did you learn? What did you decide? What surprised you? What don’t you know yet?

Do it again tomorrow. And the day after.

In a month you’ll have a knowledge base. In three months you’ll have the raw material for whatever you need to write. In six months you’ll read an entry from today and be grateful you wrote it, because you will have forgotten everything it contains.

Keep a diary.