Your best documentation is unhinged rambling

Someone on the team needed to know why a part of the system had been built the way it was. The person who built it had left. They pointed an agent at that person's old sessions and read the reasoning straight back.

That took 2 minutes, then they knew why. Nobody had ever written any of it down. No design doc, no comment in the code explaining the trade-off. The explanation existed because somebody had talked the problem through with an agent months earlier, likely in the middle of doing something else.

This has played out a few times now. Full disclosure: I've spent the last 4 months on the small team building traces.com, so I'm biased.

Some serial founders have a good habit

Despite every instinct to move on and forget it, founders who've been at it a while have to learn from their mistakes or pay for them twice. That's why a lot of us keep a "failure log". The failure log notes what didn't work and why. A brutally honest, sometimes painful record of your mistakes and decisions. I know, because I keep one myself.

For development teams, this has always been too costly and therefore has never really been a thing. We record the part that survives: code, git commits, Jira tickets, formal documentation. Maybe a presentation if we're fancy about it. But never "why this library over that" or "what happened when I tried X". It'd be too verbose and sound more like unhinged rambling than anything useful.

The problem is that the information is incredibly useful. Working in a team where you can't always get hold of everyone to talk things through, having the raw data means I can stop wasting hours or days on something somebody else already attempted.

And now, with AI generating code at speed, the attempted paths and the considerations behind them are piling up faster than anything was recording them before.

Your failure log now gets written by accident

Agent work produces that record without anyone deciding to. You direct the session, correct the behaviour, try different angles, make calls about what to do next. All of it gets stored as you go.

In a single day you may have many sessions, each documenting different aspects of your work. Each containing decisions. That's fundamentally different from git, which only holds the final commit of what you worked on. The clean version.

That's not to say you can't rebuild a failure log yourself. I've tried it. It meant a day of going through 12 months of git history with an LLM interviewing me, question after question, trying to remember what I did and why. It worked. But there's not the same depth to it, and I probably forgot something.

Nobody had to adopt a process for this. No template to fill in, no weekly discipline. The record exists because the work happened.

How it works in practice

Some time ago a new component was added to the stack, and everyone had to get it running locally. We relied on chatting about the difficulties as they came up, but nobody had a clear picture of where the time was going. So I asked our agent traces where people were getting stuck with the setup and what problems they were facing.

Going through the data, the agent found a handful of mistakes repeated across the team. The problems were environment specific, because the component was new and the docs hadn't caught up. I rebuilt the setup into single scripts. The friction went away, and nobody had to sit through a pair programming session to get there.

Setup friction leaves nothing behind anywhere else. There's no ticket for losing two hours to a broken install. No commit either, and nothing in a pipeline log.

The same question works on the code itself. Areas nobody has touched in a while, the decisions that led there, whether those decisions still apply given everything that's changed since. Modules that keep needing fixes. Static analysis can flag a module as complicated, but it has no idea that people keep losing afternoons in it.

All of these are questions about the codebase rather than about the people writing it.

None of it arrives on its own though. There's no dashboard that opens itself and tells you what to fix, so you have to show up with something you're curious about. Your agent goes and pulls it together, and that session becomes another trace in the pile.

What happens at scale?

Everything mentioned here happened inside one company's dev team, which is the only agent trace data I've had access to. Here's where I think this is going.

The industry is early days on this. But from where I'm sitting, this raw data goes a good deal further than the codebase. Connecting people across an organization to the things they've actually worked on. Customer support agents to the clients they've handled. Your own sessions and the tools you use, measured rather than guessed at. It's the groundwork for a kind of observability that doesn't exist yet.

As a builder, I look forward to spotting friction points across an organization and putting internal tools in place to fix them, with measurable improvement to show for it. I think that's really f'ing cool, and it's within reach for any org that's adopted AI to that extent.

This data was always there, hidden in everyone's head. The difference now is that it's written down.

So if your team has been working through agents for the past year, most of that record has never been read. The refactor you keep postponing has probably been documented several times over by people who were only trying to get their own work done. Whatever your AI strategy looks like on paper, this is the data it's producing. Somebody should be reading it.