Tag: Agents

A laptop glowing in a dim home office at night, showing a queue of pull requests waiting for review.

The Agent That Never Merges

Allen HutchisonLeave a comment

Most mornings now I open my laptop to a queue instead of an inbox. Overnight, while Chris and I were asleep, an agent read our open GitHub issues, decided which ones were ready to build, picked the oldest one, and opened a pull request. The tests pass. Greptile reviewed it, the agent worked through the review comments, and Greptile now scores the change five out of five. The description explains what changed and why. The commit history looks like a careful engineer wrote it. What’s left for me is to decide whether it’s good enough to ship.

I said last month that I’d keep writing about the engineering as Chris and I build Vycari. This is the second post in that series. The first was about the issue tracker, and what belongs in it now that agents do most of the work. This one is about the pipeline that works through it. It’s the part of our process I trust most, and the part I was most nervous about building, because it’s where we let an agent write code that ends up in front of you.

Two Founders, No Platform Team

There are two of us. There is no platform team, no SRE rotation, no QA function, nobody whose job is keeping the lights on so we can build product. Whatever infrastructure we need, we build ourselves, out of the same hours we need for everything else. I like this stage of a company, but it means every hour Chris or I spend on triage, boilerplate, or a stale dependency is an hour not spent on decisions only the two of us can make.

The obvious answer is to point an agent at the boring parts. The harder part is the discipline that requires. An agent that opens pull requests unsupervised can save you a lot of time or create a lot of problems, and which one you get depends on what you refuse to let it decide on its own. I’ll come back to that. First I want to explain where the tooling came from, because it didn’t start as a product decision. It started when I noticed I had built the same thing twice.

We Had Already Built This Twice

Before the company, I was maintaining Gemini Scribe, a TypeScript plugin for Obsidian I’ve been building for a couple of years, and I had started experimenting with overnight agents to do the maintenance work: keeping the docs current, writing development logs, triaging issues. An architecture audit, a daily changelog generator, a skill that reviewed pull requests against the project’s conventions. When I started on what became Vycari’s codebase, a Python backend, I began writing the same things again under slightly different names. I hadn’t noticed they were the same habits until the second or third time.

In June I pulled all of it out into its own project. Maintainerd is a marketplace of Claude Code plugins for maintainer work: audits for architecture, test quality, security, dependencies, and documentation; a daily changelog; a research radar that scans arXiv for relevant papers; a skill that opens pull requests and one that works through reviewer comments; and the pipeline this post is about. It’s open source under the MIT license and lives in the Vycari org.

The name is a pun on maître d’. A maître d’ runs the front of the house so the kitchen can cook, which is what these skills do for a repository. It also reads as Maintainer Nerd, which I didn’t notice until later and haven’t tried to fix. I wrote when we announced the company that using agents today is still a hobby. Naming my repo maintenance tooling after a restaurant host tells you which side of that line I’m on.

Every skill in maintainerd is repo-agnostic. It reads a small config contract, a JSON file plus a few markdown documents of house rules, that says what language the repo is in, where the tests live, what the lint command is, and which GitHub labels mean what. The same skill works on a Python backend and a TypeScript plugin because the repo-specific knowledge is in the config, not in the skill.

That separation is what made the rest possible. A skill hardcoded to one repo’s conventions is a one-off script. A skill that reads a contract is infrastructure you can point at a new project in an afternoon. It’s the same argument I made when I wrote that prompts are code: an instruction that hardcodes its environment isn’t reusable. Without that separation, two people could not have built the pipeline I’m about to describe in the time we had.

What the Pipeline Actually Does

The piece of maintainerd we lean on hardest is called auto-dev. It’s a state machine. An issue moves from idea to clarification to plan to implementation, one step at a time, while I’m working on something else. It has three parts.

The first is what goes in, and the last post covered most of that. The tracker holds three kinds of work: bugs we found and can’t fix right now but an agent can, follow-ups from review that weren’t worth forking a subagent to fix on the spot, and pointers to planned features whose real content lives in a design document. The pipeline only wants the first two. The design-doc issues are labeled so it leaves them alone, and a person sits down with an agent and builds those end to end.

Most of the issues the pipeline works on were written by agents, not by us. The test for whether an issue belongs in the tracker is whether a cheap model could fix it tonight without asking anything, and an agent that just watched a bug happen usually clears that bar. When one of us writes an issue by hand, we describe it to a skill whose job is to get it to that standard. It reads the relevant code first, so the issue points at real files and functions, and it asks us the two or three questions the code can’t answer, so the issue doesn’t have to ask them later.

I don’t always hold to that. Sometimes I type a few paragraphs into an issue from my phone. When an issue isn’t good enough to build from, the pipeline’s triage catches it and asks. That’s the fallback, not the plan. Either way an issue is filed unlabeled, so the pipeline evaluates it fresh rather than assuming someone already decided it was ready.

The second part is the pipeline itself, which runs on a schedule rather than on demand. A scheduled task fires the skill at a regular interval, and each run is one tick of the state machine. It does the single highest-priority piece of work available and exits. Waiting for an answer, an approval, or a review is what happens between ticks.

Each tick looks at every open issue and asks whether it’s ready to build: whether it has enough context, whether it depends on something unfinished, whether it’s in scope. Ready issues get a plan drafted and posted as a comment, and the pipeline stops there. A plan is a proposal. Only after one of us approves it does the issue become buildable, and only then, on a later tick, does the pipeline pick the oldest approved issue, write the code, run the tests, and open the pull request. All of that state lives in a small set of GitHub labels:

"stateLabels": {
  "needsInfo": "auto:needs-info",
  "planned": "auto:planned",
  "ready": "auto:ready",
  "inProgress": "auto:in-progress",
  "parked": "auto:parked",
  "skip": "auto:skip"
}

There is no database to maintain and no dashboard to check. The state is on the issue, visible to anyone who looks at the repo, and the label history is the audit log.

The third part is the human side. It’s a skill called review-queue that I run in my coding agent. It collects everything blocked on a decision that’s waiting for me to weigh in: plans waiting for approval, pull requests where a reviewer left a comment the pipeline has already addressed, issues it flagged as needing more information. I work through the list top to bottom, the way I used to work a code review queue at Google, except most of the queue was generated by something that also drafted its own response to the objections.

I run it on my desktop and on my phone against a cloud-hosted model, and the phone version is the one that changed my habits. I’ll clear three or four items while standing in a line: approve a plan, answer a question, park something that isn’t worth building yet. That time used to go to scrolling.

The pull requests themselves get more than a glance. Most of the time we spend on them goes to running the change in a local environment and testing it. When there’s nothing to see, I read the code and either agree with it or don’t. Every pull request also goes through an adversarial review agent, and it holds code I write to the same bar as code auto-dev writes.

Here is the part that matters most. There are two gates in this pipeline, and a person has to open both. Nothing gets built without an approved plan, and someone has to read the plan to approve it. And at no stage does the pipeline merge anything. It triages, it plans, it builds, it responds to review feedback on its own pull requests. It never decides that a change is good enough to ship. That decision is still the responsibility of a human, on every pull request, with no exception for the ones that look routine. The merge rule isn’t a line in a prompt. The harness the pipeline runs in doesn’t allow it to merge, so the agent couldn’t do it even if it decided to. I’ve written before about where to put the guardrails on an agent that acts on its own. The guardrail I care most about now is not the one that stops the agent from doing damage. It’s the one that stops it from making a decision a person should be accountable for.

Four Pull Requests for One Bug

It doesn’t always go that cleanly. Last night an architecture audit flagged a real problem in one of our coding convention files, and several auto-dev runs picked it up at the same time. By the time I looked there were four pull requests open, each fixing the same thing in a slightly different way, and dozens of runs still going in the runner. I closed the pull requests, went into the runner, and shut down every instance that was working on it.

The pipeline has a limit on how many pull requests it can have open at once and a rule for reclaiming a build that looks abandoned, and both of those held. What they don’t cover is a burst of runs that all start before any of them has claimed the issue. It cost tokens and an evening, and it was the first failure of any size. I’ve been raising the run cadence and the number of open pull requests it’s allowed slowly, on purpose, so that the failures show up one at a time while they’re still cheap.

The Part That Happens in Public

Gemini Scribe is a public repository with real users, and they file issues. The pipeline treats their issues the way it treats ours: it reads the relevant code, decides whether there’s enough to plan against, and if not, posts a comment asking for what’s missing.

Sometimes that’s better than what I would have done. Someone files a short bug report, and before I’ve seen it there’s a specific question underneath it, pointing at the relevant file and asking which of two behaviors they meant. That’s a faster and more useful first response than a busy maintainer usually gives, and it arrives while the problem is still fresh for the reporter.

Every comment it posts is attributed to Claude Code in the comment metadata, so anyone reading the thread can see it wasn’t me. And each run starts in a fresh virtual machine with access to the repository and nothing else: no connectors, no configured tools, none of my data. The most it can leak is what’s already public in the repo.

Sometimes it asks the reporter what the plugin should do about the bug. That’s the wrong person to ask. They wanted their bug fixed; deciding how the software should behave is my job. The pipeline isn’t broken when it does this. It’s running the same readiness triage that works fine on a private backlog, where the only people it can ask are Chris and me. In a public tracker, that same question lands on a stranger.

I haven’t fixed this yet. What makes it tolerable is the same rule as before: the worst thing the pipeline can do in public is ask a question it shouldn’t. It can’t ship a decision. Nothing it writes reaches a user’s vault unless I put it there. The never-merge rule was meant to protect code quality, and it also limits how much damage an awkward comment can do.

I still prefer this to the alternative. An issue tracker that responds within hours instead of whenever I have a free evening is better for the person who filed the issue. Getting the questions right is a tuning problem. Getting anyone answered at all, with two people, is not something I could otherwise do.

The Same Discipline We’re Selling

I said when we announced the company that an agent has to earn the trust the access requires, and that we’d hold ourselves to that before asking anyone to trust us with their calendar and mail. This pipeline is the first place I get to practice that on myself.

The rule that it never merges is enforced by the harness, but the harness is ours. We built it that way and we could build it differently. It’s a decision, applied to our own code before it’s applied to a customer’s. If I’m going to ask someone to trust an assistant with the private parts of their day, I should at least refuse to let our own agent decide what ships. Everything else is work we’re happy to hand off: triage, drafting, boilerplate, the first pass at a reviewer’s comment. That’s the same division we want the product to make for you. Simon Willison made the argument in a post about proving your code works: a computer can’t be held accountable, so the human in the loop has to be. The decision with real consequences stays with a person who can answer for it. The product is built the same way. What Pepper is allowed to do on its own is set by policies and hooks in the agent framework, not by the model’s judgment in the moment. The harness decides where the line is, and a person decides what the harness says.

There’s a second connection I didn’t expect. Part of what we believe is that nobody should have to learn our vocabulary to get value from an agent. The review-queue skill is that idea applied to me. It doesn’t ask me to understand auto-dev’s state machine or check six label filters across a dozen pull requests. It gives me one ordered list of the decisions that are mine to make. Once you decide an assistant’s job is to meet people where they are, you notice every other place a tool is asking you to come to it.

What’s Next

One more thing I’ve noticed. A pipeline like this works because someone wrote the config contract, the conventions, the review rules, and the gates, and that someone had built and shipped software for a long time before agents existed. That’s where experienced engineers add value now. You build the frameworks that let the agents succeed, and you review what comes back. It isn’t very different from being a tech lead with a team of new grads.

None of this is finished. The pipeline still gets things wrong in ways I only find by watching it, and I’ll write about the ones worth telling. Next in this series I want to cover how we grade the agent’s work when nobody is watching. It’s closer to writing a rubric than writing code. I built a first version of that for my Obsidian plugin, and doing it for a product two people are betting on is a different problem. For now, two people are building a company with a colleague that never stops, never merges, and still needs its work checked every morning. That’s the balance I wanted.

An overhead view of a wooden desk holding two laptops side by side. The left laptop is closed and covered in old stickers with a security badge on top. The right laptop is open with fresh stickers and a code editor on screen. A sheet of dated notes lies between them.

Writing My Way Out

Allen HutchisonLeave a comment

I started writing this blog two years ago because I wanted to share what I was learning about AI, and the projects it kept pushing me to take on. A year ago I wrote the first anniversary post, and it ended with a plan. Two thoughtful posts a month instead of the weekly pace that had already beaten me once. More explainer series, because the embeddings posts had been the most-read thing I published. Some shorter pieces in the spirit of Simon Willison. And permission, finally, to write about the woodworking and the guitars instead of quietly worrying that they were off topic.

I had no idea, when I wrote it, that by the time I sat down to write the next one I would not be at Google anymore.

The plan mostly worked

I published fifty seven posts between that anniversary and this one, which is a little under five a month against a goal of two. I am not sure that counts as keeping a promise so much as failing to keep it in the more enjoyable direction.

I had been afraid that I would run out of things to say. The opposite happened. If I look at my drafts and ideas folders right now, there are close to fifty posts in there that I still want to write and have not found the time for.

The explainer series happened. The Agentic Shift was planned as twelve parts, and it came in at twelve parts, thirteen if you count the introduction, running from September through April. The structure held up almost exactly as I outlined it. What I did not anticipate was how much the ground would move underneath it. I thought I was writing a textbook, assembled in public. What I actually wrote was a journal of a landscape that would not hold still long enough to be documented. I outlined Part 9 as speculation about a future problem and published it as reporting on a present one. I am really proud of that series, and I think it has stood up well.

The shorter pieces happened too, though this one is still a work in progress. Eight Reading Lists, which started as an experiment in whether I could publish something in under an hour and turned into the format I reach for when an idea is real but not yet a post. I never got to the point of posting one every week, and that is something I want to get better at this year.

And the hobbies made it in. A bill in the California legislature that wanted my 3D printer to police itself. A French band I could not stop listening to. A Starlink dish on the roof of a car. The Köln Concert, as an argument about creative constraints in software. Nobody unsubscribed, as far as I can tell, which suggests the worry was mine and not my readers’. There is more of that to come, I hope.

The post that turned out to be about something else

In April I counted my GitHub contributions for the year. There were 4,255 of them, and the post was supposed to be a light piece about the shape of a year of building in the open.

The interesting number was not the total. It was that my most productive day of the week was Saturday, by a wide margin, and the reason was not discipline. It was that Saturday was the day nobody had scheduled anything. I wrote at the time that this was a diagnosis rather than a brag, and I meant it more casually than it turned out to deserve. Given time and no meetings, I build things compulsively, and I am happier. That is a useful thing to learn about yourself. It is also the kind of thing that, once you know it, quietly reorganizes every other question you were asking.

Four months later I left, after twenty one years, most of a career, and about half my life. Twenty One Years is the post about that. Not Everyone Wants a Hobby is the post about what I left to go do, which is a company called Vycari that I am building with Chris Perry. Chris was the product lead for Colab when I ran the AI Developer organization, we were both on the founding team of Gemini CLI, and somewhere in there he became the person I most wanted to build with. He is the CEO, I am the CTO, and our first product, Pepper, is a personal assistant for your life. And Thousands of Goodbyes is the post about pointing an agent at twenty one years of my own sent mail because I could only remember about a hundred and fifty of the several thousand people I had worked with.

I did not plan any of those as a trilogy. Reading them back to back, they are obviously one thing.

What the blog was for, and what it is for now

When I started writing here, the blog had a job inside a larger job. I said as much in the first anniversary post: building in the open and writing about it was how I contributed while my calendar was still too unpredictable for mission critical work, and it let me lead by example, demonstrating the kind of developer engagement I was hoping to inspire in an organization I was running.

That is a perfectly good reason to write, and it is gone now. There is no organization. There is no example to set for anyone but Chris and me. The blog is no longer a thing I do alongside the work, and it is not marketing for the work either. It is the only continuous artifact I have from before to after, which I did not appreciate until I went looking for a through line and found that the writing was the only place one existed.

There is an obvious hazard in this, and I would rather name it now than have you notice it in six months. Founder blogs decay into product announcements. It happens gradually and the person writing them is always the last to see it. So let me be clear about what you should expect. I am going to write about Vycari and Pepper, because they are the most interesting things happening to me right now. We are building a new kind of agent harness and a new kind of agent, and there are a lot of stories in that. Some of them will be technical, like the recent post about what happened to our issue backlog once agents were filing most of the issues. Some of them will be about philosophy, like why the chat window is the wrong interface for most people and why almost nothing a consumer agent does needs a frontier model, and a few of them will be things I did not feel I could say while I worked at Google. What I can promise is that everything I write here is here because I think it is useful to share, because I think you will enjoy reading it, and because I think it articulates why Chris and I, and the company we are building, are different.

The people who read this blog are people who will happily install a CLI, configure an API key, and read a changelog for fun. They are the reason it exists. I am aware that I have just started a company premised on the argument that requiring all of that is a failure of product design, and I do not think those two things are in tension, but I understand why it might look that way. Working out loud on that confluence is most of what I expect year three to be about.

Year Two, By the Series

The Agentic Shift

Twelve parts and a retrospective, September 2025 through April 2026. The series that ate the year, and the work on this site I am proudest of.

Building the Things I Write About

The Reading Lists

Eight of them, starting in April. Short, fast, and the closest thing this blog has to a running commentary.

The Work Itself

Building Vycari

The first three posts of the next chapter. None of them is a launch announcement, which is the point.

Everything Else

The promise I made last year to stop worrying about staying on topic.

Year Three

More of the same, with one change I am now a month into.

For two years I wrote about agents from inside a company with essentially unlimited access to models, infrastructure, and the people who build them. Everything I published came with that thumb on the scale, whether or not I acknowledged it. For the past month I have been building the same class of system with two people and a budget. In the first thirty days of that, the two of us merged 657 pull requests, our agents filed most of the 449 issues that came out of them, and something like a hundred and fifty million tokens went through the product for about forty dollars. Those are not numbers I could have written from inside Google, and they are the constraints almost all of my readers have always lived with. I expect that to make the writing better, and the series on how the two of us build, which started with the issue tracker, is where I intend to find out.

I also want to keep doing what Thousands of Goodbyes turned out to be, which was the first time in two years of writing about agents that I pointed one at something that was not work and had the output make me feel something. There is a great deal more of that available, and almost nobody is writing about it, because we are all still busy benchmarking.

Beyond that, my goals are simple. I want to keep sharing what I learn. Last year I said a weekly pace was not sustainable, and then published nearly five posts a month, so this year I am going to say it out loud: at least one real piece a week, with shorter things sprinkled in on the other days. I am trying to be more active on X and LinkedIn as well, so if you see me there, say hi. I will keep bringing more of my outside interests into this space, though it will almost always have a technical angle. There are more management posts planned, a tribute to a friend who is gone, some book reviews, and more.

Two years ago I hit publish on the first one of these with no idea where it would lead. I have a much better idea now, and I am still not going to tell you it is a plan. I am looking forward to what comes next, and I am glad you have decided to come along.

An ordinary compact sedan in a driveway at dawn, sharply in focus, with a covered exotic sports car blurred in a showroom behind it.

Not Everyone Needs Superintelligence

Allen Hutchison1 Comment

Every morning, while my family gets ready for the day, we have KQED on throughout the house. Proud members since 2000. Then I listen to podcasts while I work out. Either way I am half listening, the way you do, attention drifting between whatever is playing and planning for the day in front of me.

Last week I caught a segment about frontier models, and I could not tell you now whether it was the radio or one of the podcasts. The speaker was talking about pricing and performance, I think in relation to the new GLM model. Then he started talking about cars, and I stopped half listening. These frontier models are Ferraris, the argument went. Most people don’t drive a Ferrari. Most people drive a Corolla.

I have spent a lot of time over the last few years building agents that run on Corollas. Last year I built a coding agent on Gemini Flash 2.5, mostly to find out whether I could get really good performance on a lightweight, less expensive, and faster model. Lately I have been working with Chris on Pepper, a personal assistant for your life. Pepper runs on Corollas too.

That line stuck with me throughout the day. I mentioned it to Chris on a call the next morning, and he tied it back to one of my previous posts, the chat box is a detour. He was right: they are the same argument at two different layers. That post said we handed consumers the wrong interface because a chat window was the honest expression of what models could do in 2022, and then we never revisited it. This is the same mistake one layer down. We build on the frontier tier because that is where the attention is, not because the work requires it.

What Our Product Actually Runs On

At Vycari we are building agents that handle the day to day, and Pepper is the first one. Almost all of its agentic work runs on the cheapest models the big labs sell. Orchestration goes overwhelmingly to Gemini Flash Lite 3.5, with small slices of Claude Haiku 4.5 and GPT-5.6 Luna routed alongside it to keep us honest about vendor lock-in, and to give us a baseline for our own evals. Somewhere around a hundred and fifty million tokens have gone through the system for about forty dollars, which starts looking like a rounding error once you notice that an Opus-class harness can spend forty dollars in minutes.

The work those models do is not exotic. Look at a calendar. Read an email and decide whether it needs a response. Manage my exercise log, or find my to-dos. When I wrote about how everything becomes an agent, the point underneath was that an agent brings value by deciding in the moment, with real input in front of it. Most of those decisions are small. If you were hiring a person for this job, you would not start out looking for a PhD.

That changes the shape of the pricing conversation. For a lot of use cases we have been standing in the Ferrari showroom asking for a deal, when the better move is to walk down the street to the Toyota lot. The question was never how to get the frontier tier cheaper. It was why we were standing in that showroom at all, holding a list that reads: check the calendar, read the mail, log the workout.

The Objection I Keep Hearing

I have heard the objections. Small models hallucinate. They report success on work they never did. They narrate a tool call as prose instead of calling the tool, and the turn ends clean with fabricated content sitting in it. I have watched all of that happen in production.

The objections come out of real experience, and they were true recently enough that I understand why people still reach for them. But this industry is moving so quickly that we often have to check our assumptions on the models that are available today. The cheap tier climbs about a rung a year. The lite model I run today does work that belonged to the mid tier a year ago, and that mid tier was doing work the frontier owned the year before that. The objections have not been updated since the last time they were correct.

So we ran a quick eval. Nine recorded production turns, one for each failure shape our nightly reflection kept filing, replayed against three tiers: the lite model we ship on, the mid Flash model, and Pro. Fixed thinking budget, three samples per arm, and a grader from a different vendor so we were not asking Gemini to mark Gemini’s homework. I expected a ladder. What came back was flatter than that.

That is a small eval and I am not going to pretend otherwise. What I can say is that we have now watched the platform across thousands of model calls, and what the eval showed is holding up. We are not a public product yet, so the data is thin and I would not ask anyone to take it as settled. But every time I take one of our failing cases from Flash Lite and hand it up the ladder, the results refuse to move the way I expect them to.

The headline in our own experience is that this is mostly about how a model handles missing data, not how big it is. When a tool returns real data, all three tiers ground correctly and nobody invents anything. When the data is not there, behavior splits on how the absence gets signalled. Replay a turn with a vague instruction to proceed with whatever it already has, and all three fabricate confident content, with the bigger models producing the more polished fabrication. Signal the absence clearly, and every tier including lite says it cannot see the calendar instead of inventing events.

That reorganized how I think about the problem. The fabrication I had been charging to running cheap was mostly a property of how we told the model it had nothing. Our first recommendation to ourselves was to stop shopping for a bigger model and go fix the signal.

Which puts the weight on the harness instead. The harness manages the context, decides what the model sees, and decides what it is told when a tool comes back empty. Its entire job is to put the model in a position to succeed. Get that wrong and no amount of model will save you.

Two other results fell out of it. The first is a failure where the answer is correct and well-grounded all the way through, and then comes apart in its last few words. A phrase repeats itself, or a contraction mangles. One of ours signed off with “while he recovers today while he recovers.” We call it a garbled tail, and nothing in the logs flags it, because the model finished cleanly and the turn reads as a success. It showed up on every model we tested including Pro. Low rate, stochastic, and scaling did nothing to it.

The second is that the middle is not the safe middle it looks like. On empty tools the mid tier invented more concrete detail, weather numbers and times, than either the model above it or the one below.

There is one clean exception. False completion, where the agent answers after step two of a five-step procedure and reports back as though it had finished, does improve with model size. Our own notes call it the one real model-size win, and it is the only one on the list.

I want to be careful not to oversell this, because we do pay a tax for running small. Ten modules in Pepper’s agent harness exist mostly to catch models behaving badly, roughly 2,500 lines of production code and about the same again in tests. A detector for tool calls narrated as prose. A repair pass for garbled text. A guard for the case where a tool returns a success status with a zero success count buried in the JSON, which we wrote after a model read the status, believed its own plan, and invented the record IDs out of its own request.

Six of those ten ship detect-only. They watch, they report, and they change nothing a user sees, because we do not ship a pass that suppresses model output until nightly grading proves it is effective. The failures they watch for are not exclusive to the tier we run on. Most of that code is the cost of building an agent at all, and we would be writing a version of it at any price.

None of that machinery came out of a design document. It came out of watching. Every night a high-capability model, a Ferrari, reads back every task Pepper took on that day and grades it. It correlates failures to bugs, files the issues, sometimes reproduces a failure as a failing test, and sometimes opens a pull request to fix it. That process deserves its own post and it will get one. The short version is that almost every guard in our harness exists because we watched the model fail in one specific way and wrote a defense against that one failure.

So the Ferrari does have a job here. It is just not the one that answers you.

We Don’t Have a Word for These Models

This is a naming problem, and I think it matters more than it sounds. “Small” is wrong, because the capability is not small. “Cheap” is pejorative and smuggles in the assumption I have spent this post arguing against. “Flash Lite class” is a product SKU, not a category. Nobody has named this tier because nobody is positioning for it.

I want to call them daily drivers.

A daily driver is not a lesser car. It is the car you actually chose, judged on what matters when you drive it every morning. It starts. It’s cheap to run. It does the trip. Nobody apologizes for driving one, and nobody seriously believes the Ferrari owner is having a better time getting the groceries home.

The capability question is already settled, and I published the evidence three months ago while thinking I was writing about something else. When I built a scoreboard for Gemini Scribe, my Obsidian plugin, the point was to stop grading my own agent on vibes. It scores reliability the way τ-bench does, so a task counts only if every one of five runs succeeds. The headline result, which is still published, was that the newer gemini-3.1-flash-lite solves 74.1% of that suite at solve^5 and the older gemini-2.5-flash, supposedly a tier up, solves 57.4%. Same tasks, same judge, about seventeen percentage points apart, and the lite model costs about three quarters as much per run.

Speed is part of this too, and it does not show up anywhere in a solve rate. The daily driver starts talking sooner, and the gap in time to first token is not subtle. For an assistant you are holding a conversation with, that is not a nice-to-have. It is most of what the thing feels like to use.

The Price Gap Is Widening, Not Closing

The objection I take most seriously is that all of this evaporates when frontier prices fall. If the Ferrari costs Corolla money, what is left of the argument?

Watch what actually happened this summer. OpenAI cut Luna’s price by 80% on July 30 and took 20% off the mid tier. Those are their numbers, and the same announcement adds one more line: Sol pricing remains unchanged. Three weeks later they cut Sol too, and on their own product page that second cut is described as running for the next three months.

Read the two together. The cut at the bottom is the price. The cut at the top is a sale with an end date on it. At launch the flagship cost five times what the cheap tier cost, on input and on output alike. Take the promotion out and it now costs twenty-five times, which is where the ratio lands again when the sale lapses in November.

The gap widened because the floor dropped, not because the ceiling rose. The research says to expect that. An MIT FutureTech analysis of the price-performance frontier puts the decline at five to ten times a year for a fixed level of capability, with roughly three of that coming from algorithms rather than hardware. A five-dollar subscription does not need the gap to hold still. It needs the floor to keep falling, and the floor is where the competitive energy is.

Where Cheap Doesn’t Save You

Inference is not the whole bill, and this is the part I would have wanted someone to tell me a year ago.

Grounding costs real money and it does not follow the same curve. Google gives you 5,000 free search requests a month shared across all the Gemini 3.x models, then charges $14 per thousand after that. Anthropic and OpenAI both charge $10. On a deep research task, where the agent fires dozens of queries to answer one question, the search line can run past the inference line. That inversion gets worse as tokens get cheaper, not better.

The other place the rule breaks is writing skills. A lightweight model asked to author its own procedure does it badly, and the failure is subtle. It writes instructions a smarter model could infer its way through. So our skill-authoring tool runs on a bigger model than the runtime it writes for, and the skill’s own instructions spell the asymmetry out to whatever is drafting. What you write will be executed by a lower-capability model than the one drafting it now. So be pedantic. Name the exact tool and its arguments, and do not leave a judgment call implicit. Vague instructions are instructions a smaller model will fumble.

That is the boundary in general. Reach up a tier where the output is a plan that gets reused, and stay down where the output is one decision that gets made and thrown away.

The Part I’m Most Excited About

The strategic argument for running this low is that it eventually stops depending on the labs at all. At this size there is a real path to fine-tuned open weights, and past that you are substantially immune to price hikes and can buy your own hardware.

I believe that, and I have not proved it. The abstraction is there: Pepper’s model layer is provider-neutral and already routes three vendors through one interface, so a fourth backend is a translation adapter and a route key rather than a rewrite. What is not there is a single line of code pointing at a local runtime.

When I did measure it, the answer was no, and I published that too. The local gemma4:e4b running on my own hardware clears the easy tier at 100% and then collapses: 15% on T2, 7% on T3, 11% on T4. Flash Lite stays above 65% on every tier. It almost always finishes without erroring. It just gets the answer wrong.

That is one very small model, well under what a single consumer GPU could hold, and I have not put the larger variants through the same gradient. With capable dense models now shipping in the 30-billion-parameter range, that gradient is the next thing I want to run.

A Different Company, Not a Cheaper Bill

Cheap orchestration is what makes a five-dollar subscription possible. A five-dollar subscription is what makes Pepper reachable by the people I had in mind when I wrote that not everyone wants a hobby. That is the whole thesis, and the model tier is critical to it in a way a line item on an infrastructure bill never is.

Chris’s read on the field is that most people building here run on higher-grade models, which is a large part of why they need venture money. They are burning tokens on work that does not require them. I don’t think anyone made that choice on purpose. The frontier is where the demos are, where the benchmarks are, and where the attention is. It takes a deliberate act to go shopping somewhere else.

There is an argument running right now about whether we are heading for superintelligence and what it will mean when we get there. It is a real argument and I am not dismissing it. It also has almost nothing to do with the software most people will actually use, which needs to read a calendar, check the weather, and then get out of the way. That job was solved a while ago. We just kept pricing it as though it were not.

Whoever it was I was half listening to had it right, and did not go far enough. Nobody needs a Ferrari to get to work. What nobody says out loud is that the Corolla gets faster every year while we are all reading the reviews of the Ferrari.

A mostly empty library card catalog with a few cards left in one open drawer, lit by morning light.

The Backlog Was a Coping Mechanism

Allen HutchisonLeave a comment

We gave our coding agents a rule that sounded obviously correct. No follow-up left behind. It went into our agent context files in roughly those words: anything a pull request doesn’t fully address gets filed as an issue before the review closes. No deferred cleanup sitting in a comment nobody reads again, no “we should probably handle this properly at some point” evaporating the moment the branch merges. Every loose thread becomes a tracked one, and the agents were good at following it.

The rule worked exactly as designed. That was the problem. Take the last month, which is the first real stretch where I’ve been full time on the product. In those 30 days the two of us merged 657 pull requests on our main repository, close to 30 on a typical weekday. Every one of them came out the other end of review with its own small pile of follow-ups. Honest ones, well written, entirely legitimate.

Over the same 30 days we opened 449 issues. Fifteen a day, most of them filed by our own agents rather than typed by us. Nothing in the system closed the loop. The list only grew, and it grew faster the more time I gave it.

This is the first post in a series about how the two of us actually build Vycari, which I promised when I wrote about why we started the company. I want to start here, with the issue tracker, because it’s the piece of the toolchain I had thought about the least and had to change the most. Almost everything I believed about what belongs in a bug database turned out to be a belief about human limitations rather than about software.

What We Were Really Doing When We Filed a Bug

Think about why you file an issue. You have an idea for something the project should do, and you write it down so it doesn’t fall out of your head. You’re deep in one file and you spot a bug in another, and you write it down so you don’t lose your place. You’re cleaning up a module and you notice three more that need the same treatment, so you write those down too.

The common element in all of it is that you couldn’t do the thing right then. Filing was the second best option. The tracker was a memory prosthesis, a place to park work your hands were too busy or too few to pick up, and it was built for exactly that. Bugzilla went public on a mozilla.org server in 1998, replacing Netscape’s in-house system. The shape it established is still the one we use. A durable record, an owner, a state, a conversation attached to it. Durable because the gap between noticing and fixing was measured in weeks.

The failure mode is just as old. Avery Pennarun makes the point plainly in his treatise on bug tracking and triage. Bugs pile up when filing outpaces fixing. Declaring bug bankruptcy doesn’t change the slope of that line. Every engineer I know has worked in a tracker where the honest read on most open issues was that nobody would ever look at them again. We tolerated it because the alternative was forgetting, and forgetting felt worse.

The second thing we used trackers for was breaking work apart. The vocabulary comes from the agile world. Mike Cohn defines an epic as, simply, a large user story, one you decompose into smaller stories when a team is ready to work on it. Big thing arrives, big thing gets chopped into two-week pieces, pieces get distributed across people, tracker becomes the coordination surface. That’s project management wearing an issue tracker as a costume, and for humans it works well.

Both of those jobs assumed constraints that agents removed. We spent the last few weeks stress testing that, finding out which parts of the practice were load bearing, and came out the other side with a different idea of what a tracker is for.

The Follow-Up That Doesn’t Need to Wait

Start with the follow-ups, because that’s where the pain showed up first.

Our code reviews are agentic. We run Greptile on every pull request and I’ve been genuinely happy with what it catches. An agent reads the diff against the project’s own conventions and writes up what it finds. The good ones find real things. A missing test case, an error path that swallows a failure, a helper that should have been extracted. Under the old rule, each of those became an issue. Faithful record keeping, and a queue that grew every single day.

What we’re experimenting with now is skipping the record entirely. When the review turns up a follow-up, the reviewing agent forks its own context into a subagent and hands it the problem immediately. The fork inherits the conversation that produced the finding, so it already knows the file, the convention that was violated, and why the reviewer cared. It goes off and opens its own pull request while the original review continues uninterrupted. If the pull request that spawned it hasn’t merged yet, the fork stacks its work on top of that branch.

The economics are almost embarrassing. Writing a good issue means describing context that is already loaded in the window right in front of you. Then, later, some other agent pays to rebuild that context from scratch by re-reading the same files and re-deriving the same reasoning, assuming anyone ever picks the issue up at all. Forking the context costs a fraction of that and produces a pull request instead of a promise. It reuses the cached context from the original thread, which saves real money and skips the whole bootstrapping conversation. The cache is already warm. The reasoning is already done.

I want to be careful not to oversell it. This doesn’t work for everything. Some findings send me off to do research first. Some need me to sit with the consequences, or go take a walk and figure out how I actually want to handle the situation. Some need a conversation between me and Chris about whether a feature is the right fit for the product at all. Those still get filed, and I’ll come back to what that queue looks like now. But the default flipped. Filing an issue used to be what you did with a follow-up. Now it’s what you do when the follow-up needs a person to think first.

Epics Assume a Constraint That Left

The second habit took longer to give up, because breaking work into pieces feels like professionalism itself.

An agent can produce a five thousand line pull request in ten minutes. When that’s true, the reason for chopping a large feature into eleven tracked sub-issues mostly disappears. You weren’t decomposing because the work was inherently separable. You were decomposing because a person can only hold so much in their head at once, because a sprint is two weeks long, and because four people needed to work in parallel without colliding. Those are real constraints. None of them are the agent’s.

This is not an argument against testing, or against reviewing carefully, or against building a large feature in stages you can actually validate. Every one of those matters more now, not less. Simon Willison put it well recently. Using agents well takes two skills, instructing them clearly and verifying that what came back is right. He also points out that eyeballing every line was never the best way to do the second one. The chunking that survives is the chunking that helps you verify. The chunking that dies is the chunking that existed to schedule humans.

The testing instincts I picked up early in my career are paying dividends now more than ever. I can load our project into a browser inside the coding agent and start poking at it. When something behaves wrong, the agent is sitting right there with the context already loaded, so I can just ask about it. I can drive the thing from the terminal and then go verify what actually happened in my local database or against my local copy of the service. None of that is new. It’s the same work I’ve been doing for my whole career, and agents didn’t change it.

So we stopped using issues as project management. The agent doesn’t need a burndown chart, and neither do the two of us. We did try. We briefly put the work into a GitHub project built on our issues, kanban columns and all. It fell out of date faster than either of us was willing to maintain it.

Aspirational Issues Are a Trap

The third category is the one I’d defend the longest and was most wrong about. “Someday we should support real-time collaboration.” “It’d be nice if this thing had a plugin system.”

An agent cannot reliably act on an issue like that. It doesn’t have the product context, the customer conversations, or the constraints that would make one implementation right and another one a waste of a week. What it has is enough capability to build something confidently, quickly, and wrong. Aspirational issues are the ideal input for producing exactly that.

Worse, they poison the queue around them. If a tracker is a work list something else pulls from overnight, every unbuildable item in it is a small trap laid for a system that can’t tell the difference. Those ideas are still worth keeping. They just belong somewhere that isn’t the tracker: a document, a spreadsheet, a future-features markdown file checked into the repo. Somewhere a person browses on purpose, rather than somewhere an agent shops.

What Actually Belongs in the Tracker Now

Here’s the test we use. Could a cheap, fast model with the repository’s context in hand fix this tonight, correctly, without asking me anything?

I mean cheap literally. Sonnet 5, Gemini Flash, GPT Terra. If the issue is well defined and the project’s conventions are written down where an agent can read them, the frontier model isn’t buying you much. That’s a strong forcing function on how you write the issue. It’s also the same forcing function that made issues good for humans.

Three kinds of work pass the test. Bugs I found while using the product away from my desk, which I can describe well because I just watched it happen. Most of those reach the tracker through a feedback path rather than through my hands, and the nightly reflection that turns them into issues is a post of its own. Follow-ups from review that resisted being forked. Small features that are well scoped, well understood, and simply not a priority this week. That’s it. Everything in the tracker is something a low-cost agent could pick up and drive to a pull request, prioritized so it knows which one to pick.

We have a pipeline that does exactly that. It doesn’t wait for night. It runs continuously, picking up whatever is ready whenever it becomes ready. That’s the next post in this series, and it’s the reason the discipline in this one matters so much. A pipeline that works through an unfiltered backlog is a machine for generating plausible garbage at scale.

I’m also experimenting with a task that sweeps every issue opened during the week and spawns an agent to fix the ones that qualify. That habit is new enough that I don’t know yet whether the tracker ever actually reaches empty.

Where the Big Work Goes Instead

None of this means we stopped doing big things. It means the big things stopped living in the tracker.

For a substantial feature, I work with an agent to write a product document first. That might be a short document, a mock, or a piece of a design system, whatever makes the idea concrete enough to argue about. Then Chris and I talk it through until we agree it’s actually right for the product. Only then does it become the input to a technical design document, which gets reviewed the way any other design gets reviewed and then checked into the repository alongside the code. After the design doc lands we create an issue, and the issue says one thing. Implement this design.

That issue is not food for the overnight pipeline, and it’s labeled so the pipeline leaves it alone. It’s a marker. It exists so that a person sitting down with an agent has a place to start, and so both of us can see what’s planned but not yet built. When someone picks it up, the first move is to point the agent at the design doc for context, and then build the feature end to end. Not in phases the issue dictated in advance, because an issue written before the work started is a bad predictor of how the work will actually decompose. In reasonably sized chunks of working functionality, with a lot of manual testing along the way, laddering up to the whole thing. That doesn’t mean one enormous pull request either. I’m about fifteen pull requests into a feature right now, several days of work, all of it pointed at a single issue.

The design document does the job the epic used to do, and does it better. It holds the reasoning, not just the task list, which is what an agent needs to build something faithfully. And it’s version controlled next to the code it describes, so it goes stale visibly instead of quietly. We have an agent that runs nightly to compare the claims in those documents against the code and update the ones that have drifted, which is a story for another day.

Taking Inventory

Last week I stopped adding and took inventory instead. 300 open issues on that same repository, every one of them read against the actual code rather than against its own title, then sorted by what it would really take to close.

15 were already dead. Fixed by some other pull request, obsoleted by a decision we’d since made, duplicated, or never actionable in the first place. 54 were real work an agent could simply do, so I let it. The sweep spawned subagents in waves and merged 54 pull requests over the following days. That’s 69 issues, most of a quarter of the backlog, that a machine could resolve without me in the room.

The rest is the part that stuck with me. 101 needed a decision from me. 54 needed one from Chris. 67 were epics or deliberately parked, and I left them alone on purpose. Three quarters of what we had filed wasn’t work at all. It was a queue of decisions wearing work’s clothing, and it had been sitting there looking like a backlog the whole time.

That number moved me further than any argument in this post did.

Three Jobs Instead of One

What we’ve landed on is a tracker with three purposes. It holds things we found and can’t fix right now but an agent can fix later. It holds pointers to planned work whose real content lives in a design document. It holds the follow-ups from review that weren’t worth forking a subagent to solve on the spot.

Look at what fell out. The tracker stopped being a filing cabinet for human memory. Everything in it is now addressed to a machine that will read it, act on it, and open a pull request. That makes an issue much closer to a function call than to a note to self. I’ve argued before that prompts are code and deserve the same rigor. An issue an agent will execute is the same claim, arriving from a direction I didn’t expect.

The part I’m still sitting with is how much of my old practice was compensation. Filing a bug so I wouldn’t forget it. Breaking an epic into pieces so a team could carry them. Keeping a wishlist because writing it down felt like progress. Those were all coping mechanisms for being a human with one pair of hands and a bad memory. I still have both of those things. I just have fewer reasons to build my tools around them.

Next time, the pipeline that works through this queue on its own, and what happens when it wanders into a public repository and starts talking to strangers.

An empty, glowing chat window floating apart from a cluster of worn analog interface objects like a door handle, light switch, and rotary phone, rendered as a warm painterly illustration.

The Chat Box Is a Detour

Allen Hutchison2 Comments

When ChatGPT launched in November 2022, we got a simple chat window, following a few months behind Google’s own preview of LaMDA 2 through AI Test Kitchen at Google I/O 2022, itself a chat window layered over a model. Type something in, get something back. That was the right call, and not because anyone had done deep interface research. A transformer model is trained to do one thing: predict the next token given everything before it. Feed it a prompt, get a completion, feed it more, get another. A chat window is close to the most direct expression of that loop made visible. It was an honest match for what those models were good at, an admission of what the technology could do yet, not a considered design choice.

Models have gotten more capable and efficient since, and what we ask of them has gotten far more complex. Somewhere in that gap, the chat window stopped being an honest match and became a habit. We have spent the years since watching agentic AI show up everywhere: code editors, browsers, the operating system itself, the enterprise tools people already had open. Through all of it, we kept handing people the same box. Our reluctance to move past it is a real drag on adoption, and it is one of the bedrocks of our point of view at Vycari.

Anyone who works in this technology every day already knows a specific version of this problem: the blank box itself is hard. Staring at an empty chat window and guessing how to phrase what you want, what the agent can do, which words trigger the behavior you’re after, is its own skill. It is hard for people who spend all day in this field. It is much harder for someone who does not, who has no reason to have built a mental model of what an agent can do from looking at an empty rectangle.

My co-founder Chris Perry made a version of this same argument last week, coming at it from adoption and product history rather than the interface argument here. Ours is a two person company, and we did not coordinate this; we just keep running into the same wall from different angles. It is part of why I like building with him: he thinks about a problem from the side I don’t, and by the time we compare notes we have usually converged without meaning to. His post is worth reading alongside this one.

Agents outgrew the box

Last January I wrote about how everything becomes an agent; the argument underneath was really about decision making. An agent earns its keep when the input is hard to predict ahead of time, when the alternative is branch logic encoding what a developer guessed the user would need, written months before that user showed up. Treat an agent as a piece of your program that makes the call in the moment, with the actual input in front of it, and it looks like a much bigger part of how software gets built than a chat window would suggest.

None of that requires typing into a box. An agent deciding which tool to call inside a checkout flow, a support queue, or a scheduling tool does not need a text field to do its job well. In a lot of what we build at Vycari, the agent takes a backseat to the interface on purpose. Nothing beats a solid interface that is familiar and delightful to use, and an empty chat box waiting to be filled is neither of those things for most people.

Chat is the hobbyist’s interface

I wrote a few weeks ago about why not everyone wants a hobby, and this is where that argument meets this one. A chat interface rewards people who already know how to prompt, who enjoy the back and forth, who get a small thrill from coaxing a good answer out of a model. That is real and worthwhile. It is also a hobby, and most people do not want one. They want the thing handled.

This is not a new problem for our industry. Unix was a command line. So was DOS, VMS, Ultrix, and SunOS, the whole family of operating systems that ran the serious computing of their era. Knowing the right incantation, the right flags, the right order of operations was a hobby in its own right, and it kept computers in the hands of people willing to learn that hobby. The graphical interface Xerox PARC prototyped, and that the Macintosh and then Windows brought to everyone else’s desk, did not make computers more powerful. In most ways it made them less flexible than a shell. What it did was let someone finish a task without first learning a syntax and a vocabulary of magic commands, as Chris put it. Nobody wants to read a manual to cross the next thing off their list. That was true of computing in 1984, and it is true of agents now.

Look at the products getting the most attention right now, open source projects like OpenClaw and Hermes, or the frontier lab flagships like Gemini Spark, Claude Cowork, and ChatGPT Work. Nearly all of them are built on the same premise: you will chat with the agent. My own Gemini Scribe started the same way, a chat window bolted onto Obsidian. Sometimes the chat happens over voice instead of text, but underneath the voice layer is the same box, just one you talk to instead of type into.

Meeting people where they already are

I do not think most people want to learn a new modality to get value out of AI, and I do not think they will, even though some of us genuinely enjoy it. Agentic AI is a shift on the scale of the ones we have already lived through: personal computers, the internet, the web, mobile, and now AI. I made a version of this list once before, and each of those transitions asked people to change how they worked. The ones that won met people with an interface they already understood, not the ones that made them learn a new vocabulary first. I want to be careful here: I am not arguing for skeuomorphism, for software that pretends to be the paper or the switch it replaced. I am arguing for good, intuitive design, interfaces that borrow the mental model people already carry and build the new capability on top of it. Every time this industry has moved toward a more familiar interface and a design language people already recognized, it created more value, not less, because it stopped spending people’s attention on the tool and let them spend it on the task.

That is the standard I want agent products held to. Not “does the model perform,” because at this point almost all of them do. The question is whether the software finds you where you already are, understands what you are actually trying to accomplish, and gets you back to your day.

Did you have to come back

Early in my career at Google, one of the metrics we watched for search was whether you came back. If you searched, clicked a result, and never returned, that meant you had found what you needed and moved on. I think agents deserve the same measurement. Did you get the thing done? Did you have to come back and try again? The best outcome is that you do not come back, not because the product failed you, but because you already had what you wanted and were on to the next thing.

In our own applications at Vycari, that translates into something concrete. It means the agent understood the request on the first try, without a round of clarifying questions, without asking you to drop into a chat window to add detail it should already have had. Better still, it means the agent acted on your behalf and you never had to watch it happen. In my ideal world, people using what we build will know this is agentic AI about as much as they know it is written in Python: not at all, not because we are hiding it, but because it will not be a fact that matters to them.

We should not be building toward the box. We should be building toward wherever the user already is, and toward what they are actually trying to get done. Agents are not a hobby. They are here to help people get through their day, and the sooner our interfaces act like it, the sooner more people will actually use them.

A laptop on a dark desk with thousands of small glowing blue points of light rising from the screen and spreading outward like a constellation, connected by faint threads, with a small bright cluster near the screen and the rest fading into darkness.

Thousands of Goodbyes

Allen Hutchison1 Comment

There is a problem with leaving a job you held for twenty one years, and nobody warns you about it. You cannot remember who you worked with.

Not in the sense of having forgotten them. In the sense that the list does not exist anywhere, including in your head. I left Google earlier this month after twenty one and a half years across Search, Maps, ads, mobile, Core, Core ML, and DeepMind, in roles that ranged from running large organizations to writing code with three people. Somewhere in there I met, argued with, mentored, was mentored by, shipped alongside, and occasionally annoyed several thousand people. When I sat down to write my goodbye note, I could produce maybe a hundred and fifty names from memory, and I knew with total certainty that the hundred and fifty were wrong. They were the recent ones and the loud ones. The person I had one extraordinary conversation with in a London conference room in 2008 was nowhere near that list.

This is the sort of problem that used to have exactly one answer, which is that you accept it. There is no roster of everyone who mattered to you, no internal system that will hand you the list. What you actually do is type out the names you can think of, send the note, and add a line asking people to please forward it to anyone you missed. It works, in the sense that something gets sent. It is also a sloppy solution that quietly hands the entire decision to your recency bias. The people you saw last month get a goodbye. The people who changed how you work, fifteen years ago, on a team that no longer exists, do not.

I did not want to accept it. So I did the other thing.

Letting the machine do the remembering

The tool I reached for was Antigravity, paired with the Google Workspace MCP server I had originally built as a Gemini CLI extension. That extension exists because I wanted my terminal to be able to see my calendar. It turns out that giving an agent access to your mail and calendar for ordinary convenience also gives it access to the complete archaeological record of your working life, which is not something I had thought carefully about until I needed it.

The first pass was mail. I had the agent go through twenty one years of sent messages and pull out every address I had ever written to individually. Not mailing lists, not the thirty person threads, not the announcements. One to one mail, or close enough to it, on the theory that if I had bothered to type your name into the To field, we had a relationship worth closing properly. That produced a very large and very stale list, because a lot of those people left Google years ago. So the second pass compared the addresses against an internal directory and kept only the ones still active, which cut it down considerably and also produced a small, unexpected inventory of people I had lost track of.

The second source was calendar. Mail catches the people you write to, but it misses the ones you only ever talked to in a room. So the agent went through twenty one years of my calendar looking for anything that was genuinely a one to one: a recurring weekly, a single mentoring session someone booked in 2017, a skip level, a coffee that got a calendar entry because that is how Google works. Every one of those has a person on the other side of it, and a surprising number of them never once appeared in my sent mail.

Then it merged the two. I had expected this to be the hard part and it was not, for a boring reason: Google email addresses are stable. People change teams constantly and the address follows them, so deduping on the address alone does nearly all of the work. What I was not prepared for was reading the result. Whole eras came back. I would hit a cluster of addresses and remember an entire project I had not thought about in a decade.

The last problem was purely mechanical and almost funny after all that. Gmail will not let you put several thousand addresses on a BCC line. The agent did not raise this, which is worth saying in a post that is otherwise about how capable these things are. I had to notice it and ask. Once I did, it split the list into batches that fit under the limit and drafted the same message across all of them, and I sat there and hit send, over and over, until the list was gone.

What came back

I expected silence. You send a goodbye to someone you shared one project with in 2011 and you expect it to land in an archive folder, if it lands anywhere.

One person wrote back to tell me, politely, that he had no idea who I was. I liked that reply enormously. I assume he was not the only one who felt that way and just the only one honest enough to say it, and I would rather have sent that mail and been forgotten than not sent it at all.

But most of what came back was not that, and the volume of it caught me completely off guard. A solutions engineer in Japan wrote about a Street View collaboration with Nintendo that had been his very first project at Google, and told me I was probably the first real engineer he ever encountered. We were born the same year. He had turned fifty a month earlier and was finding the number about as strange as I am. Someone I managed for only a few months thanked me for trusting him and called it one of the most engaging windows of his career. A former colleague opened by saying he had just been telling someone we both used to work with how much he missed working with me, and then reported that he still has not gotten his motorcycle license because his friends staged an intervention dinner about it. A director in Munich offered a beer if I ever make it back and reminded me of work we did together that I had honestly stopped thinking about. Someone I have known for close to twenty years said he had been meaning to call, and instead of trading contact details we ended up at lunch off campus the following Friday, which is the best possible version of what one of these notes can do.

That is the part I keep turning over. Not one of those replies exists if I send the note from memory.

A hundred and fifty, the number of names I could produce on my own, is roughly Dunbar’s number, the size of a stable social group a human can actually maintain. I do not think that is a coincidence, and I have stopped treating it as a personal failing. My recall of my own career landed almost exactly where the research says it should, which means the ceiling was never effort or affection. It was architecture. Twenty one years builds a network many times larger than the one a head is designed to hold, and the overflow is not forgotten so much as never retrievable in the first place. The agent was not being clever here. It simply was not subject to the constraint.

The people who wrote the most moving things back to me were, almost without exception, the ones my brain had not surfaced. They were on the list because software read my calendar from 2013 and did not care that it was 2013.

The gap this points at

I have spent two years writing about what agents can do, and most of my examples have been about work. Code, pipelines, research, the machinery of building software. This is the first time I have used one for something that was purely about people, and it is the only agentic project I have run where the output made me feel something.

It is also, and I want to be honest about this, a completely unreasonable thing to ask a normal person to do. To get here I needed an agent platform, an MCP server I wrote myself, OAuth credentials, a working mental model of what a tool call is, and enough patience to iterate on the approach a few times when it came back wrong. The capability was extraordinary. The cost of access was absurd.

That gap is the whole reason I am doing what I am doing next. I wrote last week that using agents today is effectively a hobby, and this is the cleanest example I have of why that matters. Everyone leaving a long job has this problem. Almost nobody gets to solve it, because solving it currently requires you to invest in the hobby first. Closing that distance, so that an ordinary person on an ordinary Tuesday can ask for something like this and simply get it, is the work.

Twenty one years of colleagues got a goodbye from me because software remembered what I could not. I would like that to be an unremarkable thing to say.

And to everyone who wrote back: I replied to every thread I could, and I am sure I still missed some. I want all of you to know how deeply your words and your wishes landed. Thank you for that.

An overhead illustration of a cluttered hobbyist workbench with soldering iron, parts drawers and a notebook of diagrams, and a single plain phone resting apart from it all at the edge of the bench.

Not Everyone Wants a Hobby

Allen Hutchison4 Comments

Yesterday I wrote about leaving Google after twenty one years and said there was a next chapter I would tell you about today. This is it.

Chris Perry and I are starting a company. It is called Vycari, we are building agents, and we are looking for people to test what we make. I want to spend most of this post on the problem rather than the product, because the problem is the interesting part and because the product is not ready for you yet.

Using agents today is a skill

Here is what it currently takes to get real value out of an agent.

You curate your skills, your triggers, and your extensions. You pick the right model for the task and make sure it is wired to the right API key. You keep the right software installed and updated on your machine. You have a sandbox configured, because obviously you have a sandbox configured. You learn which failures are the model’s fault and which are yours. You read changelogs, because the thing you learned last month is already wrong.

That is not using a tool. That is adopting a hobby.

I want to be careful here, because I do not mean that as a complaint. There is nothing wrong with AI as a hobby. A great many of the real sea changes in personal computing were driven by people who treated the work as an avocation first and a profession second, and the current moment is no different. I count myself among those people. They are my people. This blog exists because of them, and I am not going to stop writing for them.

But it is worth being honest that this is what we have built so far, and about who it excludes.

Most people do not want a hobby

Not everyone wants their tool to become a pastime. Not everyone wants to marvel at the stack, or to feel the small thrill of watching software take an action in the real world on their behalf. Plenty of capable, curious, technically fluent people simply want to get something done, and they want to use whatever makes that easiest.

I am friends with a lot of these people. They are not incurious and they are not afraid of technology. They carry a supercomputer in their pocket and use it fluently all day. They have just never been given a reason to believe that an agent is for them, because every agent they have encountered asked them to become a hobbyist first.

Chat apps are the exception that proves the point. They reached enormous audiences because there was nothing to adopt: you type, it answers, and the entire interface is a thing you already knew how to use. Step outside that box and agents are still close to magic for most people, and magic is not a compliment when you are trying to get through a Tuesday. Agent products remain genuinely hard to use, and the difficulty has very little to do with how good the models have become.

What I keep coming back to

The most useful thing I learned in two years of building agents has almost nothing to do with models. It is this: people do not bounce off AI because it is not smart enough. They bounce off because the cost of using it, all of it, the setup and the vocabulary and the remembering to go there, is higher than the problem they were trying to solve.

Which means the frontier I find interesting is not making these systems more capable. They are already more capable than almost anyone is extracting value from. The frontier is closing the enormous distance between what the technology can do and what an ordinary person can actually get out of it on a normal day, without a new app, a new habit, or a new hobby.

That gap is not a model problem. It is a product problem, a reliability problem, and a taste problem. It is also, as far as I can tell, wide open.

What we believe

We are early enough that I would rather tell you what we intend than what we have built. Four things we are holding ourselves to.

Meet people where they already are. If using the thing requires a new destination in someone’s day, we have already lost, no matter how good it is once they arrive.

Earn the trust the access requires. An agent worth having needs to see calendars, mail, and contacts, which is about as intimate as software access gets. There is no version of this business where that access to private data becomes an advertising product, and there is no version where you cannot take your data and leave.

Never make you learn our vocabulary. Nobody should have to know what a skill is, or which model answered, or that any of this is AI at all. Those are our problems. The user’s problem is that they asked for something and want it handled.

Be warm about it. An assistant you find pleasant is one you will actually use, and an assistant you actually use is the only kind that matters.

Who we are

My co-founder is Chris Perry. He is the CEO. I am the CTO. Chris used to report to me, and putting him in the CEO seat was one of the easier decisions either of us has made.

We have been circling each other for about a decade. We met when I was running Street View and imagery inside Maps and he was a PM on Google Photos, back when our two teams were trying to make those products understand each other’s pictures. Years later he turned up in the AI Developer organization I was running, as the product lead for Colab. Most recently we were both on the founding team of Gemini CLI and shipped its Workspace extension together. Somewhere in there he stopped being someone I had worked with and became someone I wanted to build with. He is writing his own version of this announcement, and his path here is different enough from mine that you should read both.

It is just the two of us right now, and we are both in the codebase. One of the underrated pleasures of leaving a large company is that you get to assemble the org chart from scratch, based on who is best at the job rather than on who has been there longest.

The part where I ask you for something

Twenty one years at Google taught me how to build systems where the hardest problems are problems of coordination. This is a different kind of hard. There are two of us, the feedback loop is measured in hours, and the person on the other end of a failure is someone who trusted us with something that mattered to them. I have not been this uncomfortable in a long time, and I have not enjoyed work this much since 2008.

We are opening to testers in the coming weeks, a few at a time and deliberately, because the failure I am most afraid of is someone relying on this and being let down. If you would like to be in that first group, put your name down here. What we want back is your honest experience, particularly the parts where it does not work.

And if the person who came to mind while you were reading this was not you but someone else, someone who would get enormous value from an assistant and would never in their life go looking for one, then you have understood exactly who we are building for.

I will keep writing about the engineering here as we go. It is going to be a good year.

Brass calipers measuring a glowing wireframe sphere floating above a dark wooden workbench scattered with paper task sheets.

How I Built a Scoreboard for My Own Agent

Allen HutchisonLeave a comment

The bug fix took an afternoon. The follow-up question took a week.

I was deep in Gemini Scribe, my Obsidian plugin that drops a Gemini-powered agent into your vault, and I had just shipped a change to the way the agent picked its tools. It felt better. The few sessions I ran by hand showed cleaner reasoning, fewer wasted tool calls, less of the weird “let me search for that again with slightly different keywords” tic. I committed, pushed, and moved on.

Then a friend asked, casually, “how much better?”

I had no answer. None I trusted, anyway. I had vibes. I had a handful of session transcripts I could squint at. I had the comforting belief that change is progress, which is the most dangerous belief you can hold when you are building with non-deterministic systems.

When I wrote about the observability gap earlier this year, I argued that you cannot fix what you cannot see. Observability lets you watch a single agent run unfold. But it does not tell you whether the next run will be better than this one. For that, you need a different instrument. You need a scoreboard.

So I built one. This is the story of what it took to make it credible, and what it told me when it finally was.

Two Reasons This Suddenly Mattered

The friend’s question was the trigger, but it was not the only reason I needed an answer. Two larger pressures had been building for a month.

The first was Ollama. In version 4.8, shipped a month ago, I added a local-model provider to Gemini Scribe. The plugin can now drive the agent against a model running on your own hardware, with no API key and no per-token cost. I wanted that, and so did a lot of users. But the moment I shipped it I had a question I could not duck. Are the local models actually good enough to use? Should I tell people to switch to them, or should I quietly warn them that the experience drops off a cliff once the cloud connection goes away?

The second was pricing. Google recently raised the price of Gemini 3.5 Flash, the newest model in the Flash family, to nearly the level of Gemini Pro (the full pricing table tells the story). For almost a year I had been recommending Gemini 2.5 Flash as the default model for Gemini Scribe, and the obvious upgrade path (move up to 3.5 Flash with the next release) suddenly looked expensive. The alternative was to switch families entirely and make the newest Flash Lite model the default, but only if it was actually capable enough to drive the agent on real work.

Both questions had the same shape. “Is model X good enough to be the default for Gemini Scribe?” Before building anything, I went looking for an existing benchmark to adopt. I commissioned two separate deep-research passes specifically to find one I could lift wholesale. Both came back with the same answer.

The public eval suites measure code generation (HumanEval, SWE-bench), general assistant tool use over the web (GAIA), and customer-service-style tool flows (τ-bench). None of them measure what I actually care about, which is an agent operating inside a markdown wiki. Opening notes by name. Following wikilinks across files. Editing frontmatter without nuking sibling notes. Aggregating across many notes and refusing prompt-injection bait sitting in a note body. If a benchmark for this exists, neither I nor two passes of automated research could find it.

So I had to build it.

Why Unit Tests Do Not Work

The instinct, if you have spent any time writing software, is to reach for unit tests. The agent took an input, it produced an output, check the output. Pass or fail. Run on every commit. We have been doing this for decades.

I am not arguing against unit tests in the abstract. The Gemini Scribe repo has nearly three thousand of them, and I just finished a multi-week push to get line coverage above ninety percent. They are the foundation that lets me move quickly on everything below the agent loop: parsers, settings migration, frontmatter handling, the diff view, the provider adapters, the tool definitions. Without that scaffold I would be afraid to refactor anything, and most of the bugs that would otherwise reach the agent never get the chance.

The other thing I had been leaning on was daily use. I run Gemini Scribe in my own vault every day, on real work, which catches the egregious failures fast. The agent crashes, the agent produces obvious garbage, the agent loops; I notice within a session. What dogfooding does not catch is the distribution. Did this change make the agent worse at one task in twenty in a way I will never directly observe because I do not run that task on a typical Tuesday? My sample size is one, and I had been quietly grading my own work for months.

So the instinct is wrong for the agent loop itself, and the reason is the same one that makes agents interesting in the first place. They do not do the same thing twice. Ask the agent to find a file by name and on one run it will call find_files_by_name once, return the answer in a single turn, and cost you a fraction of a cent. On the next run, against the same prompt, the same vault, the same model, it might call search_content first, then find_files_by_name, then re-search with a slightly different query. Same answer. Twice the cost. Three times the latency. Both runs “pass” a unit test. Both runs are real.

The problem is not that the agent is broken. The problem is that “did it work” is the wrong question. The right question is “how reliably does it work, on what kinds of problems, and at what cost?”

That question cannot be answered by a single run. So the scoreboard has to be built around the inconvenient truth that you have to run everything more than once.

Borrowing pass^k From τ-bench

I did not invent the trick that makes this tractable. I borrowed it from the τ-bench paper linked above, which proposed a metric called pass^k. A task passes at k only if all k runs pass. Not the average. Not the best. All of them.

The math is brutal in a useful way. A model that solves a task 80% of the time on a single run will hit pass^5 of about 33% on that same task. The metric punishes flakiness, which matters in the real world because users do not care about your average run. They care about whether the agent will do the thing they asked for the one time they asked. pass^k is what reliability looks like as a number.

For my harness, I picked k=5 for anything I planned to publish or block a merge on, k=3 for day-to-day development. Every task runs the full count, every time. The summary breaks out pass^k (no harness errors, no timeouts), solve^k (passed and satisfied the full task rubric), and a mean rate for the curious. Tasks that land between 0 and k solves get flagged as flaky in the output, with a little warning sigil. The flaky list is where bugs live.

Scoring What the Agent Actually Did

The harder problem, the one I spent most of the week on, was figuring out what “satisfied the full task rubric” should mean.

The naive version is to grep the final response for the right answer. That works for a few tasks. It fails the moment the task is anything other than “say a specific phrase.” Ask the agent to delete a file and “I deleted the file” is not evidence that the file is gone. Ask it to edit a note and “Done!” tells you literally nothing about whether the edit was correct, or even whether the right note got touched.

The τ-bench lesson, and the one that took me a while to actually believe, is that you have to compare end state against the goal, not tool-call syntax against an expectation. So my task definitions ended up carrying two kinds of checks. Output matchers score the text the model produced. Vault assertions score the side effects. Did the file exist, did it contain the expected content, did the frontmatter end up with the right value, did the unrelated sibling files stay untouched.

Here is what one of those tasks looks like:

{
  "id": "archive-old-notes",
  "difficulty": "T3",
  "userMessage": "Archive every note in eval-scratch tagged #old.",
  "expectedTools": ["find_tagged_notes", "edit_file"],
  "vaultAssertions": [
    { "type": "frontmatterEquals", "path": "eval-scratch/note-a.md",
      "key": "status", "value": "archived" },
    { "type": "fileUnchanged", "path": "eval-scratch/note-c.md",
      "fixture": "note-c.md" }
  ],
  "toolCallBudget": 6
}

The frontmatterEquals assertion confirms the right notes got archived. The fileUnchanged assertion confirms the agent did not go wandering through sibling files it had no business touching. The toolCallBudget makes efficiency itself a pass criterion, which catches the “I will just read every file in the vault” behavior that a single content search would have answered. Saying the right words is not enough. Doing the right thing is not enough. You also have to do it without burning the kitchen down on your way out.

The Judge Problem

A subset of my tasks are prose-heavy. “Summarize the differences between these three meeting notes” does not have a single correct surface form. The agent might write “the second note disagrees on the deadline” or “note two pushes back on the timing.” Both are right. Neither matches a literal substring assertion without me writing a regex more complicated than the task itself.

For those, I use an LLM-as-judge. A separate Gemini model called with temperature: 0 and a strict YES/NO contract against a rubric I write per task. This works, until you start asking whether the judge itself is any good.

I did not trust the answer for a while, and rightly so. So I built a calibration tool. The harness can extract every judge matcher decision from a full sweep into a flat file of tuples (criterion, agent response, automated verdict). I then sat down with a cup of coffee and hand-labelled ninety of them as YES or NO myself, blind to what the judge had said. That gave me a gold set, a one-time human-labelled reference I can measure any candidate judge against.

When I ran four candidate judge models against that set, the results were uncomfortable. The judge I had been using agreed with my human labels 92.2% of the time. The newest Flash, gemini-3.5-flash, hit 94.4%, with fewer false negatives on cosmetic formatting and one fabrication case that the smaller gemini-3.1-flash-lite missed. I switched judges.

But the more important finding was about the judges themselves. Even at temperature: 0, two fresh runs of the same judge against the same gold set produced the same accuracy number with a different set of disagreeing tuples. The pass/fail flips around. Judge nondeterminism is real. Single-run judge measurements are not to be trusted.

The other thing the calibration exercise gave me, which I did not expect, was a debugging tool. Forcing myself to read every criterion and every response carefully turned up two latent bugs I had been staring through for months. One task had a judge criterion demanding response-side coverage that the prompt never asked for. Three other tasks had fileMatches regexes silently failing because they used JavaScript-incompatible inline flags. The eval harness was not just measuring the agent. It was measuring my evaluation of the agent, and finding it wanting.

What the Scoreboard Said

With the harness real, I ran a sweep across three models on a 54-task suite, at k=5, under the calibrated judge. The headline numbers, which now live on the plugin’s docs site and auto-update on every newly blessed baseline:

The newer gemini-3.1-flash-lite solves 74.1% of tasks at solve^5. The older gemini-2.5-flash, supposedly a tier up, solves 57.4%. The local gemma4:e4b running on my own hardware solves 14.8%. A single full sweep costs about thirty cents per model in steady state.

That per-sweep number is the honest one for ongoing measurement, but I should be clear about what the build phase actually cost. Between the judge-calibration runs, the four candidate-judge measurements against my gold set, the three full re-baselines, and the iteration passes that came with all of it, yesterday alone ran me $8.12 across my Gemini Scribe API key and the dedicated judge key. That is the number to plan around if you are building your own. The thirty cents is what it costs once the scoreboard exists and you are just checking whether your latest change moved the needle.

And those are just the API numbers. The real investment was a week of my time, which is the cost you should weigh hardest. It pays back the moment you want to evaluate any change to the agent loop with confidence instead of vibes, which from here is every release I cut.

That first result answered the pricing question for me cleanly. Within a model family, the tier names mean what they say. Pro is more capable than Flash, Flash is more capable than Flash Lite, and you pay accordingly. The interesting thing is what happens across families and releases. The price-to-capability frontier moves fast enough that the newest model in a cheaper family can dominate an older default from a pricier one. That is what happened here. Gemini 3.1 Flash Lite, the newest Flash Lite, beats Gemini 2.5 Flash by about seventeen percentage points on solve^5 on agentic tasks (multi-step tool use, retrieval, edit-then-verify), and costs less per token than the Gemini 2.5 Flash it replaces. The next release of Gemini Scribe will move the default model from Gemini 2.5 Flash to Gemini 3.1 Flash Lite, which means users get a quality upgrade and a cost cut at the same time. Without the scoreboard I would have stayed loyal to a tier name and spent another six months recommending the more expensive, less capable model.

The Ollama numbers were harder to swallow but just as useful. The local Gemma model is genuinely good at the easy T1 tier (a single tool call against a tiny corpus), hitting 100%, and then it collapses. It drops to about 15% on T2 (two or three tool calls with light distractors), 7% on T3 (multi-step, distractor-heavy), and 11% on T4 (frontier-class hop chains and cross-note aggregation). Flash Lite stays above 65% on every tier. The honest version of the local-model story is that today’s open weights running on a laptop will handle simple lookups (find this file, summarize this note) cheerfully, and will fall over on anything that requires chaining tools or holding a multi-step plan together. That is useful to know. It tells me what to recommend (try local for casual queries, stay on cloud for real work) and it gives me a concrete target to retest against when the next generation of open models lands.

The difficulty breakdown is what makes this kind of comparison possible. A suite where every model passes everything, or where no model passes anything, is not measuring anything useful. The whole point is the gradient. T1 is a regression canary that any model worth running has to clear. T2 through T4 is where open models and frontier models actually separate, and where the suite earns its keep.

The Benchmark Is Open

The harness, the 54-task suite, the judge calibration set, and the methodology docs all live in the obsidian-gemini/evals directory. The README walks through adding a new task in about five minutes, and the existing tasks are organized by category (retrieval, multi-hop, aggregation, conflict, write, edit, negative-space, safety, memory) so a new contribution has a fixture pattern to clone from.

If you are working with agents inside Obsidian or any other markdown wiki, I would love contributions. Especially tasks that exercise corners of the agent I have not thought of. Weird vault layouts. Exotic frontmatter conventions. Prompt-injection payloads you have actually seen in the wild. Multi-step plans that catch the model out. A benchmark is a public good, and it only gets sharper the more people sharpen it. Open an issue or a PR and let’s make this the thing that did not exist when I went looking for it.

What I Would Tell You If You Were Starting

If you are building an agent and you have been operating on vibes, here is the short version of what I would tell you over coffee.

Start with pass^k, not single-run pass rates. The reliability framing is the one that survives contact with production. Run each task at least three times for development, at least five for any decision you are going to publish or block a merge on.

Score the side effects, not the words. The model can say it did the right thing while doing nothing of the sort. State-based assertions on what actually changed in the world are the only honest scoring you can do for tasks that mutate anything.

Make efficiency a pass criterion. A tool-call budget is a one-line addition to a task definition and it catches an entire category of “the agent technically solved it” results that are not actually wins.

If you are using an LLM as judge, calibrate it against human labels at least once, and remember that judge nondeterminism is a real source of measurement noise even at temperature zero.

Treat the scoreboard itself as a debugging tool. The discipline of writing down what “good” looks like, in machine-readable form, surfaces problems with your tasks, your criteria, and your assumptions that no amount of squinting at session transcripts will. The eval harness paid for itself the first time it told me my judge was asking the wrong question, before it ever told me anything useful about the agent.

The vibes were never going to scale. The scoreboard does. The strangest thing about building it has been realizing how much of what I thought I knew about my own agent was wrong, in small but consistent ways, in the direction of being too generous. That is not a moral failing. It is what happens when the system you are measuring does not sit still. You need an instrument. So I built one. Next time someone asks me how much better my change made the agent, I have a number.

A hand-drawn map on a workbench with a half-built mechanical instrument being assembled directly on top of it.

Agents as Building Blocks

Allen Hutchison2 Comments

There’s a thread running through the last year of my writing and my work, and I didn’t fully see it until now.

Last September, I wrote Full Circle, about going back to building after years of leading teams. I wanted to be in the driver’s seat for what I called the agentic shift. I wanted to feel the code under my fingers again, to be close enough to the technology that I could form my own opinions about where it was going.

Then I spent six months drawing the map. The Agentic Shift was twelve essays on what agents are, how they work, and what it means to build them well: anatomy, memory, tools, guardrails, multi-agent coordination, production readiness. It was a theoretical framework, written while I was getting my hands dirty on the Gemini CLI team.

And then, in January, I wrote Everything Becomes an Agent, the practitioner’s version. Not theory anymore. I’d watched Gemini Scribe grow from a chat window into a full agent. I’d seen the CLI team go from talking about code to writing and executing it. I’d noticed a pattern repeating across every AI project I touched: given enough time, they all converged on the same architecture. Tools. Loops. Policies. Judgment.

The Antigravity SDK is the second agent product I’ve worked on at Google. Gemini CLI was the first, and it’s where I learned what an agent runtime actually needs: a policy engine, a tool pipeline, lifecycle hooks, a trust model that scales from “let me approve every file write” to “here are the guardrails, go handle it.” The SDK is the next step. Taking everything I learned building one agent and making it possible for everyone to build their own.

Today we’re launching the Antigravity SDK in Preview. The official announcement covers the features (what the SDK does, how to install it, what you can build). This post is about the why. Why this SDK, why this design, and why it matters to me.

What Is an Agent SDK, Really

Here’s something I find fascinating: people have wildly different ideas about what “agent SDK” means.

For some, it’s a way to automate the coding agent. You take the AI that already lives inside your IDE (Antigravity, Cursor, Copilot), and you script it. Pipe in a task, get back a diff. The SDK is an extension of your development environment. That’s a legitimate philosophy, and there are good products built on it.

But that’s not what I wanted to build.

To me, an agent SDK gives you an agent that you can incorporate into your software. Not an extension of your IDE. A building block. Something you import into your Python project the same way you’d import a database client or an HTTP library, and then you use it to solve a problem. The agent is a component in your system, not a wrapper around your workflow.

I’ve watched this pattern play out across Gemini Scribe, the Podcast RAG prototype, and a dozen smaller projects. Software that starts as a script, grows a tools array and a while loop, and eventually looks an awful lot like an agent. I wouldn’t claim that every AI project becomes an agent. But the pattern is durable for a huge class of software problems. And if that convergence is real, if a meaningful number of AI applications end up needing tools, memory, judgment, and guardrails, then the SDK should make that convergence frictionless.

The key distinction is this: the agents you build with the Antigravity SDK aren’t extensions of your developer tools, although they can do development work. They’re independent pieces of software that happen to be implemented as agents. They live in your codebase, run on their own, and do real work.

Let me show you what I mean.

Three Agents That Prove the Point

Two of my favorite examples ship with the SDK, and we use both of them on the SDK project itself on a regular basis. They live in the examples directory on GitHub.

The first is the docstring maintenance agent. You point it at a directory, and it audits every Python file for missing or incomplete docstrings, then fixes them, all following the Google Python Style Guide. It knows which tools it’s allowed to use (read files, list directories, edit .py files in the target directory, and nothing else). It has a policy engine that enforces those boundaries. It runs, does its job, and exits.

The second is the documentation maintenance agent. Same idea, different problem: it scans your project’s documentation for staleness, checks it against the current state of the code, and updates what needs updating.

Here’s what I love about these two examples. They’re coding-related tasks, but they aren’t extensions of my IDE. They’re standalone programs. I don’t run them inside my editor. I run them from the command line, or from a CI job, or from a cron schedule. They happen to be implemented as agents because an agent is the right abstraction for “read a bunch of files, reason about their quality, and make targeted edits.” If I’d built these as scripts, I would have ended up writing a brittle classifier full of if/else branches to decide what to fix and how. The agent architecture deletes that complexity.

We use both of these on the SDK project itself. The SDK maintains its own documentation with its own agents. There’s a satisfying recursion to that.

But I want to push the point further, because the SDK isn’t just for coding tasks. Here’s a completely different kind of agent, a personal knowledge graph I wrote that connects to my Workspace MCP server and answers questions about my Drive, Docs, Gmail, and Calendar:

import asyncio

from google.antigravity import Agent, LocalAgentConfig, types
from google.antigravity.utils import interactive


async def main():
    workspace_mcp = types.McpStdioServer(
        command="node",
        args=["/Users/adh/src/workspace/workspace-server/dist/index.js"],
    )
    system_instructions = (
        "You are a Personal Knowledge Graph Agent. Your goal is to help the user "
        "navigate and synthesize information from their Google Workspace "
        "(Drive, Docs, Gmail, Calendar). You can search for documents, "
        "read emails, and check calendar events to answer questions "
        "and help the user connect the dots."
    )
    config = LocalAgentConfig(
        system_instructions=system_instructions,
        mcp_servers=[workspace_mcp],
        capabilities=types.CapabilitiesConfig(
            enabled_tools=types.BuiltinTools.read_only(),
        ),
    )
    async with Agent(config) as agent:
        print("Knowledge Graph Agent ready. Ask me anything about your Workspace.")
        await interactive.run_interactive_loop(agent)


if __name__ == "__main__":
    asyncio.run(main())

This agent has nothing to do with coding. It’s a personal productivity tool that connects to my Google Workspace via MCP and lets me query my own data in natural language. It’s about 20 lines. It’s read-only by design. And it uses the same SDK, the same patterns, the same trust model as the docstring agent.

Three examples, three completely different domains: autonomous code maintenance, documentation upkeep, personal knowledge synthesis. All built with the same building blocks. That’s the vision.

Batteries Included, Layers When You Need Them

When designing this SDK, I kept coming back to one principle: batteries included. I wanted it to be really easy to put together an agent that worked for you. Easy to grow your application when you needed more sophistication. Easy to dive into the internals when the situation required it.

Here’s what a functional agent looks like:

import asyncio

from google.antigravity import Agent, LocalAgentConfig


async def main():
    config = LocalAgentConfig()
    async with Agent(config) as agent:
        response = await agent.chat("What files are in the current directory?")
        print(await response.text())


if __name__ == "__main__":
    asyncio.run(main())

That’s it. About 10 lines of real code. That agent can read files, edit code, run shell commands, search directories, all out of the box. You didn’t have to configure tools, set up a model connection, or wire up a conversation loop. The batteries are included.

But batteries included doesn’t mean batteries only. I designed the API in three layers, and knowing which layer to reach for is part of the design.

Layer 1: Agent. The highest level. Create an agent, give it a prompt, get results. This is where most people start, and many people stay. It manages the full lifecycle (connection, conversation, tools, hooks, policies) in a single async with block. If you just need an agent that does a job, this is your entire API surface.

Layer 2: Conversation. This is the implementation layer. Conversations, hooks, policies, MCP servers, custom tools, structured output. Conversation wraps a Connection with step history, turn tracking, and convenience methods. This is where you shape behavior. You add guardrails through the declarative policy engine. You inject lifecycle hooks, and the SDK gives you three distinct types: Inspect hooks for read-only observability, Decide hooks for policy decisions (allow/deny), and Transform hooks that can modify data in flight. You wire up MCP servers and your own Python functions as tools.

Layer 3: Connection. The lowest level. Connection is the abstract interface for talking to an agent backend. ConnectionStrategy knows how to establish one for a specific runtime. Today, we ship a local connection strategy that runs the agent on your machine. On the roadmap: remote connection strategies that let the same agent code deploy to the cloud without a rewrite.

Here’s the neat thing about this layer. Because Connection is an abstraction, you could conceivably wire up other agent runtimes behind it. We do this internally. We have several different ways of talking to our agent harness, and they all work through the same Connection interface. Your agent code doesn’t know or care which one is running underneath.

The philosophy is: easy to start, easy to grow, easy to go deep. You shouldn’t need to understand the Connection layer to write your first agent. But when you need it, when you’re building something that requires custom streaming, session resumption, or a novel deployment target, it’s there, and it’s a clean abstraction, not a hack.

One detail I’m particularly proud of: the trust model adapts to the deployment context. The base AgentConfig is deny-by-default. It defaults to read-only tools, and if you try to enable write tools or MCP servers without a safety policy, the Agent refuses to start. Enforced at the framework level. LocalAgentConfig takes a different posture. Since it runs on your own machine, it enables every tool, scopes file operations to the workspaces you’ve configured, and gates shell commands behind a user confirmation prompt by default. You’re developing locally; you probably want your agent to actually do things, but you also probably want a chance to look before it runs rm -rf. The trust gradient is baked into the architecture.

Lessons Encoded

If you’ve been following along with my writing, the SDK might feel familiar. That’s intentional.

The twelve-part Agentic Shift wasn’t just an intellectual exercise. It was the blueprint. Every essay mapped a concept that eventually became a feature.

In Everything Becomes an Agent, I wrote: “If you’re writing if/else logic to decide what the AI should do, you might be building a classifier that wants to be an agent.” The SDK takes that literally. You don’t build classifiers, you define tools and let the model decide which ones to use. The complexity moves from branching logic to capability definition.

I wrote about building a “sudoers file for AI”, a permission system for agents. That became the policy engine. policy.allow("view_file"). policy.deny("*"). Declarative, composable, deny-by-default. You express what’s allowed, and the framework enforces it.

I wrote: “The real complexity isn’t in the code; it’s in the trust.” That conviction shaped the hook system. Hooks give you visibility into every tool call, before and after. Policies give you control. Together, they manage the trust relationship between you and the agent. The SDK doesn’t ask you to trust blindly; it gives you the instruments to verify.

And I wrote: “A hammer does nothing unless you swing it. But an agent? An agent can work while you sleep.” That’s the promise. The SDK is the handle.

These aren’t abstract design principles that I reverse-engineered to sound good in a blog post. They’re lessons learned from building Gemini Scribe, from contributing to Gemini CLI, from watching every project I touched converge on the same agentic patterns. I drew the map, I lived the map, and then I got to build the territory.

The Team

I want to be clear about something. I didn’t build this alone.

I did most of the design for the Python SDK (the API surface, the three-layer architecture, the philosophy behind “batteries included”), and a lot of that design came from the writing I’ve been doing this past year. But design is the easy part. The hard part is building something real, and that was a team effort.

A talented group of engineers worked with me on this. On the SDK implementation, on the test infrastructure, on the Go harness underneath that actually runs the agent, on the internal connection strategies, on the MCP bridge, on a hundred decisions that don’t show up in a blog post but absolutely show up in the quality of the software. The SDK exists because of their work, and it’s better than anything I could have built on my own.

Preview, and an Invitation

We’re shipping this as a Preview. Not “1.0.” That’s deliberate.

The API surface will change. We know that. We’ll evolve it based on feedback from you and from our own continued use of the SDK, because we use it too, every day, on the project itself. There are things we haven’t figured out yet. There are patterns we haven’t discovered. That’s the point of a preview: to learn in the open.

So here’s the invitation: build something. Build a documentation bot, a knowledge graph, a CI pipeline agent, a personal assistant. Build something I haven’t imagined. Break something. Tell us what’s missing, what’s awkward, what delights you. File an issue. Open a PR. Argue with us about the API.

Last September, I wrote that I was going back to building because “for a builder, there’s no more exciting place to be.” The Agentic Shift was the map. The SDK is the territory.

Come explore it.

The Antigravity SDK is available now as a Preview. Install it with pip install google-antigravity, read the official announcement for feature details, and find the source on GitHub.

A futuristic glowing notebook on a wooden desk with a cup of coffee and floating geometric shapes.

Reading List 6

Allen HutchisonLeave a comment

This week’s reading list is a mix of high-level theory and low-level pragmatism. I found myself bouncing between the philosophical implications of how we build AI and the immediate satisfaction of writing a good Go component.

[article] The Century-Long Pause in Fundamental Physics. The author argues that physics has stagnated by swapping “ontology-first” theory for mathematical models that merely fit data. This debate perfectly mirrors current machine learning disputes about whether LLMs build internal world models or just pattern-match at scale, which is the open empirical front currently being adjudicated in mechanistic interpretability.

[release] Onyx Has Released a New Remote Page Turner Called Tappy. I wish Amazon would support page turners for their Kindle line. It would be great if they supported a device as delightful as this one.

[blog] The agent principal-agent problem. This is a great look at one of the biggest problems with agentic development: code review. In my open source work, I now use a pattern where I work with an agent to make a change, test it locally, and create a pull request before having another agent review the code. This back-and-forth works well and keeps a good balance of mental state for the codebase and efficiency.

[article] ReMarkable Paper Pure wants to be the only notebook you’ll ever need. I have always liked the reMarkable tablets, but every time I try one I miss having my Kindle library alongside it. Reading and writing are deeply linked for me, which is why I recently got a Kindle Scribe Colorsoft and found it really hits the mark for what I want.

[blog] Just Fucking Use Go. I have been working on a project that has a Go component to it recently. This is the first time I have really started to look at the language, and it inspires me to spend more time with it.

I built my 7MB Full AI Terminal in Rust & Tauri. This is a neat open source AI terminal. It feels similar to Warp but is a lot smaller.

[article] Computer Use Is 45x More Expensive Than Structured APIs. I am not surprised at all by these findings. I think computer use will remain a last resort, and a lot of apps will expose some kind of API for an agent to use instead. My guess is that this eventually becomes the way we automate unmaintained applications that need to fit into an agentic workflow.