im·a·cto — I am a CTO

Your API is your agents' hands

Mon, 13 Jul 2026 00:00:00 GMT

Build the open API first, dogfood it, then invite others in. That used to be a pure growth bet: every team that integrates makes the product bigger than you could grow it alone. There's a second payoff now. The surface you built for outside developers is the same one your agent fleet drives. At Phoneware I put MCP servers over the platforms underneath so the fleet operates them like any other client. Design it once, for humans and agents both. Skip that and you're writing a second interface for the machines later.

Scale Is a Design, Not a Dial

Wed, 08 Jul 2026 00:00:00 GMT

The dashboard says you're running forty instances of the service. This morning it was twelve. The autoscaler did exactly its job: it saw latency climb and threw hardware at the problem, which is what you built it to do. And latency is worse. Not flat. Worse. You're paying for three times the compute to serve a slower product, and every instinct you've got says add more, because more has always been the answer. It won't work this time. Somewhere under all forty of those boxes is a single thing they're all waiting in line for, and every instance you add makes the line longer.

That's the part that feels like a betrayal. Adding capacity is supposed to help, or at worst do nothing. Here it actively hurts, and once you see why, you can't unsee it. The autoscaler multiplied the part of the system that was never the bottleneck. It spun up forty copies of the stateless shell, forty more contenders for the one place they all have to converge: the hot row, the lock, the partition, the queue everything drains through. The console has a knob for CPU and a knob for replica count. It has no knob for the thing that's actually saturated.

the dial has a ceiling nobody drew on purpose

Horizontal scaling does one trick well. It multiplies work that doesn't have to coordinate. Spread a million independent requests across a hundred workers and you get something close to a hundred times the throughput, because none of them need to talk to each other. The trick fails the instant the work does have to coordinate, and almost every real system has a piece that does.

Amdahl wrote this down in 1967, and it hasn't stopped being true: the fraction of a job that has to happen in sequence sets a hard ceiling on how much faster you can go, no matter how many processors you throw at the parallel part. If a tenth of the work is serial, you top out at ten times, forever, even with infinite hardware. Neil Gunther's Universal Scalability Law made the picture honest in the way your dashboard just did: past a certain point, the cost of all those nodes coordinating with each other doesn't merely flatten the curve, it bends it back down. Add capacity, get less throughput. Retrograde scaling. That downward bend is the curve your forty instances are living on, and it has a name and a formula that predicted your morning before you had it.

Here's the part that matters for anyone who leads a team. That ceiling was not set by the autoscaler, and it will not be moved by the autoscaler. It was set a long time before this morning, in a room, by whoever decided where the state lives and who has to touch it at the same instant. The console is downstream of a whiteboard, and the whiteboard already spent your scaling budget.

scale is a decision, and it's an early one

The industry files scale under operations. Bigger box. More replicas. A beefier managed database. A cache bolted in front of the thing that's slow. Those are all real levers, and I've pulled every one of them. Every single one is downstream of a decision made before the first line of code: where does state live, and which requests contend on it under load.

I learned that framing at iTriage, running a consumer health app at real scale. Scale showed up as a systems-design problem long before it was ever a code problem. You could write every function correctly, ship a clean diff, pass every test, and still hit a wall, because a wall like that grows out of how the pieces are arranged, not out of any one piece. Where things lived and what had to talk to what set the limit. The most durable calls I made there happened at the whiteboard, in words, before anyone opened an editor.

At GigSmart the same truth wore a product costume. A real-time marketplace is a contention problem before it's anything else. Two apps, workers on one side and requesters on the other, both converging on the same handful of moments: who's available right now, who just got claimed, whose request is still open a half-second later. I reached for Elixir because fault isolation being the floor and not a bolt-on is exactly what a marketplace needs. But the runtime was the second decision. The first one was deciding where that hot, shared state was going to live and what would serialize on it, because that call sets the ceiling, and a framework can't raise a ceiling the design already poured.

The autoscaler multiplies the part that was never the problem.

It spins up copies of the stateless shell and cannot touch the one place they all converge. That place was chosen on a whiteboard, and that's where the ceiling got set.

the diff looks perfectly scalable

Now hand this to a fleet, and watch the failure get quieter.

An agent will write you a service that looks built to scale. Stateless handlers, a tidy repo, horizontally-ready by every surface signal. It deploys clean. The tests are green, because a unit test exercises one request at a time and one request never contends with itself. The canary bakes fine at 1%, because 1% of traffic almost never has two requests fighting over the same row in the same millisecond. Every gate you trust says ship it. And the bottleneck is sitting right there in the design, in the spot where concurrent load lands on the one thing everything funnels through, invisible to all of it.

It's invisible because the mistake isn't in the lines. It's in the shape, in where the state was placed and what got to touch it at once, and you cannot catch a shape problem by reading a diff. I've argued that reviewing an agent means reading the decisions, not the characters. This is the most expensive decision of exactly that invisible kind. A fleet has no model of your load shape, your hot keys, your fan-out, the one endpoint that gets hammered on the hour. It will put the state somewhere reasonable-looking and be confidently wrong in a way no reviewer catches by reading code, because there's nothing wrong with the code. The wrongness is architectural, and architecture doesn't render in a diff.

That's the trap cheap generation sets under this specific problem. When output was scarce, a senior engineer sketched the state layout on a whiteboard first, because typing was expensive enough to make you think before you spent it. When a fleet will hand you the whole service in a minute, the whiteboard step is the one that quietly gets skipped, and it was carrying the entire scaling decision. You multiply judgment by a fleet or you multiply its absence, and the absence hides best right here, where the system runs green all the way to the day it meets real concurrency.

what you decide before the editor opens

The work reorders around the decision, in priority order, because the sequence is the whole point:

Name the hot state before the framework. Where does the contended state live, and which requests touch it at the same instant? Answer that out loud, in plain words, before anyone picks a language, a database, or a queue. The tool is downstream of that answer, every time.
Split stateless from stateful on purpose. The part you can multiply and the part you can't are different animals with different rules. Draw the line between them deliberately, so the autoscaler is pointed at work it can actually help and kept away from work it can only crowd.
Find the serial fraction and go straight at it. Amdahl's ceiling is set by the piece that has to happen in order. Shrinking that piece is worth more than any number of instances. Partition the hot key, move the contention off the critical path, make the shared thing append-only instead of read-modify-write. One point off the serial fraction beats forty boxes.
Load-test the collision, not the endpoint. A benchmark that fires independent requests at one handler proves nothing about the moment two of them want the same row. Simulate the contention on purpose, because the contention is the product under load, and it's the only test that would have warned you this morning.

So go back to the forty instances. You can keep turning the dial, and the curve will keep bending the wrong way, because the number on the console was never the variable that mattered. The fix lives back in the room where somebody decided where the state would go, and the honest move is to reopen that decision rather than buy more hardware to outrun it.

Which leaves one question worth sitting with before you touch the autoscaler again. Where does your state actually live, who contends on it under real load, and did anyone ever decide that on purpose? If the honest answer is that it just happened, that nobody drew it, then that's your ceiling, and there's no dial in the building that reaches it.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

A note on how this was made: I wrote this with Claude Opus 4.8. I brought the frame, the experience, and the calls on what mattered and what to cut; Claude did most of the drafting. I'd rather say that plainly than pretend the tool wasn't in the room.

A note

Mon, 06 Jul 2026 00:00:00 GMT

Fourth of July weekend, and I was actually off. Phone in a drawer, kids outside, something on the grill. The fleet kept opening MRs, the merge train held the line, nobody paged me. I spent a lot of years being the load-bearing wall. Turns out the quiet reward for building the tracks right is that the thing stands up fine while you're not holding it up.

The Story of Ownership

Sun, 05 Jul 2026 00:00:00 GMT

You got the offer, and somewhere on the second page, in a font a little larger than it needed to be, was the word. Owner. Maybe it read "equity partner," maybe it was a founder at the all-hands saying "we're all owners here" over a slide with a rocket on it. You felt something land. You were supposed to. Then you signed, and you never read the four pages after the word: the vesting schedule, the cliff, the paragraph about what happens to the shares you earn if you leave, the ninety days you'd have to find real cash to buy them, the tax bill that arrives whether or not you have it. Nobody reads those pages at signing. That is the design.

Ownership is the cheapest thing a company can hand you, because the word costs nothing and the slide costs nothing, and in the room both feel identical to the real thing. That's exactly why it gets handed out so freely and meant so rarely. There's the story of ownership, and there's ownership. Most companies are very good at giving you the first while the second stays precisely where it already was: with the people who held it before you walked in.

read the four pages

Go read your own grant. Not the number of shares, the mechanics around them. The standard shape is four years to vest with a one-year cliff, so if you leave in month eleven you own nothing at all, no matter what you shipped. Most people know that part. The part that quietly does the work is the post-termination exercise window. On a typical grant you get ninety days after you leave to buy the options you already vested, or you forfeit them. For a lot of people that's a five or six figure check they don't have, plus a tax bill on paper gains they can't sell to cover it. So the ownership they spent years earning expires, unexercised, and slides back to the company. Vested. Gone.

This is common enough that a wave of companies started publicly extending those windows, from ninety days out to seven or ten years, specifically because the short one was clawing hard-earned equity back from the people who'd built the thing. The ones who extended it were telling on the ones who didn't. If your window is still ninety days and walking away means you can't afford to keep what you earned, read that sentence one more time: you were renting the story of ownership. You never held the asset. You held the feeling of holding it.

A word costs nothing. A cap table costs something. Companies hand out the first and call it the second, and the room can't tell the difference in the moment.

owner-behavior at renter-prices

Here's the move, and once you see it you can't unsee it. Call someone an owner and something changes in how they work. They stay late without being asked. They treat the bug like it's theirs, because you told them it is. They carry the thing home in their head on a Sunday. That behavior is worth an enormous amount, and it's the entire reason the word gets used. The quiet trick is to buy the behavior with the word instead of the thing. You get someone to act like an owner while the ownership stays a story. That's the whole game: owner-behavior at renter-prices.

I've sat on the other side of that table. I've built teams from nothing and I've signed the grants that went to them. The thing that predicted who acted like an owner was never the size of the number. It was whether the grant was real: whether the window was humane, whether the upside was actual, whether the person was in the room when the call that moved the value got made. Give someone a real stake and they behave like it. Give them a story and, sooner or later, they read the four pages and behave like that instead. People aren't stupid. They're just slow to check.

So my rule is boring and it costs me something. If I call you an owner, I mean the cap table, the window, and the room. If I'm not willing to mean all three, I don't use the word. I'll find an honest one. Underpaying someone in dollars and topping it up with a fiction is worse than just paying them less, because the fiction has a half-life, and when it decays it takes the trust with it.

the asset that never vests

There's a layer under all of this, and it's the one that actually matters to me. The most valuable thing you're asking an owner to spend was never in the grant. It's their time.

Time is the only line on the cap table with no refresh grant.

You can re-strike options, reprice a round, raise more capital. You cannot give someone back a Saturday. Spend it like it's cheap and you've told them exactly what you think they own.

Capital renews. You can raise more, reprice it, refresh a grant, extend a window. Time is the one input on the whole cap table that doesn't come back, can't be repriced, and never refreshes. When a company spends its people's time as though it were cheaper than the money it's conserving, it's making an ownership statement louder than any offer letter: your irreplaceable thing is worth less to us than our replaceable one. Real ownership starts there, long before the equity, in whether you treat a person's time as worth more than your capital. A lot of the "we're all owners" places have that ranking backwards, and everyone can feel it even when they can't name it.

So go back to the offer, the one with the word in the slightly-too-big font. The word is free. Turn the page. Read the vesting, read the window, do the math on the check you'd have to write to keep what you earned. If the story and the substance match, you found a good one. Hold onto it. If they don't, you already know which one you were sold, and now you know why it felt so good in the room and so thin the day you left.

Jason Waldrip has spent his career building and leading engineering teams at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

A note on how this was made: I wrote this with Claude Opus 4.8. The belief is mine and so are the calls on what to keep and cut; Claude did most of the drafting. I'd rather say that than pretend the tool wasn't in the room.

Ownership Theater

Sat, 04 Jul 2026 00:00:00 GMT

Somewhere in your first week, someone told you to act like an owner. Maybe it was in the offer, dressed up as "equity." Maybe a founder said it at an all-hands, in the voice people use for things they badly want to be true. Treat this like it's yours. Think like an owner. And you nodded, because it's the kind of thing you want to be true too.

Then you read the paperwork. Four-year vest, one-year cliff, and a 90-day window to exercise your options after you leave or you forfeit them. To turn that "ownership" into actual shares, you write the company a check, at a strike price that assumes the thing is already worth something, out of money you have because you took a startup salary. Most people can't. So they walk away from the vested portion they were told made them an owner, because owning it was gated behind a payment they were never positioned to make.

That's not a loophole. It's the standard deal. And it's the first tell that the word "owner" was doing very different work than you thought.

The exit is where the story ends

Say you make it. You stay the four years, the options fully vest, you scrape together the cash to exercise, the company sells. This is the part the recruiting page was picturing. Here's what actually happens: the money doesn't split by who owns what. It splits by who holds a liquidation preference, and the people who wired in the capital wrote themselves to the front of that line. Fred Wilson, a VC who's sat on the winning side of plenty of exits, has spent years explaining the mechanics on his blog: preferred shares get paid first, sometimes a multiple of what went in, sometimes a multiple and their percentage on top. Your common stock is whatever's left after everyone senior in the stack has been made whole.

On a great exit that's still real money. On the far more common outcome, the sale that sounds impressive in the headline, the preference stack eats the number and the common holders (the employees, the people who were told to act like owners) get a rounding error. The Holloway Guide to Equity Compensation walks the mechanics in more detail than any offer letter ever will, which is exactly the point. The complexity isn't an accident. A thing you fully understood, you could negotiate.

None of this is illegal. Most of it is smart, on the side of the table that wrote it. I'm not calling the mechanics evil. I'm saying a company that runs every one of them and then asks you to "think like an owner" is asking for the behavior of ownership while handing you the story of it.

The theater

Here's the phrase I use for it: ownership theater. The whole set of rituals a company performs to get owner-behavior out of people it has carefully arranged not to make owners. Say "we're all founders here." Call the staff "partners" on the website. Put "act like it's your company" on the values poster. Hand out equity engineered, through cliffs and windows and preferences, to be worth the least it plausibly can while still counting as a number in the recruiting deck.

The tell is always the same. The company wants the feeling of ownership to do the work the structure of ownership refuses to do. Ownership makes you show up on the bad night, sweat the detail nobody asked about, treat the money like it's partly yours because it is. That behavior is worth a fortune, and it's close to impossible to fake into existence with a speech. So companies try to summon it with language instead of buying it with stake. It works for a while. Then people do the math.

You can't inspire owner-behavior out of people you've structured as renters.

They'll act like renters eventually. Not because they're cynical. Because they're paying attention.

And a renter isn't lazy. A renter is rational. If the upside is capped at "your check bounces" and the cost of caring too much is nights and weekends spent feeding someone else's liquidation preference, the correct move is to do the job well and go home at six. The company taught that lesson itself. It just doesn't like the grade.

The same judgment, pointed at people

I spend most of my time on technical calls, and I have a rule for those: earn it. Don't take on complexity before the workload forces it, because complexity is rent you pay every day it runs, and the bill lands on a different date than the decision. Fake ownership is that same mistake, pointed at the org chart instead of the architecture.

It's cheap at the whiteboard. Story-shaped equity costs nothing today and photographs beautifully in the deck. The rent shows up later, on a different desk. Your best engineer, the one who genuinely did act like an owner for three years, quietly stops. Or leaves. Or, worst case, stays and stops caring, which is the most expensive outcome of all, because now you're paying full price for a renter and filing it under retention. You got the complexity at a discount and the operating cost at full freight. Same trap, different diagram.

What real ownership costs

So do the honest version, or don't use the word. That's the whole rule.

I run my work through The Bushido Collective, and the worker side of it comes down to one line: real ownership, not the story of it. If I tell someone they're an owner, I mean the structure, not the feeling. Stake that vests like a promise instead of a trap. Terms a person can actually read and actually keep. Upside that isn't quietly rigged to evaporate at the exact moment it's meant to pay out. It costs more than theater does. That's the point. Ownership that was cheap to give was never ownership.

The other half of the same belief: time is worth more than capital. You can raise more money. Nobody gets another four years. When you ask someone to spend a slice of their one finite life building your thing, the fair trade isn't a motivational poster and a number that rounds to zero. It's a real piece of what they helped build.

So the next time someone tells you to think like an owner, ask the boring, unpopular question. Not "do I believe in the mission." Ask what happens to your stake the day you leave, and what sits in front of it the day the company sells. If the honest answer is "not much, and everyone," you already know what they meant by owner. Act accordingly. A renter would.

Jason Waldrip is a fractional CTO and CAIO. He writes about engineering leadership, incentives, and building in the age of AI agents. If you're structuring a team and want the ownership to be real instead of theater, that's a call I'm glad to take.

A note on how this was made: I wrote this with Claude Opus 4.8. The frame, the belief, and the calls are mine, drawn from how I actually build teams. Claude did most of the drafting. I'd rather tell you the tool was in the room than pretend it wasn't.

Own the model

Wed, 01 Jul 2026 00:00:00 GMT

Point a task at a frontier API and you've made two decisions: rent the intelligence, hand your data to someone else's server. Sometimes that's right. Often you reached for the biggest model because it was there, not because the job needed it. Classification, extraction, routing don't need frontier reasoning. An open-weight model you run yourself does them cheaper and keeps the data in the building, and Qwen and DeepSeek are closing the gap fast. Save the frontier for the problems that actually earn it.

Concurrency isn't a feature you add

Tue, 30 Jun 2026 00:00:00 GMT

A real-time marketplace is a concurrency problem wearing a product costume. You can bolt fault-tolerance onto the wrong runtime, supervisors you hand-wrote and queues you babysit, or you can pick one where process isolation is the floor and not the upgrade. At GigSmart I reached for Elixir for exactly that. One slow request shouldn't take down the ones next to it, and on the BEAM it doesn't, by default. Reach for it when concurrency is the product. When it isn't, the runtime is a tax you didn't need. Earn it.

Read the Decisions, Not the Diff

Sun, 28 Jun 2026 00:00:00 GMT

The agent just handed you eight hundred lines. Three files you've never opened, a migration, a new module, and a green check mark sitting under all of it. It runs. The demo works. Somewhere in your gut there's a question you'd rather not ask, because you suspect the honest answer. The question is whether you actually know what this does. Not whether it passes. Whether you could stand up in front of your team tomorrow and explain why it's built this way and not the other way. If the answer is no, you didn't ship a feature. You shipped a thing you're now responsible for and don't understand, with your name on the merge.

This is the part of the vibe-coding panic aimed at the wrong target. The danger was never that an agent wrote the code. The danger is that nobody questioned it. And "question it" gets heard as "read every line," which is both impossible at the volume a fleet produces and not what catching problems ever actually required.

you never read every line

I've approved thousands of pull requests. I did not read most of them line by line. Any engineering leader who tells you they did is lying or running a team of four. What I read was the shape of the decision: where the state lives, what this couples to, whether the approach matches what we'd agreed the problem actually was. A senior engineer reviewing a junior's PR isn't proofreading characters. They're checking judgment against intent. That skill, the one we've used on each other for as long as we've worked in teams, is exactly the one this era needs. Almost nobody is naming it as the same skill.

So here's what questioning the output actually means, and it's two moves. First, understanding why the agent made the calls it made. Not the syntax. The decisions. Why a queue here and a synchronous call there. Why it reached for that library. What it quietly decided about retries, about failure, about the case you never mentioned. Second, holding its finished design up against the spec you handed it, and finding the gap. There's always a gap.

the spec is the input, not the system

Spec-driven development is having its moment, and for good reason. The spec is the steering wheel now, and the more capable the fleet, the more elaborate that spec has to get, because every ambiguity you leave is a decision you've delegated without deciding anything. Write the spec well and you've pointed the engine at the right hill.

But the spec is the input. The system that comes back is the output, and those are not the same document. The agent fills every gap you left, confidently, in ways that drift from what you meant by degrees small enough to skim past. I've written before that agents don't go quiet when they hit missing context. They fill it. The spec tells you what you asked for. Only the diff tells you what you got, and the distance between the two is precisely the thing the green check mark can't see.

The spec tells you what you asked for. Only the output tells you what you got. The gap between them is where the work moved.

the reasoning is ephemeral now

Here's the part that's genuinely new, and the part I think most teams will learn the hard way.

When a human on my team made one of those calls, the reasoning had a home. It lived in a head I could walk over to. At Brandfolder, Ruby was crashing on large zip files, so we built that one piece of the pipeline in Go. Six months later, anyone could ask why the zip path was a different language than the rest of the app, and get the real answer, because the people who made the call were still in the building. The why survived because the person did.

When an agent makes the call, the reasoning lives in a transcript. A run log: the options it weighed, the ones it dropped, the chain that led to the design you're now holding. And that log is ephemeral. It scrolls off. It's in a session you closed, on a harness that may or may not have kept it, in a format you may or may not have the skill to read. The code stays, green and running. The reasoning that produced it evaporates, unless two things are true: your harness surfaces the trace, and you know how to dig through it.

The code is permanent. The reasoning is a log that scrolls off.

A human teammate's "why" is still in the building in six months. An agent's "why" is in a session you already closed, unless you went and captured it.

Reading an agent's decision log the way you'd read a teammate's PR description is a new literacy, and it's becoming load-bearing fast. The decision the fleet made at 2pm Tuesday is as real as any architectural call your staff engineer ever made. The difference is your staff engineer is still here to defend it.

you've held this map before

Which brings me to the thing that's supposed to feel like a crisis and, for me, mostly doesn't. The job now is to hold the map on a codebase you architected but didn't implement. To carry the why across a system you're responsible for and didn't personally type. If that sounds like a frightening new mandate, you haven't sat in the chair.

That has always been the job. At GigSmart I hired and led the team that built the platform; I made the call that a real-time marketplace is a concurrency problem and that's why we were on Elixir, but I was not the one writing every module that depended on it. My value was never that I typed the most. I argued that already, at length, about my own commit history. The value was holding the why across systems I hadn't touched in a year, built by engineers I'd guided but never stood over. Understanding the decisions my team made, and just as often the decisions they didn't make: the corner nobody addressed because my guidance there was thin. Holding that map is the oldest part of leading engineers. The fleet didn't invent the problem. It handed the problem to everyone, including the solo developer who never had a team and now suddenly does.

A fleet is a team. A fast, tireless, literal-minded team that writes no documentation unless you force it, remembers nothing past the session, and will pave a road straight off a cliff and report success. Directing that isn't a new discipline. It's the discipline good engineering leaders already had, pointed at a team that happens to be made of models.

so what do you actually do

The work reorganizes around the map, in priority order, because the sequence matters more than the calendar:

Interrogate the why before you accept, not after it breaks. Make the agent defend the call. What did it consider and drop, and why this over that? If it can't produce the roads not taken, the thinking didn't happen, and you're the one who has to do it.
Diff the design against the spec, not just the tests against the code. Green means the code does what it does. It says nothing about whether what it does is the system you asked for. The gap between spec and output is the review.
Capture the ephemeral reasoning before it scrolls away. The load-bearing "why" belongs in a decision record or in the code itself, not abandoned in a closed session. Write down the why, for the next human and for the next context window both.
Treat the agent log as a primary source. Build the harness that keeps the trace, and learn to read it like you read a teammate's PR. The reasoning is evidence. Evidence you can't retrieve isn't evidence.

None of this is anti-agent. I'd be a hypocrite. I wrote this one with a fleet at my elbow, and I'd point a fleet at nearly anything. The argument is only that generation getting free doesn't retire the engineer. It promotes everyone into the chair I've been sitting in, the one where you're accountable for decisions you didn't make with your own hands, and the only thing standing between you and a beautifully formatted disaster is whether you understood them.

So go back to the eight hundred lines. The check is green. That was never the question. The question is whether you could explain, with the session closed and no tab to peek at, why it's built this way. If you can, you're doing the actual job: you're holding the map. If you can't, the code is already yours, and the reasoning that made it is scrolling toward the bottom of a log you may never open again. Better go read it now, while it's still there.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

When the client owns the shape

Sat, 27 Jun 2026 00:00:00 GMT

GraphQL isn't an API style. It's a question about who owns the shape of the response. When one graph feeds many surfaces, iOS, Android, web, each asking for exactly the fields it needs, that ownership belongs with the client and the complexity pays for itself. At GigSmart, two apps and a web client hit one graph, and it was worth every bit of the schema. Point it at a single client talking to a single backend and you've bought resolvers and a caching headache to solve a problem you don't have. REST was fine. Earn it.

I Ran Your Demo

Fri, 26 Jun 2026 00:00:00 GMT

There's a calendar invite in your inbox for a 45-minute demo of a platform you might buy. The vendor will share a screen. Someone smooth and likeable will walk you through a flow that works perfectly, every click landing exactly where it should, the sample data clean, the latency invisible, the one feature you actually care about saved for the end so you leave on a high note. You'll nod. It'll look great. And you will have learned almost nothing about whether the thing works, because that wasn't the point of the meeting. It was never going to be.

I can say that with some confidence, because for a year that smooth likeable person was me.

the edge

My first real job in this industry was sales engineer at a CDN. I was the technical half of the sales motion, the one who got pulled in when the prospect had questions the account rep couldn't answer. I helped land Blizzard, YouTube, Disney, Rockstar. Big logos, hard rooms, smart buyers. And my job, stripped of the title, was to make our product look like the obvious answer in the time we had on the call.

I was good at it. Being good at it meant knowing exactly where the product was strong and steering every demo straight down that road. The features that lagged, the configuration that was a nightmare, the edge case where throughput fell off a cliff: those weren't on the tour. Not because I was lying. I never lied. I just built a route through the building where every door I opened led somewhere we'd cleaned up first. A good demo is a magic trick, and the trick is misdirection, not deception. You're looking exactly where I want you looking.

That's the part buyers don't internalize. The demo is a piece of software in its own right, and the thing it's engineered to do is produce a yes. It is the single least representative way to experience a product, optimized end to end by people whose income depends on you not noticing the gaps until after the contract is signed.

what the demo is built to hide

Every product has a shape. There's the part it does beautifully, the part it does adequately, and the part where it quietly falls apart under real conditions. The honest version of evaluating a tool is finding that third part as fast as possible, because that's where you'll actually live once it's in production carrying your load instead of demo load.

The demo is built to keep you out of that third part. The data is seeded so the slow query never fires. The flow avoids the integration that breaks half the time. The scale is a toy, so the thing that dies at real volume never gets the chance. I used to spend more prep time deciding what not to show than what to show. The skill wasn't presenting the strengths. Anyone can do that. The skill was routing cleanly around the weaknesses so the prospect never tripped over one and started asking the question that ends the deal.

Now I sit on the other side of that table, and I know the route exists, so I refuse to walk it.

evaluate on the gaps, not the highlight reel

Here's the move. The demo shows you the highlight reel. Your job is to evaluate on the gaps, because the gaps are what you're actually buying. A tool's strengths are table stakes, they're why it's even on your list. The decision lives entirely in what it's bad at and whether you can live there.

So I don't take the guided tour. I ask for the docs and a trial account, and I go straight at the thing the demo would have avoided. I throw production-shaped data at it, not the clean sample set. I wire up the ugly integration first, the one that touches our actual auth and our actual edge cases, instead of the toy one in the tutorial. I find the limits page and read it like a contract, because the rate caps and the unsupported configurations are where the truth lives. And I go looking for someone who ran this thing at scale and hit the wall, because a buyer who churned will tell you more in ten minutes than the vendor will in ten meetings.

The tell I trust most is simple. Ask a vendor to let you click the thing they didn't show you. Watch what happens. The good ones say "sure, here's a sandbox, break it however you want." The ones to walk away from get a little tighter, suggest a follow-up call, offer to "set that up properly for you." That hesitation is the whole answer. They know where their third part is too, and they're managing you away from it, exactly like I used to.

the buyer's job is the judgment

This connects to the thing I keep landing on from every direction. Volume is cheap. Pitches are cheap. Polished surfaces are cheaper than they've ever been, and a good demo has never been easier to produce. The scarce thing, the only thing that protects you, is the judgment to look past the surface to the shape underneath. To ask what this is bad at, and whether bad-at-that is survivable for the actual workload you have, not the impressive one on the slide.

I'm not anti-vendor. I've sold to some of the sharpest engineering orgs on the planet, and the good ones were a pleasure precisely because they were impossible to misdirect. They'd thank me for the overview, decline the rest of the tour, and go find the wall themselves. They bought on the gaps. They were right more often than the ones who bought on the demo, and they got there faster, because they spent their time where the answer actually was.

So when the invite lands, I send the same reply every time. Skip the call. Send me the docs and a sandbox with no guardrails, and I'll have your answer by Friday. If that makes someone nervous, I already have it.

Written with Claude Opus 4.8, in my voice and grounded in my own years running the other side of that table. The judgment is mine. The drafting had help, and I'd rather say so than pretend.

Let the compiler review the fleet

Wed, 24 Jun 2026 00:00:00 GMT

A fleet of agents generates more code in an afternoon than your team used to review in a week. Most of it plausible, some of it confidently wrong, all of it too much to eyeball. The compiler doesn't get tired. Static types check every line the agents wrote before you read one. It's the instinct behind the languages I build on now, Crystal and Rust: pay a little up front, stop paying at 2am. In the agent era that bill just comes due faster.

The Load-Bearing Wall

Sun, 21 Jun 2026 00:00:00 GMT

It's 11pm and the deploy is wedged. You already know why. The env var that didn't get set, the migration that ran halfway, which service to bounce first and which one to leave alone because bouncing it makes things worse. You fix it in twenty minutes. You're asleep by midnight. At standup you don't even bring it up, because to you that wasn't an incident, it was Tuesday. And somewhere under the tiredness there's a small warm feeling that says they need me. Sit with that feeling for a second. It's the most expensive thing you own.

Because here's what that night actually was. The system went down, and the only thing standing between a broken product and a working one was a specific person's memory. Not a runbook. Not a test. Not a guardrail. A person, awake, who happened to know. That's not a war story. That's a single point of failure that pays for its own coffee.

The thing nobody says about being the one who can fix it

Every hour you spend being indispensable is an hour the system spends learning it can lean on you instead of on itself. Your reliability becomes the team's liability. The better you are at quietly catching things before they become incidents, the less anyone else ever has to learn the thing you know, and the more load quietly migrates onto the one wall holding the most weight.

It looks like seniority. It reads, on a contribution graph, like the most committed person on the team. Underneath, it's a bus factor of one with a green contribution graph. The org has decided, without ever deciding, that one person being out sick is an acceptable way to stop shipping.

I have been the wall

I walked into Brandfolder to a product going down on a memory leak. For a while I was the person keeping it alive, because in a rescue someone has to be. I could have stayed that person. There's a version of that job where I'm the hero who knows where all the bodies are buried, paged forever, quietly proud of it.

The actual job was the opposite. Hire and train a team of six so the product's survival never rode on one skull again. The rescue people remember is the leak. The rescue that mattered was making myself removable from the critical path. By the time it scaled to a hundred-plus enterprise customers, the thing that kept it up wasn't me knowing the system. It was a team that did, and tracks that held whether or not any one of us was awake.

I learned the principle a step earlier, at iTriage, and it's the one I keep coming back to: the most durable thing a senior engineer ships is other engineers. I mentored through pairing, not lectures. Every patch I handed someone, I owned forever. Every principle I taught, they owned, and so did the next person they taught it to. One of those compounds. The other one pages you at 11pm for the rest of your life.

The most committed person on the team and the biggest single point of failure are very often the same person. Nobody wants to say it because it sounds like an attack. It's a compliment with a warning stapled to it.

Solo heroics have a ceiling, and the ceiling is you

There are only so many hours in your week, so many incidents you can personally absorb, so many 11pms before something gives. Go fast alone and you hit a wall that no amount of dedication moves, because the wall is you.

At CommercialTribe we collapsed a release cycle from weeks to hours across an onsite team and a remote team in Argentina. That number is impossible if release knowledge lives in one person's head in one timezone. It only happens when the capability lives in the tracks, where anyone can reach it. The win wasn't a faster engineer. It was knowledge that stopped depending on a particular human being available, and started belonging to the system.

What the agent era did to this

For most of my career, being the load-bearing wall was a slow, personal tax. You paid it in sleep and in the vacations you didn't fully take. Now the tax has a sharper edge, and it's worth being honest about why.

Volume is free now. You can point a fleet of agents at a problem and they will generate all day without coffee or ego. But a fleet can only act on what's been made legible. The architecture decision you made for a reason that lives only in your memory, the landmine you route around on instinct, the "we tried that in 2023 and here's why it bit us" that never got written down: none of it exists to an agent. It can't reason over what was never written. It will confidently undo the exact decision you'd have fought for in a meeting, because to the model that decision was never made.

I've said you multiply judgment by a fleet, never zero by a fleet, because anything times zero is still zero. The load-bearing wall is where the zero hides. Judgment locked inside one skull can't be multiplied, by a teammate or by a machine, because there's nothing there to copy.

Judgment that can't be transmitted doesn't scale.

The indispensable engineer has a bus factor of one and a leverage ceiling of one. The moat was never what you know. It's what you can hand off.

That reframes the whole "judgment is the moat" idea, and it's the part I want to land. Judgment is only a moat if it can leave your head intact. The expert whose expertise dies when they log off isn't a moat. They're a liability the org has agreed to call a strength.

Make yourself removable

This is not "document everything" busywork. It's a deliberate move to get the value out of one person and into a place the team and the fleet can both run on. In rough priority order:

Write the why, not just the what. A runbook that lists steps is for the human at 11pm. A record of why the system is the way it is is for the human at 11pm and the agent at noon. Capture the decision and the reason you'd defend it, because that's exactly the context a fleet fills in wrong when it's missing.
Pair, don't rescue. The fast fix is to do it yourself. The durable one is to teach the principle so the next person doesn't need you for the next instance. A slow handoff beats a fast fix the second time the problem shows up, and there's always a second time.
Put the knowledge in the tracks. The best place for "don't do that, it breaks under load" isn't your head. It's a test that fails when someone tries. Types, CI, and review are guardrails, and a guardrail is knowledge that doesn't need you awake to work. Every rule you encode is a 2am page you'll never get.
Measure the bus factor honestly. If one person being out stops a release, freezes a system, or means a question goes unanswered for a week, that isn't seniority. It's debt with a name on it. Find the walls. Spread the load before the wall does it for you, at the worst possible time.

None of this makes you less valuable. It does the opposite. The engineer who can hand off everything they know is the one you actually want running the next thing, because their value travels. The one who can't is stuck being the wall forever, which feels like security right up until it feels like a cage.

Build the thing that stands without you

The goal was never to be the wall holding the building up. It's to build something that stays standing when you walk out of the room. Indispensable is not the win condition. It's the failure mode wearing the costume of one.

So ask the uncomfortable question, the real one. If you went fully dark for two weeks, no Slack, no laptop, what breaks? Whatever has your name on it and only your name, that's not your job security. That's your backlog. It's the work you haven't finished, which is getting it out of your head and into something the team, and the fleet, can carry without you.

I spend most of my time now walking into orgs propped up on one or two load-bearing walls, where the whole thing runs on a couple of people who can never really log off. Turning what's in their heads into tracks the team and the fleet can run on without them is most of the job. If that's the wall you've become, or the one you've come to rely on, that's the conversation I'm here for.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

The Plan Was Never the Point

Sat, 20 Jun 2026 00:00:00 GMT

You ask the agent to scope the feature. Twelve seconds later you've got a plan. Sectioned, sequenced, a tidy risks heading at the bottom, the dependencies called out in order. It looks like the plan you'd have spent an afternoon on, and it's faster and better formatted than the one you'd have written tired on a Thursday. You skim it. Nothing jumps out. You say go. Three weeks later you're standing in front of a design decision nobody on the team can explain, because nobody on the team actually made it. The model made it. You approved it. And "approved" turns out to be a very different thing from "understood."

I keep watching smart people walk into this, and they walk in precisely because the output is good. Bad plans get scrutiny. It's the reasonable-looking ones that get a nod and a merge.

what planning was actually for

There's an old line from Eisenhower that gets quoted to death and understood about half the time: "Plans are worthless, but planning is everything." The military has known this longer than software has existed. Moltke got there in the 1800s: no plan survives first contact with the enemy. Both men spent their lives making plans. Neither thought the document was the deliverable.

Here's what they meant. When you sit down and actually plan something, you simulate it. You run the system forward in your head, hit a wall, back out, try a different route, hit another wall. You argue with yourself about the case you're pretending won't happen. By the time you've got a plan, the plan is the residue of all that. The valuable part isn't the residue. It's that you now carry a map of the territory: why this path and not the three you rejected, where the load is, which corner you're nervous about and why.

That map is what lets you adapt when reality does what reality always does and refuses to match the document. You don't re-plan from zero. You consult the model in your head, the one you built by doing the work, and you adjust. The plan was meaningless the second it met production. The planning is what's still paying you.

the artifact was always a byproduct

So here's the move the agent makes, and why it's so easy to miss. It hands you the residue and skips the fire. You get the document without doing the simulating. You hold a map of a territory you never walked.

It reads like a plan. It is, structurally, a plan. What it isn't is yours. Nothing in your head changed when that file appeared. You can recite it. You can't defend it, because defending it means knowing the three paths it rejected and why, and you were never in the room where those got rejected. There was no room. There were twelve seconds.

Reading the answer is not the same as solving it.

A plan you approved gives you the conclusion. A plan you made gives you the map that lets you adapt when the conclusion breaks. Only one of those survives contact.

I learned this on the other side of it, the slow way, before any of this was automatable. The most useful thing anyone ever told me about scale is that it's a design problem before it's a code problem: decide where state lives before you write a single line. That sounds like advice about the answer. It isn't. The value was never the answer, "state lives here." It's the argument you have with yourself getting there, consistency against availability, what you're willing to lose in a partition, where the writes actually concentrate. Win that argument honestly and you can debug the thing at 2am, because you remember exactly which tradeoff you took and what it was supposed to cost. Skip it, let something else decide where state lives while you nod, and you've shipped a system you fundamentally cannot reason about. The agent will happily make that call. It does not have to wake up for the page.

generation is free, the understanding is the work

This is the same fault line I keep coming back to, just moved upstream. I've argued you multiply judgment by a fleet, never zero by a fleet, because anything times zero is still zero. Most people picture that at the code layer: the agent writes the diff, you catch what's wrong. But the diff isn't where the zero hides. It hides in the plan, one level up, where the decisions that shape everything downstream get made in a place you weren't.

Letting the fleet generate the plan is fine. It's good, even. What's fatal is treating the generated plan as the planning. The output is the cheap part now, all of it, the plan included. The expensive part, the only part that was ever scarce, is a human carrying an accurate model of why the system is shaped the way it is. You don't get that model by reading. You get it by interrogating: arguing with the plan, poking the case it waved off, asking it to defend the route it took over the one it didn't, until the map is actually in your head and not just in the file. That's not slower planning. That's the planning, done on top of a draft instead of a blank page. The draft is a gift. Mistaking the draft for the work is the whole trap.

There's a hard version of this I've said to more than one engineer: the shortcut you hand someone today is the context gap you own forever. Same physics here, except the someone is you, and the person you're shorting is yourself in three weeks, standing in front of a decision with no idea how it got made.

so what do you actually do

You don't stop generating plans. You'd be giving up a real advantage, and the blank page was never sacred. You change what you do with the draft.

Treat the generated plan as a question, not an answer. Its job is to give you something to argue with. If you find yourself agreeing with all of it on the first read, you haven't read it, you've skimmed it. Go back and find the decision you'd have made differently, and make the plan tell you why it didn't.
Make it show its rejected paths. A plan with no roads not taken is hiding the part that matters. Ask what it considered and dropped, and why. The discarded options are where the judgment lives. If it can't produce them, that's your signal the thinking didn't happen.
Find the load-bearing decision and own it personally. Most plans have one or two calls that everything else rests on: where state lives, what's the source of truth, which thing you'll be unable to change later. Those you make yourself, by hand, with the argument fully had. Let the agent draft the other ninety percent.
Pass the empty-room test. Before you commit to it, ask whether you could defend the plan in a room where the model isn't invited to help. If you can, the map is in your head. If you can't, you have a document, and you should keep working until you have a plan.

None of this is anti-agent. I'd be a hypocrite, I wrote this one with a fleet at my elbow. It's a warning about the specific way this era lets you skip the part that was always the point and not notice, because the thing you skipped used to leave an obvious gap and now leaves a polished file in its place.

So next time the plan shows up in twelve seconds, the question isn't whether it's good. It usually is. The question is whether you could stand up tomorrow and defend it without the thing that wrote it. If you can't, you didn't make a plan. You accepted one. And the bill for the difference always comes due in the same place: at 2am, in front of a decision nobody remembers making.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

Earn It

Fri, 19 Jun 2026 00:00:00 GMT

You're in an architecture review for a product with a few thousand users and a roadmap that fits on an index card. Someone's whiteboard has Kubernetes on it. A message bus. Three services where there's currently one. The diagram is gorgeous. Boxes, arrows, a little cloud. Everyone nods, because the diagram looks like what a serious system is supposed to look like, and nobody wants to be the person who says the quiet thing: we don't have the problem this solves yet.

I've been the person who says it. It doesn't make you popular in the room. The elaborate plan reads as ambition and foresight, and the simple one reads as not thinking big enough. So the simple plan loses, and eighteen months later the team is spending half its week operating infrastructure that's babysitting a workload a single Postgres box would have shrugged at.

The impressive choice is a tell

There's a kind of design that's optimized for how it looks on a slide, or on a résumé, instead of for the problem in front of it. You can usually spot it because the justification runs backwards. Instead of "here's our constraint, here's the cheapest thing that meets it," it's "here's this powerful tool, surely we'll need it eventually." The tool comes first and the need gets reverse-engineered to fit.

I get the pull. Reaching for the powerful thing feels like seniority. It signals that you've operated at scale, that you know the patterns the big shops use. But pattern-matching to Google's architecture when you have Google's problems is wisdom. Pattern-matching to it when you have a CRUD app and four thousand users is cargo culting with extra steps. Ozan Onay made this point cleanly years ago in "You Are Not Google": companies adopt the tools of giants without ever asking whether they share the giants' constraints. The tool was a good answer to a question you don't have.

When I did reach for it

This isn't an argument for staying small or being scared of real infrastructure. I've made the expensive call plenty of times, when the workload earned it.

At CommercialTribe I migrated us onto GCP and Kubernetes. From the outside that's exactly the move I'm warning about: a VP shows up and reaches for the heavyweight orchestrator. But it wasn't résumé-driven. We had a split org, onsite engineers and a remote team in Argentina, a release cadence measured in weeks, and a stability problem that was costing us trust with every deploy. Kubernetes was the cheapest path to the thing we actually needed, which was a release process that collapsed from weeks to hours and held its footing under real load. The complexity paid for itself the first month, because the workload had already arrived. We were drowning. The orchestrator was the lifeboat, not the yacht.

That's the distinction. Same tool, opposite decision, and the only thing that changed was whether the problem was real yet.

Complexity is rent, not a purchase

Here's the part the architecture review never accounts for, because the cost shows up on a different day than the decision. When you choose the complex thing, you don't pay for it once at the whiteboard. You pay rent on it for as long as it runs.

Every service you split out is a deploy pipeline, a set of dashboards, an on-call surface, a place where the network can fail between two things that used to be a function call. Every additional language is a second toolchain, a second hiring profile, a second set of idioms your team has to hold in their heads at 2am. Every message bus is a new failure mode nobody had to reason about before. None of that shows up in the demo. All of it shows up in the year you spend operating the thing.

You pay for complexity every day it runs, not the day you choose it.

Which is exactly why it's so easy to take on. The bill arrives at a different desk, on a different date, addressed to whoever's on call.

Dan McKinley framed the budget for this better than anyone in "Choose Boring Technology." His idea: a company has a small number of innovation tokens, and every shiny, unproven, operationally-hungry choice spends one. Boring tech, the stuff with a decade of war stories and a flat learning curve, costs you nothing to run and frees the tokens for the places where novelty is actually the point. Spend them on your differentiator. Don't burn one on a queue when a database table would have done it.

This is why "just use Postgres until it hurts" is the same argument wearing different clothes. One engine that moonlights as your queue, your cache, your search, and your document store is one thing to operate, one thing to back up, one thing to reason about when it's 2am and the pager goes off. Five specialized engines, each individually defensible, add up to a system no one person can hold in their head. You reach for the second engine when the first one genuinely buckles. Not before, on the theory that you might.

Earn it

So here's the handle: earn it. Don't take on complexity before the workload forces it. Name the property you actually need, the real constraint, throughput or fault-isolation or types-as-guardrails, then pick the cheapest thing that delivers exactly that and nothing more.

When GigSmart needed a backend, I chose Elixir, and that's a less common bet than Node or Rails. But a real-time gig marketplace is a concurrency problem at its core, fault isolation isn't a feature you bolt on later, it's the shape of the thing. The constraint was real, so the choice was earned. Same year I'd have told you not to touch it for a CRUD app. At Brandfolder, Ruby was crashing generating multi-gigabyte zip files, the wrong tool for heavy IO, so we carved that one piece out into Go and left the rest of the monolith alone. We didn't rewrite the app in Go because Go is fast. We moved the one workload that demanded it. The scalpel, not the bulldozer.

The verdict on every one of those tools, on my own choices page, carries a "when I don't" right next to the "when I reach for it." The honest "when I don't" is half the credibility. Anyone can list a stack. Knowing what to leave off it is the work.

The era that makes this harder

This used to be self-limiting. Complexity was expensive to build, so the cost of construction was a natural brake on how much of it you took on. You felt the weight of the elaborate plan in the weeks it took to stand up, and sometimes that friction talked you out of it.

That brake is gone. Generation is free now. Point a fleet of agents at "scaffold me a microservices architecture with a message bus and per-service pipelines" and you'll have it by lunch, plausible config and all. The thing that used to take a quarter to build wrong now takes an afternoon to build wrong. And here's the trap: the cost of building it collapsed, but the cost of operating it didn't move an inch. A system is just as hard to run at 2am whether a human typed it or an agent did. You get the complexity at a discount and the rent at full price.

Which means the discipline matters more than it ever has, not less. When the impressive thing is nearly free to produce, the only thing standing between you and a system you can't afford to operate is the judgment to not reach for it. That's the whole game now. Volume is free; deciding what's worth running is the moat. An agent will never tell you that you didn't need the thing it was perfectly happy to build you.

So next time the gorgeous diagram is on the whiteboard and everyone's nodding, ask the unpopular question. Not "can we build this." Of course you can, that's never been cheaper. Ask what workload, here, today, makes this the cheapest path and not the most expensive one. If the answer is "we might need it someday," you haven't earned it yet. Put it back on the shelf and come find it when the problem shows up. It'll still be there. It always is.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents. If you're staring at a diagram that's bigger than your problem and want a second opinion, that's the kind of call I'm good at.

A note on how this was made: I wrote this with Claude Opus 4.8. The frame, the calls, and the stories are mine, drawn from real decisions; Claude did most of the drafting. I'd rather tell you the tool was in the room than pretend it wasn't.

Autonomy is infrastructure, not trust

Thu, 18 Jun 2026 00:00:00 GMT

"Ask forgiveness, not permission" is half a sentence. The other half is the rails that make forgiveness cheap: tests, CI, review, types. That's what lets me turn a team, or a fleet of agents, loose without signing off on every move. The guardrails catch what goes wrong before it ships. Pull them out and "move fast" is just "break things" with extra confidence. Build the rails first, then get out of the way.

The Tests Are Green

Tue, 16 Jun 2026 00:00:00 GMT

You shipped Thursday afternoon. The merge train was green, the image promoted clean, the canary baked six hours without a blip. You slept fine. Friday morning there's a support thread: customers can't get through checkout. You open the dashboards. Error rate, normal. Latency, normal. Nothing in the exception tracker, every health check green. By every signal you're paying to watch, the system is healthy. The feature is broken.

What broke doesn't throw. A user submitted a form, it validated, the response came back 200. The record never persisted, because of how the confirmation step races a read against a write that hasn't committed yet. Your tests cover the form. They cover the validation. They don't cover that combination, with that database state, at that timing, because nobody wrote that test, because nobody thought to, because the code reads fine in isolation. It is fine in isolation. That was never the question.

The question tests don't answer

Tests verify code paths. That's the job, and it's not negotiable. You write a test, the test tells you the function does what you claimed. Real, fast, valuable. But "this function does what I said" is a different question from "the checkout flow is working right now, for real people, on real phones, against production data."

Those two questions don't fail together. A team that trusts the first one to answer the second has built its confidence in the wrong place, and won't find out until a customer tells them.

I walked into Brandfolder to a product going down on a memory leak. The test suite was green the whole time. Memory isn't a code path you assert against; it's a property of the thing while it runs, under real load, over real hours. We couldn't fix it until we could see it. Once we could watch the process actually behave, the fix came fast. Before that we were tightening bolts that weren't loose while the real one backed out a turn at a time.

Monitoring is watching for what you feared

Most teams have monitoring. Dashboards, alerts, a rotation. And nearly all of it is shaped around the failures they imagined the day they built the system.

So you end up instrumented for the outage you pictured, blind to the one you didn't.

Error rate catches crashes. Latency catches slow. Queue depth catches backed-up. All worth having. None of them answer whether a person can finish a flow. Whether the submit actually persisted. Whether the third-party call that returned 200 also returned the right thing. Whether the feature you shipped last week is being used or quietly abandoned because it's broken in a way that never writes a log line. A 200 with an empty body is the happiest-looking failure in your whole system.

Your system can pass every check you're running and fail every one that matters to the person on the other end.

Observability is a different property

The word got annexed by APM vendors and flattened into "dashboards, but prettier." That's not what I mean by it.

As a property of a system, observability is this: can you look at the outputs and understand what's going on inside, including things you never thought to instrument? Here's the clean test. Something breaks in prod in a shape you didn't predict. Can you work out why from data you already have, without shipping new logging and waiting for a deploy? If the honest answer is "we'd have to add instrumentation and redeploy," then what you have is monitoring for predicted failures. Useful. Not the same thing.

What closes the gap is structured, high-cardinality data, captured at enough granularity that you can ask questions you didn't know to ask when you wrote the feature. Not "was there an error" but "what did this user's session look like across these three services in the ninety seconds before it stalled." Not "is the queue deep" but "which job class started retrying after Wednesday's deploy, and what do those payloads share." If the answer to that lives in next sprint's logging ticket, you are permanently one incident behind the thing that's actually hurting you.

The cut is simple. Monitoring tells you something went wrong. Observability tells you why, before the support ticket does it for you.

The agent era widens the gap

Every bit of this gets heavier when generation goes cheap.

When one engineer ships a handful of deliberate changes a week, there's an attention budget riding on each diff. A careful human reads it, pictures the failure, writes the test for the edge they're worried about. The coverage is a fossil of the care that went into the code. It exists because a person was nervous about exactly the right thing.

When a fleet opens dozens of clean, plausible merge requests a day, that nervousness is gone. Agents test what they exercised. They don't know about the payment provider that behaves differently on a retry, the mobile layout that's subtly wrong in a viewport no one opened, the analytics event that stopped firing after a re-render three components away. They don't know because nobody told them, and at generation-era volume there's no human reading every diff with the right fear in their gut. I've written before that you multiply judgment by a fleet, never zero by one. This is where the zero hides: in the tests nobody knew to ask for.

Which leaves exactly one honest source of truth: the running system. Real users, real product, real flows completing or not. Funnel completion, outcomes by result, conversion against the baseline. That signal is what tells you Thursday's deploy actually worked, because it's measuring the only thing that was ever the point. When generation is free, knowing what's true is the entire job.

What to actually instrument

Tests verify intent. Observability verifies outcomes. You need both, and you should stop pretending one covers for the other.

Watch outcomes, not just exceptions. A user who started a flow and didn't finish it is a signal, even when nothing threw. Instrument the funnel, not only the endpoints. Abandonment is data.
Real-user monitoring over synthetic checks. Synthetic tests prove the happy path under conditions you controlled. RUM tells you what's happening on real devices, on real connections, right now. Both earn their keep; only one knows about this morning's deploy.
One honest health endpoint. The canary is only as good as what it bakes against. Error rate and latency are necessary and nowhere near sufficient. Aggregate a real business signal into the gate: conversion, funnel completion, worker success. A slower honest gate beats a fast lying one every time.
Structure events for the question you haven't asked yet. Every request and action should carry enough context to reconstruct what happened without a redeploy: user, session, flags in effect, code version, the IDs that let you stitch a story across services. The log you'll be desperate for next month is the one you have to write this sprint.

None of this demotes tests. They're still the fastest feedback in the cycle, and they catch things in seconds that production would charge you days for. They just answer "does this code do what I said," not "is this working for the person holding the phone." Know which question you're asking. Carry evidence for both.

The signal decides

I've written about the merge train, the canary, what it takes to make every green commit shippable. All of it rests on a signal worth trusting. A canary baking against error rate and latency is a good start. A canary baking against those plus "did the core flow complete" is the difference between confidence and theater.

Your trust in the release process can't run deeper than the signal underneath it. Build shallow signals and what you've actually built is a very sophisticated machine for shipping into the dark with the lights off and a smile on.

So sit with this one. If something broke quietly in your product right now, no exception, no spike, just a flow that stopped completing, how long until you knew? An hour? A support ticket? Something you'd stumble onto in next week's numbers?

That number is the investment. And unlike most of the debt on your books, you can pay it down before it comes due.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

Teach the principle, not the patch

Fri, 12 Jun 2026 00:00:00 GMT

The hardest part of being a senior engineer isn't solving the hard problem. It's watching someone struggle toward it slower than you could, and keeping your hands in your pockets. Unblocking means pointing at the answer. Carrying means they need you again next time. At iTriage I learned the most durable thing a senior engineer ships is other engineers. The shortcut you hand them today is the context gap you own forever.

The context your agents don't have

Tue, 09 Jun 2026 00:00:00 GMT

An undocumented codebase has always been a tax on new hires. In the agent era it's a different problem. Agents don't go quiet when they hit a gap in context — they fill it. Confidently, and wrong. The architectural decision everyone knows because they were in the room? That doesn't exist to the fleet. They'll reverse it, build on top of it in ways that silently invalidate it. Write down why the system is the way it is. Not for the humans. For the context window.

Write it down. Not the what. The why.

Mon, 08 Jun 2026 00:00:00 GMT

Every decision your team made and never wrote down is a debt, payable by asking someone. Works until it doesn't: departures, six-month velocity gaps, agents that need context to not be dangerous. At Brandfolder, I knew why the zip pipeline was in Go. That knowledge survived because the people did. Decision records aren't documentation theater. They're the context a fleet can reason over, and the resilience that outlasts the team that built it.

A monorepo is a context boundary

Sun, 07 Jun 2026 00:00:00 GMT

The debate used to be tooling overhead and org preferences. Those tradeoffs are real. But there's a second argument now that flips the calculus. When a fleet of agents has to reason over a change, can they see the whole system? Split repos mean split context: a fix in service A that quietly breaks service B goes undetected because nothing is looking at both. One repo isn't a preference. It's the context boundary the whole system needs, for humans and agents both.

Release on Demand

Thu, 04 Jun 2026 00:00:00 GMT

There's a calendar invite on your team's shared calendar called something like "Release Window." Maybe it's every other Thursday at 2pm. There's a change freeze the day before. A go/no-go thread. A rollback plan nobody's tested. And a quiet, shared understanding that you do not, under any circumstances, ship after lunch on a Friday. That invite is not a process. It's a confession.

It says: we don't trust our own releases, so we've agreed to only do the scary thing on a schedule, in daylight, with everyone watching. Call it discipline if you want. It's really a workaround for a capability the team never built.

The doom loop nobody names

Here's the trap, and most teams are living in it without naming it. There's a gap between "this commit passed CI" and "this code is safe to ship." For a lot of teams that gap is measured in calendar weeks. Every change pays it. So you do the rational thing: you batch. You wait until you've got enough changes to justify the ceremony, then you ship them all at once.

Except batching is the thing that makes releases dangerous. A release with one change in it is easy to reason about. A release with forty changes in it is a haystack with an unknown number of needles. So the big batch needs more testing, more soak, more sign-off, which makes it slower to ship, which means the next batch is even bigger by the time it goes. Round and round. The schedule tightens. The freezes get longer. The hotfixes start going straight to prod outside the pipeline entirely, because the pipeline is the slow path and prod is on fire.

I've watched teams answer this with more process. A freeze before the freeze. A release captain. A go/no-go meeting with more people in the room every quarter. Every one of those is management layered on top of the fear, never a fix for the thing causing it.

That's the loop: the less you trust a release, the bigger you make it, and the bigger you make it, the less it deserves your trust. The schedule is just where you've parked the anxiety.

The schedule isn't keeping you safe. It's keeping a problem comfortable enough that you never have to fix it.

Faster doesn't fix it

The instinct is to make the engine faster. More CI workers. Parallel test shards. And now a fleet of agents that can open a few dozen correct-looking merge requests a day without coffee or ego. All real, all good. None of it touches the actual problem.

A faster engine on the same track just gets you to the same bottleneck sooner. If anything, cheap generation makes the gap worse: more commits piling up behind the same release window, more haystack per batch. You can't out-produce a trust problem. The fastest way to ship more code safely was never "type faster." It's to close the gap between green and shippable so completely that the schedule has nothing left to protect.

And the data's been clear on this for years. The DORA research (the Accelerate body of work) keeps finding the same counterintuitive result: the teams that deploy most often are also the most stable. Speed and safety rise together, because the same practices that make releases fast (small changes, strong automation, fast feedback) are the ones that make them safe. The teams shipping on demand have lower change-failure rates than the ones shipping on a quarterly calendar, not higher.

So the goal flips. Make every green commit shippable, and make the rare miss recoverable before a user ever notices. Two stacks do that work: a confidence stack on the way in, a recovery stack for when something slips through anyway.

Main is the train

Start with the thing that makes a release a non-event: stop treating "release" as a separate step at all.

Every commit on main is a candidate release. No release branch, no cut, no separate deploy job that someone runs. The way code gets to main is through a merge train (that's GitLab's name for it; GitHub ships the same thing as a merge queue, and the principle predates both as Graydon Hoare's Not Rocket Science Rule): an integration gate that tests each change against the projected merge result. Your change rebased on top of main plus everything queued ahead of you, all run as the commit that will actually land. If the train's pipeline is green, the exact commit that hits main has been proven integrated against the exact state it's joining.

Which kills a whole category of theater. Re-running CI on main after the merge tells you nothing the train didn't already prove. And a change that would break main never gets there. It fails its car on the train and gets pulled before it ever touches the branch. No "main is red at 2pm" scramble. No revert dance. No bisecting at midnight to find whose merge poisoned the well. The bad car comes off the track; the train keeps moving.

This is the difference between a green pipeline and a green build, and it matters more than it sounds. A pipeline that retries flaky tests until they pass, lets important checks warn-and-continue, and sets a coverage target low enough to clear in its sleep, that's a green light wired to nothing. A signal is only real once it has a measurement and a threshold. Green has to mean green, or every layer you build on top of it is built on a lie.

Build once, promote via overlay

When a change earns its way onto main, the train builds the container image one time and tags it by the commit SHA. That image is the artifact. It runs in dev immediately. And here's the part that makes promotion boring: shipping to prod doesn't rebuild anything. It re-points a config overlay (a Kustomize overlay, in practice, committed to the same repo) at the same SHA that's already running in dev. Same bytes, different environment.

Promotion is a routing decision, not a build decision.

The thing you tested in dev is the literal thing that runs in prod. Not a rebuild from the same commit and a hope. The same image.

That cleanly splits two worlds that most setups jam into one giant deploy job. The slow-moving substrate (the cluster, the network, IAM, the shape of your manifests) changes on the order of a quarter and belongs to Terraform, ideally run through something like Terraform Cloud so applies are deliberate and auditable instead of run from someone's laptop. The fast-moving payload (image refs, rollouts, the things that change every commit) belongs to Git, reconciled continuously by a GitOps controller. Argo CD is the common one: it watches the repo, diffs the declared state against what's actually running in the cluster, and makes the cluster match. Change the overlay in Git, Argo CD notices and applies it. Flux does the same job if you prefer it. TF for the building, Git for the lights. The rule is simple: if it changes per release, Git owns it; if it changes per quarter, Terraform owns it. Match the tool to the rate of change and the six-hour deploy job that rewrites your infrastructure mid-flight just stops existing.

One more piece falls out of this: tags are markers, not artifacts. The cluster never deploys a tag. The tag is a point-in-time anchor that says "this commit's state was live in prod," useful for audit, for rollback targeting, and for release notes. It records what shipped and when. Nothing reads it to decide what runs.

The canary is the only honest soak

Most teams have a staging or beta environment that exists to "soak" changes before prod. I've shipped behind plenty of them, and here's what I learned the hard way: be honest about what one actually does. It runs the wrong shape of data and the wrong shape of traffic, so passing it only tells you a change survives conditions no real user will ever create. It's a dress rehearsal in an empty theater. The thing that's green in staging breaks the minute it meets production load and production-shaped data, and you find out from a customer.

The only place real confidence accrues is real traffic. So make prod itself the soak, one slice at a time. A new release enters at 1% of traffic and walks itself up: 1 to 5 to 25 to 50 to 100, and at every step it bakes against live signal (error rate, latency, worker health, plus a real-user-monitoring read on whether people are completing flows). Any red, at any stage, rolls back automatically to the last good release.

This is exactly what a progressive-delivery controller like Argo Rollouts exists to do. It owns the canary as a literal state machine: weight traffic up a step, run an analysis check against your metrics, advance if it's clean, roll back on its own if it isn't. You point the analysis at one internal health endpoint that aggregates the signals you care about, and the controller polls it at each gate. (If you're on a service mesh, Flagger covers the same ground.)

The reason this has to be automated is human nature. Nobody has the patience to sit and stare at the 1% dashboard and calmly decide it looks fine. They glance, it looks okay, they wave it through. So don't make them. The canary is a state machine. The signal decides; the train moves. A human only gets paged when something's actually wrong, into context that's already populated.

That same logic extends to the one thing you genuinely can't canary: infrastructure. You can't give 1% of users a new VPC CIDR. But the train can carry the infra change and apply it to dev as part of the car's pipeline, then check that dev came back up green before the change is allowed to merge. If the change broke dev, the merge request doesn't merge. So main and dev are green by construction. A broken state can't reach them, because the proof of merge-worthiness is "dev survived this." Dev becomes the canary for ops.

Stack those layers and they overdetermine the outcome. Real-shape pre-merge tests, an honest readiness probe, the statistical canary, an eyes-on check for the visual breakage that never throws an exception, internal users before external on every flag. Each catches a different class of failure. For a bug to reach prod and stay there, it has to evade all of them. That's what "shippable by default" actually means: not that nothing ever breaks, but that breaking and staying broken requires beating five independent defenses.

Recovery is the other half of courage

Confidence on the way in is only half the deal. The other half is knowing that when something slips through, and it will, you can pull it back before it matters. Recovery, ordered by how fast each lever is:

Flag kill, in milliseconds. If the bad behavior is behind a real backend flag (LaunchDarkly, Unleash, or anything that speaks OpenFeature), flip it off. No deploy, no rebuild. The code's still in prod, just not running. This is why a real flag system matters: without it, a "feature flag" is just a config value read at startup, and flipping it costs you a whole deploy. Real flags decouple deploy from release. You ship the code dark and turn it on when you're ready.
Overlay rollback, in minutes. Re-point the prod overlay at the previous SHA. Same train, opposite direction. The prior image is already built and already proven, so there's nothing to rebuild. Your delivery tooling unwinds the canary back to the last good release on its own.
Hotfix on the same train. Not a separate "emergency" pipeline that's less tested precisely because it's used least. The fix rides the normal train, and because the train is signal-driven, a clean change advances through the canary fast. The urgent path and the everyday path are the same path. That's the point.

When rollback is "git revert" instead of "reprovision the world and pray," shipping stops feeling dangerous. You can build a fleet of agents on top of this, each one watching a signal and taking a bounded action (open a fix MR off a Sentry spike, close a coverage gap, quarantine a flaky test), and you don't have to bet on the agents being right. You bet on the same gates you already trust to ship human work. The train is the safety net for all of it.

Burn the calendar invite

Go back to that Release Window on the calendar. Once main is the train, once every green commit is the artifact, once the canary does the soak that beta only pretended to and rollback is a one-line revert, what is the window even protecting you from? The freeze was insurance against a release you couldn't trust. Build the trust into the track and the insurance is just friction you're still paying premiums on.

I've said before that if you can't release any time, any day, Friday afternoon included, you don't have a courage problem, you have a tracks problem. This is what the tracks look like up close. The courage isn't something you summon in the go/no-go meeting. It's something the system gives you for free, every commit, all day, because it has earned the right to.

So here's the real question: if shipping were genuinely safe at any hour, with no schedule and no war room, how much of your release process would you keep? Whatever survives that cut is the real work. The rest was just managing fear.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

A note on how this was made: I wrote this with Claude Opus 4.8, built from a release architecture I'd designed and spec'd out. I brought the frame, the calls, and the hard-won opinions; Claude did most of the drafting. I'd rather say that than pretend the tool wasn't in the room.

Just use Postgres

Tue, 02 Jun 2026 00:00:00 GMT

Two engineers, a new project, and the architecture already has Redis, a queue, and Elasticsearch before a single user signs up. Postgres does all three. Not forever, but right now at your actual load. At Brandfolder we added Go when Ruby was genuinely hurting us on heavy IO. Not before. Earn the complexity. Wait until the first tool is actually failing you.

Volume Is Free Now. Judgment Is the Whole Game.

Fri, 29 May 2026 00:00:00 GMT

By the time I left, I'd written something like one in five commits in the company's entire history. Eight years, the most of anyone, by a wide margin. For a long time I wore that like a badge. I've since decided it was the clearest symptom of a problem I should have fixed years earlier. And the problem is about to get a lot more common, because the thing I was good at just became free.

There was no hour of the clock that didn't have my name on a commit. Better than a quarter of them landed after 7pm; plenty at 2 and 3 in the morning. I touched nearly every part of the codebase: the app, the libraries, the infrastructure, the build system, the deploy path. If something broke at 11pm, I fixed it before standup. I told myself this was dedication. What it actually was, was a bus factor of one dressed up as a work ethic.

The volume trap

Here's the trap, and I walked straight into it: output feels like value. Commits, lines, features shipped, fires put out at midnight. All of it is visible, countable, and deeply satisfying. So you optimize for it. You become the person who can produce the most, fastest, across the most surface area. And because you can, you do, and the org happily lets you, because why wouldn't it.

The problem is that you've made yourself the load-bearing wall. The more you produce, the more the system depends on you specifically, and the less anyone else needs to understand. Your productivity becomes the org's fragility. I was a fifth of the company's commits not because I was five times better than anyone, but because the system had quietly arranged itself so that the foundation ran through one person. That's not a win. That's a single point of failure with a green contribution graph.

I was a fifth of the company's commits. That's not a trophy. It's a single point of failure with good PR.

And then the floor moved

For my entire career, producing code was the bottleneck. Hiring was about adding hands. Velocity was a headcount problem. The whole industry was organized around the scarcity of people who could write working software fast.

That scarcity is gone. An agent will write you a few dozen well-formatted, plausible commits a day, all day, without coffee or ego. The thing I spent a decade getting elite at, producing a lot of correct code quickly, is now something you rent by the token. Volume has been commoditized. If your value, or your company's, is measured in output, you're now competing with a machine that out-produces you and never sleeps, and you will lose that race.

Generation is now the cheap part.

When anyone can produce the code, producing the code stops being where the value is. The bottleneck moved. Most orgs haven't noticed yet.

Judgment is the moat

What an agent can't do, yet, and I'd argue structurally, is hold the map. It can't tell you which problem is a "move fast and break things" problem and which one is a "slow down, this is load-bearing" problem. It goes fast at both. It doesn't know which clean-looking diff is quietly wrong, because it has no model of why the system is the way it is. It will confidently pave a road straight off a cliff and report success.

That discernment (knowing what's worth building, what's dangerous to touch, what "done" actually means, when the plausible answer is wrong) is judgment, and judgment is the thing that didn't get commoditized. It got more valuable, because now it's the scarce input. The engineer's job just moved from producing to directing: fewer people, each steering a fleet of tireless producers, each responsible not for how much they typed but for what they caught and what they refused to ship.

This is the part people get wrong about AI and engineering. It doesn't replace the person with the map. It's a force multiplier on judgment, and a multiplier is only as good as what you multiply. Multiply great judgment by a fleet of agents and you get something genuinely new. Multiply no judgment by the same fleet and you get a beautifully formatted disaster, faster. Anything times zero is still zero.

The train can only go as fast as its tracks

I've said this so many times my team could finish the sentence: the train can only move as fast as the tracks it's built on. In the agent era it's not a nice metaphor anymore, it's the whole strategy. Agents are a faster engine. Dramatically faster. But a faster engine on bad track doesn't get you there sooner. It derails sooner.

The faster you can generate code, the more load your foundation has to bear: your tests, your CI, your deploy path, your guardrails, your ability to verify that a thing actually works. Cheap generation doesn't reduce the need for a strong foundation. It's the single biggest argument for one. The companies that win the next decade won't be the ones generating the most code. They'll be the ones whose tracks can survive the new speed.

How it actually fails: not with a bang

The failure mode I'd warn every leader about isn't dramatic. Nobody dies in an outage. You die by entropy. The agents keep the lights on. The demos work. The line on the dashboard stays up. But quarter over quarter the code drifts further from anyone's actual understanding. Decisions get made by whoever's least wrong in the room. And one day a "simple" change takes three weeks, because nobody alive knows why the thing was built the way it was, and the only one who did was a person (or a model) nobody bothered to ask the right questions of.

It's invisible right up until it's expensive. Which is exactly why it's the one to design against now, while it's cheap.

So what do you actually do

If output is commoditized and judgment is the moat, the work reorganizes around that. In priority order, not on a timeline, because the sequence matters more than the calendar:

Get the map out of people's heads. The cross-system context that lives in one senior engineer's skull is your biggest risk and your most valuable asset. Externalize it: architecture docs, decision records, runbooks, written intent. The infrastructure you need to survive losing someone is the infrastructure that lets agents be useful in the first place. Build it before you need it.
Make agents the volume layer and humans the judgment layer — on purpose. Stop measuring engineers on output; the machine wins that and the metric is now meaningless. Measure them on what they catch, what they decompose, and what they have the taste to reject.
Kill every bus-factor-of-one. No system gets exactly one person who understands it. Two humans know it, or it's documented well enough that a mid-level engineer and an agent can own it together. That, precisely, is what surviving any departure means.
Overinvest in the tracks. Tests, CI, deploy, verification. This is the line item everyone underfunds and the one that decides whether speed helps you or kills you. A faster engine makes a strong foundation more important, not less.
Make verification the core competency. When generation is free, the scarce, valuable skill is proof: does this actually work, end to end, the way a real user will hit it? Not "the tests are green." Knowing what's true is the whole job now.

The optimistic part, and I mean it

This sounds like a eulogy for the engineer. It's the opposite. This era is more forgiving than the one it replaced, if you orient to it correctly. A small, judgment-dense team pointing agents at the right problems will outrun a large team of people typing, every time. The winners won't be the orgs with the most engineers. They'll be the ones that figured out the job changed: from how much you can produce to how well you can direct, verify, and decide.

And the personal version of the same truth is the one I keep coming back to: judgment travels with the person who has it. The map lives in a head, not a repository. Whatever I build next, that's the asset I'm bringing, and unlike a fifth of someone's commit history, it's portable.

I spent eight years proving I could be the engine. The lesson I'm taking forward is that the engine was never the hard part. Build the tracks.

Jason Waldrip has spent his career leading engineering at consumer-scale software companies. He writes about engineering leadership, infrastructure, and building in the age of AI agents.

A note

Wed, 27 May 2026 00:00:00 GMT

Drove to the office this morning, realized it was in shambles, and turned right back around to work from home. Some days the best architecture decision is the commute you don't make.

"Add AI" is not a roadmap item

Tue, 26 May 2026 00:00:00 GMT

Watching teams bolt a chatbot onto a product with no idea what problem it solves. AI is a tool, not a strategy. If you can't say what gets faster or cheaper or better, you're shipping a press release.

The deploy path is a product

Thu, 21 May 2026 00:00:00 GMT

If you can't release at any time, 24/7, Friday afternoon included, you don't have a culture problem. You have a tracks problem. Fix the rails and the courage shows up on its own.