What happened, including the bad weeks

The situation

Tilltap sells kitchenware online. Four people: two engineers, a designer who also does support, and a founder who also does everything else. The storefront is one repository, the admin back end is another, and both deploy to a single box over SSH by way of a shell script that one engineer wrote and one engineer understands.

Their problem was not writing code. It was that every change had to pass through the same person, and that person had opinions about Fridays. Code review happened when there was time. There was rarely time. The last security pass anyone could remember was a penetration test a customer had asked for.

What they filed

The first request was deliberately unglamorous: "Refunds partially applied to an order leave the order total wrong on the customer's invoice." A type, a priority, one sentence. The project sat at Manual autonomy, which meant nothing at all would run until a human pressed a button.

What the plan looked like

The Project Manager read both connected repositories and came back with five tasks and a dependency graph: a Database Engineer task to add a migration for the adjusted totals, two Backend Developer tasks that could not start until the migration existed, a Frontend Developer task for the invoice template, and a QA Engineer task at the end that depended on all four.

The founder read the plan summary. He is not an engineer and he understood it, which was the point of the exercise. He approved it. The four buildable tasks dispatched, each onto its own branch in its own git worktree, and each came back with a structured report — summary, files modified, risks, assumptions, recommendations, blockers.

The Code Reviewer rejected one of them. The backend task had introduced a new helper for currency rounding when the repository already had one, three directories away. The task was re-run with that noted, and merged on the second pass.

What went wrong

Two things, and the second one was their fault rather than ours.

A deploy failed at 2am. Six weeks in, at Supervised autonomy, a change to the order-search query shipped and the health check failed. The rollback command ran, the last healthy commit was restored, and an investigation opened with read-only access. By morning there was a written cause — the query relied on an index that existed on the engineer's machine and not on the server — and a follow-up request already filed with the fix. Nobody was woken. But the box did serve errors for the ninety seconds between the deploy and the health check failing, and their customers can count.

They moved to Autonomous too early, on the wrong project. They set the admin back end to Autonomous in week seven because the storefront had been calm for a month. The admin repository had a schema migration in its queue that the founder would very much have liked to read first. Nothing broke. He did not enjoy it. They dropped that project back to Supervised, left the storefront at Autonomous, and put requires_review on the Database Engineer role permanently. That flag is still there.

What changed

Review stopped being a thing they intended to do. Every merged change now carries a reviewer's verdict and a security note, because the gate is not optional and nobody has to be free on a Thursday for it to happen.

The founder's summary was less about speed than about the shape of the week: the deployment process is no longer stored in one engineer's head, and that engineer took a holiday in month three without leaving a phone number.

The situation

Northwind Freight moves pallets around northern England. Their dispatch system — routing, driver assignment, proof of delivery — was built by a contractor who finished in 2022 and did not stay. It runs. It has no tests worth the name, two directories that appear to do the same job, and a deployment script with a comment in it reading # do not run twice.

Nine engineers work there, none of whom wrote it, all of whom are slightly afraid of it. It had reached the state where a change request was answered with an estimate and a facial expression.

What they filed

"Drivers assigned to a cancelled run still receive the pickup notification." A bug, high priority, filed by an engineering lead who already knew roughly where the fault was and wanted to see whether the plan would find it.

What the plan looked like

It did not find it. Not on the first attempt.

The repository contains both dispatch/ and dispatch_v2/. The first is deprecated and still present; the second is what actually runs. The Project Manager planned against dispatch/, produced three tasks that were internally coherent and entirely irrelevant, and the engineering lead rejected the plan.

She then did the thing that fixed it permanently: she edited the project's Architect role contract to state, in two sentences, that dispatch/ is dead and dispatch_v2/ is live. The re-plan targeted the right module. No later plan has made the same mistake, because the instruction is now a file in their repository rather than a fact in somebody's head.

What went wrong

The security pass found something nobody wanted to find. In the first month, the Security Engineer flagged three findings across the requests it touched. One was a duplicated CSRF check, one was a missing authorisation condition on an internal endpoint, and one was a card number written into application logs by a payment-callback handler that had been running since 2022. That last one was not in scope for any request. It was found because the security role reads the surrounding code, and it produced a request of its own the same day.

Finding it was good. Having it there for three years was not, and the study would be dishonest without saying so.

An investigation stalled on a permission. A deployment failed its health check in week three, rolled back correctly, and opened an investigation. The investigation then got nowhere for forty minutes, because their diagnostics command needed sudo and the investigation had read_only access, which is the default and which they had not looked at. They rewrote the diagnostics command to work without elevation. The next investigation completed in four minutes.

Runs were burned on the wrong module. The rejected first plan still cost agent runs. Planning is not free, and a repository with two directories that look alike will cost you a plan before it costs you anything worse.

What changed

Three role contracts are now Northwind-specific — the Architect, the Code Reviewer and the Security Engineer — and they encode the things a new starter used to learn by breaking something. The engineering lead describes the contracts, not the agents, as the thing they actually bought.

The dispatch system is still the dispatch system. But work on it now arrives as a plan they can argue with, on branches they can throw away, behind a review gate that does not get skipped when the week gets busy.