One of my reports sent me Addy Osmani's post on software factories. I finished it and got stuck on "human in the loop."
It is a handy phrase because it covers a lot of ground. An engineer might work through the plan, tell the agent which boundaries to leave alone, and review a change they understand. Another engineer might get 2,000 lines of finished code and six more PRs waiting. We can say both were involved, although the second person did not get much of a chance.
The queue does not go away
Businesses have good reasons to want the speed. I have seen careful engineering turn into weeks of discussion while the thing a customer needed sat untouched. Nobody buys our software because the architecture meeting went well.
An agent can take a bug, trace the call path, try a fix, and learn from the failed test while I am stuck in a meeting. I want the team to use that time. Asking an engineer to spend the same afternoon doing each step by hand would be silly.
The catch shows up once those changes start arriving faster than the team can read them. We saved time during implementation and gave some of it back to the reviewer, who now has to figure out why the agent touched twenty files and whether all twenty belonged in the change.
More PRs keep appearing, so the team gets faster at clearing them. Better tooling and smaller diffs help. On a long afternoon, the green checks may look close enough because everyone would like to go home.
Please let CI check the types
I do not need a person re-running the compiler in their head. The type check and test suite should finish before review, along with the security scans we trust. If the payments package needs an odd command that everyone forgets, put the command in CI and be done with it.
Those checks give the reviewer time for the harder part. A passing test tells me the code behaved the way the test expected. I still have to ask whether we captured the right behavior in the first place.
An agent can work against a weak test until it gets a green build. It can add another weak test too. Everything agrees at that point: the code, the tests, and the agent's explanation of both. We may have left out the rule that matters to the customer.
I have caught myself relaxing when I see a clean row of checks, so I understand the pull. The PR looks finished. It took the agent a few minutes to write and will take me much longer to read. That mismatch can make review feel like paperwork even when the decision inside the diff deserves attention.
I would rather argue with the plan
A bad assumption is cheap while it lives in a short plan. Once the agent repeats it through controllers, tests, and a migration, changing direction becomes a project of its own.
For a change with some weight to it, I want the engineer to explain the problem and where they intend to make the cut. I also want to know what behavior has to survive. This might take a page, or ten minutes with the person who remembers why the current code looks wrong but has worked for four years. We can disagree then and still do something useful with the disagreement.
I will read the finished change. I am not interested in replacing code review with plan review and trusting the agent with the rest. I want the reviewer to reach the diff knowing what the code is supposed to do, instead of searching through it for the decision. Two thousand lines are a rotten place to discover the architecture.
Copy can move faster than billing
I am comfortable giving an agent room when I can check the result without much interpretation. Replace a deprecated API in one component, run the focused test, and keep the diff inside that component. I do not need to turn that into an event.
A billing change deserves more time. Authentication does too. I want someone involved before an agent writes a data migration that could leave half the records in one shape and half in another. That person needs to understand how we undo it and which production signal they will watch after release.
This sounds obvious when the examples sit next to each other. The line gets messy in a real backlog. A small UI change can affect a public contract. A dependency bump can change authentication behavior. Teams need their own rules, written down with enough detail that review does not depend on who picked up the PR or how close they are to vacation.
I cannot ask someone to own a forced yes
An engineer cannot own a release when the deadline has removed their option to stop it. They can click Approve, but the project made the decision before the review began.
Managers create this problem more often than we admit. If I ask a team to double its output with agents, leave the review queue alone, and keep praising velocity, I have told them what I value. A rubber stamp should not surprise me later.
The engineer still owns the call they make. I do not want accountability watered down until nobody owns anything. I do want to look at the time they had, the context they received, and whether saying no would have carried a penalty. The last name in the Git history cannot explain all of that.
After a failure, I care about the gate that missed it. The reviewer may have made a bad call, or our tests may have given them confidence they should not have had. We need enough honesty to discuss both without turning the incident into a hunt for the person who clicked last.
I care what the team can explain next month
PR count tells me how much work entered the system. I also watch what comes back: rollbacks, incidents, and changes that take days to diagnose because nobody recognizes the code.
That last part worries me more with AI. A codebase can keep working while the team's understanding of it gets thinner. The tests stay green right up until the failure they did not cover, and then the engineer on call has to learn the generated architecture at 2 a.m.
I will let some agent loops run without waiting for me. I will keep a person close to the work when the blast radius grows or when a machine cannot prove the result. There is no single setting for the whole factory, which is inconvenient but seems to be the job.
Before I count an approval as human oversight, I want to know that the reviewer understood the change and could have stopped it. Their name in the Git history does not answer either question.