Caleb Chong

Writing / 2026

Stop batching the small stuff

How AI-assisted development helps me take small, reversible product changes to a review-ready PR, and could give other non-engineers the same path.

NoteProduct ManagerAIProduct deliveryTesting

The ticket is bigger than the change

A recurring pattern in internal operations tooling: someone spots a small usability gap. One of mine: custom rate cards had accumulated so many configurable dimensions that most cards used only a handful, and the sparse grid was hard to read for both the ops teams configuring them and the commercial teams viewing them. The fix was a way to hide rarely used dimensions. The code change is small. The ticket, planning slot, handoff, and context switch cost more than the change itself, so it waits until enough papercuts accumulate into a cleanup batch.

Ticket economics drive this more than whether a PM can write code. Batching is rational when every change must travel through a scarce engineering queue, but users live with small defects because none of them is worth a sprint on its own.

AI-assisted development changes the cost of bounded work. I can write a scoped brief, implement through an AI coding agent, try the change in an isolated environment, test it, and open a PR. That loop currently produces about one internal-tooling PR a week for me, covering copy, layout, interactions, and minor bugs that would previously have waited for a batch. It has also shipped more than ten fixes to live documentation owned by other teams, where outdated references were misleading users and the AI assistants instructed from the same pages.

On well-bounded UI work, I can often get to review before the issue would have received a planning slot. The reviewer receives a small implementation with test evidence instead of starting from a ticket.

That suggests a broader team model. Designers and other non-engineers who understand a workflow could use the same path, provided the blast radius is small and the underlying platform guardrails already hold.

The loop that keeps it safe

  1. Scope it. Name one user-visible outcome, the non-goals, and what must not break.
  2. Plan the tests. Draft the test plan with the scope, then have a model generate a second plan independently of my draft. A fresh set of eyes, even a synthetic one, thinks to test things the author does not: edge cases, likely regressions, data assumptions, and checks that still need a person.
  3. Implement it. Use the same checkout that will produce the PR and prove the change in an isolated environment.
  4. Run and record. Let the agent cover repeatable paths and preserve screenshots, network results, and structured evidence.
  5. Review it. Click through the real workflow, inspect the ambiguous states, and open the PR with the evidence attached.

I built an agent-usable validation path around this loop. It runs the exact branch under review, on my laptop or a cloud instance, executes the test plan end to end: working examples, then regression checks on adjacent flows, and records screenshots and session recordings of each expected behaviour passing. Running it against my own new cross-domain functionality has caught a couple of small defects that a green happy path missed.

The agent stops before any persistent change unless the test explicitly allows it. It handles setup, interaction, assertions, and evidence capture, leaving the decision outside the automation.

What makes the loop fast

A disposable environment. Some changes run on a laptop; others need a cloud environment or a more elaborate setup. Either works if the environment belongs to the change, is isolated from other work, and can be refreshed quickly. A shared staging stack that drifts with every branch slows the feedback loop and makes the evidence hard to trust.

When I control the environment, I can see the change, break it, and update it without joining another queue. A patch I cannot exercise myself teaches me nothing about whether the product should change.

Proximity to the user. Internal operations tooling is the cleanest case because I sit close to the people doing the work. I can see where the workflow breaks, put a change in front of them, and watch them try the job again. An empty state that tells an operator what to do next or a label that finally uses their language is modest work. It is also the reason to fix papercuts when they appear.

Local tests cannot tell me whether the workflow is right; sitting near the people who run it can.

Pace layers decide ownership

I am borrowing this framing from “Pace Layers and AI Integration”, which applies Stewart Brand’s idea of pace layers to software. AI works best where change is frequent, failures are contained, and outcomes are verifiable.

Most of my self-serve iterations sit near the fast end: UI, copy, layout, and the last mile of a workflow. Platform structures such as object models, permissions, and financial invariants often move more slowly. The right pace follows blast radius, recovery time, dependencies, and how confidently the result can be verified, wherever the code sits.

For a product team, this becomes an ownership rule. Stable lower layers make faster layers cheap to touch. If the guardrails are vague, a UI tweak may be a hidden systems change and needs engineering involvement from the start.

One higher-risk workflow redesign made the boundary clear. Its proposed interface would capture and preview an operator’s intended outcome, while a slower service would validate and commit the result under existing controls. An AI coding agent helped implement a shared test corpus: the interface and service had to produce the same result from the same rule. Engineering owns the architecture and release.

I used the same tools differently for a separate migration. An AI coding agent built a portable simulator; I tested and corrected it against controlled configuration and usage inputs. The replay showed that configurations which looked equivalent could behave differently after migration. AI made that breadth of exploration affordable, while sign-off on the migration itself stayed with the people accountable for it.

These larger projects sit outside the papercut lane. The tools still accelerate analysis and implementation, but the review and ownership expand with the risk.

Where self-serve stops

Engineering review remains part of the cost. The bet is that reviewing a bounded, tested diff is cheaper than staffing the same change from a ticket. If the team starts rubber-stamping, the workflow has failed.

A green AI test is evidence, not a verdict. Models miss visual judgement, unusual data, and changes that are technically correct but wrong for the user. Human review stays focused on those.

Scope creep is the practical stop signal. A migration, a new abstraction, unclear data integrity, or a debate about product behaviour takes the work out of the self-serve lane. The same applies when an isolated environment cannot prove the change or when important invariants live only in tribal knowledge.

The test-plan prompt

This is the generic version of the prompt I use:

Change brief:
- What should now be true for the user
- What must not change
- Surfaces, roles, and data states involved
- How I will know this is done

Write a test plan with:
- The happy path
- Edge cases: empty, error, permission, and stale-data states
- Likely regressions in adjacent UI
- A low, medium, or high risk rating for each case
- Test-data assumptions
- Cases an LLM can execute and cases that need a person
- The expected result for each case

Start with one papercut

Pick one reversible UI issue that would otherwise wait for a cleanup batch. Run the loop once. Compare the engineering review time and rework with the cost of waiting for a sprint slot.

When this works, the sparse rate card that opened the article reaches a reviewer as a small, tested diff. Engineering still owns shipping, with that diff as its starting point.