Agentic Software Engineering

About Me

It Started as Next-Token Prediction

The model picks the most likely next token and repeats. Pattern matching over enormous text corpora. One token ≈ ¾ of a word.

Add Thinking: Reason Before Answering

Chain-of-thought prompting (and o1-style reasoning) gives the model space to work through a problem before committing to an answer. This changed things substantially.

Add Tool Calls: Act in the World

The model can pause mid-response, call a tool, and continue from where it left off. Text in, real actions out.

Harness → Provider: The Loop

The harness (pi, Claude Code…) orchestrates a tight request/response loop with the model provider. The model never touches your machine — it only describes what to do.

The Models Finally Got Good

We went from autocomplete to reasoning agents capable of multi-step work. A useful measure: feedback loop length — how long an agent runs before needing your input.

• Longer feedback loop = more complex tasks the agent can handle
• We went from autocompleting a line to 30-minute unsupervised tasks

Feedback Loop Length Is Everything

METR tracks how long a model can work unsupervised on a research task — doubling roughly every 7 months. (Kwa et al. 2025, metr.org)

Planning Before Code

Agree on what to build before any code is written. OpenSpec is a lightweight framework that structures this as a folder of Markdown artifacts. GitHub (41k ⭐) →

The Bottleneck Moved

Agents cut coding time roughly in half, which surfaced the next constraint. Reviewing and validating is now the slow part.

The Validation Bottleneck

Writing code got faster. Reviewing it didn't. The constraint shifted but didn't disappear.

• We can produce code faster than we can validate it
• Macroscope on every PR — automated sanity checks, style, logic issues
• But automated review isn't enough — humans are still the quality gate
• The review burden is now the constraint on velocity

Macroscope — Example Review

A real Macroscope comment on a PR — logic analysis, not just style.

Agentic Exploratory Testing

To speed up validation, we're using agents for exploratory testing — beyond the traditional testing pyramid.

• Agent reads the PR diff and documentation
• Agent reasons about which UI components are affected
• Agent scripts Playwright on the fly to test those paths
• Agent "looks" at screenshots to verify behavior

Exploratory Testing — Example Output

A GitHub Issue filed autonomously by the exploratory testing agent — with a full reproduction journey.

Dev Meets Ops

A production error becomes a GitHub Issue becomes a PR — mostly without a human kicking it off.

Triage Agent — Example Output

The actual GitHub Issue filed autonomously — root cause, affected versions, and a fix suggestion, all included.

The GitHub Switchboard

GitHub is the coordination layer. Agents plug into every stage — but humans set direction and hold the quality gate.

You Are the Orchestration Layer

Each agent works through its own think→act→observe loop. You move between them: kick off a task, check progress, correct course.

Context Is the New Code

Model quality is fixed — what you can control is what you put in the context window.

• Markdown "Skills" — reusable capabilities. Example: analyze_user_feedback.md knows how to summarize thousands of support tickets
• Shared repos with helper scripts that fetch from BigQuery, telemetry, web search — all into the context window
• PMs use Cursor + agents for competitive analysis, feature prioritization — not just engineers anymore

Model Economics

Match the model to the task. Prices via OpenRouter (per million tokens, input / output).

The Death of "The Zone"

Working this way has real downsides worth being honest about.

• No more 4-hour deep dives — constant context switching
• Pattern: assign Agent A a 30-min task → switch → assign Agent B → review A → correct → loop
• More code review, less flow state
• Skill atrophy is real — you lose something. The ability to write a module from scratch, the muscle memory of debugging without hints. Leaning on agents too hard and you'll notice the gap when the model is wrong and you can't tell.
• As a tech lead, it lets me get closer to the code — but some engineers will struggle with the shift

Building the Wrong Things

When building anything takes hours instead of weeks, the cost of a bad idea shrinks — and you build more bad ideas.

• Because it's easy to build, we build features that don't solve real problems
• AI-written code still needs human-led maintenance — you still own the debt
• 20% more productivity can mean 40% more technical debt if you're not careful
• Code sprawl is real — more surface area for bugs, more to understand

The Reality Check

The bottleneck moved — from coding to validation. That doesn't mean it doesn't work. It means it's more complicated than just pumping out code faster. METR RCT: experienced devs were 19% slower with AI →

Code Is the Universal Language

Agents work best when problems are expressed in code. That makes coding a more central skill, not a less relevant one.

• Code is what agents speak fluently. Turn everything into code — config, specs, infra, tests
• More people will code (at least a little) because the barrier to entry dropped
• Jevons paradox: making development cheaper increases demand for software — and therefore for more engineers, not fewer
• We need people who understand systems: judgment, architecture, context — that doesn't automate away

The Self-Healing System

When observability and development run on the same agent loop, the SDLC closes itself.