Sixty hours beneath the abstractions

The AI systems I work with every day barely existed two years ago, and the practice around them changes monthly. So this summer I ran a six-week deep dive to check my working knowledge against the frontier: an agentic engineering course taught by an AWS Applied AI alum and a current member of OpenAI’s Codex team. Lectures, weekly build assignments, a capstone, done in the margins of a COO’s day (and night) job.

The fee was the cheapest part. The real price was the late nights and weekends, sixty hours of them, and a year of my own practice put under examination.

For the past year I’ve been building AI into how I work: BI pipelines, automations, agents doing real jobs, and shared services that several of my own tools run on. The meter says forty million tokens of delivered work this year, five billion counting everything the tooling read along the way.

Built the disciplined way: specs before code, data models designed up front, QA on everything that shipped. Coding agents met me at that level. I bring the spec and the architecture, the harness writes and runs the code, working software comes out.

Which is exactly the problem. A year of building at that depth never once required me to understand the intelligence itself: why a search misses, or how an agent decides it’s done. The abstractions keep the one layer that matters out of sight. And that layer has started carrying money. I sign off on this technology at work, and I underwrite it as an investor. I could build on these systems. I wanted to judge them from underneath: which parts are genuinely hard, and which are just sold that way.

So I went underneath. Six weeks, one system at a time, each instrumented, traced and evaluated like something headed for production: a query router; a tool-calling agent loop written by hand, to see the machinery bare; retrieval over a 450-document corpus; a multi-agent research system, a manager delegating to specialist agents; evaluation suites scored against gold answers I worked out by hand; and finally an agent granted real access — a shell, persistent memory, external tools over MCP — inside boundaries I designed.

Two things from those sixty hours, out of dozens.

The first came in week one. I built a system that sorts incoming questions into four categories and routes each to different instructions, the same pattern that routes support tickets or payment requests and refunds. The engineering took under an hour. Nearly everything else went on a question that turned out not to be technical at all: deciding where a fact ends and a definition begins. Both readings of the same question were defensible. One had to be written down, because every downstream measurement of whether the system “worked” was defined against that choice.

I’d been calling this a technical build. The load-bearing decision in it was policy. Nobody in most org charts owns decisions like that one, and the systems ship anyway.

The second came in the final week, and it’s the one I’d put in front of a board. I built an agent that evaluates ML experiment runs against a shipping policy: hard constraints, tradeoff weights, the kind of gate a real platform team runs. Before testing it, I did the check by hand: nine candidate runs against four thresholds, worked out on a spreadsheet, so the right answers existed before the agent’s. Then I ran it.

The recommendation came back beautifully. Correct format, every claim citing an evidence file, caveats included, confident. And the central numbers in it were invented. The agent had fabricated the latency figures — plausible values, internally consistent, shaped to justify its pick — because my own pipeline had a gap: the summary it read reported values for the runs that failed the gates and a bare “pass” for the runs that mattered. It had rich evidence about the rejects and a void where the winners should be, and it filled the void.

The report looked better than an honest one would have. I only caught it because the hand-checked answers existed first.

I traced the fault, fixed the pipeline, ran it again. The agent then read the real numbers, applied a judgment rule I’d written in plain English, and returned the recommendation my gold labels said was right, caveats and all. The rule was never the problem. What the model was given to read was.

Multiply that finding by every AI quality number currently arriving in board packs and diligence rooms. Confident, fluent output, whether or not it’s right. And the difference does not show on the surface.

Someone has to decide what “correct” means, what evidence a recommendation must carry, which failures are unacceptable, and when a system may act on its own. Those are operating decisions. In most companies right now they’re being made by default, by nobody.

For twenty years, “technical enough” for an operating executive meant reading a dashboard. I think that bar has moved.

I’m not describing a personal quirk. Tobi Lütke wrote to all of Shopify that reflexive AI usage is now a baseline expectation, and that teams asking for headcount must first show why AI can’t do the job. Since Paul Graham named founder mode, going deep is respectable again, and coding agents have made it cheap: what used to cost a career detour now costs evenings. But the deeper shift is in method, not access. For my whole career, executives have worked from represented evidence — the PRD, the deck, the report someone else stands behind. With AI, the represented version and the real one can differ in ways that don’t show on the surface. I’ve stopped accepting the representation. The homework is what running the experiment yourself looks like.

I don’t mean executives need to become engineers: sixty hours didn’t make me one. What the hours bought is narrower: I can work beneath the abstractions. I know what an evaluation actually proves, what a trace shows, what questions separate a governed system from a polished one, and what good answers sound like.

The commitment is the point. Watching a lecture about evaluation teaches you the vocabulary. Scoring your own system against answers you worked out by hand, and catching it fabricating — that teaches you the reflex. The reflex is what you carry into the next vendor meeting.

What’s changed: I read traces before I read roadmaps. I ask what the judge’s evaluation looks like, because the judge is a model too. I ask what happens at the exit condition. And when a system hands me something confident and beautifully formatted, I now treat the polish as a reason to check, because I’ve watched polish and fabrication arrive in the same document.

What follows is a series that opens the machinery: the decisions I now think operators should own.

COO in energy, previously Twitter and Stripe. I build what I write about.