Skip to content
Khaled.dev
← Back to work

Solo engineer — runtime, tool protocol, eval harness · 2025

AI Agent Harness

A typed, tool-calling agent runtime with deterministic replay, budget guards, and pluggable skills — the substrate my other agents run on.

AI Agent Harness

Stack

  • TypeScript
  • Anthropic SDK
  • Zod
  • Bun
  • SQLite

Links

Overview

What it is

A from-scratch agent harness built around a strict tool-call loop: every step is logged as a structured event, every tool is a typed function with a JSON schema, and every run can be replayed deterministically from its event log for debugging.

It enforces token and wall-clock budgets per run, supports parallel tool dispatch when calls are independent, and degrades gracefully when a model or tool is unavailable. Skills are loaded from a manifest so capabilities can be added without touching the core loop.

The problem

What needed solving

Debugging agents is miserable: a run fails on step 14, you change one prompt, and you can never reproduce the exact path that broke. Costs are unpredictable too — a single runaway loop can burn a day's token budget before anyone notices.

I needed a substrate where every run was reproducible, every cost was bounded, and new capabilities could be added without destabilizing the core loop.

The approach

How I built it

I modeled every step as a structured event written to an append-only log. Because the model and tools are the only sources of nondeterminism and both are captured, any run can be replayed deterministically straight from its log.

Tools are typed functions with JSON schemas; independent calls dispatch in parallel. Per-run token and wall-clock budgets are enforced by the loop itself, and skills load from a manifest so capabilities are additive.

The outcome

Results

0
Mean tools per run
0%
Replay reproducibility
0
Budget overruns since launch
  • Reduced the time to root-cause a failed agent run from hours of re-prompting to a single deterministic replay.
  • Budget guards turned 'why did this run cost $40' incidents into a hard ceiling that never gets crossed.

Highlights

4 things worth calling out

Deterministic replay from a structured event log

Per-run token and wall-clock budget enforcement

Parallel dispatch of independent tool calls

Manifest-loaded skills — add capabilities without editing the core loop

Gallery

A closer look

AI Agent Harness screenshot