AI Pitlane

Why a pitlane

The pitlane is the one stretch of a racetrack with a speed limiter. It is where the fastest machinery on the circuit gets worked on — wheels off, parts swapped, sent back out — under rules nobody argues with: a hard speed cap, defined lanes, and a crew chief who decides when the car moves.

That is the design brief. Coding agents are fast machinery, and most orchestration tools respond by making them more autonomous — daemons, retries, auto-escalation. AI Pitlane goes the other way. Pipelines are explicit graphs. Every shipped graph launches from a button, most cap paid spend at $0.00, and nothing runs unattended unless a human turns three separate keys. The interesting part is that slowing the lane down did not make it less capable — it made the pipeline itself something an AI can safely be handed the keys to (§ AI drives it).

Concretely, it is a local-first Python package: a stdlib HTTP dashboard on 127.0.0.1:9191 where you draw and launch graphs, a one-shot queue drainer for terminals and timers, and a stdio MCP server exposing the same surface to AI clients. Two runtime dependencies (watchfiles, requests); the repo's own docs call adding a third "a decision, not a convenience."

The graph is the program

A pipeline is a validated DAG stored as plain JSON. A node is {id, role, kind, config}; an edge says what feeds what. The registry defines 8 roles and roughly 80 plugin kinds — supervisors, implementers, reviewers, scanners, routers, halt detectors, budget enforcers, and DAG primitives like foreach and collect.

The load-bearing decision is that the backend behind a node is a field, not an architecture. An implementer node might be kind: claude_cli today and kind: gemma_local (a Gemma on a local vLLM server) tomorrow; the swap never touches an edge. Local models, Claude, Codex, and OpenAI or Cerebras specialists sit in the same graph as peers.

Graphs live at ai-queue/operations/projects/<name>.graph.json, and every save auto-commits, so the pipeline's history is git history. Editing the file is equivalent to editing in the browser. Two rules keep the canvas honest:

• Every node executes. There is no label or note kind — an orphan node has in-degree zero, runs early, and costs whatever its backend costs. Notes go in annotations; architecture drawings go in design docs, kept off the executable canvas on purpose.
• Gating is per node, not per lane. A router gating its first hop does not gate the rest of that branch. The wiring check catches ungated routers, unreachable lanes and missing required config, and links each finding to the offending node.

What a run actually is

Preview launch lists nodes, warnings, scan targets and the target repo before anything moves. ▶ Launch then runs the graph in a transient systemd-run --user unit, so Abort kills the entire process tree — including any vendor CLI still billing underneath it. On the canvas, nodes light up as they go: pulsing outline running, solid green complete, red failed, grey skipped because a router chose another branch.

Everything that happened is written to ai-queue/operations/mode_runs/<run_id>/state.json with per-node outputs. The dashboard only reads that file — there is no hidden state to drift from it, and the repo's standing rule is that nobody edits that directory by hand. It is the record of what happened, the same way a timing sheet is.

The speed limiter

Budgets are enforced, not observed. Every project ships with its paid caps at zero; free, local plugins run under those caps and paid ones are refused — recorded as the node's error — until a human raises the cap. Two enforcer kinds ship:

• litellm — a daily USD cap checked against a local LiteLLM proxy's running spend. The failure mode is the right way around: if the proxy is down and the spend is unknowable, paid calls are refused, not allowed. Free calls never consult it.
• claude_counter — a blunt dispatches-per-day cap for CLI vendors that don't meter cleanly.

Both run in hard or soft mode; either refuses the call at the cap, and soft additionally names a configured fallback in the error for a router — or a human — to act on. Nothing is re-dispatched automatically.

Giving it work

The working loop is queue-based. A work-order is a markdown file with frontmatter — slice_id, status, priority, depends_on — plus a goal, acceptance criteria, and a files_in list whose paths are inlined from the target repo so the agent sees real code, not a summary. Orders are born draft; nothing is eligible until a human flips one to pending.

Point a project at any git repo you are willing to git reset --hard (an implementer will commit there), drop an order in its queue directory, and either press Launch or run the drainer:

.venv/bin/python -m orchestrator.drain --dry-run   # what would run; changes nothing
.venv/bin/python -m orchestrator.drain             # run ONE eligible slice, then exit
.venv/bin/python -m orchestrator.drain --reap      # mark provably-dead runs failed; requeues nothing

drain takes the coordinator lock, so it cannot race the dashboard; if the lock is held it exits 0 and does nothing. The implementer commits to the target repo and the loop stops at a draft PR — review stays human. poll_issues closes the loop from the other end, turning labeled GitHub issues into work-orders.

One honest gap, filed and documented rather than papered over: there is no crash recovery. An interrupted run stays running until --reap marks it failed, and its work-order needs a human to set it back to pending. In a tool this shaped, a requeue that happens by itself would be a bug.

When no kind fits, write one

The registry's ~80 kinds are each tied to a purpose — a scanner wraps a specific tool, an implementer drives a specific agent. The escape hatch is primitive.python_script: the blank node. Drag it onto the canvas and write Python straight into its config. The contract is three names — inputs (a dict of upstream node id → that node's output) and config are already in scope; assign result to whatever downstream nodes should see:

# `inputs` and `config` are already in scope.
result = {"ok": all(r.get("passed") for r in inputs.values())}

The snippet runs in its own subprocess interpreter with a real timeout_s (default 60), nothing auto-imported beyond the standard library, and no orchestrator internals reachable — so a snippet that loops forever or calls sys.exit has a blast radius of one node, not the run. It plays the role n8n's Code node and ComfyUI's custom nodes play elsewhere; the shipped testbed graph uses one as its audit gate, deciding in plain Python whether a slice's checks actually passed.

When a snippet grows into something you'll reuse, promote it to a real plugin. The dashboard ships a Monaco-based plugin editor at /editor: a New Plugin form scaffolds a registered (role, kind) skeleton into the plugin pack, lifecycle badges track it from scaffold-only to implemented, and the toolbar runs Validate, Smoke Test and Reload Registries so a new kind shows up in the palette without restarting the server. The same loop is exposed over MCP (scaffold_plugin → validate_plugin → test_plugin → reload_plugin_pack) — which matters in the next section.

Have an AI talk right to it

The browser is one client. The other is a stdio MCP server — line-delimited JSON-RPC over stdin/stdout — exposing 124 tools that cover the same surface as the dashboard: projects, nodes, edges, templates, launches, runs, artifacts, plugins, triggers, health. It is not a chat feature bolted to the side; it is the second door into the same room. Registering it with an MCP-capable client is one line:

claude mcp add -s user ai-pitlane -- "$PWD/.venv/bin/python" -m orchestrator.mcp_server

After that you can stop drawing graphs and start describing them. A working session looks like this:

Three conventions make that safe to hand to a model:

• Mutating tools accept dry_run and report what they would have done. An AI client can rehearse an entire graph edit before writing a byte.
• Destructive tools require confirm: true — launch_project, delete_project, delete_artifact, restore_graph_snapshot. A model has to say it twice, structurally.
• The guardrails don't care who is calling. Budget enforcers refuse a paid dispatch over the cap whether the launch came from a human's mouse or a model's tool call. The AI gets the same pitlane, with the same speed limiter.

There is a genuinely useful recursion in here: an AI assistant designing, wiring, launching and monitoring pipelines of other AI agents — Claude assembling a graph in which a local Gemma does the work and a local reviewer checks it. Read-only MCP resources (orchestrator://projects, orchestrator://runs/latest, orchestrator://project/<name>/graph) give the client ambient context without tool calls, and apply_graph_diff applies a whole multi-op edit as one transaction. And because the plugin-authoring loop is on the tool surface too, an AI client that finds no kind for the job can scaffold one, implement it, smoke-test it, reload the registry, and wire the new node into the graph it is building — end to end, without a human opening an editor.

What you can do with it

The patterns those shapes unlock, in rough order of how often I reach for them:

• A zero-dollar code factory. Local vLLM or llama.cpp endpoints behind gemma_local-style kinds mean a full pick → implement → review loop that never touches a paid API. The 3090 bake-off published on this site was driven by exactly such a graph — the benchmark harness, the runs and the report generator wired as pitlane nodes.
• Capped burst of paid quality. Keep the graph local, swap the implementer's kind to claude_cli for one hard task, with a $5 hard cap and a dispatch counter standing between the graph and the invoice.
• A red team you can rerun. Point red_team at a directory; a scout scans, a local model verifies findings so one hallucinated vulnerability doesn't pollute the report, and the output lands as a markdown artifact you can diff between runs.
• Issues in, draft PRs out. Label a GitHub issue, let poll_issues turn it into a work-order, drain one slice, review the draft PR over coffee. The human stays exactly where a human is useful: writing the order and judging the result.
• Consensus wiring. foreach / collect primitives plus routers let two vendors attempt the same slice and a third judge them — the multivendor graph is the worked example.

What it refuses to do

• Serve the network. The dashboard binds loopback and has no auth — putting it on a public interface is explicitly a "do not." The MCP server is stdio; it exists only for the client that spawned it.
• Run a daemon. Launch, Abort and status work without one; the old long-running watchdog is being retired rather than grown. Timers exist for those who opt in, three keys deep.
• Hold your keys. Plugins name an environment variable (api_key_env) rather than embedding a secret, so a graph can be committed and a key cannot. Secrets set in the browser land in ~/.config/ai-pitlane/secrets.env, mode 600, outside the repo, and are never read back to the page. Vendor CLIs keep their own logins; the orchestrator checks claude auth status and never touches the credential. Committed graphs won't even carry a username — a leading ~ in a repo path expands at load time.
• Retry on your dime. A refused or failed paid call is an error for a router or a human. Nothing re-dispatches itself.

Kick the tires in ten minutes

git clone https://github.com/freePerro/ai-pitlane.git
cd ai-pitlane
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"
.venv/bin/python -m pytest orchestrator/tests -q   # 1,678 tests, ~35 s, no network, no model
.venv/bin/python -m orchestrator                   # dashboard on 127.0.0.1:9191

The suite stubs every subprocess and model call, so it passes on a machine with no vendors installed at all. First launch of the default graph with no local model server running fails with a structured error — that's the expected outcome, it costs nothing, and the run record shows exactly which node said why. From there: /help on the dashboard for tutorials and generated references, /connections for what's wired up, and infra/mcp/README.md for the full MCP tool list.

Grounded 2026-08-30 against the public repo at github.com/freePerro/ai-pitlane (Apache-2.0). Where this page and the code disagree, the code wins. Companion pages: the 3090 coding-stack brief and the bake-off it drove.