This series documents what we learned building a production AI agent runtime and Aiden — StackGen’s multi-tenant orchestration platform for enterprise SRE and platform teams. Every post is grounded in shipped behavior and production failures, not demo polish.

Start with a pack

Pack For
Go agent runtime Runtime definition, Go vs Python, platform split
SRE on-call Triage, RCA, observability
Evidence-gated RCA checklist Operator review of agent write-ups
“Done” checklist When not to trust agent completion

Topic hubs

Dive by theme:

Suggested for you

Every post, month by month

August 2026 23 posts
  • Simple vs plan on AppWorld skills smoke: plan wins some tasks, ties on others, and always costs more. Choose the mode that fits the job — both belong in the toolkit.

    · ai-agents, evaluation, multi-agent, orchestration, appworld, benchmarking, tokenomics, workflows, aiden

  • Single-agent MCP loaded 453 tools (~17k peak). A planner peaked lower, spent ~2× total tokens, and sometimes won quality. Report pass@1 and pass@3 — that is not pass^k.

    · ai-agents, evaluation, multi-agent, orchestration, mcp, appworld, benchmarking, tokenomics, workflows

  • Unattended AI agent evaluation fails when the model asks the user or reports done with zero MCP calls. Put those rules in the harness, not the framework.

    · ai-agents, evaluation, benchmarking, appworld, mcp, multi-agent, workflows, production

  • Run AppWorld locally with the Aiden agent runtime: docker-compose, MCP HTTP wiring, health gates, and lessons from replacing custom shims with the official three-server stack.

    · ai-agents, evaluation, benchmarking, appworld, mcp, docker, workflows, orchestration, production

  • Stop duplicate agent workers: a handoff gate fixed fairness 5/5 and cut planner token tax on AppWorld evals — partial pass% rose while strict TGC remains a separate harness goal.

    · ai-agents, evaluation, multi-agent, orchestration, workflows, handoff, subagent, aiden, production

  • AI agent eval failure modes on AppWorld: budget stops before evaluate, PII placeholders poison tool calls, 422 wrong methods, and prose PASS vs judge FAIL.

    · ai-agents, evaluation, reliability, pii, redaction, verification, workflows, aiden, production

  • Agent orchestration tax after a fair eval: on AppWorld tasks, planner paths cost ~1.6× tokens and ~3× iterations — measure coordination cost separately from benchmark success.

    · ai-agents, evaluation, benchmarking, multi-agent, tokenomics, orchestration, reactree, workflows, aiden, production

  • Fair agent evals for planner vs single-agent: match domain tool access on workers before you compare tokens, pass rate, or declare a routing winner.

    · ai-agents, evaluation, benchmarking, multi-agent, workflows, orchestration, mcp, aiden, production

  • From vibes to contracts: how we rebuilt agent evals around eval sets, rubrics vs criteria, a grader stack, and pass^k reliability.

    · ai-agents, evaluation, reliability, sre, rca, adk, pass-at-k, incident-response, aiden, production

  • Canary-first evals for live SRE investigate: check the canary before burning judge tokens, and never treat a draft RCA as done.

    · ai-agents, sre, evaluation, consistency, tokenomics, incident-response, observability, aiden, production, rca

  • AI SRE agent benchmarks: wall time, tool calls, tokens, and ReAcTree tax — fair A/B numbers so you know when orchestration is worth the cost.

    · ai-agents, sre, benchmarking, tokenomics, reactree, multi-agent, incident-response, observability, aiden, production, evaluation

  • Reasoning effort is not a free upgrade for tool-heavy AI agents. Live SRE A/Bs: blanket high timed out, adaptive low→high dug deeper.

    · ai-agents, sre, reasoning, multi-agent, orchestration, incident-response, openai, golang, aiden

  • Claim-aware evidence packing: don't accuse AI agents of inventing what you truncated. Why hallucination guards fail when they drop the receipts.

    · ai-agents, golang, runtime, hallucination, evidence, production

  • Single-agent vs multi-agent for SRE triage: a fair A/B, what each shape wins at, and a decision framework so you stop defaulting to either.

    · ai-agents, multi-agent, orchestration, sre, reactree, incident-response, workflows, golang, aiden

  • Aiden the easy way: one OpenTofu module from a vague GitHub issue to a review PR — Specify, Research, Plan, without wiring every resource by hand.

    · ai-agents, github, workflows, aiden, terraform, opentofu, sdlc, beginners

  • Aiden the hard way: turn a vague GitHub issue into a review PR — GitHub integration, agent, workflow, webhook, and status poll wired by hand.

    · ai-agents, github, workflows, aiden, terraform, opentofu, sdlc, beginners

  • What are SRE AI agents? AI for incident triage, diagnostics, and RCA with bounded autonomy — not a chatbot and not open-ended remediation demos.

    · ai-agents, sre, incident-response, on-call, production, aiden

  • What is an AI agent runtime? The production loop that plans, calls tools, and manages context — not a GenAI platform, chatbot, or notebook demo.

    · ai-agents, runtime, golang, production, aiden

  • SRE for agentic systems: uptime isn't enough. Judgment SLOs and how to measure agentic drift when the agent can be 'up' and still wrong.

    · ai-agents, sre, observability, golang

  • PII redaction for AI agents needs two views: a protected model history and authorized operator visibility when debugging tool calls.

    · ai-agents, pii, privacy, security, observability, audit, aiden, production

  • Can you interrupt or redirect an AI agent mid-response? Yes. How to send steer or cancel signals mid-stream and change the active task without restarting.

    · ai-agents, hitl, human-in-the-loop, feedback, orchestration, ux, aiden, production

  • Prompt caching for AI agents is an architecture problem: stable prefixes, early compaction, and references beat copying a turbulent payload.

    · ai-agents, prompt-caching, llm, tokenomics, context-management, aiden, production

  • AI agent loop detection can erase a good answer. Preserve the best evidence-backed result when a stalled run ends — don't throw the work away.

    · ai-agents, loop-detection, reliability, orchestration, sre, aiden, production

July 2026 16 posts
  • AI agent root cause analysis fails when a dig finds a lead and discards it — peer noise, wrong fire-time windows, and transcript gates for AI SRE.

    · ai-agents, sre, root-cause-analysis, incident-response, on-call, evaluation, prompt-engineering, aiden, production, observability

  • When an AI SRE agent hits a dead end, be creative — don't invent. Search harder instead of hallucinating rule IDs, metrics, or a tidy RCA.

    · ai-agents, sre, root-cause-analysis, incident-response, on-call, prompt-engineering, aiden, production, llm-hallucination

  • AI agent root cause analysis for SRE: curiosity before confidence. Soft prompts don't stop bad RCAs — checklists, hard gates, and batched validation do.

    · ai-agents, root-cause-analysis, sre, incident-response, on-call, evaluation, prompt-engineering, production, aiden, compound-ai

  • Is the AI agent task actually done? Why production agents need an independent completion check — not a self-graded 'I'm finished.'

    · ai-agents, verification, llm-as-judge, production, golang, aiden, sre, budgets

  • When your AI agent scorecard lies: measure telemetry quality before you trust reliability, correctness, cost, or latency scores.

    · observability, ai-agents, telemetry, evaluation, sre, production, langfuse

  • Hypothesis-driven AI SRE root cause analysis: climb identity and onset before deploy theories, keep parallel branches, prove first and narrate last.

    · sre, incident-response, root-cause-analysis, ai-agents, on-call, hypothesis-driven-debugging, production, aiden

  • From demo to deploy: production-ready AI agents need receipts, not fluent demos — evidence gates, HITL tiers, and eval checklists for enterprise pilots.

    · ai-agents, production, sre, evaluation, hitl, workflows, aiden, compound-ai, enterprise-agents

  • AI agent learning loop: daily digests become human-approved workflow and policy changes — not a bigger vector store.

    · ai-agents, learning, governance, workflows, policy, hitl, aiden, production, enterprise-agents

  • Beyond Confluence runbooks: why GitOps triage steps matter for AI agents — version-controlled procedures that change with your stack.

    · runbooks, gitops, confluence, sre, incident-response, ai-agents, golang

  • Your RCA agent doesn't need another runbook — it needs a map. Topology, gates, and verify-first navigation beat a forty-page notebook.

    · ai-agents, sre, evaluation, compound-ai, workflows, golang

  • LLM tokenomics for production agents: token budget strategies, context compression, and FinOps loops that keep sessions finishing.

    · llm, finops, ai-agents, context-window, production, aiden

  • Evidence-based verification for AI agents: don't trust self-report — pull proof from ArgoCD, Datadog, and systems of record, then let Go own pass/fail.

    · ai-agents, sre, verification, observability, production, golang

  • Evidence-gated RCA for AI SRE agents: prove with receipts, then narrate. Fixed stages, structural evals, and token-aware tool loops.

    · ai-agents, compound-ai, orchestration, evaluation, sre, workflows

  • AI incident triage for SREs — what actually helps on-call versus demo theater. Canonical copy now on StackGen.

    · sre, incident-response, on-call, ai-agents, production, golang

  • Production agent tool calls need layered JSON repair — why one pass fails on malformed LLM output and what we learned in Go middleware.

    · ai-agents, production, go, reliability, tool-calls

  • Contributing back while building a commercial product: how we shipped a proprietary platform and still merged PRs into the agent framework we depend on.

    · open-source, community, ai-agents, go, engineering

June 2026 12 posts

More on this site

Posts outside the numbered series (e.g. cloud entitlements, web→LLM metrics) live on the homepage archive.

FAQ

Why build an enterprise AI agent platform in Go?

Go gives you static typing, simple deployment, and concurrency primitives that map cleanly to multi-stage agent workflows. This series covers when that trade-off beats Python-first AI frameworks in production.

Where should I start reading?

Use a starter pack: Go agent runtime (definition, why Go, platform split) or SRE on-call (triage, RCA, observability). Then open the searchable series archive and follow series_order. Each post is self-contained but builds on prior lessons.

Who is this series for?

Staff engineers, platform teams, and SREs shipping agentic workflows to production — not tutorial readers looking for a hello-world chatbot.