Retrieving a past note can help an agent remember an incident, but it does not change the workflow that produced a recurring mistake. By “learning loop” here, I mean using reviewed operational history to propose a change to that workflow or its policy.

For example, repeated denials, a recurring sequence of site reliability engineering (SRE) steps, or an expensive integration could lead to a proposed change. These are patterns to investigate, not proof that a change will help. A human should be able to approve, edit, or dismiss the proposal.

In Aiden, our agent platform, we use a diary → insight → human gate → materialize path for agent workflows. “Materialize” means creating a reviewable draft, not changing production automatically. This is an operating model rather than a measured claim that proposals have already reduced incidents.

The Problem: Digests Nobody Reads

Enterprise agent platforms generate history whether you want it or not: audits, tool denials, workflow runs, operator thumbs-up and thumbs-down, spend per session. Many teams also summarize that activity into something diary-shaped — a daily or weekly digest per agent.

Then the digests sit unread.

Three patterns the loop is intended to surface:

  1. Recurring failure with no owner. The same error class appears every maintenance window. The diary records it. Nobody promotes a guardrail.
  2. Repetitive human choreography. Discovery → triage → root-cause analysis (RCA) often in that order. Humans already invented a composite playbook; the platform never proposes one.
  3. Policy friction that looks like security. Legitimate team requests bounce for weeks. Operators work around with tickets. The policy never gets a refinement proposal.

Adding memories may improve recall, but reducing repeated incidents or policy friction would require approved changes and outcome measurement.

The Diary: Bounding the Context Firehose

A diary entry is a bounded summary of what an agent did — not a raw firehose of every tool payload. Think: who ran what, what failed, what cost money, what operators liked or hated. Bound it on purpose. Unbounded history into a summarizer is how offline jobs recreate the same context blow-ups you fight in live sessions (tokenomics as an operating model).

The diary is an observation layer. Learning starts when something turns observations into proposals.

The Architecture of a Learning Loop

Three jobs — map them to whatever runtime you already run:

Job Responsibility
Summarizer Bounds the token context and extracts boring facts (failures, sequences, spend, feedback) — honest about what it sampled.
Evaluator Compares those facts against a small taxonomy (table below) and emits proposals only — never silent deploys.
Materializer Turns an approved proposal into a typed draft artifact humans already know how to review (workflow draft, policy draft, doc PR, ticket). 
Daily digests + feedback + cost signals
            │
            ▼
        Summarizer
            │
            ▼
        Evaluator  →  Insight queue: proposed | approved | dismissed
            │
            ▼ (human approve)
        Materializer → workflow · policy · persona · knowledge (draft)

What the human actually reviews

A reviewer needs a concrete record. This is an illustrative proposal, not a product schema or an observed count:

{
  "pattern": "repetitive_workflow",
  "trigger": "High CPU alert on database cluster",
  "evidence": ["run_445", "run_481", "run_512"],
  "proposal": {
    "type": "create_composite_playbook",
    "steps": ["fetch_metrics", "check_query_hotspots", "page_oncall_if_replica_lag"]
  },
  "rationale": "Operators ran these three steps in sequence 14 times this week."
}

A concise proposal makes review more feasible; complex policy changes may still need longer analysis.

Reviewing changes through Git or a decision queue

How you approve matters as much as that you approve.

  • Prefer GitOps—reviewed changes in Git—when the change is code. Policies, workflow definitions, and persona prompts that already live in git should materialize as a draft pull request (or equivalent reviewable diff) against that repo. Same review bar as Rego, runbooks, or Terraform — AI does not get a side door.
  • A dashboard approval can help triage proposals, but is not enough by itself for production policy. A dashboard “Approve / Dismiss” queue works for WIP and noise control. Shipping deny-rule text with only a button click — and no reviewable artifact — is how you recreate shadow IT with nicer UX.
  • A weekly queue too large to review carefully recreates the human-in-the-loop (HITL) paradox. Cap WIP; shorter queues get real read time.

Four boundaries we use:

  1. Proposals are not deployments. An insight is a draft change with a rationale and evidence pointers — not a silent rewrite of production policy.
  2. Humans stay on the gate. Approve, dismiss, or send back — via PR review or an equivalent durable decision.
  3. Evidence must be boring and checkable. Timestamps, agent names, denial classes, “we saw this N times” — not a novel about how the model felt.
  4. Materialization is typed. “Create a composite workflow,” “refine a deny rule,” “adjust a persona,” “cache a missing fact” — vague “improve the agent” tickets do not count.

This is the same spirit as evidence-gated RCA: prove with artifacts, then narrate. Here the artifact is a reviewable proposal, not an investigation key.

Diary learning loop from bounded digest and evaluator proposals through a human approval decision to typed draft changes and normal review

The evaluator proposes only; approval and ordinary change review precede any production change.

Insight Shapes Worth Detecting

Keep the taxonomy small enough that operators recognize themselves:

Pattern operators feel What a good proposal looks like
Same failure every week Guardrail, persona clarification, or workflow pre-check
Same stage sequence every time Composite / referred workflow instead of tribal knowledge
Legitimate work repeatedly denied Policy refinement or scoped exception — not “turn policies off”
Repeated rediscovery tax Cached knowledge or launch-time context so day-2 is not day-1 again
Missing integration or skill Capability request with examples — not a silent tool invent
One path always over budget Routing or stage simplification proposal with cost receipts

These categories do not require a large agent hierarchy. A bounded summarizer, an evaluator restricted to the taxonomy, and a draft-producing materializer are enough to test the approach. Whether the proposals help still depends on review quality and measured outcomes.

Failure modes to monitor

Confident nonsense proposals

A sparse diary can support an overconfident workflow proposal. Treat confidence as a user-interface hint, not permission to deploy; inspect the cited runs and favor narrow, reversible changes first.

Learning that bypasses governance

Automatically merging policy text would let the agent change its operating rules without normal review. Tie materialization to the same review culture you use for Rego, runbooks, or Terraform.

Digest bloat that eats the week

If the offline path stuffs every audit event into the prompt, you will relearn context budget the hard way. Sample with intent; tell the model what it cannot see.

Queue theater

A backlog of 200 “proposed” insights with no reviews would create an appearance of improvement without any reviewed change. Cap WIP. Prefer a short weekly review ritual.

What approval is meant to protect

The intended benefits depend on proposals being reviewed and adopted, not merely generated:

  • Less repeat work: a reviewed guardrail may prevent the same mistake from recurring; check subsequent runs to see whether it did.
  • More visible policy friction: recurring denials become reviewable proposals instead of informal workarounds.
  • Accountable learning: operators own the accept/dismiss decision; the platform does not silently rewrite the org.

The approval record should show the evidence, the decision, and the resulting draft. That makes later corrections possible.

A small pilot

  1. Write one honest weekly digest for your highest-traffic agent — even if it is a scripted rollup.
  2. Add a single human-reviewed queue (spreadsheet is fine) with columns: pattern, evidence, proposed change, decision.
  3. Materialize only approved rows — as draft policy, draft workflow, or a draft PR when those artifacts already live in git.
  4. Measure reviewed insights per week, not generated insights per week.
  5. Disable automatic policy application unless it meets the same review bar as other production changes.

Related: Pensieve memory is about forgetting and curated recall. This post is about organizational learning — changing the system the agent runs in, not only the vectors it searches.

Are your agents proposing improvements your team actually reviews — or just writing diaries into the void? Find me on GitHub or LinkedIn.

We build incident-triage agents at StackGen; the SRE offering is at ai.stackgen.com.