Reasoning effort is not free. Reasoning vs generate asked which model class belongs on a tool-heavy job. This post tests a configuration choice: route planning and tool-query work to different models within the existing hierarchical mode. A planner chooses subgoals; a dig worker executes queries and returns observations. Confirm workers actually used a normal chat model by checking response ids and reasoning token fields (Completions vs Responses).

Cousins: hierarchical vs single-agent · what is ReAcTree?.

What the evidence supports

  • Hierarchical plan already separates roles: root = planning, digs default = tool_calling.
  • An all-reasoning roster ignored that split. Digs correctly requested tools and still ran on a high-effort Responses preview — first digs in the ~8–13 minute class, multi-million dig tokens on a prior rematch.
  • Hybrid roster: reasoning (medium) for planning / scientific reasoning; generate for digs / efficiency / summarizer.
  • Same burn-rate alert, same Grafana MCP plane: hybrid plan finished in ~6 minutes with a measured Theory. Dig Expert.Do times landed in the ~12s–2.5m band. Routing logs showed plan → Responses, digs → generate.

The mistake

We did not need a new ExecutionMode. The harness already does this:

User → plan root (task_type=planning)
         → create_agent dig (task_type=tool_calling)
              → observability tools + notes
         → Theory synthesis

The cost came from the model map. Every good_for_task entry, including tool_calling, selected the same high-effort Responses reasoning model. Thus every dig worker paid for reasoning before its Grafana calls. The runtime correctly followed the configuration; the configuration failed to distinguish planning from measurement.

That is not “reasoning is bad at SRE.” It is reasoning spent on the wrong turn.

The hybrid roster

A planning model coordinates narrower tool-using workers

Count worker costs and check their evidence before treating a cheaper roster as a win.

Task class Seat Effort
planning, scientific_reasoning, general_task Responses reasoning preview medium (not high for the first pass)
tool_calling, terminal_calling, efficiency Generate (gpt-5.4-class) none
summarizer, frontdesk Generate mini or full none

This roster spends additional reasoning on selecting probes and uses a generate model for repeated tool calls. A worker may still need reasoning if a probe requires complex interpretation; measure its output quality rather than forbidding that model class outright.

In this harness, planning and tool_calling were already separate task types, so no new execution mode was needed. If a different runtime requires the parent to measure before delegation, preserve that safety check; the model roster does not replace tool permissions or completion gates.

Receipt (obfuscated)

Job: hierarchical plan on a live API availability burn-rate card. One Grafana tool plane. Same prompt across seats.

Roster Wall Digs What closed
All-generate control ~0.5m 1 Thin undetermined this pass (dig never attached real probes)
All-reasoning high (prior rematch) ~13m 2 Expensive dig tax; multi-million dig tokens
Hybrid ~6m 4 Probable Theory: one hot route all 5xx, matching app error type

Hybrid log assertion (the part that matters more than wall):

  • Root GetModel → planning → Responses preview
  • Every dig GetModel → tool_calling → generate
  • Dig tool credits actually fired (alert rule, firing instances, metric/log probes)

We stopped a fresh all-reasoning rematch after confirming that its first dig selected Responses for tool_calling. The ~13m number is from a prior run, not a simultaneous controlled comparison. The hybrid ~6m run also used four digs rather than two, so these wall times cannot isolate the effect of model routing.

What operators should copy

  1. Audit the roster. If tool_calling maps to your highest-effort reasoner, you are taxing digs on purpose.
  2. Keep dig task_type off planning. Host prompts that set digs to planning reintroduce the tax even with a good default.
  3. Remap summarizer and efficiency too. Fallbacks that inherit the reasoner quietly rebuild the bill.
  4. Start medium on the planner. Raise only if Theory quality regresses. High everywhere is how the basement floods.
  5. Verify from the export. Worker spans should show a normal chat model (or zero / off thinking). Planner spans may show Responses ids (resp_…) plus reasoning tokens (how to read Completions vs Responses).

Blindspots

  • A dig that explicitly requests planning still pulls the reasoner. Policy must forbid that.
  • The all-generate control finished in ~0.5m but produced a thin Undetermined answer without real probes. It is not an acceptable faster close to the same task. The hybrid route is an attempt to retain planner reasoning while avoiding high effort on every dig; its benefit needs repeated quality-matched runs.
  • Quality still depends on dig tool allowlists. A fast undetermined close is not a win.

Test this routing against all-generate and all-reasoning configurations on repeated identical incidents. Record per-role model, effort, tool outcome, total tokens, and diagnosis quality; a faster dig that never reaches a useful probe is not a savings.

FAQ

Do I need a new execution mode for hybrid plan?

No. Hierarchical plan already sends the root to a planning task type and dig workers to tool_calling by default. The fix is which model each role gets: a reasoning model for planning, a normal chat model for digs.

Why did an all-reasoning plan bench burn so much time?

Every task type, including tool workers, pointed at the same high-thinking Responses model. Digs asked for tools and still got a slow reasoning model for Grafana loops.

What did the hybrid model map change?

Planning and synthesis stayed on a medium-thinking reasoning preview. Digs, efficiency helpers, and summarizers moved to normal chat models. Dig run times dropped from many minutes to tens of seconds or a couple of minutes.