Orchestration and State Management

Lesson 8: Orchestration and State Management

This is the lesson that decides whether your agent is a toy or a system. Multi-agent orchestration is genuinely easy to start and genuinely hard to keep coherent โ€” because the hard part is never "spawn a worker". It is state: who owns which unit of work, what happens when one fails, and how the orchestrator knows what is ready.

Part 1 โ€” Custom agents as delegation targets

A custom agent is a named definition โ€” prompt plus tool scope โ€” that the runtime can delegate to inside a single session. When a user's request matches an agent's expertise, the runtime runs it as a sub-agent in an isolated context and streams lifecycle events back to the parent.

import { CopilotClient } from "@github/copilot-sdk";

const client = new CopilotClient();
await client.start();

const session = await client.createSession({
  model: "gpt-5.4",
  customAgents: [
    {
      name: "researcher",
      displayName: "Research Agent",
      description: "Explores codebases and answers questions using read-only tools",
      tools: ["grep", "glob", "view"],
      prompt: "You are a research assistant. Analyze code and answer questions. Do not modify any files.",
    },
    {
      name: "editor",
      displayName: "Editor Agent",
      description: "Makes targeted code changes",
      tools: ["view", "edit", "bash"],
      prompt: "You are a code editor. Make minimal, surgical changes to files as requested.",
    },
  ],
  onPermissionRequest: async () => ({ kind: "approve-once" }),
});
PropertyRequiredDescription
nameโœ…Unique identifier
promptโœ…The agent's system prompt
displayNameโ€”Human-readable name shown in events
descriptionโ€”What the agent does โ€” this is what the runtime matches intent against
toolsโ€”Tool names the agent may use. null or omitted = all tools
mcpServersโ€”MCP servers specific to this agent
inferโ€”Whether the runtime may auto-select this agent (default true)
skillsโ€”Skill names to preload eagerly into the agent's context
modelโ€”Model to use while this agent runs
reasoningEffortโ€”Reasoning effort while this agent runs
tools: null means "all tools", not "no tools". The default is full inheritance of the session's tool surface. Use explicit lists to enforce least privilege โ€” a "researcher" that silently holds edit and bash is a researcher only by name, and the model will use them when the prompt is ambiguous.

Two session-level knobs control which agent is live:

const session = await client.createSession({
  customAgents: [ { name: "researcher", prompt: "..." }, { name: "editor", prompt: "..." } ],
  agent: "researcher",          // pre-select โ€” active from the very first prompt
});
// Equivalent to calling session.rpc.agent.select() after creation, minus the extra round-trip.

You can also pin a different model and reasoning effort per agent โ€” useful for making a cheap explorer and an expensive editor. One precedence subtlety is worth knowing because it is a frequent "why did my effort setting not apply?" question: when reasoningEffort is omitted, the SDK sends no per-agent override. The runtime then resolves in order โ€” a per-call client option, the resolved model's default, or the agent definition โ€” and inherits the parent session's effort only when the sub-agent runs the same model as the parent. If the sub-agent resolves to a different model, it falls back to that model's default instead.

The orchestrator pattern: agent-exclusive tools

The most useful orchestration primitive here is defaultAgent.excludedTools. It hides tools from the default agent โ€” the built-in agent that handles turns when no custom agent is selected โ€” while leaving them available to sub-agents. The effect is that the main agent cannot do the heavy work itself; it must delegate.

import { CopilotClient, defineTool, approveAll } from "@github/copilot-sdk";
import { z } from "zod";

const heavyContextTool = defineTool("analyze-codebase", {
    description: "Performs deep analysis of the codebase, generating extensive context",
    parameters: z.object({ query: z.string() }),
    handler: async ({ query }) => {
        return { analysis: "..." };
    },
});

const session = await client.createSession({
    tools: [heavyContextTool],
    defaultAgent: {
        excludedTools: ["analyze-codebase"],
    },
    customAgents: [
        {
            name: "researcher",
            description: "Deep codebase analysis agent with access to heavy-context tools",
            tools: ["analyze-codebase"],
            prompt: "You perform thorough codebase analysis using the analyze-codebase tool.",
        },
    ],
});
FilterScopeEffect
availableToolsSession-wideAllowlist โ€” only these tools exist for anyone
excludedToolsSession-wideBlocklist โ€” blocked for everyone
defaultAgent.excludedToolsMain agent onlyHidden from the main agent, available to sub-agents

Precedence: session-level availableTools/excludedTools apply first, globally; defaultAgent.excludedTools is applied on top, restricting only the main agent. If a tool appears in both a session-level exclusion and defaultAgent.excludedTools, the session-level exclusion wins โ€” the tool is unavailable to everyone. Mechanically: excluded tools are still registered (handlers are available), they are simply absent from the main agent's tool list.

Token angle โ€” this is the highest-leverage orchestration trick in the SDK. The reason to hide a heavy tool from the main agent is not purity, it is context. A tool that returns thousands of tokens of analysis pollutes the orchestrator's window for the rest of the session, triggering compaction and degrading every subsequent decision. Forcing the heavy read into a sub-agent keeps the orchestrator's context small and cheap while the expensive tokens are burned once, in a context that gets discarded.

Part 2 โ€” Watching the tree

session.on((event) => {
    if (event.type === "subagent.started") {
        console.log(`Started ${event.data.agentDisplayName}`);
    }
    if (event.type === "subagent.completed") {
        console.log(`Completed ${event.data.agentDisplayName}`);
    }
});

Plus subagent.failed, subagent.selected, and subagent.deselected. For rendering, remember that sub-agent assistant events carry an envelope-level agentId โ€” so a tree UI keys off agentId and the message stream stays separable per agent. To filter tools or arguments before a sub-agent's calls run, use onPreToolUse / onPostToolUse, which fire correctly for sub-agent activity, and the onSubagentStart / onSubagentStop hooks for injecting context into a child's first prompt or replacing its response.

Part 3 โ€” State: a taxonomy before any API

Almost every broken orchestration traces to state living in the wrong place. Decide this before writing code:

Kind of stateWhere it belongsWhy not elsewhere
Conversation historyRuntime (session checkpoints)You cannot out-engineer the runtime's compaction
Coordination state โ€” what work exists, who owns it, dependenciesExplicit store: SQL todos (the canonical pattern) or your own tableImplicit shared memory does not exist between sub-agents
Durable business state โ€” orders, tickets, usersYour database, behind a toolSession state is not transactional and can be compacted away
Session metadata โ€” tenant, feature flags, stagesession.metadata RPCSurvives resume; does not pollute the model's context
Plan stateplan.md in the session workspaceRuntime-managed, visible in plan mode
Artifacts โ€” generated filesSession files/ or your object storeContext is for reasoning, not storage
Anything in a tool handler's memoryNowhere. Externalise it.In-memory tool state is not persisted and is lost on resume
The one-line rule: the session is a reasoning context, not a system of record. Everything that must survive a resume, a compaction, a crash, or a second worker goes in a store you control and is reached through a tool.

Part 4 โ€” The blackboard: SQL todos as coordination state

The canonical coordination model in this SDK is a small SQL todo table. It is not ceremony โ€” it is what makes "who does what next" answerable by a query instead of by a conversation.

CREATE TABLE todos (
    id TEXT PRIMARY KEY,
    title TEXT NOT NULL,
    description TEXT,
    status TEXT DEFAULT 'pending'
);

CREATE TABLE todo_deps (
    todo_id TEXT,
    depends_on TEXT,
    PRIMARY KEY (todo_id, depends_on)
);
pending  ->  in_progress  ->  done
                          \->  blocked

A worker's contract is five lines long, and the strictness is the point:

  1. Claim exactly one ready todo by setting status = 'in_progress'.
  2. Work only on that todo's scope.
  3. Store its result in the conversation or the task output.
  4. Set status = 'done' when complete.
  5. Set status = 'blocked' with a reason when it cannot proceed.

And the orchestrator finds ready work with one query โ€” the piece that makes dependencies real rather than aspirational:

SELECT t.*
FROM todos t
WHERE t.status = 'pending'
  AND NOT EXISTS (
      SELECT 1
      FROM todo_deps td
      JOIN todos dep ON td.depends_on = dep.id
      WHERE td.todo_id = t.id
        AND dep.status != 'done'
  );
Why a blackboard beats conversation for coordination. A sub-agent is stateless across calls โ€” it does not remember the previous worker's reasoning. Passing coordination through conversation means every worker pays to re-read every other worker's narrative, and correctness depends on the model's summarisation of it. A todo table makes dependencies explicit, cheap to read, and queryable by your code as well as the model's. Three practical wins: idempotency (a claimed row is a lock), resumability (restart and the table still says where you were), and observability (a dashboard reads the same table).

Part 5 โ€” Fleet mode

Fleet mode is the runtime's built-in pattern for dispatching multiple sub-agents in parallel via the task tool, using SQL todos as the shared coordination state. The wire method is session.fleet.start:

const result = await session.rpc.fleet.start({
    prompt: "Refactor each SDK package independently, then summarize the changes.",
});

if (result.started) {
    console.log("Fleet mode started");
}

From a plan-mode UI you get there by returning the exit action โ€” autopilot_fleet rather than autopilot โ€” when the approved plan already contains independent work items.

Good fitsAvoid
Multi-file refactors where each worker owns a file, package, or language SDKSequential tasks where step 2 needs step 1's concrete output
Batch reviews where each worker checks a separate diff, module, or alert groupTightly coupled edits where workers contend for the same files
Parallel research across independent repositories or feature areasSmall tasks one agent finishes quickly anyway
Documentation refreshes, one worker per pageWork needing continuous shared reasoning rather than clear ownership
Migrations where each worker validates its own sliceโ€”

The rules that keep a fleet from thrashing

  • Decompose before starting. Fleet mode dispatches; it does not plan. If the split is unclear, you get contention, not parallelism.
  • Minimise dependencies between todos. Every dependency is lost parallelism โ€” and the ready-work query is doing real work only when the graph is mostly wide.
  • Give every todo a durable ID, a clear title, and a complete description.
  • One owner per todo, one todo at a time per worker.
  • Background workers for genuinely parallel work; synchronous calls for validation gates. And do not dispatch a single background sub-agent โ€” use a synchronous call, or batch several in parallel.
  • Give each worker complete context. Sub-agents are stateless across calls. Include file paths, commands, expected outputs, and constraints in the prompt; do not assume it can infer them from the parent's history.
  • Avoid overlapping file ownership unless the parent will reconcile conflicts explicitly.
  • Require a structured report: what changed, how it was validated, what remains blocked.
  • The parent verifies the combined result. Fleet mode does not remove the need for parent review โ€” parallel workers can produce mutually inconsistent assumptions that the orchestrator must reconcile.
Fleet mode's honest limitations. The binding is experimental in several SDKs โ€” pin both SDK and CLI. Sub-agent lifecycle hook inputs identify the parent session but do not yet carry a shared tool-call ID for correlating with subagent.* events, so cross-referencing hook data to events is currently imperfect. Plugin-provided sub-agent types are configured at the runtime layer via --plugin-dir; there is no SDK-level registration helper. Java native typed bindings for session.fleet.start were not found on the audited branch. And whether the SQL todos pattern is a stable contract for SDK consumers is still an open question in the docs โ€” treat the schema as a convention you own, not an API you depend on.

Part 6 โ€” Steering and queueing: state transitions on a live agent

Two ways to send a message while the agent is already working, selected by the mode field on MessageOptions:

ModeBehaviourUse case
"immediate" (steering)Injected into the current LLM turn"Actually, don't create that file โ€” use a different approach"
"enqueue" (queueing, default)Queued, processed after the current turn finishes"After this, also fix the tests"
const session = await client.createSession({
    model: "gpt-5.4",
    onPermissionRequest: async () => ({ kind: "approve-once" }),
});

// Start a long-running task
await session.send({ prompt: "Refactor the authentication module to use sessions" });

// Steer it mid-turn
await session.send({ prompt: "Actually, use JWT tokens instead of sessions", mode: "immediate" });

// Queue follow-ups โ€” each starts its own full turn, FIFO
await session.send({ prompt: "Add unit tests for the auth module", mode: "enqueue" });
await session.send({ prompt: "Update the README with setup instructions", mode: "enqueue" });

How they actually work โ€” the part that determines your guarantees

Steering ("immediate")Queueing ("enqueue")
Mechanism Added to the runtime's ImmediatePromptProcessor queue; injected into the conversation before the next LLM request within the current turn Added to the session's itemQueue as a QueuedItem; processQueuedItems() runs when the session goes idle
OrderingBest-effort within the turnStrict FIFO, one full turn each
If the turn ends firstAutomatically moved to the regular queueโ€”
If steering was pending at turn endโ€”Moved to the front of the queue
Completionโ€”Processing continues until the queue is empty, then the session emits idle
Steering is best-effort, not a rollback. If the agent has already committed to a tool call, steering takes effect after that call completes โ€” still within the same turn, but the side effect has already happened. For anything irreversible, enqueue a corrective turn instead, or gate the dangerous call behind onPreToolUse so a steer can still change the decision.

Provenance: telling agent input from human authorisation

await session.send({
  prompt: "Here are the results you asked for.",
  source: "agent-sender-id",       // serialises as agent-<id>
  mode: "enqueue",
});

The typed APIs also support user and system. Use system for internal application context โ€” not as a substitute for an identified agent. Provenance lets the runtime distinguish agent input from human authorisation while keeping its agent-message steering behaviour.

Derive the sender ID from trusted application metadata, never from message text. If the sender identity comes from the content of the message, anything that can write a message can impersonate a sender. Two more cautions: a successful send acknowledgement confirms acceptance, not consumption โ€” do not auto-resend because no reply appeared, and remember sendAndWait can complete on an idle event with no assistant message at all. And remote backends do not necessarily preserve source end to end: Mission Control may include it in its local echo without carrying it in the remote HTTP request, so a local source event is not proof the remote worker received that provenance.

Part 7 โ€” Handoffs and context lifecycle

When a workflow changes phase โ€” research is done, now implement โ€” you have two options. Resume the same session with a reconfigured model and tools (Lesson 3), or hard-reset the conversation with clearContext, seeding the fresh window with a prompt. The second is the sharper tool: it preserves session identity, configuration, and the event log while removing the model-facing conversation, which is exactly what you want when the previous phase's transcripts are now noise.

// In-session handoff tool: terminal on success, never deferred
defineTool("clear_context", {
  description: "Clear the conversation and start a fresh context window",
  parameters: z.object({ prompt: z.string() }),
  isTerminal: true,
  defer: "never",
  handler: async ({ prompt }) => {
    const { messagesCleared } = await session.rpc.history.clearContext({ prompt });
    return `Cleared ${messagesCleared} messages.`;
  },
});

Part 8 โ€” Anti-patterns, ranked by how much time they cost

Anti-patternWhat happensFix
Coordination through conversationWorkers re-read each other's narratives; correctness rides on summarisationSQL todos / an owned table
Module-level state in a tool handlerWorks until the first resume or a second worker โ€” in-memory state is not persistedExternalise to a store; keep handlers stateless
Non-idempotent side effectsA retried turn duplicates work โ€” once per resume, once per retry, once per nudgeIdempotency keys, upserts, "claim row then act"
Fanning out with no ownership mapTwo workers edit the same file; reconciliation lands on the parent, lateOne owner per unit; explicit dependency edges
Dispatching a single background sub-agentCoordination overhead for zero parallelismSynchronous call, or batch several
Sub-agents assumed to know the parent's contextThey are stateless across calls and will guessFull context in the worker prompt
Unbounded autonomy with no budgetAn autopilot nudge loop or an agent-stop loop runs until someone noticessessionLimits.maxAiCredits + a bounded retry counter (Lesson 9)
Trusting the workers' summary"Completed successfully" is a self-report, not evidenceParent verifies the combined result; require a verifiable handle

๐Ÿง  Knowledge Check

1. You want the main agent to be unable to perform a heavy analysis itself, so it must delegate. Which setting?

2. Why is an explicit coordination store (the SQL-todo blackboard) better than letting sub-agents coordinate through conversation?

3. The agent has just committed to a destructive tool call. You send a steering message (mode: "immediate"). What happens?

Further Reading