Sessions as Durable State

Lesson 3: Sessions as Durable State

A session is the durable unit of an agent run. It has an identity, a conversation history on disk, a plan file, and a working directory. Treat the session ID the way you treat a primary key in a database, because that is functionally what it is โ€” and in multi-tenant systems it is also your access-control boundary.

Session IDs are an interface, not an implementation detail

You can let the runtime generate an opaque ID, but in anything resembling production you should not. Structured IDs make auditing, cleanup, and authorisation trivial:

PatternExampleWhy
โŒ Randomabc123No ownership information, unauditable
โœ… User-scopeduser-alice-pr-review-42Find and delete every session for a user
โœ… Tenant-scopedtenant-acme-onboardingEnforce tenant isolation by parsing the ID
โœ… Time-scopedalice-deploy-1706932800000Time-based retention and cleanup
function createSessionId(userId: string, taskType: string): string {
  const timestamp = Date.now();
  return `${userId}-${taskType}-${timestamp}`;
}

const sessionId = createSessionId("alice", "code-review");
// โ†’ "alice-code-review-1706932800000"
Session IDs are a security boundary, so validate before you trust them. The runtime scopes state per session but does not enforce your application's authorisation. If a caller supplies a session ID, parse the user/tenant segment and verify the caller is entitled to it before resuming โ€” otherwise you have handed out a conversation-history read primitive.

What actually gets persisted

State lands in ~/.copilot/session-state/{sessionId}/, or under COPILOT_HOME when you override baseDirectory:

~/.copilot/session-state/
โ””โ”€โ”€ user-123-task-456/
    โ”œโ”€โ”€ checkpoints/           # Conversation history snapshots
    โ”‚   โ”œโ”€โ”€ 001.json           # Initial state
    โ”‚   โ”œโ”€โ”€ 002.json           # After first interaction
    โ”‚   โ””โ”€โ”€ ...
    โ”œโ”€โ”€ plan.md                # Agent's planning state (if any)
    โ””โ”€โ”€ files/                 # Session artifacts
        โ”œโ”€โ”€ analysis.md
        โ””โ”€โ”€ notes.txt
DataPersisted?Note
Conversation historyโœ… YesFull message thread
Tool call resultsโœ… YesCached for context
Agent planning stateโœ… Yesplan.md
Session artifactsโœ… Yesfiles/ directory
Provider / API keysโŒ NoSecurity โ€” you must re-provide on resume
In-memory tool stateโŒ NoTools must be stateless
The two โŒs are design constraints, not trivia. Keys never touch disk, so a resume that forgets provider fails. And because in-memory tool state is lost, any tool whose correctness depends on a previous call must externalise that state (a database, a cache, or the conversation itself). This is the root cause of most "it worked, then I resumed and it went weird" bugs.

Creating, resuming, listing, deleting

// Create with a meaningful ID
const session = await client.createSession({
  sessionId: "user-123-task-456",
  model: "gpt-5.4",
  onPermissionRequest: approveAll,
});

// Resume later โ€” same ID, different process, different machine
const resumed = await client.resumeSession("user-123-task-456", {
  model: "claude-sonnet-4",
  reasoningEffort: "high",
});

// Housekeeping
const sessions = await client.listSessions();
const repoSessions = await client.listSessions({ repository: "owner/repo" });
await client.deleteSession("user-123-task-456");   // permanent
await session.disconnect();                        // detach, keep state

disconnect() and deleteSession() are different operations and conflating them is a data-loss bug waiting to happen. Disconnect releases the connection while leaving state on disk; delete removes the session permanently.

Resume is a reconfiguration point

The genuinely useful insight here: resumeSession accepts almost the entire session config again. That means you can adapt a live workflow without losing its history.

OptionEffect
modelSwitch models โ€” escalate to a stronger one for a hard phase
reasoningEffortAdjust thinking budget
systemMessageOverride or extend the system prompt
availableTools / excludedToolsChange the tool surface
providerRequired for BYOK sessions โ€” keys are never persisted
mcpServers, customAgents, agentRe-attach or re-select agents and MCP servers
skillDirectories, disabledSkillsChange skill availability
infiniteSessionsCompaction behaviour
workingDirectory, configDirRelocate execution context
capi.enableWebSocketResponsesTransport fallback (see pitfall below)
// BYOK: the provider MUST be re-provided, because keys are never persisted
const resumed = await client.resumeSession("user-123-task-456", {
  provider: {
    type: "azure",
    endpoint: "https://my-resource.openai.azure.com",
    apiKey: process.env.AZURE_OPENAI_KEY,
    deploymentId: "my-gpt-deployment",
  },
});
The 400 you will eventually hit: a resumed session reporting 400 input item ID does not belong to this connection is specific to the WebSocket transport for the Responses API (on by default). The fix is capi: { enableWebSocketResponses: false } on the resume call โ€” which is also worth setting proactively when you run behind a proxy. The equivalent env var is COPILOT_CLI_DISABLE_WEBSOCKET_RESPONSES, and note it has the opposite polarity.

Infinite sessions and compaction

Long-running agents will eventually exceed the context window. Rather than letting that fail, the runtime compacts โ€” summarising older history in the background and continuing. It is on by default, and you tune it with a threshold:

const session = await client.createSession({
  infiniteSessions: {
    backgroundCompactionThreshold: 0.80,   // start compaction at 80% of the window
  },
  onPermissionRequest: approveAll,
});

Watch for the pair session.compaction_start and session.compaction_complete. Compaction is where accuracy silently degrades: details can be summarised away, and a resumed-plus-compacted session may not remember a specific identifier it saw 60 turns ago.

Token angle: compaction is the runtime spending tokens to buy back context. A session that compacts repeatedly is a session whose individual tool results are too large. Before you raise the threshold, look at your biggest tool payloads โ€” you will usually find a tool returning a whole file or a whole API response where a projection would do. Fixing the tool is cheaper than paying for the summariser every cycle.

Clearing context without losing the session

Sometimes you want a hard reset of the conversation while keeping the session identity, configuration, system messages, and event log. That is clearContext, and it is a tool-handler primitive โ€” the runtime rejects it unless a tool call is in flight.

import { approveAll, CopilotClient, defineTool } from "@github/copilot-sdk";
import type { CopilotSession } from "@github/copilot-sdk";
import { z } from "zod";

const client = new CopilotClient();
let session: CopilotSession;

session = await client.createSession({
  onPermissionRequest: approveAll,
  tools: [
    defineTool("clear_context", {
      description: "Clear the conversation and start a fresh context window",
      parameters: z.object({ prompt: z.string() }),
      isTerminal: true,       // success ends the turn โ€” essential here
      defer: "never",         // must always be available, never lazily loaded
      handler: async ({ prompt }) => {
        const { messagesCleared } =
          await session.rpc.history.clearContext({ prompt });
        return `Cleared ${messagesCleared} messages.`;
      },
    }),
  ],
});

The required prompt becomes the first user message in the fresh window, and a successful clear emits session.context_cleared with the count and the seed message. This is the primitive behind handoffs and host-managed context lifecycle policies.

Why isTerminal: true is not optional here. Terminality ends the current agent turn when the tool succeeds (a failure, denial, rejection, timeout, or validation error stays visible to the model so it can recover). Without it, the loop may make another model call against the newly cleared window before starting your seeded turn โ€” burning a turn on an empty context. Use isTerminal only for tools whose success should end the turn; leave it unset everywhere else.

Deployment topology: where the session state lives

Two patterns dominate, and the choice trades isolation against cost:

Pattern 1 โ€” CLI per userPattern 2 โ€” shared CLI
IsolationStrong โ€” separate process, separate credentialsLogical only โ€” your app must enforce it
CostHigher (one runtime per user)Efficient (one runtime, many sessions)
RequiresProcess orchestration, per-user lifecyclemode: "empty", per-session tokens, app-level access checks
Use whenRegulated data, per-user credentials, hard boundariesMany light users, you control tools and auth

Pattern 2 additionally wants baseDirectory to isolate runtime state per process/pod/tenant, and sessionIdleTimeoutSeconds so abandoned sessions get reaped rather than leaked.

const client = new CopilotClient({
  mode: "empty",
  baseDirectory: `/var/lib/my-app/copilot/${runtimeInstanceId}`,
  sessionIdleTimeoutSeconds: 900,
  connection: RuntimeConnection.forUri(process.env.COPILOT_RUNTIME_URL!),
});
Containerised deployments: mount the state directory. ~/.copilot/session-state/ must be writable, so a container with an ephemeral filesystem loses every session on restart โ€” and resume silently becomes create. Mount a persistent volume, or route storage through sessionFs so session file I/O goes to your own storage layer (the right answer when disk is ephemeral or you need tenant-aware paths).

Session filesystem (sessionFs)

sessionFs registers a custom session filesystem provider so session-scoped file I/O is routed through your storage instead of local disk. The isolation subtlety is worth stating precisely: the runtime does not partition files by session ID for you. Sessions accessing the same files in the same provider namespace share remembered grants. Your application must use distinct namespaces per tenant and authorise access to each.

๐Ÿง  Knowledge Check

1. A BYOK session worked yesterday. Today you resume it and get an authentication failure. Why?

2. Which is not persisted in session state?

3. You expose a clear_context tool. Why must it set isTerminal: true?

Further Reading