Production Deployment โ Auth, Scaling, and Multi-Tenancy
Lesson 10: Production Deployment โ Auth, Scaling, and Multi-Tenancy
Everything so far ran as one user on one machine. Production means many users, credentials you do not control, storage that disappears, and a bill you have to explain. This lesson is the deployment layer โ and it is mostly a story about identity, because in this SDK identity determines isolation, billing, and capability all at once.
The authentication priority order
When several credentials are configured, exactly one wins. Knowing the order is how you debug "it's using the wrong account":
- An explicit SDK token (passed in code)
- Direct Copilot API environment authentication
- Environment-variable GitHub tokens โ server-to-server installation tokens use this path
- Stored Copilot CLI credentials (the interactive sign-in)
- GitHub CLI credentials
| Method | Use case | Copilot subscription needed? |
|---|---|---|
| GitHub signed-in user | Interactive apps where users sign in with GitHub | Yes |
| GitHub OAuth app | Acting on behalf of users via OAuth | Yes |
| Environment variables | CI/CD, automation, server-to-server | Yes |
| Server-to-server | Organisation-attributed automation, direct org billing | No user subscription โ org policy required |
| BYOK | Your own API keys (Foundry, OpenAI, Anthropic, more) | No |
Token types โ one of these is a trap
| Prefix | Type | Supported |
|---|---|---|
gho_ | OAuth user access token | โ |
ghu_ | GitHub App user access token | โ |
github_pat_ | Fine-grained personal access token | โ |
ghp_ | Classic personal access token | โ Not supported (deprecated) |
For a web app acting for a signed-in user, pass their token explicitly and make sure the SDK does not fall back to whatever the machine happens to be logged in as:
import { CopilotClient } from "@github/copilot-sdk";
const client = new CopilotClient({
gitHubToken: userAccessToken, // from your OAuth flow
useLoggedInUser: false, // do NOT use stored CLI credentials
});
useLoggedInUser: false is the line that makes multi-user apps correct. Omit it and a request from user A can silently run as whoever the host is signed in as โ a cross-tenant leak that looks like a working feature until someone reads the audit log. In multi-user server mode, pass a per-session gitHubToken so each session runs with the correct GitHub identity.
Rotating session-scoped tokens โ the right pattern for services
Do not store one long-lived token per service and hope. Set a token provider per session; the runtime calls it for the effective GitHub host and tells you whether the request is initial or refresh:
const session = await client.createSession({
gitHubTokenProvider: async ({ host, sessionId, reason }) => {
const token = await acquireGitHubToken({ host, sessionId, reason });
return {
kind: "token",
accessToken: token.value,
expiresIn: token.secondsRemaining,
};
},
});
| Rule | Detail |
|---|---|
expiresIn is required | The positive number of seconds remaining when the callback completes. Production GitHub tokens typically last 8 hours, so 8 * 60 * 60 is a common value. |
| Return a tagged result or an explicit cancellation | Not a bare string |
| Do not set both a static per-session token and the provider | Pick one โ otherwise the priority order becomes your bug |
The callback receives host (the effective GitHub host), sessionId, and reason (initial or refresh). The session ID is absent only when a cloud session has not yet received its ID.
BYOK โ your own keys, and the memory rule
const session = await client.createSession({
sessionId: "user-123-task-456",
model: "gpt-5.2-codex",
provider: {
type: "azure",
endpoint: "https://my-resource.openai.azure.com",
apiKey: process.env.AZURE_OPENAI_KEY,
deploymentId: "my-gpt-deployment",
},
});
provider on every resume. This is a security feature, and it is the single most common BYOK bug: the session resumes fine, then fails at the first model call because the credential is gone. Related: the BYOK callback field was renamed getBearerToken โ bearerTokenProvider, and ProviderTokenArgs now carries sessionId for per-session token scoping. If you are on an older SDK, that rename is a silent break.
For Azure, prefer managed identity over API keys โ no secret in your environment, no rotation chore, and the credential is bound to the workload's identity rather than to a string someone copied into a config file.
Server-to-server โ organisation-attributed automation
When the automation belongs to an organisation rather than a person, use a GitHub Actions or GitHub App installation token. Two properties make this the right choice for org automation: it needs no user subscription (organisation policy is what gates it), and usage is attributed to the organisation for direct billing. Installation tokens travel via the environment-variable path in the authentication priority order.
Multi-tenancy โ the non-negotiable baseline
import { CopilotClient, RuntimeConnection } from "@github/copilot-sdk";
const client = new CopilotClient({
mode: "empty",
baseDirectory: `/var/lib/my-app/copilot/${runtimeInstanceId}`,
sessionIdleTimeoutSeconds: 900,
connection: RuntimeConnection.forUri(process.env.COPILOT_RUNTIME_URL!),
});
const session = await client.createSession({
sessionId: `user-${user.id}-${crypto.randomUUID()}`,
model: "gpt-5.4",
availableTools: ["custom:lookupOrder", "custom:createTicket"],
gitHubToken: user.githubToken,
sessionLimits: { maxAiCredits: 30 },
onPermissionRequest: permissionHandler,
});
| Option | What it buys you |
|---|---|
mode: "empty" | No ambient OS tools, no bundled skills, no implicit plugins โ required for shared/multi-user scenarios |
availableTools | Explicit capability allowlist per session |
baseDirectory | Sets COPILOT_HOME for a runtime instance โ isolates state, credentials, and session data per process, pod, worker, or tenant boundary |
sessionIdleTimeoutSeconds | Reaps abandoned sessions instead of leaking them |
sessionFs | Routes session file I/O through your storage โ for ephemeral disk, object storage, or tenant-aware paths |
Per-session gitHubToken | Correct identity per tenant |
sessionLimits | Per-tenant budget ceiling |
mode: "copilot-cli" OS filesystem access is shared through the host process, which is why that mode must not be used for multi-user server mode. Session-level isolation means user-specific model and state information is scoped to a session rather than global shared state โ it does not mean the runtime enforces your authorisation. You must still validate that a caller is entitled to the session ID they present before resuming or deleting it.
Two topologies, one trade-off
Pattern 1 is recommended when you need the strongest boundary or genuinely separate process credentials per user. Pattern 2 is the efficient default for many light users, and it is only safe when the application โ not the runtime โ controls tools, auth, and session IDs.
Where the CLI runs
// SDK spawns and owns the CLI (default, stdio)
const a = new CopilotClient();
// SDK spawns the CLI with extra arguments
const b = new CopilotClient({
connection: RuntimeConnection.forStdio({ args: ["--plugin-dir", "./plugins/x"] }),
});
// Connect to an already-running headless CLI server
const c = new CopilotClient({
connection: RuntimeConnection.forUri("http://127.0.0.1:8080"),
});
Mode C is the production shape for containers and serverless: the CLI runs as its own process or sidecar, and your application scales independently. Remember the consequence from Lesson 1 โ extra args are only forwarded when the SDK spawns the CLI, so with forUri you pass flags to the runtime's own command line.
Deployment checklist
| Concern | Do this |
|---|---|
| Container filesystem | ~/.copilot/session-state/ must be writable โ mount a persistent volume or route storage through sessionFs. An ephemeral filesystem silently turns every "resume" into a "create". |
| Runtime state isolation | baseDirectory per pod/worker/tenant |
| Abandoned sessions | sessionIdleTimeoutSeconds, plus your own retention sweep using structured session IDs |
| Session IDs | Encode ownership and purpose; validate the caller is entitled to them before resume/delete |
| Credentials | Per-session token provider with expiresIn; never a shared long-lived token |
| Capability | mode: "empty" + availableTools |
| Approvals | Real permission handler โ approveAll throws under managed settings and destroys the guardrail story everywhere else |
| Budget | sessionLimits.maxAiCredits per session, plus workflow limits for any multi-agent work |
| Telemetry | telemetry.otlpEndpoint wired; decide captureContent deliberately |
| Versions | Pin SDK and CLI together โ they are a pair, and several surfaces are experimental |
| Evals | Replay-based suite in CI with a cost ceiling (Lesson 9) |
cloud option) change where state lives and how provenance propagates โ remember from Lesson 8 that Mission Control may include a message source in its local echo without carrying it in the remote HTTP request. Validate the remote path against your isolation requirements before depending on it.
๐ง Knowledge Check
1. A multi-user web app signs users in via OAuth. Which SDK configuration is correct, and what breaks without it?
2. Why must a session-level gitHubTokenProvider return expiresIn, and what value is typical?
3. Your containerised deployment "loses" sessions on every restart. What is the most likely cause and the correct fix?