modelcontextprotocol · claude · Sep 24, 2026
diff --git a/README.md b/README.md
@@ -63,7 +63,8 @@ npx @modelcontextprotocol/conformance client --command "<client-command>" --scen
 
 - `--command` - The command to run your MCP client (can include flags)
 - `--scenario` - The test scenario to run (e.g., "initialize")
-- `--suite` - Run a suite of tests in parallel: `all`, `core`, `extensions`, `backcompat`, `auth`, `metadata`, `draft` (scenarios targeting the in-progress draft spec), or `sep-835`
+- `--suite` - Run a suite of tests in parallel: `all`, `core`, `extensions`, `backcompat`, `auth`, `metadata`, `draft` (scenarios targeting the in-progress draft spec), `custom` (the scenarios loaded with `--scenario-file`), or `sep-835`
+- `--scenario-file <path>` - Load your own scenarios from a JavaScript module; repeatable (see [Writing Your Own Scenarios](#writing-your-own-scenarios))
 - `--spec-version <version>` - Filter scenarios by spec version (e.g., `2025-11-25`, `2026-07-28`; `draft` is accepted as an alias for the current draft identifier). The draft version selects the latest dated release plus any draft-only scenarios. When omitted, the version is inferred from the scenario's spec applicability (draft-only scenarios run at the draft version, everything else at the latest dated release); an explicitly requested version outside a scenario's applicability window skips the scenario (exit 0) unless `--force` is passed
 - `--force` - Run a scenario even if it is not applicable at the requested `--spec-version`
 - `--requirements <revision>` - Run exactly what a spec revision requires, frozen at its release (see [Conformance Requirements](#conformance-requirements))
@@ -256,6 +257,53 @@ Two things to know:
 A scenario cannot be listed both wholesale and per-check — the wholesale entry already
 excuses everything, so the pair is contradictory and is rejected.
 
+## Writing Your Own Scenarios
+
+You can run client scenarios that are not part of this repository: checks that belong to your own product, or a scenario you are trying out before proposing it here. Put them in a JavaScript module and pass it with `--scenario-file`:
+
+```bash
+npx @modelcontextprotocol/conformance client \
+  --scenario-file ./my-scenarios.mjs \
+  --command "<client-command>"
+```
+
+[`examples/scenarios/trace-id.mjs`](./examples/scenarios/trace-id.mjs) is a complete, commented scenario, and [`trace-id-client.mjs`](./examples/scenarios/trace-id-client.mjs) is a client that passes it. From a checkout of this repository, after `npm install && npm run build`:
+
+```bash
+node dist/index.js client \
+  --scenario-file examples/scenarios/trace-id.mjs \
+  --scenario example/trace-id \
+  --command "node examples/scenarios/trace-id-client.mjs"
+```
+
+**This interface is experimental** and may change between releases.
+
+### The scenario object
+
+The module's default export is one scenario or an array of them. A scenario plays the server and judges what the client sent. It imports nothing from this package.
+
+| Member             | What it is                                                                                                                                                                                                                                                                                                           |
+| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `name`             | Parts of letters, digits, `.`, `_` and `-`, each starting with a letter or digit, with `/` between parts. Prefix it (`acme/login`) so that it cannot clash with a built-in scenario, now or later.                                                                                                                   |
+| `description`      | One sentence.                                                                                                                                                                                                                                                                                                        |
+| `source`           | `{ introducedIn: '<spec version>' }`, the first spec version the scenario applies to, and optionally `removedIn`.                                                                                                                                                                                                    |
+| `start(ctx)`       | Starts the server and returns `{ serverUrl }`. `ctx.createServer(handlers)` serves one handler per method, `(params, request) => result`, which may be async, with the lifecycle of `ctx.specVersion` around them. An optional `context` object in the return value reaches the client as `MCP_CONFORMANCE_CONTEXT`. |
+| `getChecks()`      | Returns the checks, synchronously. It is called after the client has finished and before `stop()`. `server.recorded` holds every request and notification the client sent, in order, without the lifecycle messages.                                                                                                 |
+| `stop()`           | Closes the server.                                                                                                                                                                                                                                                                                                   |
+| `allowClientError` | Optional. `true` if the client is expected to exit with an error.                                                                                                                                                                                                                                                    |
+
+A check has an `id`, a `name`, a `description`, a `timestamp` and a `status`: `SUCCESS`, `FAILURE`, `WARNING`, `SKIPPED` or `INFO`. `FAILURE` and `WARNING` fail the run. `errorMessage`, `details` and `specReferences` are optional. Use one `id` per check whether it passes or fails; expected-failures entries refer to it. Give your ids a prefix of your own: `traceability` counts ids of the form `sep-<number>-…`. The runner adds a `wire-schema-valid` check when it has seen traffic. More conventions are in [AGENTS.md](./AGENTS.md).
+
+### How loaded scenarios run
+
+- **Selection.** With `--scenario-file`, only the loaded scenarios can be selected. `--scenario <name>` runs one and prints each check; without `--command` it starts in interactive mode and prints the server URL, so any client can be pointed at it. Without `--scenario`, all loaded scenarios run as the `custom` suite.
+- **Marked as custom.** Every run names the loaded scenarios and says that they are not part of MCP conformance. `list --scenario-file <path>` and the suite summary mark them `(custom)`.
+- **Spec versions.** `--spec-version` selects and skips them by their `source`, as for built-in scenarios. The same scenario serves every spec version it applies to; the example client speaks the lifecycle of `2025-11-25`, the default.
+- **Not part of conformance.** Loaded scenarios never count towards a requirement set or a tier: `--scenario-file` cannot be combined with `--requirements` or with a built-in suite. [Expected failures](#expected-failures) work for them as for any scenario.
+- **Trust.** The module is loaded with `import()`, which runs its code. Load only files you trust.
+
+A scenario that has proved useful can be proposed for the suite: see [CONTRIBUTING.md](./CONTRIBUTING.md).
+
 ## GitHub Action
 
 This repo provides a composite GitHub Action so SDK repos don't need to write their own conformance scripts.

diff --git a/examples/scenarios/trace-id-client.mjs b/examples/scenarios/trace-id-client.mjs
@@ -0,0 +1,19 @@
+/**
+ * A client that passes examples/scenarios/trace-id.mjs. The runner appends
+ * the server URL to the command. It uses the SDK, so it speaks the lifecycle
+ * of spec version 2025-11-25, which is the default.
+ */
+import { argv } from 'node:process';
+import { URL } from 'node:url';
+import { Client } from '@modelcontextprotocol/sdk/client/index.js';
+import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
+
+const client = new Client({ name: 'trace-id-client', version: '1.0.0' });
+await client.connect(new StreamableHTTPClientTransport(new URL(argv[2])));
+await client.listTools();
+await client.callTool({
+  name: 'echo',
+  arguments: { text: 'hello' },
+  _meta: { 'com.example/traceId': 'trace-0001' }
+});
+await client.close();
diff --git a/examples/scenarios/trace-id.mjs b/examples/scenarios/trace-id.mjs
@@ -0,0 +1,104 @@
+/**
+ * Example of a scenario kept outside the suite: a check that belongs to one
+ * product, not to the spec. Here a team requires its client to tag every
+ * tool call with a trace id in `_meta`. The README, "Writing Your Own
+ * Scenarios", says how to run it.
+ *
+ * A scenario plays the server. `start()` serves the handlers below and
+ * returns the URL the client under test is given; `getChecks()` judges what
+ * the client sent. Nothing is imported from the suite.
+ */
+
+const TRACE_ID = 'com.example/traceId';
+
+const SPEC_REFERENCES = [
+  {
+    id: 'MCP-Tools',
+    url: 'https://modelcontextprotocol.io/specification/2025-11-25/server/tools#calling-tools'
+  },
+  {
+    id: 'MCP-Meta',
+    url: 'https://modelcontextprotocol.io/specification/2025-11-25/basic/index#_meta'
+  }
+];
+
+class TraceIdScenario {
+  // Prefix your names so they can never clash with a built-in scenario.
+  name = 'example/trace-id';
+  description = 'Example: the client tags every tool call with a trace id';
+  // The first spec version the scenario applies to.
+  source = { introducedIn: '2025-06-18' };
+  server = null;
+
+  async start(ctx) {
+    // One handler per method. It receives the request's `params` and returns
+    // the result. The suite supplies the lifecycle for `ctx.specVersion`.
+    this.server = await ctx.createServer({
+      'tools/list': () => ({
+        tools: [
+          {
+            name: 'echo',
+            description: 'Return the text it is given',
+            inputSchema: {
+              type: 'object',
+              properties: { text: { type: 'string' } },
+              required: ['text']
+            }
+          }
+        ]
+      }),
+      'tools/call': (params) => ({
+        content: [{ type: 'text', text: String(params.arguments?.text) }]
+      })
+    });
+    // `context`, if you return one, reaches the client as MCP_CONFORMANCE_CONTEXT.
+    return { serverUrl: this.server.url };
+  }
+
+  async stop() {
+    await this.server?.close();
+  }
+
+  // Called after the client has finished and before stop().
+  getChecks() {
+    // Every request and notification the client sent, in order, without the
+    // lifecycle ones (initialize, notifications/initialized, server/discover).
+    const calls = (this.server?.recorded ?? []).filter(
+      (request) => request.method === 'tools/call'
+    );
+    const untagged = calls.filter(
+      (call) => typeof call.params?._meta?.[TRACE_ID] !== 'string'
+    );
+    const called = calls.length > 0;
+    const tagged = called && untagged.length === 0;
+
+    // One id per check, whether it passes or fails.
+    return [
+      {
+        id: 'example-tool-called',
+        name: 'ToolCalled',
+        description: 'The client called a tool',
+        status: called ? 'SUCCESS' : 'FAILURE',
+        timestamp: new Date().toISOString(),
+        specReferences: SPEC_REFERENCES,
+        ...(!called && { errorMessage: 'The client never sent tools/call' })
+      },
+      {
+        id: 'example-trace-id-sent',
+        name: 'TraceIdSent',
+        description: `Every tool call carries _meta["${TRACE_ID}"]`,
+        status: tagged ? 'SUCCESS' : 'FAILURE',
+        timestamp: new Date().toISOString(),
+        specReferences: SPEC_REFERENCES,
+        ...(!tagged && {
+          errorMessage: called
+            ? `${untagged.length} of ${calls.length} tool calls had no trace id`
+            : 'The client never sent tools/call'
+        })
+      }
+    ];
+  }
+}
+
+// Export one scenario, or an array of them.
+export default new TraceIdScenario();
diff --git a/src/index.ts b/src/index.ts
@@ -36,6 +36,12 @@ import {
   resolveSpecVersion
 } from './scenarios';
 import type { SpecVersion } from './scenarios';
+import {
+  applyScenarioFiles,
+  isCustomScenario,
+  listCustomScenarios,
+  type ScenarioFileOptions
+} from './scenarios/custom';
 import { ConformanceCheck } from './types';
 import {
   AuthorizationServerOptionsSchema,
@@ -206,6 +212,31 @@ function filterScenariosBySpecVersion(
   return allScenarios.filter((s) => allowed.has(s));
 }
 
+function collect(value: string, previous: string[] = []): string[] {
+  return [...previous, value];
+}
+
+async function applyScenarioFilesOrExit(
+  options: ScenarioFileOptions
+): Promise<string[]> {
+  try {
+    const names = await applyScenarioFiles(options);
+    if (names.length > 0) {
+      console.error(
+        `Loaded custom scenarios, which are not part of MCP conformance: ${names.join(', ')}`
+      );
+    }
+    return names;
+  } catch (error) {
+    console.error(error instanceof Error ? error.message : String(error));
+    process.exit(1);
+  }
+}
+
+function customLabel(name: string): string {
+  return isCustomScenario(name) ? ' (custom)' : '';
+}
+
 const program = new Command();
 
 program
@@ -222,6 +253,11 @@ program
   .option('--command <command>', 'Command to run the client')
   .option('--scenario <scenario>', 'Scenario to test')
   .option('--suite <suite>', 'Run a suite of tests in parallel (e.g., "auth")')
+  .option(
+    '--scenario-file <path>',
+    'Load your own scenarios from a JavaScript module (repeatable). The module is executed.',
+    collect
+  )
   .option('--timeout <ms>', 'Timeout in milliseconds', '30000')
   .option(
     '--expected-failures <path>',
@@ -243,6 +279,8 @@ program
   .option('--verbose', 'Show verbose output')
   .action(async (options, cmd) => {
     try {
+      const suiteGiven = options.suite !== undefined;
+      await applyScenarioFilesOrExit(options);
       const timeout = parseInt(options.timeout, 10);
       const verbose = options.verbose ?? false;
       const outputDir = options.outputDir;
@@ -267,7 +305,11 @@ program
       // Handle suite mode
       if (options.suite || options.requirements !== undefined) {
         if (!options.command) {
-          console.error('--command is required when using --suite');
+          console.error(
+            suiteGiven || requirements
+              ? '--command is required when using --suite'
+              : '--command is required to run the loaded scenarios. To start one in interactive mode, name it with --scenario.'
+          );
           process.exit(1);
         }
 
@@ -279,6 +321,7 @@ program
           auth: listAuthScenarios,
           metadata: listMetadataScenarios,
           draft: listDraftScenarios,
+          custom: listCustomScenarios,
           'sep-835': () =>
             listAuthScenarios().filter((name) => name.startsWith('auth/scope-'))
         };
@@ -393,7 +436,7 @@ program
           const status = failed === 0 && warnings === 0 ? '✓' : '✗';
           const warningStr = warnings > 0 ? `, ${warnings} warnings` : '';
           console.log(
-            `${status} ${result.scenario}: ${passed} passed, ${failed} failed${warningStr}`
+            `${status} ${result.scenario}${customLabel(result.scenario)}: ${passed} passed, ${failed} failed${warningStr}`
           );
 
           if (verbose && failed > 0) {
@@ -452,7 +495,7 @@ program
         console.error('\nAvailable client scenarios:');
         listScenarios().forEach((s) => console.error(`  - ${s}`));
         console.error(
-          '\nAvailable suites: all, core, extensions, backcompat, auth, metadata, draft, sep-835'
+          '\nAvailable suites: all, core, extensions, backcompat, auth, metadata, draft, custom, sep-835'
         );
         process.exit(1);
       }
@@ -938,6 +981,11 @@ program
   .option('--client', 'List client scenarios')
   .option('--server', 'List server scenarios')
   .option('--authorization', 'List authorization server scenarios')
+  .option(
+    '--scenario-file <path>',
+    'Also list the client scenarios a JavaScript module defines (repeatable). The module is executed.',
+    collect
+  )
   .option(
     '--spec-version <version>',
     'Filter scenarios by spec version (cumulative for date versions)'
@@ -946,7 +994,8 @@ program
     '--requirements <revision>',
     'List exactly what a spec revision requires, frozen at its release'
   )
-  .action((options) => {
+  .action(async (options) => {
+    const custom = await applyScenarioFilesOrExit(options);
     const specVersionFilter = options.specVersion
       ? resolveSpecVersion(options.specVersion)
       : undefined;
@@ -995,7 +1044,7 @@ program
       }
       clientScenarioNames.forEach((s) => {
         const v = getScenarioSpecVersions(s);
-        console.log(`  - ${s}${v ? ` [${v}]` : ''}`);
+        console.log(`  - ${s}${v ? ` [${v}]` : ''}${customLabel(s)}`);
       });
     }
 
@@ -1023,6 +1072,8 @@ program
         console.log(`  - ${s}${v ? ` [${v}]` : ''}`);
       });
     }
+    // A loaded module may have left a timer or a socket open.
+    if (custom.length > 0) process.exit(0);
   });
 
 program.parse();