test(conformance): assert the CMA beta-header resource-family contract (#561)

The path decides which beta a request needs, with one documented exception; the code says what the regex matches, and only a request says whether the exception is reached.
2026-09-28 06:03:00 +08:00 · 2026-09-27 00:31:46 +08:00
parent 0d481f4e1f
commit 68738a51c5
2 changed files with 130 additions and 0 deletions
@@ -48,6 +48,7 @@
 - `GET /v1/sessions/{id}/events` now answers the published collection envelope `{data, prev_page, next_page}` instead of `{data, has_more, first_id, last_id}`. Three of those four field names occur **nowhere** in the published contract — a search of the published documentation finds zero occurrences of `has_more` and `first_id` against 34 of `next_page` — so a client built against the published contract read `next_page` from this listing, found nothing, and could not page at all: the listing cut its page at `limit` and reported the cut in a vocabulary the caller does not speak. `next_page` is now a followable cursor carrying `{order, filter: {session_id}, after_id}`, which is what `contracts/anthropic-cma/pagination.md` had already described this route as doing; `prev_page` is always `null` because the scan is forward-only and an invented predecessor would not resolve. The local `after_id` and `limit` are unchanged. A `page` the route cannot have issued — malformed, issued for a different session, replayed under a different ordering, or naming an event that is not in the log — is a `400` rather than a silent restart from the first event, because a restart is answered as though it were the page after the caller's position and the caller cannot detect it. The SDK's `events()` return type and its accepted options follow the route. The one canonical `/v1` collection still on the local envelope is `/v1/sessions/{id}/artifacts`, filed separately so each conversion is verified on its own.

 ### Added
+- Added the first `tests/conformance/` suite: the CMA beta-header contract is now asserted by executing requests, including that memory-store routes require their own beta, that a canonical resource refuses that beta, that combining both is refused with its own code, and that the two-beta listing exception is admitted under either. No runtime behaviour changes.

 - Refuses an unimplemented query parameter on the vault and memory-store listings, the two routes that could not adopt admission until they honoured `limit`/`page`. A parameter neither listing read was ignored, so `?include_archived=true&status=archived` answered a page as if the second half of the request had been understood — the failure mode `src/api/routes/query-params.ts` exists to name, and the one thing those two routes did differently from the thirteen that already refuse by name. The admission list is derived from the parameter-name constants the readings use (`COLLECTION_LISTING_QUERY_PARAMS`) rather than repeating the strings in both handlers, because the strings and the readings are one fact and a second copy is the one that gets forgotten; the tests assert the two collections advertise the **identical** list, and that the published `/v1/vaults` mount refuses identically to the local `/v1/credential-vaults` one, since one router serves both. `beta` is still accepted and still not advertised, because its compatibility semantics are not modelled. This also consolidates `memory-stores.ts`'s two separate imports from `query-params.ts` into one. Docs updated in `docs/api.md`, `contracts/anthropic-cma/credentials.md` and `contracts/anthropic-cma/memory-stores.md`.

@@ -0,0 +1,129 @@
+/**
+ * CMA beta-header contract — the resource family is decided by the path.
+ *
+ * This is the first suite under `tests/conformance/` and the first slice of the
+ * official-SDK conformance work: it is deliberately **provider-free** and needs
+ * no dependency, because the shape layer is the part of the contract that can be
+ * decided today by executing real requests (see
+ * `local-notes/work/06-official-sdk-conformance.md`).
+ *
+ * The reason this belongs in a conformance suite rather than in a unit test of
+ * the middleware is the point the plan makes about it: the resource-family rule
+ * and its single exception **cannot be confirmed by reading the code**. Reading
+ * `isMemoryListPath` tells you what the regex is; only a request tells you that
+ * `GET /v1/memory_stores/:id/memories` really is admitted under either beta while
+ * every other memory route is not.
+ *
+ * Not covered here, and named so the gap is visible rather than implied: the
+ * official `@anthropic-ai/sdk` is still not a dependency, no OpenAPI baseline
+ * exists, and the semantic layer (a real turn, `stop_reason`, the custom-tool
+ * loop) needs a model provider. Those are the next slices.
+ */
+
+import { afterEach, describe, expect, it } from 'vitest';
+import { mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { createServer } from '@/api/server.js';
+import { Database } from '@/core/db/database.js';
+import { SessionManager } from '@/core/session/session-manager.js';
+
+const VERSION = '2023-06-01';
+const MANAGED_AGENTS_BETA = 'managed-agents-2026-04-01';
+const AGENT_MEMORY_BETA = 'agent-memory-2026-07-22';
+
+function makeApp() {
+  const tmpDir = mkdtempSync(join(tmpdir(), 'ma-conformance-beta-'));
+  const db = new Database(join(tmpDir, 'test.db'));
+  db.runMigrations();
+  db.exec(`INSERT INTO environments (id, name, config) VALUES ('env_default', 'local', '{}')`);
+  db.exec(`INSERT INTO agents (id, name, definition) VALUES ('agent_a', 'a', '{}')`);
+  const app = createServer({
+    db,
+    sessionManager: new SessionManager(db),
+    agents: [{ name: 'a', model: 'm', system: 'p' }],
+    reloadAgents: () => ({ agents: [], errors: [] }),
+  });
+  return { app, db, tmpDir };
+}
+
+type TestContext = ReturnType<typeof makeApp>;
+const contexts: TestContext[] = [];
+
+function app() {
+  const ctx = makeApp();
+  contexts.push(ctx);
+  return ctx.app;
+}
+
+function request(path: string, beta: string, method = 'GET') {
+  return app().request(path, {
+    method,
+    headers: { 'anthropic-version': VERSION, 'anthropic-beta': beta },
+  });
+}
+
+async function errorOf(res: Response): Promise<{ type?: string; code?: string; message?: string }> {
+  const body = (await res.json()) as { error?: { type?: string; code?: string; message?: string } };
+  return body.error ?? {};
+}
+
+describe('CMA beta-header contract', () => {
+  afterEach(() => {
+    for (const ctx of contexts.splice(0)) {
+      ctx.db.close();
+      rmSync(ctx.tmpDir, { recursive: true, force: true });
+    }
+  });
+
+  it('admits the documented combination for a canonical resource', async () => {
+    const res = await request('/v1/agents', MANAGED_AGENTS_BETA);
+
+    expect(res.status).toBe(200);
+  });
+
+  it('refuses the canonical beta on a memory-store route and names the one it wants', async () => {
+    const res = await request('/v1/memory_stores', MANAGED_AGENTS_BETA);
+
+    expect(res.status).toBe(400);
+    const error = await errorOf(res);
+    expect(error.type).toBe('invalid_request_error');
+    expect(error.code).toBe('unsupported_anthropic_beta');
+    // The message is part of the contract for a human reading a failed call: it
+    // has to name the beta this path actually requires.
+    expect(error.message).toContain(AGENT_MEMORY_BETA);
+  });
+
+  it('refuses the memory beta on a canonical resource and names the one it wants', async () => {
+    const res = await request('/v1/agents', AGENT_MEMORY_BETA);
+
+    expect(res.status).toBe(400);
+    const error = await errorOf(res);
+    expect(error.code).toBe('unsupported_anthropic_beta');
+    expect(error.message).toContain(MANAGED_AGENTS_BETA);
+  });
+
+  it('refuses a memory-store request that combines both betas', async () => {
+    const res = await request('/v1/memory_stores', `${MANAGED_AGENTS_BETA}, ${AGENT_MEMORY_BETA}`);
+
+    expect(res.status).toBe(400);
+    const error = await errorOf(res);
+    // A distinct code, not the generic unsupported-beta one: this request can
+    // never be valid for the path, which is a different fault from a caller who
+    // used the wrong resource family.
+    expect(error.code).toBe('conflicting_memory_store_beta');
+  });
+
+  it('admits the documented memory-listing exception under either beta', async () => {
+    // The one route CMA defines as equivalent under both betas. The store does not
+    // exist, so the status is not the assertion; what is asserted is that the beta
+    // check did not refuse the request, which is the exception itself.
+    for (const beta of [MANAGED_AGENTS_BETA, AGENT_MEMORY_BETA]) {
+      const res = await request('/v1/memory_stores/ms_absent/memories', beta);
+      const error = await errorOf(res);
+      expect(error.code, `${beta} should be admitted on the memory listing`).not.toBe(
+        'unsupported_anthropic_beta',
+      );
+    }
+  });
+});