Interface: ConformanceTestConfig<TBus, TConnector, TAgent>

This bus is used to emit events and handle requests in the adapter’s domain (e.g., ‘claude-code’, ‘codex’). The conformance suite observes events on this bus to verify behavior.

Namespace Isolation: Each adapter should use a dedicated namespace to prevent cross-talk between concurrent test suites.

Example

import { MakaioBus } from '@makaio/bus-core';

const scopedBus = MakaioBus.scoped('claude-code');

capabilities?

optional capabilities?: object

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:124

Optional capability flags indicating adapter-specific features.

These flags allow conformance tests to:

Skip tests for unsupported features (e.g., replace mode)
Verify optional behaviors when supported
Provide clear capability documentation

Default: All capabilities are false (core features only)

When to Enable

supportsReplace: Adapter implements ‘replace’ delivery mode (clears pending queue before enqueuing message)
supportsInterrupt: Adapter exposes interrupt() method (cancels active turn mid-execution)
supportsUsageMetrics: Adapter emits agent.usage events (token counts, cost tracking)

supportsInterrupt?

optional supportsInterrupt?: boolean

Supports interrupt() method for cancelling active turns.

When enabled, the adapter can abort mid-turn execution (e.g., during a long-running tool call or reasoning phase).

Claude Code: true (exposes interrupt via MCP SDK) Stateless Adapters: false (no turn state to interrupt)

supportsReplace?

optional supportsReplace?: boolean

Supports ‘replace’ message delivery mode.

When enabled, the adapter clears the pending message queue before enqueuing a new message. This is useful for “superseding” interactions.

Claude Code: true (implements replace via MCP SDK) Basic Adapters: false (enqueue/immediate only)

supportsUsageMetrics?

optional supportsUsageMetrics?: boolean

Supports usage metrics via agent.usage events.

When enabled, the adapter emits token counts and cost data after turns or at session completion.

Claude Code: true (exposes MCP usage data) Free-Tier Adapters: false (no usage tracking)

Examples

capabilities: {
  supportsReplace: true,      // Claude Code's replace mode
  supportsInterrupt: true,     // Claude Code's interrupt()
  supportsUsageMetrics: true,  // Usage tracking
}

capabilities: undefined // or omit entirely
// Tests will skip replace, interrupt, tool approval, and usage tests

cleanup?

optional cleanup?: () => void | Promise<void>

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:326

Optional cleanup hook for adapter-specific test infrastructure.

Use this for resources created by createTestConfig() that outlive individual connectors/adapters (for example shared test servers).

Returns

void | Promise<void>

createAdapter?

optional createAdapter?: (options?) => Promise<AIAdapter<TBus, TConnector, TAgent>>

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:298

Factory function for creating full adapter instances (orchestration tests).

Orchestration vs Agent Tests:

Agent tests (createAgent): Direct agent instantiation, bypass session manager
Orchestration tests (createAdapter): Full pipeline via MakaioBus.request()

When provided, enables orchestration-level conformance tests that verify:

Bus request/response flow (AdapterSubjects.sendMessage, etc.)
Session manager lifecycle
Event transformation pipeline (SDK to Adapter to Global)
Multi-adapter coordination

Important: Each call should return a fresh instance with unique adapterId for test isolation. The factory receives optional init options to allow passing a specific adapterId.

Parameters

options?

AIAdapterInitOptions

Returns

Promise<AIAdapter<TBus, TConnector, TAgent>>

Fresh AIAdapter instance configured for orchestration testing

Example

createAdapter: (options) => createClaudeAdapter({
  adapterId: options?.adapterId ?? `test-${crypto.randomUUID()}`,
  ...options
})

createConnector

createConnector: (options?) => Promise<TConnector>

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:58

Factory function for creating fresh agent instances.

Important: Return a new instance on each call. The conformance suite decides when to instantiate based on isolation requirements.

DO NOT instantiate eagerly or return a singleton—tests may run concurrently or require pristine state.

Parameters

options?

CreateTestAgentOptions

Returns

Promise<TConnector>

Fresh connector instance configured for testing

Example

createConnector: () => new ClaudeCodeConnector({
  bus: testBus,
  agentId: `test-${Date.now()}`,
  sessionId: undefined, // Let adapter generate
})

options?

optional options?: object

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:173

Test configuration and environment options.

These options tune test behavior without affecting adapter functionality. All fields are optional with sensible defaults.

concurrency?

optional concurrency?: number

Maximum concurrent test FILES for this adapter.

Controls how many conformance test files run simultaneously when using the programmatic test runner (scripts/test-adapters.ts). Each adapter can specify its own concurrency based on rate limits or resource constraints.

How it works:

Test runner creates a per-adapter queue with this concurrency limit
Test files for this adapter run through the queue
Multiple adapters can run in parallel, each with its own queue

vs testConcurrency:

concurrency: Limits concurrent TEST FILES
testConcurrency: Limits concurrent OPERATIONS within test helpers

Default Value

2 (when not specified)

Example

// Rate-limited API (e.g., Gemini free tier)
concurrency: 1  // Run test files sequentially

// High-throughput API
concurrency: 4  // Run 4 test files in parallel

defaultTimeout?

optional defaultTimeout?: number

Default timeout for tests in milliseconds.

Applies to async operations like agent.start() and message delivery. Individual tests can override via Vitest’s { timeout: ... }.

Default Value

10000 (10 seconds)

Example

defaultTimeout: 30000 // 30s for slow CI environments

primaryModel?

optional primaryModel?: TestModelRef

Cheap/fast model for cost-sensitive test execution.

Used by most conformance tests that make real API calls. Derived from the default provider preset’s fastModel (or defaultModel as fallback).

Claude: { definitionId: 'anthropic', modelName: 'haiku' } Gemini: { definitionId: 'gemini', modelName: 'gemini-2.5-flash' }

Example

primaryModel: { definitionId: 'anthropic', modelName: 'haiku' }

secondaryModel?

optional secondaryModel?: TestModelRef

Secondary model for model-switching tests.

Used as the target model in model-change conformance tests. Derived from the default provider preset’s defaultModel.

Claude: { definitionId: 'anthropic', modelName: 'sonnet' } Gemini: { definitionId: 'gemini', modelName: 'gemini-2.5-pro' }

Example

secondaryModel: { definitionId: 'anthropic', modelName: 'sonnet' }

testConcurrency?

optional testConcurrency?: number

Maximum concurrent test operations for this adapter.

When set, test infrastructure serializes context creation and key operations through a per-adapter queue. Useful for adapters with aggressive rate limits (e.g., Gemini free tier).

Default: undefined (no throttling - tests run fully concurrent)

Example

testConcurrency: 1 // Serialize all test operations

tmpDir?

optional tmpDir?: string

Temporary directory for file operations during tests.

Used for tests that create artifacts (logs, cached responses, etc.). Should be isolated per test suite to prevent collisions.

Default Value

OS temp dir + random suffix

Example

tmpDir: '/tmp/conformance-tests-claude-code'

Example

options: {
  defaultTimeout: 30000,               // 30s for slow CI
  fastModel: 'claude-3-haiku-20240307', // Fast model for tests
  tmpDir: '/tmp/conformance-tests',    // Isolated temp directory
}

testProviderContext?

optional testProviderContext?: object

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:318

Unresolved provider context for the test provider.

Contains credential refs (e.g. env:ANTHROPIC_API_KEY) built from the test provider definition’s credentialEnvVars. Used by orchestration tests that call startAgent directly (bypassing the conformance createConnector path where resolveTestConfig handles credential ref building).

The conformance test infrastructure registers a credential channel handler that resolves env: refs from process.env so connectors can call resolveConnectorCredentials() without a real credential store.

credentialEnvVars?

optional credentialEnvVars?: Record<string, string>

Maps credential keys to environment variable names for subprocess adapters. E.g., { apiKey: 'ANTHROPIC_API_KEY' }.

credentialRefs

credentialRefs: Record<string, string & $brand<"CredentialRef">>

Credential references resolved at the connector layer, not on the bus.

definitionId

definitionId: string

Provider definition ID (e.g., 'anthropic', 'alibaba').

endpointOverrides?

optional endpointOverrides?: object

Endpoint URL overrides keyed by protocol.

endpointOverrides.anthropic?

optional anthropic?: string

endpointOverrides.openai?

optional openai?: string

providerConfigId

providerConfigId: string

Provider config UUID. Links back to the ProviderConfig that produced this context.

Methods

registerToolApprovalHandler()

registerToolApprovalHandler(connector, context): () => void

Defined in: ../../../adapters/core/src/types/conformance-test-config.ts:69

The handler will only receive approval requests from the given connector (filtered by agentId). Returns an unsubscribe function for cleanup.

Parameters

connector

TConnector

The connector to scope approval handling to

context

ToolApprovalContext | (() => Promise<ToolApprovalContext>)

Optional approval context (e.g., allowed tools, policies)

Returns

Unsubscribe function to remove the handler

() => void