Capability & Scenario Registry

A capability is a prompt × model × tools that produces an outcome from an input. Each capability is verified by scenarios — an input, the tool requests/responses it should drive, and an expectation whose shape depends on the Shared test ladder (test-pyramid.md): at S3 the expectation is the tool trajectory + agent output; at S5 it is the resulting database state. MCP tools carry their own S1 contract + S2 approval, inherited by every capability that calls them.

Capability	Type	Owner	Tools	Ladder coverage S1→S5	Eval health

Ladder: S1contract S2approval S3selection + trajectory S4integration S5outcome · S?not defined S?n/a

The registry is loaded at runtime from the canonical YAML records — capability definitions in agent/src/redgate_assistant/capabilities/, plus tools and per-scenario files in evals/records/ — the same files the offline evals read. A capability (prompt × model × tools) is verified by scenarios (input + tool req/resp + a level-dependent expectation). Click any row for its detail / scorecard.
The builder requires the dev server: start evals/offline/run_builder_server.py (needs AWS creds) and open the printed URL. It enumerates and serves the records from disk, and a variant's Run registers the capability against the agent, builds, and evaluates its scenarios with the real agent — scores, agent output and reasoning shown inline.

Capability & Scenario Registry

Add an MCP tool

Add an MCP server

Build & register a capability

Scenarios — how this capability is verified

Experiment — prompt × model × tools