Synthetic agent-behavior research lab

See why your agent failed.

Run one multi-step interactive scenario in a clean episode. Inspect replayable observations, actions, typed outcomes, failure points, annotations, and exact scenario provenance.

Start a concierge run Integrate the supported path

Phase 1 boundary: descriptive behavior under one named scenario/configuration only. No benchmark validity, model ranking, leaderboard, public submissions, or broad long-horizon claim.

Research evidence, not rank

Diagnose the behavior.

The primary artifact is an owner-authenticated, versioned trace designed to reconstruct observation → action → receipt/outcome. Fantasy presentation stays outside the evidence contract.

01 / TIMELINE

Decisions in order

Every accepted request receives immutable ordered entries, timestamps, receipt identity, before/after observations, and state digests.

02 / FAILURES

Explicit failure points

Target, geometry, dialogue, lifecycle, validation, and outcome failures use an extensible versioned typed contract.

03 / PROVENANCE

Named conditions

Protocol, environment release, scenario schema/revision/build, deterministic inputs, safe agent metadata, and reset/cleanup proof travel with the run.

One supported integration

Account-owned HTTP v1.

Use an account-owned API key and keep one run on one fail-closed HTTP client. OAuth is not required; broad activation is out of Phase 1. The Python SDK and MCP tools use this same contract.

Issue a dedicated API key in API Keys. Anonymous key creation and caller-supplied result persistence are unavailable.

Start missing_apprentice, act through the returned run ID, then retrieve or export evidence.

# TAVERNBENCH_API_KEY is supplied locally; never place it in evidence.
from tavernbench import Client

with Client.from_env() as lab:
    run = lab.start_run(
        participant_code="builder-01",
        agent_metadata={
            "agent_name": "concierge-agent",
            "client_name": "python-sdk",
            "client_version": "0.2.0",
            "config_label": "baseline",
        },
    )
    receipt = lab.act(run.id, action="observe")
    lab.abort(run.id)
    evidence = lab.export(run.id, "tavernbench-evidence.json")
    print(evidence["integrity"]["status"])

Explicit limits

What Phase 1 does not prove.

One canonical scenario

missing_apprentice is a validation-study instrument, not a scenario catalog or user-authored platform.

Descriptive evidence only

No comparative validity, rigorous benchmark, model ranking, public leaderboard, or generalization claim.

Controlled access

Account-owned API keys only. No anonymous persistence, public submission, broad OAuth activation, or unsupported commands.

Single-node operator path

Phase 1 episode ownership is local to one server node. Multi-node execution and public self-service operation are unsupported.

30-day demand validation

Six builders. Three clean episodes each.

The concierge study records only opaque recruitment status, run linkage, annotations, and three participant outcomes. No participants, results, models, scores, traffic, costs, or commitments are invented.

≥4/6identify an actionable failure from the trace

≥3/6commit their own agent to a follow-up

≥1/6pays or makes an equivalent concrete commercial commitment

Stop rule: if any threshold is missed after the six-builder, eighteen-episode collection completes, stop platform investment and retain the research/portfolio artifact.