● IN ACTIVE DEVELOPMENT EXPERIMENTAL PREVIEW TRACE DIAGNOSTICS MVP

Diagnostic trace audit.
Observable facts, not marketing claims.

A standalone, local-first service for account-scoped coding-agent trace diagnostics. Inspect JSONL execution logs to detect runaway tool loops, token consumption anomalies, and observable failure states.

Mandatory Development Notice: Agent Trials is currently an isolated diagnostic MVP. Uploaded traces record observable metrics only and do not represent independently verified execution. Direct agent execution is disabled until an isolated hardware sandbox is verified. 20-task benchmark suites remain unverified until Codex validation is complete.

Inspect Trace Schema Review Intended Architecture Request Pilot Access

agent_trials · trace_analyzer (isolated) ● ISOLATED RUNTIME

$ agent-trials ingest --trace coding_session_049.jsonl
[PARSE] Ingested 142 observable agent steps...
[BOUND] Direct execution status: DISABLED (Sandbox pending)
[STATS] Observable Trace Metrics:
        • Total Steps: 142 · Elapsed: 18.4s
        • Tool Invocations: 89 (42 read, 18 write, 29 exec)
        • Token Usage: 48,210 tokens
        • Observed Exceptions: 3 (recovered via retry)
[AUDIT] Detected 1 potential infinite retry loop on 'git status'
---------------------------------------------------------------
DIAGNOSTIC REPORT EMITTED · Observable trace metrics only

OBSERVABLE TRACE SCHEMA

JSONL Trace Ingestion Format

The diagnostic engine accepts newline-delimited JSON where each line represents an observable agent step, tool call, or error.

Input JSONL Trace Record Per-Step Telemetry

{
  "step_index": 12,
  "type": "tool_call",
  "tool": "filesystem",
  "name": "edit_file",
  "path": "/workspace/src/auth.py",
  "tokens": 420,
  "duration_ms": 14.2,
  "error": null
}
{
  "step_index": 13,
  "type": "tool_call",
  "tool": "shell",
  "name": "run_command",
  "command": "pytest tests/test_auth.py",
  "tokens": 850,
  "duration_ms": 820.5,
  "error": "AssertionError: 401 != 200"
}

Diagnostic Synthesis (Report) Observable Metrics

{
  "diagnostic_id": "diag_8a92cfb140",
  "observable_metrics": {
    "total_steps": 14,
    "tool_calls_count": 8,
    "unhandled_errors": 1,
    "token_efficiency_ratio": "0.82",
    "repeated_tool_patterns": [
      {"pattern": "edit_file -> run_command", "count": 3}
    ]
  },
  "verification_boundary": {
    "direct_execution_performed": false,
    "trace_tamper_evident": false,
    "status": "OBSERVABLE_METRICS_ONLY"
  }
}

Zero Fabrication Guarantee: Values not directly observable in the uploaded trace (such as host CPU seconds, peak RAM, or unrecorded inference costs) are marked explicitly as null rather than estimated or guessed.

ARCHITECTURAL BLUEPRINT INTENDED CAPABILITIES

What Agent Trials is being built to solve.
Engineering discipline for agent evaluation.

🔄

Tool Loop Detection

Flags recursive agent loops where an LLM repeatedly invokes the same failing tool command with identical arguments without making progress.

⚡

Token Burn Anomalies

Identifies steps where context window bloat causes sudden exponential token consumption increases without corresponding output gains.

🛑

Safety Boundary Violations

Detects unauthorized attempts to access prohibited paths (e.g. .env, ~/.ssh) or make unapproved external network requests.

⏱

Latency Bottleneck Mapping

Measures step durations to highlight whether an agent's wall-clock time was consumed by slow model generation, long compile commands, or test timeouts.

🔒

Isolated Account Scope

Traces are partitioned by user account. No shared global data repository, zero leakage between independent developer investigations.

📊

JSON Diagnostic Reports

Generates programmatic JSON diagnostic summaries at /api/runs/{id}/report for automated ingestion into CI/CD quality gates.

PILOT WAITLIST

Join the Agent Trials Diagnostic Pilot.

No payment is collected. We strictly enforce honest claims: benchmark features will not be advertised as available until independently verified.

Diagnostic trace audit.Observable facts, not marketing claims.