Tasks — US5: Test View Dashboard — Explorer (drilldown) + Compare (3 variants) (Priority: P2)¶

Framework: Dash (Plotly). An earlier draft of this list targeted Streamlit; visual deltas against the handoff were unacceptable and US4 has already been rebuilt on Dash (see tasks-us4.md). Explorer and Compare mount into the same app.py scaffold as additional view modules. All state references below assume the US4 dcc.Store(id="app-state") payload + pattern-match callbacks; there is no st.session_state.

⭐ Primary Source of Truth¶

The design handoff bundle is AUTHORITATIVE for every visual, interaction, and layout decision in this task list. When spec.md and the handoff differ, the handoff wins. Spec.md defines what the Explorer must contain functionally; the handoff defines how it must look, behave, and sequence. Compare is introduced by the handoff and has no spec counterpart — the handoff is the ONLY source of truth for that view.

Always consult, in this order of precedence:

design_handoff_holodeck_eval_dashboard/README.md — Views §2 (Explorer) + §3 (Compare) + §State mapping. (Ignore any "Streamlit notes" — we're on Dash now; map those to dcc.Store + @callback.)
design_handoff_holodeck_eval_dashboard/Evaluation Dashboard.html — open in a browser; click the Explorer and Compare tabs and try every interaction before writing code
design_handoff_holodeck_eval_dashboard/explorer.js — Explorer component source (run list collapse, case list, collapsibles, tool-call rendering, JsonPretty highlighting, 500-byte collapse threshold). Since Dash is React-backed, components map almost 1:1 to html.*/dcc.* trees.
design_handoff_holodeck_eval_dashboard/compare.js — Compare component source; the 3 variants, case matrix, delta pills, compare tray, + button lifecycle — ALL come from here
design_handoff_holodeck_eval_dashboard/styles.js — exact CSS for chat bubbles, tool-call panels, config-diff highlighting, heatmap cells. Copied verbatim into assets/02-holodeck.css by US4 T344; extend (don't fork) that file here.
design_handoff_holodeck_eval_dashboard/data.js — the sampleConversation map (data.js:160–178) feeds the Explorer detail panel; US4 T313 ports this as SEED_CONVERSATIONS
Our spec.md / plan.md — only for FR-032 (the spec says ≥4KB collapses, the handoff says 500B — the handoff wins per T410)

Rule for every implementation task below: before writing code, open the corresponding handoff JS file and read the referenced line ranges. Every code task cites explorer.js:nn-nn or compare.js:nn-nn — treat those citations as mandatory reading.

Feature: 031-eval-runs-dashboard Spec: spec.md — User Story 5 (P2) Plan: plan.md Contract: contracts/cli.md

Goal: Replace the US4 stubs with two full views:

Explorer (views/explorer.py) — 3-column drilldown: runs list (340px, collapsible to 48px) → cases list (340px) → case detail panel (flex-1). Detail panel sections: case header, agent config snapshot (collapsible JSON + tool chips), conversation thread (chat bubbles + inline tool-call panels with collapse-when-large), expected-tools coverage, per-metric evaluations grouped by kind with judge reasoning.
Compare (views/compare.py) — select up to 3 runs via the floating Compare tray; render one of three layout variants (side-by-side / baseline+deltas / matrix-first) switched by a segmented control; Plotly heatmap case matrix with regression/improvement rings.

Independent Test: - Explorer: From Summary, click any run row → Explorer tab opens with that run pre-selected (via dcc.Store state + dcc.Location.search sync) → cases list populated → click a case → detail panel renders agent config, conversation, tool calls (with collapse-when-bytes>500), expected-tools indicators, and evaluations grouped by kind. Total drill-in: ≤ 3 clicks (SC-006). - Compare: From Summary table, click + on three runs → Compare tray shows 1/3, 2/3, 3/3 slots → click "Open Compare" → all three variants render via segmented control → each variant shows summary stats + config diff + case matrix with heatmap coloring and regression/improvement outlines.

TDD discipline: Data-assembly helpers (select_run, build_case_detail, run_stats, compute_compare_callouts, case_matrix_dataframe) are unit-tested. Dash layout builders return plain Python trees and ARE testable by shape/className/id. Visual fidelity is verified against the HTML prototype (Evaluation Dashboard.html) via Chrome MCP (Phase 8).

Dependency: US4 scaffolds app.py, assets/CSS, state.py helpers, seed data, and stub views/explorer.py + views/compare.py. US5 replaces the stubs and upgrades the US4 compare-tray placeholder in app.py.

Phase 1: Setup¶

None — all infrastructure ships with US4. US5 edits existing files.

Phase 2: Foundational¶

None.

Phase 3: Explorer — data assembly (TDD)¶

Reference: design_handoff_holodeck_eval_dashboard/explorer.js:186–337 (CaseDetail component).

Tests first¶

T401 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: select_run(runs, run_id) -> EvalRun | None — given a list and an id (filename stem or run-XXX), returns the match; None when missing; runs[-1] fallback semantics left to the caller
T402 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: list_case_summaries(run) -> list[CaseSummary] with fields [name, passed, geval_score, rag_avg_score] — geval pulled from the case's first kind=="geval" metric, rag avg from mean(score for m in kind=="rag"), None when absent (mirrors explorer.js::CaseList)
T403 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: build_case_detail(run, case_name, conversations_map) -> CaseDetail returning a dataclass with fields: header (pass/fail, run_ts, prompt_version, model, temperature, commit), agent_snapshot (model.provider/name/temp/max_tokens, embedding.provider/name, claude.extended_thinking, prompt.version/author/file_path/source/tags, tools list with kind+name), conversation (user, assistant, tool_calls: list[ToolCallView]), expected_tools_coverage (list of {name, was_called}), evaluations (dict keyed by standard|rag|geval → list of metric rows)
T404 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: ToolCallView dataclass — name: str, args: dict[str, Any], result: Any, result_size_bytes: int (copied from the persisted ToolInvocation.bytes when available, else computed via len(json.dumps(result, default=str)) for legacy-mode fallback), large: bool (True when result_size_bytes > 500 — handoff explorer.js:156), duration_ms: int | None, error: str | None. This view object projects one ToolInvocation (US1 Migration B) per row with the minimum data the Dash layout needs; it is NOT the persisted on-disk model.
T405 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: expected-tools coverage — case-insensitive comparison against tools_called; returns both matched (configured + called) and missed (configured, not called) counts; optionally exposes unexpected (called but not configured)
T406 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: missing/absent conversations_map key — build_case_detail falls back to the data.js pattern of using refund_eligible_standard as the default conversation (matches explorer.js:197); if neither present, returns an empty conversation without raising
T407 [P] [US5] (TDD) tests/unit/dashboard/test_explorer_data.py: evaluations dict ordering — keys always iterate in geval, rag, standard order (matches handoff's detail-panel rendering order explorer.js:204–208), empty groups are omitted from the output, not left as empty lists

Implementation¶

T408 [US5] Create src/holodeck/dashboard/explorer_data.py with: select_run, list_case_summaries, build_case_detail, and dataclasses CaseSummary, CaseDetail, ToolCallView, AgentSnapshot, MetricRow
T409 [US5] Data source precedence inside build_case_detail. The interleaved ConversationTurn union was originally drafted for US1 Migration C but has been deferred — the handoff's Explorer renders a simple user→tool_calls→assistant shape that does not need role-tagged turns (see US1 tasks-us1.md "Deferred — conversation discriminated union"). The three data-source paths are therefore:
1. Real-run mode (primary): consume test_input (user turn), agent_response (assistant turn), and run.report.results[i].tool_invocations: list[ToolInvocation] (US1 Migration B, US1 T010e–T010j). Each ToolInvocation maps 1:1 to a ToolCallView — the amber tool-call panel renders its name, args, result, and uses bytes for the large-result collapse at the LARGE_TOOL_RESULT_BYTES threshold (T410). error surfaces as a coral "Tool failed: " strip inside the panel.
2. Legacy real-run fallback: when tool_invocations is empty but the name-only tool_calls: list[str] is not (runs persisted before US1 Migration B shipped), render chat bubbles from test_input + agent_response AND emit one amber panel per name with a muted "args/result not captured — re-run after upgrading to this HoloDeck version" subtitle.
3. Seed mode: fall back to seed_data.SEED_CONVERSATIONS[case_name] (US4 T313) when both tool_invocations and tool_calls are empty. Dev-only path guarded by HOLODECK_DASHBOARD_USE_SEED=1. Document the precedence in explorer_data.py's module docstring so reviewers see the data flow without tracing calls.
T410 [US5] Centralise the tool-call large-result threshold constant LARGE_TOOL_RESULT_BYTES = 500 (matches handoff, keeps US5 consistent with explorer.js:156; note this differs from FR-032's 4KB threshold — the handoff wins here because it's the user-validated experience)

Phase 4: Explorer — Dash view¶

Reference: open Evaluation Dashboard.html in a browser, click Explorer tab. All layouts below return html.Div trees; interactions are wired via @callback with Input/Output/State + dash.ALL pattern-match IDs.

Phase 5: Compare — data assembly (TDD)¶

Reference: design_handoff_holodeck_eval_dashboard/compare.js.

Tests first¶

T424 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: run_stats(run) -> RunStats dataclass with fields pass_rate, passed, total, duration_ms, geval_avg, rag_avg, total_tokens, est_cost. Cost computation precedence: (a) when every TestResult.token_usage (US1 Migration C) is populated, sum input_tokens and output_tokens across test results and multiply by the rate from a PRICING_TABLE: dict[str, tuple[float, float]] constant (per-model input/output USD-per-1M-tokens); (b) when any token_usage is None — legacy runs or backends that did not report tokens — fall back to the handoff's synthetic formula duration_ms/1000 × rate (rate 0.018 for sonnet, 0.012 otherwise; matches compare.js:23–42). total_tokens is None in the fallback path so the UI can surface "estimated from duration" instead of a token count. Aggregates match handoff arithmetic to 4 decimals when on the fallback path.
T425 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: compute_case_matrix(runs) -> DataFrame — rows = union of case names across all runs sorted alphabetically, columns = one per run (run_id, score, passed, regression: bool, improvement: bool); score derived from first geval metric, else rag avg, else 1 if passed else 0 (matches compare.js::caseScore)
T426 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: regression/improvement flags — for runs[1:], regression = baseline.passed and not this.passed, improvement = (not baseline.passed) and this.passed (matches compare.js:225–226)
T427 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: compute_config_diff(runs) -> list[ConfigRow] — rows [label, values_per_run, all_same: bool] for prompt_version, model_name, temperature, tags_joined, git_commit, extended_thinking (matches compare.js:94–101)
T428 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: compute_compare_callouts(runs) -> list[Callout] — for each non-baseline run, list up to 3 regressions and 3 improvements by case name with +N overflow count (matches compare.js::CompareV3:353–364)
T429 [P] [US5] (TDD) tests/unit/dashboard/test_compare_data.py: compute_summary_rows(runs) -> list[StatRow] — returns rows for [pass_rate, passed_ratio, geval_avg, rag_avg, duration_ms, total_tokens, est_cost], each with a delta_polarity: "normal"|"invert" field (invert for duration, total_tokens, and est_cost; matches compare.js:85–92). total_tokens is omitted from the row list entirely when all runs' RunStats.total_tokens is None (synthetic-only mode) — we don't surface a column with nothing but dashes.

Implementation¶

T430 [US5] Create src/holodeck/dashboard/compare_data.py with run_stats, compute_case_matrix, compute_config_diff, compute_compare_callouts, compute_summary_rows, the COMPARE_PALETTE = ["#7bff5a", "#5ae0a6", "#ffcf5a"] constant (baseline, run-1, run-2), AND a module-level PRICING_TABLE: dict[str, tuple[float, float]] mapping model name (e.g. "claude-sonnet-4.5", "gpt-4o") to (input_usd_per_1M_tokens, output_usd_per_1M_tokens). Pricing values are best-effort documented placeholders — comment the source URL inline; update them when vendor pricing changes. PRICING_TABLE misses (unknown model) fall to the synthetic duration-based formula.
T431 [US5] Expose a single helper delta_pill_class(value, *, invert=False) -> str returning "hd-delta-pos"|"hd-delta-neg"|"hd-delta-neutral" so layout builders can spray CSS classes consistently (matches compare.js::deltaClass). Used by T436/T437/T438 to render delta pill spans.

Phase 6: Compare — Dash view¶

Reference: open Evaluation Dashboard.html, click Compare tab, try all three layout variants.

Phase 7: Wiring + smoke tests¶

T442 [US5] In app.py, replace the US4 compare-tray placeholder with compare_tray.render(state, runs) (T435) — rendered above html.Div(id="view-container") on every view when queue is non-empty. Put it inside app.layout directly (not in a callback output) and use a separate callback Input("app-state","data") → Output("compare-tray","style") to toggle visibility (display: none when queue empty) — avoids re-rendering the whole tree on every state change
T443 [US5] In Summary's runs table (US4 T361), replace the placeholder + column with the real CompareAddButton (T441). Since dash_table.DataTable can't host arbitrary Dash components in cells, render the runs table via html.Table (the US4 T361 implementation already landed on this path to avoid the DataTable markdown workaround) and inject CompareAddButton.render(run_id, queue) as the first cell of each row. The user journey Summary click + → Compare tray appears → Open Compare → variants render then works end-to-end via the shared pattern-match callback
T444 [US5] In Explorer's runs column (T413), mount the same CompareAddButton on each run row — same callback, zero additional wiring
T445 [US5] (optional, slow) Extend tests/integration/dashboard/test_app_smoke.py:
- Import holodeck.dashboard.app under monkeypatch.setenv("HOLODECK_DASHBOARD_USE_SEED","1")
- Call render_explorer({"explorer_run_id": "run-020", ...}, build_seed_runs()); assert the returned tree contains a .hd-explorer-grid class, 3 columns, and the detail panel has 5 sections (html.Details count + case header div == 5)
- Call render_compare({"compare_queue": ["run-018","run-020","run-023"], "compare_variant": 1}, runs); assert it returns a .hd-compare-v1 tree and contains a dcc.Graph with id="chart-compare-matrix". Repeat for variants 2 and 3 (.hd-compare-v2, .hd-compare-v3)
- @pytest.mark.slow

Phase 8: Visual fidelity — Chrome MCP side-by-side inspection (Explorer + Compare)¶

Why this phase exists: Explorer and Compare have the most complex layouts in the app. Component-tree tests confirm structure; only a real browser side-by-side with the prototype confirms visual match. The HTML prototype is the ground truth (§Primary Source of Truth, item 2).

Setup (same as US4 Phase 8): - Terminal A: holodeck test view --seed → http://127.0.0.1:8501 - Terminal B: python -m http.server 8000 -d docs/design-docs/031-eval-runs-dashboard/design_handoff_holodeck_eval_dashboard → http://localhost:8000/Evaluation%20Dashboard.html

Explorer parity (T446–T453)¶

T446 [US5] Open both tabs and baseline screenshots: mcp__claude-in-chrome__tabs_create_mcp with the prototype URL, click the Explorer tab button in the prototype header via mcp__claude-in-chrome__click, take_screenshot → visual-baselines/prototype-explorer.png. Then tabs_create_mcp with http://127.0.0.1:8501/?tab=explorer, take_screenshot → visual-baselines/dash-explorer-v1.png
T447 [US5] 3-column layout parity — confirm Explorer shows 3 columns with widths matching the handoff: Runs (340px when expanded, 48px when collapsed) · Cases (340px) · Detail (flex-1). Use mcp__claude-in-chrome__javascript_tool to run document.querySelectorAll('.hd-explorer-grid > *').forEach(e => console.log(e.getBoundingClientRect().width)) on both tabs; values on the Dash side should match the prototype to the pixel (CSS grid columns are deterministic). If off, iterate on T412
T448 [US5] Runs column collapse — on the Dash tab, click #explorer-runs-toggle; screenshot at 48px; click again to expand; screenshot. Confirm vertical rotated RUNS <count> text appears in collapsed state (matches explorer.js:48–62). Fix T414 if missing
T449 [US5] Case detail: section sequence parity — get_page_text on the Dash detail panel (.hd-explorer-detail); extract the five eyebrow labels in order. MUST be: [case header] (no eyebrow — pass/fail + name) → AGENT CONFIG SNAPSHOT → CONVERSATION → EXPECTED TOOLS → EVALUATIONS. Wrong order = T417–T421 sequencing bug. Reference: explorer.js:210–335
T450 [US5] Conversation thread: tool-call panels — click open the conversation <details>; confirm every .hd-tool-call panel has: caret (native <summary> marker) + TOOL eyebrow + name() monospace + <size>B byte count in the header row; args <details> open, result <details> collapsed with Expand (<size>B) summary text when size > 500B; amber-tinted panel background (rgba(255,207,90,.08) per T419 CSS). Reference: explorer.js::ToolCall:152–184. Screenshot, diff against prototype, fix T419 if the amber tint or collapse behavior is wrong
T451 [US5] Expected-tools indicators — confirm check/cross glyphs render in accent green (#7bff5a) / coral (#ff9d7e) respectively, with called / not invoked note text, and the <summary> row shows the N/M matched pill on the right (via flex justify-content: space-between). Reference: explorer.js:285–302
T452 [US5] Evaluations: group order + reasoning block — confirm eyebrow labels GEVAL, RAG, STANDARD appear in that order (T407 guarantees this) and that G-Eval metric rows have a .hd-judge-reasoning block below the score. Screenshot with reasoning visible
T453 [US5] Drill-in click count (SC-006) — from a fresh Summary load, measure clicks to reach a test-case detail:
1. click a run row in the Summary <html.Table> runs list
2. click a case in Explorer cases column
3. (case detail already visible — first case auto-selected by T413's "clears explorer_case_name" logic) Total ≤ 3 clicks. If the cases column requires an extra interaction step (e.g. the auto-first-case fallback from T411 is missing), the contract is broken

Compare parity (T454–T462)¶

T454 [US5] Populate compare queue from Summary — on the Dash tab, go to Summary; click + on 3 runs via mcp__claude-in-chrome__click on [id^="{\"type\":\"compare-add\""] selectors. Confirm the floating compare tray appears with 3 slot pills, first tagged base, and that the Open Compare → button enables. Reference: compare.js::CompareTray:512–549. Screenshot the tray → visual-baselines/dash-compare-tray.png; diff against the prototype tray
T455 [US5] Tray is sticky across tabs — switch to Explorer (click the Explorer tab). Confirm the tray remains visible and functional (position: fixed + z-index: 50 survives view-container re-render because the tray is mounted OUTSIDE #view-container). Switch to Compare. Confirm the tray remains visible. If the tray disappears on tab switch, T442 (app.py layout position) is wrong — it must be a sibling of #view-container, not a child
T456 [US5] Empty state CTA — clear the queue via the tray's Clear button; open the Compare tab. Confirm the empty-state panel shows the three-rectangles SVG (or the absolute-positioned div stack per T433) + headline "Pick runs to compare" + two CTAs (Compare latest 2 runs primary, Compare latest 3 runs ghost). Reference: compare.js::CompareEmpty:435–457. Click Compare latest 2 runs — queue should populate with the 2 newest runs and variant-1 rendering should appear
T457 [US5] Variant toolbar parity — get_page_text on .hd-compare-toolbar. Confirm: eyebrow COMPARE + <N> runs · baseline <version> + layout label + radio group with three options side-by-side, baseline + deltas, matrix-first + Clear button. Reference: compare.js:484–501
T458 [US5] Variant 1 — Side-by-side — set compare-variant radio to the first option via click; take_screenshot of the full page. Confirm:
- Column headers per run with palette dot, slot label, timestamp, version-in-color, model, commit
- Summary block with up to 7 rows (pass rate, passed, geval, rag, duration, total_tokens if available, cost) and delta pills on non-baseline cells
- Config diff block with differing cells having amber left-border + changed badge
- Case matrix Plotly heatmap at bottom (dcc.Graph #chart-compare-matrix present) Save → visual-baselines/dash-compare-v1.png; compare with the prototype's V1 screenshot
T459 [US5] Variant 2 — Baseline + deltas — click radio option 2; screenshot. Confirm the baseline card is visually emphasized (1.4fr grid column; large pass-rate number in accent), delta cards are compact with 5 delta-rows each. Reference: compare.js::CompareV2:253–343
T460 [US5] Variant 3 — Matrix-first — click radio option 3; screenshot. Confirm compact run-card strip at top + callouts block (regressions/improvements listed by case name with +N overflow) + matrix dominant below. Reference: compare.js::CompareV3:347–431
T461 [US5] Case matrix heatmap parity — on any variant, hover over a cell via mcp__claude-in-chrome__hover — confirm Plotly tooltip shows case_name / run_label / score / pass|fail. Confirm regression cells have a coral dashed outer outline and improvement cells a green dashed outline. Reference: compare.js::CaseMatrix:182–250. If the outlines are missing, fig.add_shape(..., line=dict(dash="dash")) calls in T439 need review
T462 [US5] Delta polarity sanity — pick two runs where run[1] has LONGER duration than baseline; confirm the duration delta pill shows coral (hd-delta-neg) because duration polarity is inverted (lower = better). Same for est_cost and (when present) total_tokens. Reference: compare.js::deltaClass:17–21. If polarity is wrong, fix T431. Additionally, confirm the cost cell shows a tooltip or caption indicating whether it was computed from token_usage × PRICING_TABLE (real) or from the synthetic duration × rate fallback — so users know when the number is provisional

Global sweep (T463–T466)¶

T463 [US5] Console cleanliness — mcp__claude-in-chrome__read_console_messages with pattern: "(error|warning)" on the Dash tab while navigating Summary → Explorer → Compare and toggling all 3 variants. Zero errors from our modules. Dash/werkzeug framework warnings are acceptable but should be reviewed
T464 [US5] Navigation record using mcp__claude-in-chrome__gif_creator: name it dashboard_tour.gif — record the journey from Summary → click row → Explorer with case selected → add + to queue ×3 → Compare tab → toggle all three variants. Commit alongside visual-baselines/
T465 [US5] URL-state shareability test — with a filter applied on Summary AND a case open in Explorer AND a compare queue populated, copy the full URL via mcp__claude-in-chrome__javascript_tool running window.location.href. Open that URL in a new tab (tabs_create_mcp). Verify the app restores: filter state, Explorer's selected run+case, and the compare queue. Requires that US4's URL-sync callback (T346) serializes the Explorer + Compare keys in addition to filter params — extend url_search_from_state / state_from_url_search if explorer_run_id / explorer_case_name / compare_queue don't round-trip. If the compare queue does not survive, the serde on the URL-sync side needs work
T466 [US5] Accessibility check — mcp__chrome-devtools__lighthouse_audit on the Dash app; confirm Accessibility score ≥ 85 (contrast on the terminal-green theme is the likely risk area). File follow-up tasks for any blocker-level issues; do not block US5 merge on this, but capture the score

Outputs of Phase 8: visual-baselines/ directory contains prototype + Dash screenshots for each view/variant; dashboard_tour.gif documents the UX; any deltas logged and resolved against T411–T441.

Dependencies¶

US4 fully complete: app.py, assets/CSS, state.py helpers, seed data, Summary view, CLI view command, views/explorer.py + views/compare.py stubs.
T401–T407 (Explorer TDD) blocks T408–T410.
T408–T410 blocks T411–T423.
T424–T429 (Compare TDD) blocks T430–T431.
T430–T431 blocks T432–T440.
T441 (CompareAddButton component) blocks T443, T444 (Summary + Explorer use it).
T435 (compare tray component) blocks T442.

Parallel Opportunities¶

# Data-assembly TDD — Explorer and Compare are independent:
Task: "tests/unit/dashboard/test_explorer_data.py (T401–T407)"
Task: "tests/unit/dashboard/test_compare_data.py (T424–T429)"

# View modules run in parallel (separate files):
Task: "src/holodeck/dashboard/views/explorer.py (T411–T423)"
Task: "src/holodeck/dashboard/views/compare.py (T432–T440)"
Task: "src/holodeck/dashboard/components/compare_tray.py (T435)"
Task: "src/holodeck/dashboard/components/compare_add_button.py (T441)"

Acceptance Scenario Traceability¶

US5 (Explorer — Spec AC1–AC8)¶

AC	Covered by
AC1 (click run → list cases)	T402, T413, T415
AC2 (agent config snapshot in detail)	T403, T418
AC3 (chat-style conversation)	T419
AC4 (tool calls with formatted JSON args + result)	T404, T419
AC5 (expected tools with match indicators)	T405, T420
AC6 (metric results with reasoning)	T421
AC7 (errors displayed prominently)	covered in T417 case header for per-case errors
AC8 (large results collapsed)	T404 (`large` flag), T410, T419

Design-handoff Compare (additive to spec, introduced by handoff README §3)¶

Handoff requirement	Covered by
3 layout variants switchable via segmented control	T434, T436, T437, T438
Compare queue max 3, baseline = first added	T435, US4 T345
Delta pills with inverted polarity for duration + cost	T431, T436, T437
Config-diff highlighting (amber left-border, `changed` badge)	T436 (T427 supplies data)
Heatmap case matrix with regression/improvement outlines	T425, T426, T439
Floating Compare tray visible across tabs	T435, T442
`+` button for run rows that cycles "add / in-slot-N / remove"	T441, T443, T444
"Compare latest 2 / latest 3" empty-state shortcuts	T433

Implementation Strategy¶

Recommended order (matches user directive: scaffold → Summary → Explorer → Compare)¶

Explorer first (T401–T423). Smaller surface (1 view module, ~300 LOC equivalent), uses only the seed data and US4 state.
Port explorer.js section by section: case detail panel → runs column → cases column → wiring. Verify each section against the HTML prototype before moving on. Because Dash layouts are pure functions over state, you can preview any section in isolation via python -c "import ...; print(render_section(...))".
Compare second (T424–T444). Larger surface but orthogonal to Explorer.
Build data layer (T424–T431) — fully TDD'd.
Ship the floating Compare tray (T435, T442) first so users can build queues from Summary even before the Compare view itself lands.
Add empty state + toolbar (T433, T434).
Build variants in order: V1 side-by-side → V2 baseline+deltas → V3 matrix-first. Each variant reuses the shared case-matrix heatmap (T439).
Wire compare buttons into Summary + Explorer (T441, T443, T444) — the + must be everywhere the user sees a run. A single pattern-match callback handles all three surfaces.
Smoke tests (T445).
Visual fidelity sweep via Chrome MCP (T446–T466) — every view and variant diffed against the HTML prototype in a real browser. No visual delta is acceptable at US5 merge.

Parallel team strategy¶

Two developers can split cleanly: - Dev A: Explorer (T401–T423) + Compare tray (T435) - Dev B: Compare data layer + variants (T424–T440)

They converge on the wiring step (T441–T444).

Visual fidelity checklist¶

This checklist is enforced via the Phase 8 Chrome MCP tasks (T446–T466). For each item below, capture before/after screenshots via mcp__claude-in-chrome__take_screenshot with the HTML prototype and the Dash app loaded side-by-side. Commit both under docs/design-docs/031-eval-runs-dashboard/visual-baselines/. No item may remain un-ticked at US5 merge: - [ ] Colors match (accent #7bff5a, fail #ff9d7e, warn #ffcf5a) - [ ] JetBrains Mono font used for all numeric values - [ ] Eyebrow labels uppercase, 10px, .15em letter-spacing, accent-soft color - [ ] Card borders + gradient backgrounds - [ ] Pills use correct tier colors (green ≥ 85%, yellow 65–85%, coral < 65%) - [ ] Compare palette dots are visually distinct (#7bff5a, #5ae0a6, #ffcf5a) - [ ] Delta pills use correct polarity (inverted for duration/cost) - [ ] Regression dots on Summary pass-rate chart are coral with dark outline - [ ] Heatmap case matrix gradient matches coral→dark→green scheme - [ ] Regression/improvement ring outlines are dashed, not solid