Skip to content

Complete feature 035 — OpenAI Agents SDK backend

Status: In progress; T0–T3 complete (4/12 tasks). The T3 sample ran live on Azure OpenAI (E8); remaining live provider evidence for T1, T2, and T3 (OpenAI platform, collector, Ollama) stays with T10. Created: 2026-09-06. Owner: Backend maintainers, with serving, deployment, retrieval, and documentation maintainers for their respective tasks. Baseline: b585e80 plus the uncommitted reconciliation from this session. Requirements: 035 specification and acceptance matrix.

Objective and observable acceptance criteria

Finish the agreed 035 parity scope and close its outstanding acceptance gaps. An OpenAI or Azure agent must support the declared tools, hooks, handoffs, serving, deployment, and telemetry behavior. Claude behavior and provider-neutral execution contracts must remain intact.

Completion requires all twelve tasks below, with linked evidence for each acceptance gate. Every functional requirement and success criterion must have a disposition: verified, superseded by a recorded decision, or explicitly deferred with an owner and exit criterion. A missing credential, skipped test, implemented adapter, or checked historical task does not prove acceptance. The original specification contains superseded requirements. This plan does not claim that its deferred sandbox and hardened-profile scope is delivered.

Canonical records and scope

This file is the active execution checklist for completing 035. The full plan remains the detailed design reference for tasks A–K. The old TODO and reconciliation report retain the audited baseline and historical counts. Do not maintain another parallel completion checklist in those records. Update this plan with implementation decisions, test commands, results, and remaining blockers after each task.

In scope:

  • Remaining full-parity tasks D, E, G, I, J, K and the partial F4/H1 acceptance work.
  • Baseline acceptance for implemented A, B, C, F and the MVP/SK cleanup gates.
  • Open LiteLLM migration acceptance in its task record, including H-009.
  • Documentation corrections in the documentation plan.
  • Requirement reconciliation, reproducible fixtures, final verification, and archival of completed records.

Existing deferrals remain separate: P3/Envoy, sandbox mode, computer-use, interactive human approval, MCP-server tool guardrails, modify, prompt-tool execution, and namespace unification. Final Semantic Kernel decommission remains H-008 in the debt tracker. This plan closes the native-backend migration and its inference acceptance. It does not replace the eleven vector-store connectors or remove the SK package. T0 must give each deferred item a durable destination. A deferral must not be labelled implemented.

Constraints and decisions

  • Use BackendSelector and the contracts in src/holodeck/lib/backends/base.py.
  • Preserve current routing: OpenAI/Azure use OpenAI Agents; Anthropic/Ollama use Claude.
  • Keep the openai: namespace and the Azure /openai/v1 Responses mapping established by #338.
  • Use typed boundary adapters and validated Pydantic configuration. Do not bypass known types with Any or getattr.
  • Preserve multimodal execution, evaluation-model overrides, tool events, and GenAI instrumentation.
  • Keep network calls and blocking I/O outside Temporal workflow execution and async event loops respectively.
  • Verify SDK integration points against the installed, locked SDK before implementation. Do not copy obsolete SDK assumptions from the draft spec.
  • Do not change unrelated working-tree edits or add a repository-wide import linter as part of this feature.
  • The user authorized plan creation and T0 contract reconciliation. Future live deployment validation follows the explicit target-authorization rule in Reliability.

Ordered work and verification

Task checkboxes mean all listed acceptance and verification criteria are satisfied, not just that source code exists. Dependencies require upstream implementation and local gates to be ready. They do not require live gates assigned to T10 to finish first. Record “implementation ready; live acceptance pending” in progress when that distinction applies. T10 evidence then closes the upstream live gates. Proposed new test files are labelled as additions. Other file paths below are repository-relative.

T0 — Establish the final requirement contract

Dependencies: None. Owner: Backend maintainers.

  • T0 complete.

Create acceptance-matrix.md beside this plan, with one row per FR-001–FR-111 identifier actually present and SC-001–SC-013. Each row records scope, owning task, required test/observable result, decision reference, and evidence status. Resolve the known specification drift in place: backend enumeration/AF/ADK assumptions, openai: naming, Azure client construction, effort max → xhigh, hooks, hosted-tool count, and tracing suppression. Preserve the original requirement identifiers and record why each changed requirement was superseded. Map P3 and sandbox criteria to explicit deferrals instead of silently deleting their success criteria. Reconcile /ready versus the current health API and SC-008's ten-turn target versus integer memory-derived capacity before writing acceptance tests. Give each deferred item a debt/spec link, owner, and observable exit criterion; H-008 already covers final SK removal.

Gate: Every requirement has a destination. No mandatory in-scope behavior disappears into a generic “follow-up.” Run make harness-check; review the matrix against the complete spec and all six reconciled records.

T1 — Prove and correct fallback and tracing behavior

Dependencies: T0. Owner: Backend/observability maintainers. Old tasks: F4, H1.

  • T1 complete (local gates; live provider/dashboard evidence is T10).

Inspect openai_agents_fallback.py, openai_agents_backend.py, and openai_agents_tracing.py under src/holodeck/lib/backends/. Exercise fallback through the SDK Runner, including retry exhaustion, retryable/non-retryable errors, streaming before/after the first event, and both-attempt traces. Define one bounded retry order and correct the wrapper if its internal catch bypasses that order. Test OpenAI/Azure and provider-upload overrides with observability enabled and disabled, repeated initialization, and mixed-provider initialization in both orders. Resolve SDK process-global processor ownership so configuration isolation is explicit and safe. Do not fix mixed-provider behavior by silently ignoring later agents.

Gate: Deterministic Runner-level tests demonstrate ordering, no duplicate streamed output, and provider-upload suppression without starving permitted OTel spans. Extend existing fallback/backend/tracing tests. Record the process-global tracing policy in the spec and guide.

Result (2026-09-06): Decisions D13 (per-backend tracing-policy router owning the SDK processor list) and D14 (primary client retries, then one fallback, no Runner repeat) recorded in the spec; guides and changelog updated. The first-initialization flag was replaced: register_tracing_policy / unregister_tracing_policy and active_tracing_policy in openai_agents_tracing.py; every run tags RunConfig.trace_metadata["holodeck.tracing_policy"]. Evidence E5 in the matrix.

T2 — Complete LiteLLM inference acceptance

Dependencies: T0. Owner: Retrieval/observability maintainers. Old tasks: LiteLLM T2/T3/T6/T8; H-009.

  • T2 complete (local gates; live three-provider and collector evidence is T10).

Resolve the shared embedding service versus per-tool dimensions contract in tool_initializer.py and litellm_support.py. Test factory-to-provider dimension forwarding and mismatch handling, including tools with different configured dimensions. Choose and document the provider-error boundary, then test propagation or wrapping without losing useful causes. Resolve semantic-convention stability against the installed integration. Verify emitted embedding/chat spans through an in-memory exporter, including token attributes, redaction, and capture disabled. Preserve LLMContextGenerator retry, truncation, and concurrency behavior and the Claude context-generator selection path.

Gate: Focused tests cover the resolved contracts. T10 supplies the live grounded/query and contextual-ingest evidence required to close H-009. No rollback flag or SK inference path is reintroduced.

Result (2026-09-06): Decision D15 recorded in the spec: per-tool embedding_dimensions builds a dedicated LiteLLMEmbeddingService that sends dimensions to the provider; EmbeddingServiceError (cause chained) is the provider-error boundary and both tools propagate it instead of storing placeholder vectors; semantic conventions are not pinned, the default LiteLLM shape is documented, and the opt-in is documented and tested. LLMContextGenerator and Claude context-generator selection are unchanged. Guides, changelog, and the LiteLLM plan record updated. Evidence E6 in the matrix.

T3 — Deliver subagents, skill tools, and handoff events

Dependencies: T0. Owner: Backend/model maintainers. Old tasks: D1–D3.

  • T3 complete (local gates; live handoff and skill routes verified on Azure OpenAI, E8; the plain OpenAI run remains T10).

Add validated openai.agents configuration, SDK subagent construction, parent-tool inheritance, explicit tool restrictions, model inheritance, and Claude-literal rejection. Add SkillTool to the tool union, with inline and SKILL.md forms, following the full plan's contracts. Emit handoff/parent-link events through the existing backend and AG-UI contracts. Preserve ordinary tool start/end/error events as well as thinking events. Record how the recommended handoff prefix and its opt-out work. Define the portable setting_sources warning required by FR-071.

Gate: Unit and session integration tests prove tool/model inheritance, restricted skills, invalid configuration, and ordered AG-UI events. Generate schemas and verify schema synchronization. Add bounded fixture-based handoff tests; T10 covers a live handoff.

Result (2026-09-06): Decision D16 recorded in the spec: openai.agents entries (OpenAISubagentSpec) and the net-new SkillTool (type: skill, inline or SKILL.md, YAML-only allowed_tools) become SDK handoff-target agents built from the parent's built tool surface by openai_agents_subagents.py (inherit-all, explicit restriction by config name, unknown/disallowed names fail load, inherit model reuse, Claude literals rejected at config load, prefix once with opt-out, skills verbatim). openai_agents_events.py maps SDK run items and stream events onto the existing ToolEvent contract (start/end/parent_link/subagent_message/thinking), the session exposes tool_events, and handoffs are recorded in ExecutionResult. FR-071 logs setting_sources is a Claude-only concept; ignored on openai. once per load. Schema regenerated. sample/openai-agents-full/ is a runnable agent exercising every shipped backend surface (function, RAG, MCP, subagents, skills, disallowed tools, effort, budget, fallback, tracing, tests). Live run on 2026-09-06 (Azure OpenAI gpt-5-mini, agent.azure.yaml): 7/7 cases pass across five consecutive runs, in-memory and with both RAG tools on Qdrant; see E8 in the matrix for the three harness bugs it exposed and fixed on the same branch. Guides (OpenAI backend, Tools), examples index, changelog, and the full plan updated. Evidence E7 in the matrix.

T4 — Deliver hosted tools and safety gates

Dependencies: T0. Owner: Backend/model maintainers. Old tasks: G1–G2.

  • T4 complete.

Add validated hosted-tool configuration and the five factories retained by the full plan. Cover web search, file search, code interpreter, image generation, and hosted MCP. Reject computer-use configuration with an actionable deferred-feature error. Apply the code-interpreter opt-in and permission filtering before constructing tools. Accept Azure configurations according to the recorded runtime-capability policy and preserve useful unsupported-capability errors.

Gate: Per-factory tests verify exact SDK arguments, invalid parameters, safety opt-in, permission conflicts, and Azure error handling. Generate schemas. T6 must complete hosted rejection behavior before the hosted-tools checkpoint closes.

T5 — Implement default credential boundaries

Dependencies: T0, T1; integrate T3 adapters before closure. Owner: Backend/security maintainers. Old tasks: J1–J2.

  • T5 complete.

Attach default output redaction to HoloDeck-built tools and preserve the recorded opt-out warnings. Test the model-visible result and exported spans using synthetic credential-shaped values, without using real credentials. Implement child-process environment scrubbing for HoloDeck-managed MCP and function subprocess paths, with explicit opt-out behavior. Do not mutate the process-wide environment during concurrent requests. Document the limits for arbitrary Python tools that spawn children outside managed paths. Keep hosted and MCP-server guardrail limitations visible and distinguish them from OTel redaction.

Gate: Tests observe sanitized model context, sanitized spans, child environments, opt-outs, and concurrent requests without cross-request leakage. Reconcile any boundary that cannot meet FR-089 in T0's matrix; do not claim arbitrary Python execution is contained.

T6 — Complete YAML hooks and rejection semantics

Dependencies: T3, T4, T5. Owner: Backend/model maintainers. Old tasks: E1–E2.

  • T6 complete.

Implement validated hook configuration and the event/action mappings in the full plan. Cover log/notify/script observation, session start, stop, tool success/failure, and unreachable matchers. Run internal tracking/redaction before user hooks and preserve declaration order and terminal-action behavior. Function-tool rejection must continue the run with the rejection result; input rejection must abort with the specified message. Reject unsupported MCP-server guardrail configuration at load. Implement the agreed hosted-MCP rejection path without introducing the deferred interactive approval loop. Treat modify as the documented inert warning, not silent mutation.

Gate: Tests cover each event/action pairing, failure wrappers, ordering, script lifecycle, function/input rejection, hosted rejection, and unsupported MCP configuration. Ensure hook scripts use the credential boundary from T5. Generate and validate schemas.

T7 — Complete serving lifecycle and capacity

Dependencies: T0, T1; final integration after T3–T6. Owner: Serving/backend maintainers. Old task: I1.

  • T7 complete.

Extend src/holodeck/serve/server.py and shared validation for OpenAI/Azure startup credential preflight, readiness, and configuration echo. Implement active-turn capacity with explicit override or the D11 formula: full finite cgroup MiB divided by the configured estimate (default 100), with an assumed 1024-MiB fallback and an actionable error below one. Do not subtract Claude's subprocess reserve. Idle sessions must not consume active-turn slots. Release slots on success, error, timeout, cancellation, disconnect, and streaming-generator closure. Return the agreed 429 problem response and Retry-After; verify tool-init endpoints and non-init-tool errors.

Gate: Concurrent REST and AG-UI tests exercise the exact limit, overflow, recovery, readiness transitions, and collect-all-errors startup behavior. Extend existing serve/tool-init tests and verify Claude regressions. Test startup failures before the listener accepts traffic.

T8 — Complete packaging and Azure deployment behavior

Dependencies: T7. Owner: Deployment maintainers. Old tasks: I2–I3.

  • T8 complete.

Reuse the existing conditional-Node Docker branch. Verify OpenAI/Azure images with and without Node-requiring MCP tools. Retain supported ACA 1-CPU / 2-GiB defaults and the same resolved capacity echo used by serve (default twenty). Preserve SC-008's local 1-CPU / 1-GiB ten-turn regression; the cloud twenty-turn gate remains T10. Verify credential preflight, non-root/read-only corpus behavior, scratch volumes, and internal-ingress defaults/warnings. Use the current wheel in local image acceptance so published base images cannot mask the checkout under test.

Gate: Unit tests verify generated Docker/ACA configuration. Local container checks demonstrate actual filesystem, startup, and shutdown behavior. T10 supplies authorized cloud evidence; Dockerfile text inspection alone does not close deployment acceptance.

T9 — Make examples and documentation match delivered behavior

Dependencies: T1–T8 for final claims; prerequisites/examples can be prepared earlier. Owner: Documentation/backend maintainers. Old tasks: K1, K3; docs plan.

  • T9 complete.

Create tracked test fixtures for the representative agent and its tool implementation. Do not depend on ignored /sample files or mutable external samples for CI acceptance. Resolve the external sample location/version in the docs without writing to another repository without authorization. Complete provider tabs, the warehouse example prerequisites, LiteLLM mapping, and the hook/guardrail semantics matrix. Resolve each guide-template discrepancy by implementing it or recording a specific justified design change. Correct stale SK/serve/deploy and quick-start runnability claims in current documentation and Unreleased notes. Preserve historical changelog entries.

Gate: Both backend quick-start configurations load from a clean fixture directory. Commands reference real prerequisites and pinned/sample identities. Run make harness-check and uv run mkdocs build --strict; inspect rendered backend/provider pages and navigation.

T10 — Establish complete acceptance evidence

Dependencies: T1–T9 implementation and local gates ready; upstream live-acceptance checkboxes can remain open. Owner: Backend maintainers coordinating service owners. Old tasks: K2, all open checkpoints.

  • T10 complete.

Fill T0's matrix with named tests and results for all in-scope FRs/SCs. Cover single/multi-turn execution, multimodal input, streaming, each retained tool type, hosted tools, handoffs, hooks, budget/fallback, and errors. Exercise NLP, G-Eval, and RAG evaluation result compatibility and model overrides. Measure validation timing against SC-004 using a stated fixture/environment. Verify installation without the OpenAI extra in an isolated environment, not through an already-loaded SDK mock. Run real OpenAI and Azure acceptance and the OpenAI/Azure/Ollama retrieval checks required by the LiteLLM plan when credentials are available. Include hierarchical-document contextual ingest and Claude context-generator regression acceptance from LiteLLM T5. The collector smoke must observe LiteLLM embedding and contextual-chat spans, token attributes, redaction, and capture-disabled behavior, not only backend spans. Run the authorized ACA loop for startup/readiness, capacity/overflow, known-answer retrieval, and OTel/provider-trace destinations. Record provider capability restrictions explicitly; unavailable hosted features do not count as passed tests.

Gate: Every retained success criterion has passing evidence for the candidate revision or an explicit unresolved blocker. Cloud evidence records the selected target, image digest, configuration identity, concurrency/memory results, response, and redacted telemetry. Use Reliability for the deployment loop and its existing authorization rules. Missing access keeps this task open while independent local work continues.

T11 — Close and archive feature records

Dependencies: T10. Owner: Backend/harness maintainers. Old task: K checkpoint and historical plan closure.

  • T11 complete.

Run final repository checks listed below on the assembled candidate and resolve introduced failures. Review the final diff, acceptance matrix, and actual runtime scope together. Close H-009 only when its acceptance evidence is complete; retain H-008 and other deferred items with accurate status. Replace historical open gates with links to the evidence that resolves them. Mark superseded proposals archived rather than implemented. Update the spec, inventory, architecture/quality evidence, and all feature indexes consistently. Move completed feature plans to docs/exec-plans/completed/035-openai-agents-backend/ and repair incoming/outgoing links. Update destination paths in docs/generated/spec-migration.json for moved migrated records. New files have no migration-baseline entry to invent.

Gate: The retained 035 scope is verified, all twelve tasks are checked, and remaining excluded work is discoverable outside its completion claim. The feature can then be marked shipped, with the retained scope and deferrals stated explicitly.

Dependencies and suggested PR boundaries

T0 precedes implementation. T1, T2, T3, and T4 can proceed independently after that contract is settled. T5 starts after T1 and integrates T3 adapters. T6 follows T3–T5. T7 can develop in parallel with tool work, then integrates it. T8 follows T7. T9 prepares fixtures early and completes after runtime behavior stabilizes. T10 and T11 are final gates.

Prefer one reviewable PR per task, splitting schema/adapters from integration only when each intermediate change remains usable. A task stays open until all dependent acceptance is satisfied. Merge order follows the dependencies above. For delegated work, assign disjoint files. Sequence edits to shared models, the selector, backend initialization, schemas, and tests.

Verification commands and evidence format

For each Python change, run focused Black, Ruff, MyPy, and relevant pytest tests through uv run. Run pytest with -n auto. Use the existing quality map to select consumers of shared contracts. After model changes, run make schema and make schema-check. At final integration, preserve CI/pre-commit requirements and run:

uv run pre-commit run --all-files
uv run pytest -n auto --cov --cov-branch --cov-report=xml
make security
make schema-check
make harness-check
uv run mkdocs build --strict

Keep live service tests separately identifiable. Record skips and unavailable services, rather than treating a green summary as complete live acceptance. Do not rerun broad suites after they pass unless the candidate changes or a new concern warrants it.

Each task records: commit/configuration identity, changed behavior, command, result/count, artifact reference, and remaining limitation. Do not store credentials or raw sensitive traces in the evidence.

Progress and decisions

  • 2026-09-06: Created this execution plan from the six reconciled records. No implementation tasks were completed by plan creation.
  • The earlier audit recorded 324 focused test passes plus harness and strict MkDocs passes. Those results cover the audited baseline only.
  • T0 records decisions D01–D12 and all 69 requirement dispositions in the spec/matrix. Deferred scope has owners and observable exit criteria in H-008 and H-011–H-022; H-009 remains mandatory migration acceptance.
  • /ready remains canonical but must gain prerequisite/tool-state gating and HTTP 503 semantics. D11 corrects the invalid ACA 1-CPU / 1-GiB proposal to supported 1-CPU / 2-GiB while preserving the local ten-turn target and adding the cloud twenty-turn gate.
  • 2026-09-06 (T1): the fallback wrapper's internal catch was found to sit inside the SDK Runner's retry loop, so runner-managed retries would repeat the primary-then-fallback pair; D14 pins the order to the OpenAI client's provider-managed retries followed by one fallback, with HoloDeck never enabling ModelSettings.retry. The process-global mirror flag silently skipped later agents (an Azure agent after an OpenAI agent inherited upload; observability-disabled Azure still uploaded); D13 replaces it with a router that applies each backend's policy per trace. Only traces and spans with no HoloDeck identity keep SDK default behaviour; events for a torn-down backend are dropped immediately. An independent Codex review of the first T1 draft reproduced escapes across two rounds (span under a caller-owned outer trace started before the run, span finished outside its scope or after its trace ended or after trace-map eviction, in-flight trace after unregister) and the SDK's conversation_locked compatibility retry; span identity is now pinned at span start, withdrawal is immediate, and the retry bound is documented in D14, each with regression tests. Codex's reproduction script yields zero uploads on the committed revision.

Validation results and remaining blockers

Plan-only validation: make harness-check and git diff --check passed on 2026-09-06. Independent coverage review identified a live-acceptance dependency cycle and missing explicit LiteLLM live checks. Both were corrected. The review covered task scope, dependencies, authorization boundaries, and completion criteria. T0 verification on 2026-09-06: all original 56 FR and 13 SC identifiers preserved exactly; 69 matrix rows have every required field. All 181 local fragment links across the spec and active records resolve. make harness-check, git diff --check, and uv run mkdocs build --strict passed. Independent contract review covered the full spec and six reconciled records; its two matrix gaps (first-event fallback boundary and trace run/capture identity) were corrected.

T1 verification on 2026-09-06: uv run pytest tests/unit/lib/backends tests/unit/serve tests/integration/test_multi_turn_dual_backend.py -n auto -q — 946 passed (6 new Runner-level fallback tests, 23 router tests, 18 policy/lifecycle/capture tests); Black, Ruff, and MyPy clean on the three changed source modules; make harness-check and uv run mkdocs build --strict passed. Test files remain outside the repository's MyPy gate. Remaining T1 limitation: real OpenAI dashboard and collector observation for FR-100–FR-102/SC-011 and live 429 fallback behaviour are T10 evidence.

T2 verification on 2026-09-06: uv run pytest tests/unit/lib/test_litellm_support.py tests/unit/lib/test_tool_initializer.py tests/unit/test_vectorstore_tool.py tests/unit/tools tests/unit/lib/backends/test_otel_redaction.py tests/unit/lib/observability tests/unit/lib/test_llm_context_generator.py tests/unit/lib/backends/test_claude_backend.py tests/unit/lib/backends/test_openai_agents_tracing.py tests/integration/test_multi_turn_dual_backend.py -n auto -q — 699 passed (7 dimension/boundary initializer tests, 4 shim boundary tests, 9 exporter-level LiteLLM span tests, tool propagation and failure-telemetry redaction tests); independent adversarial review found six issues (Azure deployment-alias dimensions guard, unredacted error.*/exception-event telemetry, LiteLLM callback_list fixture leak, two documentation claims, an untyped helper), all fixed before commit; tests/integration/test_litellm_rag_integration.py skipped without credentials (8 skipped); Black, Ruff, and MyPy clean on the six changed source modules; make harness-check and uv run mkdocs build --strict passed.

T3 verification on 2026-09-06: uv run pytest tests/unit/lib/backends/test_openai_agents_subagents.py tests/unit/lib/backends/test_openai_agents_events.py tests/unit/lib/backends/test_openai_agents_backend.py tests/unit/lib/backends/test_openai_agents_tool_adapters.py tests/unit/lib/backends/test_validators.py tests/unit/models/test_skill_tool.py tests/unit/models/test_openai_config.py tests/unit/models/test_agent.py tests/unit/lib/test_skills.py tests/unit/lib/backends/test_claude_backend.py tests/unit/chat tests/unit/serve -n auto -q — see the recorded count below; make schema and make schema-check pass; Black, Ruff, and MyPy clean on the changed source modules; make harness-check and uv run mkdocs build --strict passed. Remaining T2 limitation: live OpenAI/Azure/Ollama grounded ingest/query, contextual ingest, and collector observation are T10 evidence.

T3 live follow-up on 2026-09-06 (same branch, PR #380): cd sample/openai-agents-full && uv run holodeck test run agent.azure.yaml against Azure OpenAI (gpt-5-mini deployment, text-embedding-3-small) — 7/7 passed on four consecutive runs; handoff to researcher, handoff to analyst with inherited tools, and skill route exercised real handoffs with inherited and restricted tools (analyst called get_inventory + reserve_stock after transfer_to_analyst; researcher and summarise skill called knowledge_base_search). The run exposed and fixed three harness defects: (1) retrieval_context never built on this backend because the test runner matched only SK/Claude tool names — faithfulness and contextual_* were silently skipped (d21a14c; faithfulness now scores 1.0 on the four retrieval cases); (2) vectorstore ingest failed on Qdrant with 400 Bad Request (readable point IDs) and search failed on qdrant-client 1.18 (AsyncQdrantClient.search removed) — fixed with UUIDv5 IDs and a query_points shim (5879801, 3bdecb0; H-010 tracks structured-key preservation); (3) the loader has no ${VAR:-default} syntax, so sample env references were literal. Sample-level fixes: G-Eval compares with expected_output; subagent prompts assert ownership of the request at low reasoning effort. Independent adversarial review of the Qdrant fix found two issues (native UUID IDs re-hashed; structured business key lost), both fixed. Plain provider: openai run still pending (no credits on the available key) and remains T10 with the OpenAI dashboard/collector checks.

Serve follow-up on 2026-09-06 (E9, same branch): holodeck serve with the same sample validated by raw AG-UI SSE requests and a CopilotKit browser flow. It exposed a serve defect on this backend — turn 2 on a thread returned empty because the backend and its stdio MCP servers were initialised in turn 1's HTTP request task — fixed by running every serve session's backend inside its actor task and surfacing is_error results in the chat executor. This is the validation loop now recorded in AGENTS.md: live sample run, raw serve requests, then user-driven CopilotKit validation.

Apart from the Azure OpenAI sample run above, live-provider, container, and cloud acceptance have not run under this plan. Access to provider credentials, a collector, Docker, and an authorized ACA target will be needed for the corresponding T10 evidence.