Technical debt tracker¶
Review date: 2026-09-06. Owner identifies the responsible module maintainers, not an assigned individual. Update an entry when its evidence or status changes. Close it only with a linked result.
| ID | Gap / evidence | Owner | Exit criterion | Status |
|---|---|---|---|---|
| H-001 | Spec inventory had historical task drift and inconsistent status vocabulary, including complete. |
Feature maintainers | Reconcile each affected row against merge and acceptance evidence. | Resolved 2026-09-06 by the drift reconciliation: statuses normalised to shipped/pending/draft/archived, 23 feature plans moved to completed/, remaining scope named per row. Historical checkboxes are intentionally unticked. |
| H-002 | Architecture has no global dependency-layer enforcement. Some models import library logic. | Architecture maintainers | Agree actual allowed edges, record exceptions, and add focused structural checks without inventing a new runtime architecture. | Open |
| H-003 | Makefile includes serial test aliases, placeholder package helpers, and Sphinx-style docs targets despite MkDocs. | Developer tooling | Align affected helpers with current package and documented commands, with command-level verification. | Open |
| H-004 | No automatically isolated per-worktree logs/metrics/traces environment. | Observability maintainers | Provide a reproducible local setup and a demonstrated query against a representative request. | Open |
| H-005 | Legacy context generation could repopulate root instruction files. | Repository harness | Removed command scaffolding and generators; the harness checker validates both entry points. | Resolved in workflow cleanup |
| H-006 | Repository-source links broke when published as relative docsite links. | Documentation maintainers | Publishing hook resolves existing source links. Regression tests and strict MkDocs build passed. | Resolved in harness adoption |
| H-007 | NLTK 3.10.3 had no patch for PYSEC-2026-3740, so make security carried a scoped audit exception for the inspected evaluator paths. |
Evaluation and dependency maintainers | Lock a patched release and remove the exception. | Resolved 2026-11-09: nltk 3.10.4 locked via constraint-dependencies; exception removed from make security and docs/SECURITY.md |
Feature 035 follow-ups — reconciled 2026-09-06¶
| ID | Gap / evidence | Owner | Exit criterion | Status |
|---|---|---|---|---|
| H-008 | Final SK inventory: connectors, chunking, telemetry defaults, and tests still depend on SK. | Retrieval and backend maintainers | Replace remaining runtime/test dependencies, preserve retrieval behavior, remove package/lock constraints, and verify SK-free installation. Create a dedicated implementation spec before this work. | Resolved 2026-11-10 by 042 T16 (PR #440): semantic-kernel is out of pyproject.toml and uv.lock; no file under src/ names it and it is not importable in the dev environment (guard); a clean base wheel install has no semantic_kernel and runs holodeck --help. Existing Qdrant, Chroma and Postgres layouts are frozen in legacy fixtures and checked by the native adapters' compat tests. See the 042 execution plan. Evidence that is still open after the removal is tracked as H-023 to H-031 |
| H-009 | LiteLLM acceptance gaps: dimension plumbing, error contract, telemetry evidence, and provider documentation. | Retrieval and observability maintainers | Resolve each documented contract difference and supply focused and provider acceptance evidence. | Open; focused contracts and documentation closed by 035 T2 (2026-09-06), live provider evidence pending T10 |
| H-010 | Structured vectorstore sources on provider: qdrant whose id_field values are not UUIDs or integers are stored under UUIDv5 hashes (vectorstore_tool.py _coerce_record_id), so StructuredQueryResult.id carries the hash; the original key survives only when listed in meta_fields. A warning is logged once per tool. |
RAG maintainers | Persist the original record key on the structured record schema (ask-first schema change) and return it through StructuredQueryResult.id, with a regression test on Qdrant. |
Open |
Feature 042 follow-ups — 2026-11-10¶
These entries hold what the 042 execution plan leaves open. Its plan stays in active/ until H-023, H-024 and H-031 close or the maintainer accepts them.
| ID | Gap / evidence | Owner | Exit criterion | Status |
|---|---|---|---|---|
| H-023 | Live evidence is missing for three native adapters. Pinecone: the contract suite ran on Pinecone Local (end of life) with a fake control plane. Azure AI Search: a fake service and the real SDK over a stub transport. Cosmos DB NoSQL: the emulator for everything except hybrid search, which ran on a fake. The credential-gated cases in make verify-live skip without credentials. |
Retrieval maintainers | Run make verify-providers with PINECONE_API_KEY, AZURE_AI_SEARCH_ENDPOINT and AZURE_AI_SEARCH_API_KEY, and a Cosmos DB account with full-text search; record the result in the 042 plan. |
Open: needs credentials |
| H-024 | The manifest and fingerprint have not run on hosted Pinecone or Azure AI Search (they ran on Qdrant, pgvector, Chroma, Weaviate and the Cosmos DB emulator on 2026-11-10). Concurrent writers are narrowed, not eliminated, on stores with no compare-and-set (see the vector stores guide). | Retrieval maintainers | Run the migration tests on each store, or document the exact race for each store that lacks compare-and-set and accept it. | Open |
| H-025 | A vectorstore or hierarchical_document tool still falls back to in-memory storage when its store cannot be reached (base_tool.py _create_adapter_with_fallback). Only a missing client library and a retired provider raise. A wrong host stores the knowledge base in memory and logs one warning. |
Tools maintainers | Decide with the maintainer whether an unreachable store should fail; if so, fail at initialization with the connection error and update the guide and the verify fallback check. | Open |
| H-026 | uv.lock pins pypdfium2 5.15.1, which PyPI marks as yanked ("Closing of remaining objects on interpreter exit broken due to an oversight"). 5.15.2 exists. pdfplumber, reached through markitdown[pdf], requires it. Checked 2026-11-10 on the PyPI JSON API. |
Packaging maintainers | uv lock --upgrade-package pypdfium2, then make verify-profiles. |
Resolved 2026-11-10 (PR #443): the lock pins 5.15.2. |
| H-027 | The three copies of agent.schema.json under src/holodeck/templates/*/schemas/ are hand-kept. They differ from schemas/agent.schema.json by about 570 diff lines (for example the optimizer axes are missing), and make schema and make schema-check do not cover them. |
Configuration maintainers | Generate the copies in make schema and check them in make schema-check, or stop shipping them. |
Resolved 2026-11-10 (PR #443): make schema writes the copies and make schema-check checks them. |
| H-028 | Temporal dev-server startup is flaky on the self-hosted CI runner when many runs overlap. | CI and Temporal maintainers | Start the dev server with a bounded retry and a longer timeout, or serialize the lane, and show a clean run of overlapping jobs. | Open |
| H-029 | The make verify-live AG-UI turn that asks the OpenAI model to repeat a code word fails now and then: the model sometimes declines. A re-run passes. |
Verify-suite maintainers | Reword the prompt or accept two attempts, with the retry shown in the report. | Open |
| H-030 | Two evidence steps of 042 need a person: the CopilotKit browser step of the live validation loop in AGENTS.md, and the provenance review (FR-029) of the Node.js install (NodeSource) in the image template. | Deployment maintainers | Run the CopilotKit flow against holodeck serve and share what the UI showed; review and record the Node.js source in the image template. |
Resolved 2026-11-10 (PR #443): the Node.js install uses the signed NodeSource apt repository, and the CopilotKit flow ran against holodeck serve (both recorded in the 042 plan). |
| H-031 | The pre-commit job on PR #442 fails on MyPy errors in scripts/verify/profile_probe.py (VectorstoreTool and DatabaseConfig calls, and a None passed to NumericProposer.begin). make verify-fast type-checks only src, so it did not catch them. mypy scripts/verify/profile_probe.py run alone passes, so the hook's MyPy settings differ; reproduce with pre-commit run mypy --all-files. Azure OpenAI, Ollama and a Temporal server were also not part of make verify-live (AC-03). |
Verify-suite maintainers | Fix the probe so the hook passes; add scripts to the MyPy run in verify-fast; run the live loop on Azure OpenAI and Ollama when credentials exist. |
Partly resolved 2026-11-10 (PR #442): the probe is fixed and the hook passes. Open: scripts in the verify-fast MyPy run, and an Azure OpenAI live run. Ollama and Temporal ran live on 2026-11-10 (042 plan, AC-03). |
The 035 completion plan schedules H-009 and retains H-008 as a separate decommission effort. The 035 reconciliation audits the 035 portion of H-001. Other feature inventories remain outside this audit.
Deferred 035 scope — T0 destinations¶
These entries preserve the existing exclusions in the completion plan. They are durable backlog destinations, not new implementation authorization or delivered capabilities. FR-084 still forbids silently bypassing approval gates. FR-089 still requires managed child-process scrubbing. H-013 and H-021 cover the broader interactive-approval and arbitrary-execution boundaries.
The harness checker detects layout, link-target, discovery, and generated-inventory drift. Semantic freshness still requires code inspection and named test evidence. No scheduled cleanup service is configured by this change.