Skip to main content

Knowledge Base — acceptance criteria

Status: Draft Last updated: 2026-05-21 Audience: Engineers verifying feature completeness before merge; QA gating PR review.

These criteria are the gate for declaring the Knowledge Base feature shippable for v1 (Phases 1 + 2 + 3 combined). Each criterion has a phase tag. A phase is "done" when all of its tagged criteria pass.

The criteria here mirror spec.md §24 and are duplicated into this standalone file so reviewers can scan them without traversing the full spec.

Phase 1 — foundation​

  • A1. A newly-created Work has an initialized empty .content/kb/ structure in its data repo, including all class folders and a valid .index.yml. (Backfill job also produces this for pre-existing Works.)
  • A2. A user can POST /api/works/:id/kb/documents with a class + body and see the document persisted in both DB (row) and Git (sidecar .yml + .md pair at .content/kb/<class>/<slug>.{yml,md}).
  • A3. GET /api/works/:id/kb/documents returns the created document with all spec §6.1 fields populated.
  • A4. Lock semantics: POST /api/works/:id/kb/documents/:docId/lock with mode='full' sets the locked + lockMode columns and (in Phase 2) prevents the scheduled regeneration pipeline from mutating the doc.
  • A5. All Phase 1 migrations apply cleanly on a fresh database (sqlite for tests + postgres for CI integration).
  • A6. All Phase 1 migrations apply cleanly on a copy of staging (with non-empty data).
  • A7. The workId XOR organizationId CHECK constraint rejects both-set and neither-set inserts.
  • A8. Cascade-delete behavior on Work deletion removes all child KB documents, uploads, tags, citations, chunks.
  • A9. OpenAPI documentation generated by apps/api/ includes every new endpoint with request/response schemas matching the contracts package.
  • A10. All entity classes pass the existing pnpm lint / pnpm type-check / pnpm test pipeline at the monorepo root.
  • A11. PR review loop completes per workspace NN #14 / #18 (CodeRabbit / Codex / etc. all green or explicitly acknowledged) and CI is green per NN #19 before merge.

Phase 1B — workbench MVP + ingest pipeline​

  • A12. A user can upload a PDF via the workbench drag-and-drop; within 60 seconds the upload row reaches extractionStatus='succeeded' and a corresponding WorkKnowledgeDocument exists with non-empty Markdown body, persisted in both DB and Git.
  • A13. A user can open the workbench, see the upload in the Originals tab, see the extracted Markdown in the KB tab, and edit the Markdown in the Tiptap editor with autosave at the documented debounce.
  • A14. Viewer fallback: a 30 MB PDF renders in PDF.js; a 30 MB XLSX shows the download fallback per the spec §14.5 thresholds.
  • A15. Activity log emits kb.upload.created → kb.upload.normalized → kb.upload.extracted → kb.document.created in order for a successful ingest.
  • A16. Failed extraction sets extractionStatus='failed' + populates extractionError. The upload retry endpoint successfully re-runs extraction.
  • A17. Dedup: uploading the same file twice (identical SHA-256) reuses the existing WorkKnowledgeUpload row and emits a dedup activity-log entry.

Phase 2 — agent integration + retrieval​

  • A18. A user can @kb:brand/voice (or @kb:<docId>) in an AI conversation message; the assistant's response is grounded in that document and the response message carries a citations field referencing it.
  • A19. An org-admin can POST /api/organizations/:orgId/kb/documents with class='legal' and within 5 minutes every Work in the org with kbConfig.inheritance.legal='inherit' shows the document materialized in its .org/legal/ overlay.
  • A20. A Work that has its own legal/privacy.md overrides the org-level one for that path; agent retrieval returns the Work's version.
  • A21. kb.context.truncated is logged when retrieval context exceeds Work.kbConfig.retrievalConfig.maxContextTokens; truncation precedence is legal → glossary → brand → style → seo → personas per the spec.
  • A22. Embedding generation succeeds for a new document within 60 seconds; chunk rows are created with non-null embedding vectors and the hybrid chunker respects H2 / H3 boundaries when present.
  • A23. Lexical-only retrieval continues to work when no embedding provider is configured for the org (graceful fallback).
  • A24. Locked documents are excluded from scheduled-regen mutations; CommunityPrService does not propose changes to a locked doc; the doc's history shows no AI-authored commits after the lock was applied.
  • A25. kb.lock.violation is logged when a direct Git push mutates a locked doc; the workbench shows the diff banner with accept / revert actions.
  • A26. Cost tracking: a generation that retrieves N chunks creates exactly N WorkKnowledgeCitation rows and increments the kb-embedding category in UsageLedgerEntry by the embedded token count.
  • A27. LLM-eval suite produces stable scores on a fixture set across three consecutive runs (variance < specified threshold).

Phase 3 — polish​

  • A28. Video uploads are normalized to MP4 (size sanity check, playable in the workbench <video> viewer) and a transcript Markdown doc is created in the same KB folder.
  • A29. Audio uploads are normalized to MP3 with a transcript.
  • A30. Embedded-app outputs: an agent can write outputs/q2-deck/index.html + assets; the workbench renders it in a sandboxed iframe; the path is recorded in WorkKnowledgeDocument.metadata.
  • A31. "Promote to KB" action on an AI conversation message creates a new output-class document with source='agent' + generatedByAgentRunId populated.
  • A32. MCP kb.list / kb.read / kb.search tools return correct results for an authorized API key against a Work the key has access to. Calls against unauthorized Works return 403.
  • A33. CLI: ever works kb upload <work-id> <file> produces the same end state as a workbench upload (DB + Git both updated, activity-log entries identical except for actorType=cli).
  • A34. Wikilink renamer: renaming brand/voice.md to brand/voice-and-tone.md updates every [[voice]] / [brand voice](../brand/voice.md) reference in other KB documents and re-commits to Git in one operation.
  • A35. Reconciliation job runs daily and reports zero false-positive drift on a no-change repo; reports the correct delta on a changed repo.
  • A36. Orphan sweeper detects + tombstones + (after 7 days) deletes objects whose DB rows have been removed; recovers a tombstoned object if its SHA-256 is re-uploaded within the grace period.
  • A37. Every PostHog event kind listed in spec §23 fires for the appropriate user actions; no body content present in any event payload.

Cross-cutting (every phase)​

  • A38. Permissions: every endpoint rejects unauthorized roles per spec §20 (verified by automated e2e tests).
  • A39. Quotas: every documented limit in spec §21 returns 402 with the correct upgrade hint when exceeded.
  • A40. Security: no KB body content appears in webhooks, telemetry, or activity-log payloads (verified by a static-grep CI check on event-emitter call sites).
  • A41. End-to-end test in apps/web-e2e (Playwright) covers the full happy path: create document → edit → upload → extract → search → @kb: mention → lock → restore-from-history.

Phase exit checklist​

Before declaring a phase complete:

  1. All criteria for that phase tagged above pass.
  2. JIRA child ticket for the phase moves to Done.
  3. Each new feature spec / architecture doc referenced by this work is committed and indexed (per platform spec-driven-development convention).
  4. CHANGELOG / docs/log.md (or whichever the platform uses) gets a dated entry referencing the spec + the PR(s) that landed the phase.
  5. Smoke test in staging: real Work, real upload, real generation cycle, real deployment — passes end-to-end.
  6. Operator sign-off recorded in the JIRA ticket.