Skip to main content

Overview

A workspace’s topics are strands with origin = 'summary'. After a session summary completes, the summarisation worker queues a topic_graph job (visible in admin jobs). Pasting a session summary queues the same job, which first splits the paste into paragraphs (pasted summaries). The rag worker runs the merge step (kair_voice_core::topics) with its agent LLM: every summary paragraph is aligned with the workspace’s existing topics and recorded as an appearance, like a commit onto a file.

Outcomes

One model call per session lists the workspace’s topics (E1, E2, …; at most 40, shortlisted by shared words above that) and the session’s paragraphs (P1, …). The model returns one decision per paragraph: Without existing topics there is no model call: every paragraph is new. A paragraph the reply skips, or a decision naming an unknown topic, becomes new (repaired in the worker log). Machines never rewrite a topic sentence; only a person revision may (enforce_strand_revision_change_rules).

Ordering and re-indexing

  • Candidates are the workspace’s active summary topics that appear in any other session. A late upload that matches a topic only later sessions hold joins it as its earliest appearance (outcome new on that strand) instead of starting a duplicate; later appearances are not rewritten.
  • Merges run under a per-workspace advisory lock, so two sessions cannot both create the same topic.
  • Regenerating a summary re-runs the merge: the session’s earlier appearances and drafts are removed, and topics it created that no other session has continued are deleted. Topics other sessions have continued keep their history.
  • A failed merge is logged and leaves the summary in place; regenerating the summary runs it again.

Conflicts

Only wording and identity are merge conflicts; a disagreement is history and never waits for anyone. Open conflicts are listed at GET /workspaces/{workspace_id}/topic-conflicts and decided by whoever can edit the summary of the session that raised them (can_manage_recording; an admin when that session is gone): POST /topic-conflicts/wording/{draft_id} and POST /topic-conflicts/identity/{strand_id} record the decision and re-project the workspace’s graph. Listings show only conflicts from sessions the caller may read.

Organisation stitching

Workspaces are linked, not copied. After every merge, a workspace with cross_workspace_matching proposes corresponds_to relations from its topics to topics of the organisation’s other opted-in workspaces that share at least two entities with an entity Jaccard of 0.4 or more (kair_voice_core::topics::stitch). A pair with any earlier corresponds_to, including a rejected one, is never proposed again, and check_strand_relation_org_and_optin still refuses relations across organisations or without both opt-ins. Organisation admins, platform admins and the organisation’s moderators review them: GET /organisations/{organisation_id}/topic-links (proposals first, with the shared entities) and POST /topic-links/{relation_id} (accept or reject, then both workspaces are re-projected). Only accepted links reach the graph as CORRESPONDS_TO edges; agents follow one only into workspaces already in the query’s scope, and the API’s topic history never shows another workspace’s topic. Organisation admins (and platform admins) opt workspaces in or out under Settings → Linked topics: GET /organisations/{organisation_id}/workspace-matching lists them and PUT /workspaces/{workspace_id}/cross-workspace-matching ({enabled}) sets the flag. Opting in proposes links straight away; either way the workspace is re-projected, so an opt-out removes its cross-workspace edges.

Connections

Connections only ever compare a workspace with the opted-in workspaces of its own organisation, and show nothing until the workspace itself has opted in.
  • Linked topics: GET /sessions/{session_id}/topic-connections (session page and moderator dashboard) and GET /workspaces/{workspace_id}/topic-connections list accepted links from Postgres, from being the caller’s own topic. Links whose workspaces no longer both take part are hidden.
  • Shared entities: GET /workspaces/{workspace_id}/shared-entities (workspace_ids, min_workspace_count default 2, limit) returns entities mentioned by topics of several compared workspaces, with topic, session and workspace counts, from the topic graph. GET /workspaces/{workspace_id}/shared-entities/topics?name_normalised= lists the topics mentioning one. Both endpoints count only topics tied to sessions the caller may read (the same visible_sessions rule as workspace topics). Linked-topic responses drop any edge whose from or to topic is not visible under that rule. The /workspaces/{id}/connections page draws shared entities as bubbles.

Topic graph (Neo4j v2)

With rag.neo4j_v2 set, the same topic_graph job rebuilds the workspace’s topic graph after every merge (kair_voice_rag::topic_graph), in one Neo4j transaction from Postgres:
  • Topics are upserted; a topic keeps its embedding until its title or statement changes. Embeddings and theme titles/reports use <title>: <statement>, since a title alone (“Maintenance”) carries little meaning. Sessions, entities, themes and all edges are rebuilt.
  • Entities come from the paragraphs’ entities (never people), keyed per workspace by normalised name.
  • Themes are Louvain communities over topics, edges weighted by shared entities. A theme needs two topics; its title is its most connected topic’s sentence and its report lists the members’ sentences. Topics sharing no entity have no theme.
  • topic_embeddings is a cosine vector index on topic sentences, created with the embedding size and rebuilt when that size changes. Constraints and indexes are created at worker start-up.
  • Proposed and accepted relations between the workspace’s own topics are projected, plus accepted CORRESPONDS_TO links to other workspaces while both workspaces still take part in matching.
  • A failed projection is logged and leaves Postgres untouched; rebuild with the backfill.

Documents

Uploaded documents enter the graph as their own nodes, never as topics: they have no session appearances to merge, and would otherwise land in conflict review. The same projection writes them in the same transaction and under the same workspace lock:
  • Each document contributes the paragraphs of its latest structured document summary. A summary counts once it has paragraphs, so a re-run that is pending or failed keeps the old paragraphs in the graph. A document whose summary belongs to a deleted session, and a document that is gone from Postgres, is removed from the graph. session_id is the upload session, or '' for a workspace-level upload.
  • Only the anonymised paragraph title and statement leave Postgres: no chunk text and no file name. Paragraphs keep their embedding while <title>: <statement> is unchanged; topics and paragraphs are embedded in one call.
  • MENTIONS link a paragraph only to entities the workspace’s topics already created. Documents never create entities, so themes, shared entities and entity_topics are unchanged.
  • document_paragraph_embeddings is a cosine vector index created and resized like topic_embeddings; Document.document_id and DocumentParagraph.paragraph_id are unique, and both labels are indexed by workspace_id.
  • The document_graph job (queue kair.document_graph) re-projects the whole workspace. The summarisation worker queues it when a document summary completes, and the API when a document is deleted; a still-pending job of the same workspace is reused. Without rag.neo4j_v2 it completes without doing anything. It shows in the admin jobs list, cannot be re-run from there, and the reconciler republishes it from jobs.workspace_id.

Queries and API

The moderator agent gets four topic tools and two document tools when the topic graph is configured: The agent cites a document as [doc:<document_id>], never by name, since file names can name people. Answers are stored with the ids. cited_documents resolves the current file names, limited to the asker’s scope (and, for a session-pinned scope, to documents summarised in that session); the agent query response, the stream’s completed event and the conversation detail carry them as documents: [{document_id, file_name}], and the UI swaps each citation for the name, or a neutral “no longer available” label. Every tool filters by the verified scope (workspaces, or one session) before it expands; a session-scoped query sees only that session’s appearances. A document is reachable only when its workspace is in the scope; under a session-pinned scope (a share holder) its summary must also come from that session, and a bare session scope reaches no documents. An unknown, malformed or out-of-scope document id gets the same calm refusal, so it never reveals whether the document exists. API (tag Graph): GET /workspaces/{workspace_id}/topics, GET /topics/{strand_id}, GET /sessions/{session_id}/topics, GET /workspaces/{workspace_id}/themes. Each checks workspace membership, then drops whatever comes from sessions the caller may not read (can_view_session_summary); a topic only hidden sessions hold is reported as not found. After the merge and projection, the job runs strand enrichment and queues pattern_synthesis (both best effort). Pattern synthesis reads the session’s topics (sentence and outcome) and takes its entities from the session summary’s paragraph entities.

Backfill and migration

kair-voice-topic-backfill (in the API image, configured like the workers via CONFIG_FILE): Migrating a workspace: set rag.neo4j_v2, run resummarise, wait for the summaries, then project once more to settle the graph after the last merge.

Erasure

strand_revisions stays append-only, except inside a transaction that sets kair.strand_erasure = 'on'. Data-subject erasure (erase_session_topics) uses it:
  • The sessions are marked erased first, under the advisory lock of every workspace they sit in (ascending id). A merge, a summary paragraph write or a graph projection that waited on the lock sees the mark and writes nothing.
  • Appearances and drafts from the erased sessions are deleted, along with their summary paragraphs.
  • Topics the erasure left without an appearance are deleted with their revisions. Merged strands record a person’s same-topic decision and are never deleted.
  • Revisions the erased sessions wrote (source_session_id) are scrubbed to the scrub marker, as are revisions of the same topic repeating that text (an accepted wording keeps the statement). A topic whose head was scrubbed gets a new head from its earliest remaining paragraph.
  • Deleting a session through the API runs the same erasure in the delete transaction; source_session_id and origin_session_id are ON DELETE SET NULL. The topic graph is re-projected afterwards, and dropped when the workspace is deleted.
  • Graph projection holds the workspace lock from its Postgres reads until the Neo4j commit; only topics with an appearance in a live session are projected or readable by the agent.
  • Deleting a document (API or data-subject erasure of the uploader) deletes its summaries with it, paragraphs and evaluations cascading. The API then queues a document_graph job; erasure re-projects its workspaces, so the graph keeps nothing from the document.
After erasure the worker re-projects the organisation’s workspaces, so the graph never holds an erased session. Moderator-agent strands (origin = 'moderator') are not covered yet.