Skip to content

Answer from the documents the asker may read - #113

Merged
davidmckayv merged 1 commit into
CopilotKit:mainfrom
Hotragn:answer-from-what-the-asker-may-read
Aug 21, 2026
Merged

Answer from the documents the asker may read#113
davidmckayv merged 1 commit into
CopilotKit:mainfrom
Hotragn:answer-from-what-the-asker-may-read

Conversation

@Hotragn

@Hotragn Hotragn commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What this changes

connectors/sync-persistence.ts has been writing documents, chunks and document_acls since the
beginning and nothing ever read them back. A deployment that connected a source got rows in
PostgreSQL and still no citation, while the Knowledge coworker answered as though something were
behind it.

This is the read half — item 1 of the three @NathanTarbert laid out on #59.

createKnowledgeSearch matches a question against chunks.content and returns one citation per
document: the title, the canonical URL the connector stored, and the passage that matched. It reaches
the model as a server-executed tool beside the MCP ones, so a run needs no browser.

The ACL is evaluated in SQL, not in this process. That is the part worth reviewing closely. A read
path that fetches rows and filters them in the server has already fetched them: the wrong document is
in memory, it is one refactor from being returned, and the cost scales with the corpus rather than
with the answer. So the predicate is the boundary:

  • readable when some ACL row allows one of the asker's principals and no ACL row denies one, which
    makes deny beat allow whatever order the planner reaches them in;
  • a document with no ACL rows at all is readable by nobody. Absence is the refusal, as it is for
    every other grant here, so ACLs that failed to sync leave a document invisible rather than public;
  • soft-deleted documents are excluded, because a connector's delete means the source removed it.

group: principals are matched even though users.groups is written by nothing today (#82, #92), so a
group ACL starts working the day groups arrive from an IdP instead of needing to be found and changed.
Until then such a row matches nobody, which denies rather than permits. Asserted both ways in the
tests so that day is a passing test rather than a new one.

The three questions from #59, and what I did absent an answer

Stated here rather than buried, because each is a decision and each is cheap to reverse.

  1. Ranking is full-text search, not vectors. chunks.embedding is vector(1536) and nothing in
    this repository has ever written one: connectors/contract.ts has the adapter supply embeddings,
    no adapter exists yet, and there is no embedding model in the tenant package, .env.example or the
    docs. Full-text search needs no configuration a deployment does not already have and can be
    demonstrated end to end today. When an adapter starts writing embeddings the ranking changes behind
    this signature and the ACL predicate does not move. If you would rather slice 1 carried an
    embedding client and its configuration, this is the piece to send back.
  2. No per-Bot grant. The query is the access control: the search runs on the asker's own
    principals, so no Bot can return a document the person asking could not have opened themselves. A
    grant would refine who may ask, which is a real thing to want and a separate decision. I have not
    invented a grant table for it.
  3. No new policy intent. Refusing retrieval for a particular Bot would need one, and an intent is
    a boundary change rather than a feature, so I would rather you named it than I did.

The tool is offered only when there is something to search, because a tool in the list is a sentence
in the prompt and a step the model may spend.

Part of #59. Does not close it: items 2 and 3 remain, and item 2 is @guidovizoso's #97.

Where it runs

  • New state that outlives a request? None. No caches, no Map, nothing held between calls. Every
    answer is a query.
  • What happens on the second replica? Identical. Each replica runs the same three statements
    against the same tables and holds nothing, so which process answers cannot change the answer. This
    is deliberately the opposite shape to the InMemoryKnowledgeRepository that Take back the two features that only worked on one machine #21 would have taken
    back.
  • Anything serialised? Nothing needs to be. The only write is the knowledge.searched audit
    insert, which has no ordering requirement against anything else.
  • Anything fanned out to a browser? No. The result is the tool's return value on the run that
    asked for it.
  • New listener, port, or schedule? No — and that is the line between this and item 3. A
    connector schedule is the thing that raises "what do a hundred copies of it do", and it is not here.
  • Cost, since it is not free: two small reads per run in addition to the search — one indexed
    limit 1 to see whether anything is worth offering, and one primary-key lookup for the asker's
    groups. Both are per-run rather than captured at boot, for the reason plugins/tools.ts gives about
    grants: a source connected this morning should work this afternoon, not after a restart.

Boundary and audit

  • Every call writes knowledge.searched, which audit.ts already declared and nothing had ever
    written.
  • The row is written after the search, and this is the one place that is the right way round.
    Nothing is being acted on: no document changes, nothing leaves the deployment, and the interesting
    fact is what came back rather than what was attempted. Every acting path still writes first.
  • A search that matched nothing writes a row too, so "this Bot was asked six times last week and found
    nothing" is answerable.
  • The row names the query and the document ids and never quotes a passage. content is already on
    the redaction list in audit.ts, and a snippet is the document's text under another name; a trail
    that quoted it would have copied the corpus into a table with a different audience. There is a test
    asserting the passage is absent from the payload.
  • Nothing new is trusted from the client. The asker is the actor the run was already resolved to; the
    query text is the only thing the model supplies, and it reaches PostgreSQL as a bound parameter.
  • An empty result returns a sentence rather than an empty string, because a model handed "" fills the
    gap from training and cites a document that does not exist. Same reasoning as Have Knowledge admit no source is connected, instead of citing one #58.

Changelog

Added under UnreleasedAdded: "A Bot can answer from a connected source, as the person asking."

Proof

Ran locally — server/tests/knowledge-tool.test.ts, 6 pass / 0 fail / 22 expect() calls. No
database needed, so these are the assertions I can stand behind directly: an empty result becomes a
sentence, a row is written either way, the search is asked on behalf of the person rather than the
Bot, no passage reaches the payload, and a malformed call is answered instead of ending the run.

 6 pass, 0 fail, 22 expect() calls

server typecheck clean, biome format and biome lint clean on all five files.

Not run locally — server/tests/knowledge-search.integration.test.ts, 11 tests. This machine has
no PostgreSQL: Docker's engine will not start here and the embedded one is an apt install inside the
image rather than something reusable on a host. So CI is the first place the query itself executes,
and I would rather say that than imply a green suite I did not see. It is the same disclosure I made
on #109, where the integration tests then passed in CI.

What those 11 cover, since they are the ones that matter: an allowed document comes back with its link
and a marked passage; a document nothing allows does not; a document allowed to somebody else does
not; a deny beats an allow on the same document; a deny on a group beats an allow on the person; a
group allow reaches somebody in that group and nobody else; a soft-deleted document is excluded; a
document matching in several passages is one citation; a blank question asks the database nothing; and
the count is bounded however large a limit is requested.

I will post the CI result here rather than leaving the caveat standing.

What is not covered

  • Ranking by meaning, per question 1 above. This finds documents that share words with the
    question, which is a real answer and not the same answer.
  • A schedule. Nothing here syncs anything; it reads what a sync wrote. That is item 3 and you
    asked for it to be discussed first.
  • The dead second design. server/src/knowledge/acl.ts, repository.ts and types.ts are
    untouched — this depends on none of them, and Remove the knowledge path until it's supported. #89 removing them stands on its own. I did not want to
    fold a deletion you had opened into a feature PR.
  • Chunk-level ACLs. ACLs are per document, which is what the schema models.
  • The tool is offered to every Bot when documents exist, per question 2. Safe because of the ACL
    filter, but "which Bots may ask" is a decision I have left to you.

`connectors/sync-persistence.ts` has been writing `documents`, `chunks` and
`document_acls` since the beginning and nothing ever read them back. A
deployment that connected a source got rows in PostgreSQL and still no
citation, while the Knowledge coworker answered as though something were
behind it.

This is the read half. `createKnowledgeSearch` matches the question against
`chunks.content` and returns one citation per document: title, the canonical
URL the connector stored, and the passage that matched.

The ACL is evaluated in SQL, not in this process. A read path that fetches
rows and filters them here has already fetched them: the wrong document is
in memory and one refactor from being returned, and the cost scales with the
corpus rather than the answer. A document is readable when some row allows
one of the asker's principals and no row denies one, so deny beats allow
whatever order the planner reaches them in, and a document with no ACL rows
at all is readable by nobody. Absence is the refusal, as it is for every
other grant here, so ACLs that failed to sync leave a document invisible
rather than public.

`group:` principals are matched even though `users.groups` is written by
nothing today, so a group ACL starts working the day groups arrive from an
identity provider rather than needing to be found and changed. Until then such
a row matches nobody, which denies rather than permits.

Reached as a server-executed tool beside the MCP ones, so a run needs no
browser. It is offered without a per-Bot grant because the query is the access
control: the search runs on the asker's own principals, so no Bot can return a
document the person asking could not have opened themselves. It is offered only
when there is something to search, because a tool in the list is a sentence in
the prompt and a step the model may spend. Every call writes
`knowledge.searched`, naming the query and the documents returned and never
quoting their text.

Ranking is PostgreSQL's full-text search, not vectors. `chunks.embedding` is
`vector(1536)` and nothing in this repository has ever written one:
`connectors/contract.ts` has the adapter supply embeddings, no adapter exists
yet, and there is no embedding model in the tenant package or the environment.
Full-text search needs no configuration a deployment does not already have.
When an adapter starts writing embeddings the ranking changes behind this
signature and the ACL predicate does not move.
@Hotragn
Hotragn force-pushed the answer-from-what-the-asker-may-read branch from 9c16316 to b118c8e Compare August 21, 2026 21:39
@Hotragn

Hotragn commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Closing the loop on the verification caveat, as promised — and correcting myself twice.

Every check is green, including the integration tests I could not run here:

tests                1047 pass / 0 fail / 2354 expect() calls  (1052 across 102 files)
format, lint, types  pass
build                pass
migrations           pass
image                pass
verify               pass

All ten ACL cases pass against a real pgvector database, including the two that matter most — a deny beats an allow on the same document and a group allow reaches somebody in that group and nobody else.

Two corrections.

I wrote "11 tests" in the integration file above. It is ten. Miscounted my own list.

More usefully: the first CI run failed, and the reason is worth recording rather than quietly force-pushing over. I had passed the asker's principals as one JSON array and read them back with jsonb_array_elements_text($1::jsonb), which PostgreSQL refused:

PostgresError: cannot extract elements from a scalar

The driver already encodes a parameter, so $1::jsonb arrived as a JSON string rather than an array. It is now one bound parameter per principal via sql.join, which has no encoding to get wrong, and the reasoning is in a comment at the call site so the next person does not reach for the tidier-looking version. The ACL predicate itself did not change.

That is exactly the failure the local-PostgreSQL gap was going to cost, so the caveat earned its place. Worth noting it was a query-construction bug and not a boundary one: the tests that assert who may read what were the tests that caught it.

@davidmckayv
davidmckayv merged commit ba8dd2f into CopilotKit:main Aug 21, 2026
6 checks passed
davidmckayv added a commit that referenced this pull request Aug 21, 2026
Reverts #113, which was merged by accident, and says why rather than only undoing it.

It added a knowledge search over `documents`, `chunks` and `document_acls`: our own copy of a
customer's corpus, ranked here, with an ACL predicate of our own writing deciding who may see what.
The code is careful and the ACL filter is in the right place. The design is the thing being taken
back.

A Bot answers from a live system by calling that system's own search, as the person asking. The
vendor decides what they may see, because the vendor is the only thing that actually knows: an index
here is a permission model we have to keep in step with theirs, and every gap between the two is an
answer assembled from documents somebody cannot open. It is also a second copy of their data to
secure, to keep current, and to remember to delete when somebody leaves.

Retrieval has a place later, over the tool catalogue rather than over documents: choosing which of a
hundred tools to call is a search problem, and one about our own metadata rather than a customer's
files. That is a different thing wearing the same word.

The write half of the index goes with #97, which removes the connector that filled it. The three
tables and the `knowledge/` modules are read by nothing after this and should be dropped in a change
that says so, rather than as a side effect of this one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants