Skip to content

[Feature] Preserve minimal provenance on durable task evidence #35

Description

@kvnloo

What outcome should OpenMuse handle?

Durable task evidence should retain enough provenance to tell what was actually observed, where it came from, and how it was acquired.

Today Evidence is intentionally small:

{
  id,
  kind: "mail" | "file" | "web" | "user",
  title,
  excerpt,
  url?
}

That works while each kind has one obvious source, but the current repo is already crossing that boundary:

Those records can have the same URL/excerpt shape while meaning different things. A search excerpt is provider-returned evidence about a page; a browser read is an OpenMuse observation of the page itself. Treating both as the same provenance makes later verification, stale-data handling, debugging, and Learning/export harder.

The invariant I think we need is:

evidence content is not its provenance.

Proposed behavior

Add the smallest provenance envelope that is useful across existing evidence producers, without turning Evidence into a generic knowledge graph.

For example, optional fields along the lines of:

  • observedAt
  • source / acquisition such as browser, monitor, search, mail, user
  • a stable source/entity reference when one exists (messageId, browser session, monitor, provider, etc.)

The exact schema is less important than preserving these distinctions:

  • observation identity vs source/container identity;
  • direct observation vs provider-supplied excerpt;
  • source data vs authority.

I would keep provider-specific payloads out of the core record. Search-provider names or query IDs can live in a small optional provenance object if needed, while the main evidence shape stays stable.

How would we verify it works?

  • A direct browser read and a search result for the same URL remain distinguishable after task restart.
  • Two observations from one persistent browser session have distinct evidence IDs but share the correct source/session provenance.
  • Monitor evidence identifies the monitor and observation time without using the monitor ID as the observation ID.
  • Search adapters can change providers without changing the core evidence schema.
  • Existing mail/file/user evidence remains backwards-compatible.
  • UI can continue rendering title/excerpt/url without knowing provider details.

This seems worth deciding while #22/#28 add search and before more connectors or Automatic Learning depend on the current evidence shape.

AI-use note: I used an AI assistant to trace the current evidence producers and draft this issue; I verified the cited browser, monitor, mail, UI, and #22 search paths against current source/PR heads before filing.

Activity

  1. hachemchaabi commented on Sep 23, 2026

    @hachemchaabi

    Hi, I would love to work on this. Could you please assign it to me?

  2. kvnloo commented on Sep 23, 2026

    @kvnloo
    ContributorAuthor

    Thanks! I'd be glad to see this explored.

    I'd keep the first increment deliberately small: preserve acquisition provenance on the existing "Evidence" record rather than introducing a broader evidence/knowledge abstraction.

    The main invariants I'd try to pin down are:

    • two browser observations can have distinct evidence IDs while sharing the same browser session/source;
    • a direct browser observation and a search-provider excerpt for the same URL remain distinguishable;
    • monitor evidence retains its monitor/source identity and observation time;
    • existing mail/file/user evidence stays backwards-compatible.

    An optional provenance/source field plus "observedAt" may be enough for the first pass. I wouldn't add confidence scores, inference graphs, or provider-specific payloads yet.

    If you open a PR, happy to review the shape against those invariants.

    AI-use note: I used an AI assistant to help inspect the current evidence producers and draft this scope suggestion; I verified the referenced behavior against the current issue and repository code.

  3. hachemchaabi commented on Sep 23, 2026

    @hachemchaabi

    @kvnloo

    Opened #43 for this. It keeps the first pass minimal: an optional provenance: { acquisition, observedAt, sourceId?, provider? } on the existing Evidence, covering the four invariants you listed. Happy to adjust the shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions