Skip to content

Allow multiple schemas for one shot #393

@ppinchuk

Description

@ppinchuk

It might be good to allow users to provide multiple schemas, which would set up multiple parsers for the one shot extraction.

Key considerations:

  • Auto generating queries, heuristic keywords, and keywork scores will become tricker. Might be able to get around this by only merging the "features" and "$descriptions" into a "main" schema and using that as the template
  • Allowing custom prompts for text collection and text extraction will have to support a per-schema approach. Likely the safest way to do this is to use a dict keyed by the schema filename (no extension?), but it's kind of ugly. Could also disable custom prompts for a multi-schema input but this seems unnecessarily restrictive. Maybe instead the schema input itself is a dict and then keys can match between those? No matter what, this is kind of a bookkeeping nightmare
  • The auto generated collection/text extraction prompts as well as the parsers should be on a 1-per-input schema basis, with the data getting concatenated at the very end like we do for wind/solar

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions