Skip to content

explain_clean computes build_contexts twice — once on df and once on cleaned #32

Description

@JohnnyWilson16

In explain.py:

contexts = build_contexts(df, cfg)          # L160
cleaned, report = run_pipeline(df, cfg)      # L161
...
post_contexts = build_contexts(cleaned, cfg) # L192

build_contexts is called on the original df at L160 to compute role model selections. Then run_pipeline at L161 also calls build_contexts internally (via build_engine_cache at cleaner.py). Finally L192 calls it a third time on the cleaned output.
That's 3 full profiling passes. On wide datasets with 100+ columns, this is a measurable performance hit.
Fix: Accept an optional contexts parameter in explain_clean or refactor run_pipeline to return the cache it built so explain_clean can reuse it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions