fix(sampling): sampling span missing for strategies that override sam…

[cptnm3]

Pull Request

Issue

Fixes #1487

Description

Move the sampling lifecycle boundary from BaseSamplingStrategy.sample() into the SamplingStrategy base class.

SamplingStrategy.sample() now owns:

• requirement merging and deduplication
• sampling_id creation
• sampling_loop_start dispatch and effective loop budget resolution
• validation of hook-modified loop budgets
• exception handling and error-path lifecycle closure
• sampling_loop_end dispatch

Concrete strategies now implement _sample_impl() for their sampling algorithm. Shared helpers centralize sampling iteration and repair payload construction and hook dispatch.

Migrate BaseSamplingStrategy, BudgetForcingSamplingStrategy, SOFAISamplingStrategy, and majority-voting to the new contract.

This ensures every top-level sample() call emits exactly one enclosing sampling lifecycle, regardless of whether a strategy uses the base sampling loop, implements its own loop, or fans out into multiple inner samples.

Intentional behavior changes:

• Majority voting emits one enclosing lifecycle for the top-level majority-vote operation instead of one lifecycle per inner sample.

• Budget Forcing and SOFAI emit sampling iteration and repair events for their strategy-specific attempts and repairs.

• Budget Forcing and SOFAI do not perform repairs after the final allowed failed iteration, since the repaired action/context cannot be consumed, aligning them with BaseSamplingStrategy.

• sampling_loop_end fires for failures during lifecycle setup, including requirement merging, start-hook execution, and invalid hook-modified loop budgets, allowing lifecycle consumers to close error paths.

Add firing-site and regression coverage for:

• strategies overriding only _sample_impl()
• sampling_id propagation and correlation
• start-hook exceptions and lifecycle setup failures
• hook-modified effective loop budgets
• invalid effective loop budgets and error-path closure
• Budget Forcing iteration and repair events
• SOFAI S1/S2 iteration and repair behavior
• majority-voting iteration and repair events
• a single enclosing lifecycle for majority voting

Assisted-by: IBM Bob

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

[github-actions]

This comment is managed by a bot. Editing it is fine — checking off boxes, adding notes — but please leave the HTML comment marker on the first line alone, otherwise checklist updates will break.

Sampling Strategy PR Checklist

Use this checklist when adding or modifying sampling strategies in mellea/stdlib/sampling/.

Base Class

• Extends appropriate base class:
  • BaseSamplingStrategy if your changes are mostly modifying the repair and/or select_from_failure functions
  • SamplingStrategy if your changes involve a new sample method
  • Other defined sampling strategies if your implementation is similar to existing implementations

Return Value

• Returns a properly typed SamplingResult. Specifically, this means:
  • ModelOutputThunks in sample_generations are properly typed from the Component and the parsed_repr is the expected type.

Integration

• Strategy exported in mellea/stdlib/sampling/__init__.py

[ajbozarth]

Just got back from vacation, I'll deep dive review this by EOW. Thank you for the contribution