Skip to content

fix: avoid duplicate originals in augment_text_with_ids - #842

Open
emecii wants to merge 1 commit into
QData:masterfrom
emecii:fix/augment-text-with-ids-duplicates
Open

fix: avoid duplicate originals in augment_text_with_ids#842
emecii wants to merge 1 commit into
QData:masterfrom
emecii:fix/augment-text-with-ids-duplicates

Conversation

@emecii

@emecii emecii commented Sep 7, 2026

Copy link
Copy Markdown

What does this PR do?

Summary

Fixes Augmenter.augment_text_with_ids() returning every original input twice. The method appended the source text before augmentation and then appended it again alongside the generated variants, which silently over-represented originals in the returned training data.

Additions

  • Adds regression coverage for two inputs and multiple generated variants, including ID alignment.

Changes

  • Retains each original input once, followed by only its generated augmentations.

Deletions

  • Removes the duplicate original entry and a no-op list.extend attribute access.

Validation

  • pytest tests/test_augment_api.py -k 'not back_translation and not back_transcription' -q — 12 passed, 2 model-dependent tests deselected.
  • black --check, isort --check-only, and flake8 on the changed files — passed.

AI assistance

This PR was prepared with AI assistance. The behavior and regression test above were verified locally.

Checklist

  • The title of this pull request summarizes its contribution.
  • This description explains the changed behavior and rationale.
  • Relevant regression coverage was added.
  • The full existing test suite was run locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant