fix: clamp MMR's iteration count to the number of candidate words

[dripston]

Problem

update_topics() with MaximalMarginalRelevance raises a ValueError when a topic's number of extracted candidate words is less than or equal to top_n_words — reported as Length of weights not compatible with specified axis, though the exact wording is numpy-version-dependent (I get attempt to get argmax of an empty sequence on numpy 2.4.6 with the same root cause — see Testing below). This happens "randomly for certain trials," as the reporter noted, precisely because it depends on how many words a given topic actually extracted, which varies per topic/per hyperparameter trial.

Root cause

bertopic/representation/_mmr.py's mmr():

keywords_idx = [np.argmax(word_doc_similarity)]
candidates_idx = [i for i in range(len(words)) if i != keywords_idx[0]]

for _ in range(top_n - 1):
    ...
    mmr_idx = candidates_idx[np.argmax(mmr)]
    keywords_idx.append(mmr_idx)
    candidates_idx.remove(mmr_idx)

The loop runs top_n - 1 times, removing one entry from candidates_idx per iteration, with no check against how many candidates actually exist. If top_n (from MaximalMarginalRelevance(top_n_words=...)) is >= the number of candidate words for that topic, candidates_idx empties out before the loop ends, and the next iteration's np.argmax(mmr) (or an equivalent empty-array reduction, depending on numpy version) raises.

Fix

n_iterations = min(top_n - 1, len(candidates_idx))
for _ in range(n_iterations):

When there are enough candidates (the common case), min(...) is a no-op and behavior is unchanged. When there are fewer, the loop stops once every candidate has been selected, returning all of them instead of crashing — which matches mmr()'s own contract of returning up to top_n keywords.

Testing

• Reproduced the crash directly against mmr() (isolated from the full BERTopic/Optuna pipeline in the issue): 3 candidate words, top_n=10 → ValueError: attempt to get argmax of an empty sequence on this environment's numpy (2.4.6). Confirmed the fix resolves it and returns all 3 words.
• Added tests/test_representation/test_mmr.py covering: top_n exceeding available words, top_n exactly equal to available words, a single candidate word, and the existing "more candidates than top_n" case (to confirm no behavior change there).
• Verified all 4 new test cases pass against the fixed code and that the primary regression case fails (with the empty-array error) against the unfixed code.
• Could not run these through pytest directly in this environment — tests/conftest.py eagerly imports umap, which isn't installed here, so pytest can't collect anything under tests/ at all (fails before reaching my file). Ran the equivalent assertions as a plain script instead; please run pytest tests/test_representation/test_mmr.py in CI/a configured dev environment to confirm under the project's actual test runner.
• python3 -m py_compile passes on both changed/added files.

Fixes #2266

🤖 Generated with Claude Code