fix: ensure minimum sigma of 1.0 in Hill Climbing for fine search grids (#86)

[mohammed18salah]

When epsilon is very small and the search grid is fine (e.g. np.arange(-10, 10, 0.01)), the noise sigma calculated as max_positions * epsilon can fall well below 0.5. Since discrete positions are represented as integer indices and noise is rounded, a sigma < 0.5 causes the noise to round to zero on almost every step, making the optimizer permanently stuck at its initial position.

This fix enforces a minimum sigma of 1.0 (one index step), ensuring the optimizer always has a chance to move to an adjacent grid point, regardless of epsilon magnitude.

Fixes #86

Motivation

When using fine search grids (e.g., np.arange(-10, 10, 0.01)) which have thousands
of discrete index positions, setting a small epsilon causes the noise sigma
(max_positions * epsilon) to fall below 0.5. This makes the rounding of discrete
noise always evaluate to zero, leaving the optimizer permanently stuck at its starting
position and unable to explore the search space at all.

Description of the changes

The fix is a single-line change in _iterate_discrete_batch inside
hill_climbing_optimizer.py.

Before:
sigmas = maximum(sigmas, 1e-10) # Too small — still causes stuck behavior

After:
sigmas = maximum(sigmas, 1.0) # Ensures at least 1 index step of noise

By enforcing a minimum sigma of 1.0 (one index unit), the Gaussian noise generator
will always have a non-zero probability of producing a value >= 0.5, which allows
the optimizer to step to an adjacent grid point. This preserves the original intent
of epsilon (controlling exploration scale) while preventing a complete halt when
epsilon is very small.

Tested on: sphere function with np.arange(-10, 10, 0.01) (2000 indices per dim).

• epsilon=0.001 → Best score: -9e-26 ✅ (previously stuck)
• epsilon=0.0001 → Best score: -9e-26 ✅ (previously stuck)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.