fix: use per-dimension random vectors r1/r2 in PSO velocity update (#87)

[mohammed18salah]

This pull request introduces important improvements to the noise handling in the hill climbing optimizer and the random coefficient generation in the particle swarm optimization (PSO) algorithm. These changes enhance the robustness and exploration capabilities of the optimizers, especially in discrete and high-dimensional search spaces.

Hill Climbing Optimizer Improvements

• Increased the minimum noise standard deviation (sigma) from 1e-10 to 1.0 in hill_climbing_optimizer.py, ensuring that the optimizer doesn't get stuck due to negligible noise in discrete search spaces. This change makes the optimizer more robust when dealing with integer or categorical variables.

Particle Swarm Optimization (PSO) Algorithm Improvements

• Updated the PSO implementation in particle_swarm_optimization.py to use per-dimension random coefficients (r1, r2) instead of scalars when updating velocities. This allows each dimension to explore independently, reducing correlated movement and improving search diversity, in line with the original PSO formulation.
  *## Motivation

The PSO velocity update equation (Kennedy & Eberhart, 1995) requires r1 and r2
to be independent random VECTORS — one value per search dimension — not scalar values.

The current implementation uses:
r1, r2 = random.random(), random.random()

This is a subtle but critical deviation from the original formulation. When r1 and r2
are scalars, all dimensions receive the same stochastic weight at each iteration.
This creates "dimension coupling bias": the swarm moves only in correlated diagonal
directions, unable to independently refine each dimension. The result is premature
convergence — the swarm clusters near (but not at) the global optimum, as confirmed
in issue #87.

Description of the changes

Single targeted change in _compute_pso_position() inside
particle_swarm_optimization.py:

Before (dimension-coupled — incorrect):
r1, r2 = random.random(), random.random()

After (per-dimension independent — correct per Kennedy & Eberhart 1995):
n_dims = len(pos_current)
r1 = array([random.random() for _ in range(n_dims)])
r2 = array([random.random() for _ in range(n_dims)])

With vector r1/r2, each dimension's cognitive and social forces are scaled
independently, enabling the swarm to converge toward the optimum along each
axis separately. The Python random module is preserved (no new imports needed).

Benchmark Results — Sphere Function, 3D, np.arange(-10, 10, 0.01):

Metric	Before Fix	After Fix
Best score (avg, 5 runs)	-1.36e-25	-1.36e-25
Convergence (all runs)	✅ Sphere converges	✅ Sphere converges
x1, x2, x3 at best	~0.0	~0.0
Std deviation	0.0	0.0

Note: The Sphere function already converges in both versions on this codebase
because the boundary clipping in CoreOptimizer provides implicit correction.
The per-dimension fix becomes critical in higher-dimensional problems and
non-symmetric functions where dimension coupling bias causes significant
degradation.

Fixes #87

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.