Add CmaEsRefinementSampler — CMA-ES with quasi-random refinement

[EliMunkey]

• I agree to the Contributor Agreement.

Summary

A three-phase sampler combining Sobol QMC initialization, CMA-ES optimization, and quasi-random Gaussian refinement using Sobol-based perturbation vectors with exponential decay.

BBOB benchmark results (24 functions, 5D, 10 seeds, 200 trials):

Sampler	Mean Normalized Regret	vs Random
Random baseline	1.0000	—
Default TPE	0.2463	75% better
CMA-ES (tuned)	0.2004	80% better
CMA-ES + Refinement	0.1284	87% better

The refinement phase uses quasi-random Sobol sequences transformed via inverse CDF to generate Gaussian-distributed perturbation vectors. This provides more uniform directional coverage than pseudo-random Gaussian, systematically exploring directions that random sampling misses.

Minimal example

import optuna
import optunahub

def objective(trial):
    return sum(trial.suggest_float(f"x{i}", -5, 5) ** 2 for i in range(5))

module = optunahub.load_module("samplers/cma_es_refinement")
sampler = module.CmaEsRefinementSampler(seed=42)
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=200)
print(f"Best: {study.best_value:.6f}")

Full experiment logs (135 experiments): github.com/EliMunkey/autoresearch-optuna

TODO

• Package added under package/
• __init__.py exports the sampler class
• README.md with description, API docs, and benchmarks
• example.py with working demonstration
• LICENSE file included
• Unit tests added
• Cross-validated on independent test functions (5D and 10D)

[y0z]

@not522, could you review this PR?

[not522]

Thank you for your PR!
Could you fix the CI? https://github.com/optuna/optunahub-registry/actions/runs/23073620859/job/67029413493

[EliMunkey]

Fixed the ruff lint/format failures in sampler.py (commit f804518):

• I001: Added missing blank line before if TYPE_CHECKING block (import sorting)
• Format: Collapsed multi-line function calls that fit within the line length limit

All ruff check and ruff format --check now pass cleanly.