Optimization

Mathematical Principles & Theorems

Solves multi-dimensional optimization \(\min_{\mathbf{x}} f(\mathbf{x})\). (1) Gradient Descent: updates along negative gradient \(\mathbf{x}_{k+1} = \mathbf{x}_k - \alpha \nabla f(\mathbf{x}_k)\); (2) Momentum Gradient Descent: \(\mathbf{v}_{k+1} = \beta \mathbf{v}_k + \alpha \nabla f(\mathbf{x}_k), \mathbf{x}_{k+1} = \mathbf{x}_k - \mathbf{v}_{k+1}\); (3) Simulated Annealing: probabilistic escape from local minima via Metropolis acceptance \(P(\text{accept}) = e^{-\Delta E / T}\); (4) Benchmark loss functions: Rosenbrock banana function, Rastrigin multimodal function, Ackley landscape.

Operating Instructions

• Select benchmark 2D objective landscape (Rosenbrock, Rastrigin, Ackley, Sphere).
• Choose optimization algorithm: Gradient Descent, Momentum, or Simulated Annealing.
• Set learning rate \(\alpha\), momentum \(\beta\), and initial coordinates using numeric inputs.
• Click Run to watch the optimizer navigate contour lines and track loss reduction over iteration steps.