Introduction

NLS_Solver.jl is a pure Julia package that solves:

• classical nonlinear least squares:

\[\min\limits_{\theta} \frac{1}{2}\|r(\theta)\|^2\]

• bound constrained nonlinear least squares:

\[\begin{align*} \min\limits_{\theta} & \frac{1}{2}\|r(\theta)\|^2 \\ & \theta_l\le\theta\le\theta_u \end{align*}\]

Usage

This package is easy to use. Problems are solved by calling a generic solve() function of the form:

solve(problem, algorithm_conf) -> algorithm_result

• problem are problem dependant data,
• algorithm_conf defines the chosen algorithm,
• algorithm_result is the returned structure that stores the solver result.

Detailed examples are provided below:

• Unconstrained nonlinear least squares
• • Problem definition
  • Choose a solver
  • The solve() function
  • Using solver result
  • Annex: direct sub-typing of AbstractNLS
• Bound constrained nonlinear least squares
• • Problem definition
  • Choose a solver
  • The solve() function
  • Using solver result
• Nonlinear regressions
• • Synthetic data
  • Solve the problem
  • Fit result
  • More fit examples

Detailed documentation

• API
• • Index
  • Documentation
  • • Public
    • Private

Algorithms and references

For the moment two methods are implemented. The classical Levenberg-Marquardt method for unconstrained problems and a modified Levenberg-Marquardt method for bound constrained problems.

These implementations are mainly based on these references:

• Madsen, N. (2004). Methods For Non-Linear Least Squares Problems. imm3215.pdf

  This reference is a must read about nonlinear least squares algorithms.

• Nielsen, H. B., & others, (1999). Damping parameter in marquardt's method. tr05_99.pdf

  This second reference provides some details about damping parameter. It also gives an useful list of test functions (unconstrained case).

• Kunisch, K., & Rendl, F. (2003). An infeasible active set method for quadratic problems with simple bounds. SIAM Journal on Optimization, 14(1), 35–52. epubs.siam.org

  This reference presents an efficient method to solve bound constrained quadratic problems. This is the method we use in our modified Levenberg-Marquardt method.