Bound constrained nonlinear least squares

In this part we see how to solve problems of the form:

\[\begin{align*} \min\limits_{\theta} & \frac{1}{2}\|r(\theta)\|^2 \\ & \theta_l\le\theta\le\theta_u \end{align*}\]

Compared to the unconstrained case, there are essentially two differences:

• the solve() function has an extra parameter, bc that store bound constraints.
• The solver category is different and Abstract_Solver_Conf is replaced by Abstract_BC_Solver_Conf, (where BC stands for bound constrained).

Further details can be found there:

solve(nls::AbstractNLS, θ_init::AbstractVector, bc::BoundConstraints, conf::Abstract_BC_Solver_Conf)

As before, you can reproduce the results using the sandbox/example_Rosenbrock.jl file.

Problem definition

Identical to the unconstrained case:

nls = create_NLS_problem_using_ForwardDiff(2 => 2) do θ
  sqrt(2)* [ 1-θ[1], 10*(θ[2]-θ[1]^2) ]
end

Choose a solver

Algorithm and its parameters are defined by sub-typing Abstract_BC_Solver_Conf. For the moment there is only one implementation, a bound constrained version of the classical Levenberg-Marquardt method:

conf = LevenbergMarquardt_BC_Conf()

We also need a starting point for the unknown parameter vector $\theta$. Here we start at point $(3,3)$:

θ_init = Float64[3,3]

The solve() function

To solve the problem one must call the solve() function. For bound constrained problems, this function has the following prototype:

function solve(nls::AbstractNLS,
               θ_init::AbstractVector,
               bc::BoundConstraints,
               conf::Abstract_Solver_Conf)::Abstract_Solver_Result

There is an extra bc argument which is used to define bound constraints. By example if constraints are $2 \le \theta_i \le 4$ we can proceed as follows:

θl = Float64[2,2]
θu = Float64[4,4]

bc = BoundConstraints(θl,θu)

result = solve(nls, θ_init, bc, conf)

Using solver result

The solve() function returns a Abstract_BC_Solver_Result sub-typed structure that contains algorithm result.

You can check if the method has converged

@assert converged(result)

get the optimal $\theta$

θ_solution = solution(result)

2-element ReadOnlyArrays.ReadOnlyArray{Float64, 1, Vector{Float64}}:
 2.0
 3.999999999974335

and its associated objective function value

objective_value(result)

1.0000000000000002