AdaptiveRegularization.jl Questions: Stopping Tolerances and Saving Sequences

[RS-Coop]

Hello,

I have a two questions about using ARCqKOp from AdaptiveRegularization.jl

First, how do I go about implementing a stopping criteria based on the gradient norm?

Second, how can I save the sequences of function values and gradient norms? For example, I would like to be able to plot the function value vs. iteration counter.

Thanks for the help!

[tmigot]

Hi @RS-Coop !
Thank you for the question and very sorry for the delay.

You can modify the stopping criterion used by using Stopping. Indeed, the basic way is to call the solver is to create a NLPModel as follow:

using AdaptiveRegularization, ADNLPModels
# Rosenbrock
nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])
stats = ARCqKOp(nlp, verbose = true)

but you can also do

using AdaptiveRegularization, ADNLPModels, Stopping
nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])
nlp_stp = NLPStopping(nlp)
ARCqKOp(nlp_stp, verbose = true);

Stopping allows a couple of useful things like modifying the stopping criterion:

# Check optimality: https://solverstoppingjulia.github.io/Stopping.jl/dev/howstopcheckoptimality/
nlp = ADNLPModel(x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2, [-1.2; 1.0])
function new_optim_check(nlp, current_state; kwargs...)
  gx = current_state.gx
  @info gx[1] + sqrt(abs(gx[2]))
  return gx[1] + sqrt(abs(gx[2]))
end
nlp_stp = NLPStopping(nlp, optimality_check = new_optim_check, atol = 1e-3, rtol = 1e-5)
ARCqKOp(nlp_stp, verbose = true);
# We can check optimality afterwards
nlp_stp.current_state.current_score <= nlp_stp.meta.optimality0 * nlp_stp.meta.rtol + nlp_stp.meta.atol

and you can save intermediary values into a file following this tutorial https://solverstoppingjulia.github.io/Stopping.jl/dev/checkpointing/ .

JSO-solvers usually have a callback system but this is still work in progress for this package, JuliaSmoothOptimizers/AdaptiveRegularization.jl#88 .
This package is relatively recent and the documentation should expand slowly but surely.