Introduction to JSOSolvers #44
Conversation
tmigot
requested review from
abelsiqueira,
amontoison,
dpo,
geoffroyleconte and
paraynaud
|
That's a detail but maybe you could provide links to the algorithms in the table? Otherwise LGTM. |
|
That's a good suggestion. Link to the code https://github.com/JuliaSmoothOptimizers/JSOSolvers.jl/blob/main/src/lbfgs.jl or the docstring https://juliasmoothoptimizers.github.io/JSOSolvers.jl/latest/reference/#JSOSolvers.lbfgs-Union{Tuple{NLPModels.AbstractNLPModel},%20Tuple{V}}%20where%20V ? |
|
I'd say to the docstring, but as you prefer. |
|
@paraynaud, they should be executed. You can review the preview here: https://juliasmoothoptimizers.github.io/JSOTutorials.jl/introduction-to-jsosolvers/ |
|
|
||
| The solvers `tron` and `trunk` both have a specialized implementation for input models of type `AbstractNLSModel`. | ||
|
|
||
| The following example illustrate this specialization. |
There was a problem hiding this comment.
I think a better example would be a data-fitting NLS, so you can plot the solution.
There was a problem hiding this comment.
Thanks @abelsiqueira for the review! I added a data-fitting example on top of the existing examples and using keywords.
| - `verbose::Int = 0`: if > 0, display iteration details every `verbose` iteration. | ||
|
|
||
| Refer to the documentation of each solver for further details on the available keyword arguments. | ||
|
|
There was a problem hiding this comment.
There should be an example for basic usage as well.
|
Once the build has completed, you can preview your PR at this URL: https://juliasmoothoptimizers.github.io/JSOTutorials.jl/ |
|
Once the build has completed, you can preview your PR at this URL: https://juliasmoothoptimizers.github.io/JSOTutorials.jl/ |
There was a problem hiding this comment.
I made more comments about the NLS problem. Instead of using some random data, I choose a more traditional NLS problem from https://www.itl.nist.gov/div898/strd/nls/nls_main.shtml
The text will need an update to match the new problem.
|
|
||
| We build a nonlinear model `m` with unknown parameters α and β, and generate randomly some observations of this model. | ||
| ```julia | ||
| m(α, β, x) = α * cos(x) + β * sin(x) # nonlinear models with unknown α and β |
There was a problem hiding this comment.
Since this is linear in alpha and beta, this is actually LLS.
Instead, we can use a classic NLS, like Thurber: https://www.itl.nist.gov/div898/strd/nls/data/LINKS/DATA/Thurber.dat
| m(α, β, x) = α * cos(x) + β * sin(x) # nonlinear models with unknown α and β | |
| m(β, x) = (β[1] + β[2] * x + β[3] * x^2 + β[4] * x^3) / (1 + β[5] * x + β[6] * x^2 + β[7] * x^3) # nonlinear models with unknown β vector |
| ndata = 100 # size of data sample | ||
| data = [rand() * 2 - 1 for i=1:ndata] | ||
| obs = [m(0.5, 0.5, xi) for xi in data] + rand(ndata) / 10; # observations |
There was a problem hiding this comment.
Here is the data: https://gist.github.com/abelsiqueira/8ca109888b22b6ab1e76825f0567c668
| ndata = 100 # size of data sample | |
| data = [rand() * 2 - 1 for i=1:ndata] | |
| obs = [m(0.5, 0.5, xi) for xi in data] + rand(ndata) / 10; # observations | |
| url_prefix = "https://gist.githubusercontent.com/abelsiqueira/8ca109888b22b6ab1e76825f0567c668/raw/f3f38d61f750b443fb4307efbf853447275441a5/" | |
| data = CSV.read(download(joinpath(url_prefix, "thurber.csv")), DataFrame) | |
| x, y = data.x, data.y |
| ```julia | ||
| F(w) = [m(w[1], w[2], xi) - yi for (xi, yi) in zip(data, obs)] | ||
| w0 = [0., 0.] | ||
| nls = ADNLSModel(F, w0, ndata) | ||
| ``` |
There was a problem hiding this comment.
| ```julia | |
| F(w) = [m(w[1], w[2], xi) - yi for (xi, yi) in zip(data, obs)] | |
| w0 = [0., 0.] | |
| nls = ADNLSModel(F, w0, ndata) | |
| ``` | |
| ```julia | |
| F(β) = [m(β, xi) - yi for (xi, yi) in zip(x, y)] | |
| β0 = CSV.read(download(joinpath(url_prefix, "thurber-x0.csv")), DataFrame).beta | |
| ndata = length(x) | |
| nls = ADNLSModel(F, β, ndata) |
| using Plots | ||
| scatter(data, obs, c=:blue, m=:square, title="Basic nonlinear regression", ylab="m(α, β, x)", xlab="x") | ||
| mtest = [m(stats.solution[1], stats.solution[2], xi) for xi=-1:0.01:1] | ||
| plot!(-1:0.01:1, mtest) |
There was a problem hiding this comment.
| using Plots | |
| scatter(data, obs, c=:blue, m=:square, title="Basic nonlinear regression", ylab="m(α, β, x)", xlab="x") | |
| mtest = [m(stats.solution[1], stats.solution[2], xi) for xi=-1:0.01:1] | |
| plot!(-1:0.01:1, mtest) | |
| using Plots | |
| scatter(x, y, c=:blue, m=:square, title="Nonlinear regression", lab="data") | |
| plot!(x, t -> m(stats.solution, t), c=:red, lw=2, lab="fit") |
| nls = ADNLSModel(F, w0, ndata) | ||
| ``` | ||
|
|
||
| As shown before, we can use any `JSOSolvers` solvers to solve this problem. For instance, we use `trunk` with a time limit of `60` seconds. |
There was a problem hiding this comment.
| As shown before, we can use any `JSOSolvers` solvers to solve this problem. For instance, we use `trunk` with a time limit of `60` seconds. | |
| As shown before, we can use any `JSOSolvers` solvers to solve this problem, but since `trunk` has a specialized version for unconstrained NLS, we will use it, with a time limit of `60` seconds. |
|
Thanks @abelsiqueira for the suggestion! I added it to the tutorial. |
|
Once the build has completed, you can preview your PR at this URL: https://juliasmoothoptimizers.github.io/JSOTutorials.jl/ |
#3