Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ makedocs(;
doctest = false,
linkcheck = true,
checkdocs = :exports,
warnonly = [:missing_docs],
plugins = [bib],
format = Documenter.HTML(
assets = ["assets/favicon.ico", "assets/citations.css"],
Expand Down
8 changes: 8 additions & 0 deletions docs/src/api/native.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Native Line Search Algorithms

## Result Type

```@docs
LineSearchSolution
```

## No Line Search

```@docs
Expand All @@ -9,6 +15,7 @@ NoLineSearch
## Derivative-Free Line Searches

```@docs
GoldenSection
LiFukushimaLineSearch
RobustNonMonotoneLineSearch
```
Expand All @@ -17,4 +24,5 @@ RobustNonMonotoneLineSearch

```@docs
BackTracking
StrongWolfeLineSearch
```
22 changes: 22 additions & 0 deletions src/LineSearch.jl
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,28 @@ include("strong_wolfe.jl")

include("line_searches_ext.jl")

"""
LineSearchSolution(step_size, retcode)

The result returned by a line-search solve.

# Fields

- `step_size`: accepted step length for the current search direction.
- `retcode`: a `SciMLBase.ReturnCode` describing whether the line search found
an acceptable step.

# Examples

```julia
using LineSearch
using SciMLBase

sol = LineSearchSolution(0.5, SciMLBase.ReturnCode.Success)
sol.step_size
sol.retcode
```
"""
@concrete struct LineSearchSolution
step_size
retcode::ReturnCode.T
Expand Down
9 changes: 9 additions & 0 deletions src/backtracking.jl
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@ This is a modification of the algorithm described in Nocedal Wright (2nd ed), Se
`autodiff` is the automatic differentiation backend to use for the line search. This is only
used for the derivative of the objective function at the current step size. `autodiff` must
be specified if analytic jacobian/jvp/vjp is not available.

# Examples

```julia
using ADTypes
using LineSearch

alg = BackTracking(autodiff = AutoForwardDiff(), order = 3, maxiters = 100)
```
"""
@concrete struct BackTracking <: AbstractLineSearchAlgorithm
autodiff
Expand Down
18 changes: 16 additions & 2 deletions src/golden_section.jl
Original file line number Diff line number Diff line change
@@ -1,8 +1,22 @@
"""
GoldenSection(; tol = 1e-7, maxiters = 100)

A derivative-free line search that minimizes a unimodal function by successively
narrowing the interval containing the minimum using the golden ratio.
A derivative-free line search that minimizes a unimodal merit function by
successively narrowing the interval containing the minimum using the golden
ratio.

# Keyword Arguments

- `tol`: interval-width tolerance used to stop the golden-section search.
- `maxiters`: maximum number of interval-refinement iterations.

# Examples

```julia
using LineSearch

alg = GoldenSection(tol = 1e-8, maxiters = 200)
```
"""
@kwdef @concrete struct GoldenSection <: AbstractLineSearchAlgorithm
tol = 1.0e-7
Expand Down
8 changes: 8 additions & 0 deletions src/li_fukushima.jl
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ equations [li2000derivative](@cite).
inside GPU kernels. However, this particular version doesn't support `stats` and
`reinit!` and those will be ignored. Additionally, we fix the initial alpha for the
search to be `1`.

# Examples

```julia
using LineSearch

alg = LiFukushimaLineSearch(lambda_0 = 1.0, beta = 0.5, maxiters = 100)
```
"""
@kwdef @concrete struct LiFukushimaLineSearch <: AbstractLineSearchAlgorithm
lambda_0 = 1
Expand Down
9 changes: 9 additions & 0 deletions src/line_searches_ext.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@ differentiation for fast VJPs or JVPs.
specified if analytic jacobian/jvp/vjp is not available.
- `initial_alpha`: the initial step size to use. Defaults to `true` (which is equivalent
to `1`).

# Examples

```julia
using LineSearch
using LineSearches

alg = LineSearchesJL(method = LineSearches.Static(), initial_alpha = 1.0)
```
"""
struct LineSearchesJL{M, A, AD <: Union{Nothing, ADTypes.AbstractADType}} <:
AbstractLineSearchAlgorithm
Expand Down
10 changes: 9 additions & 1 deletion src/no_search.jl
Original file line number Diff line number Diff line change
@@ -1,7 +1,15 @@
"""
NoLineSearch(alpha)
NoLineSearch(; alpha = true)

Don't perform a line search. Just return the initial step length of `alpha`.

# Examples

```julia
using LineSearch

alg = NoLineSearch(alpha = 1.0)
```
"""
@kwdef @concrete struct NoLineSearch <: AbstractLineSearchAlgorithm
alpha = true
Expand Down
8 changes: 8 additions & 0 deletions src/robust_non_monotone.jl
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ Robust NonMonotone Line Search is a derivative free line search method from DF S
Defaults to ``fn_1 / n^2``.
- `maxiters`: the maximum number of iterations allowed for the inner loop of the
algorithm. Defaults to `100`.

# Examples

```julia
using LineSearch

alg = RobustNonMonotoneLineSearch(M = 10, gamma = 1e-4, maxiters = 100)
```
"""
@kwdef @concrete struct RobustNonMonotoneLineSearch <: AbstractLineSearchAlgorithm
gamma = 1 // 10000
Expand Down
22 changes: 19 additions & 3 deletions src/strong_wolfe.jl
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,28 @@ Strong Wolfe line search satisfying both Armijo (sufficient decrease) and
curvature conditions. Based on Nocedal & Wright, "Numerical Optimization" (2006),
Algorithms 3.5 and 3.6.

`autodiff` is the automatic differentiation backend to use for computing the
directional derivative. Must be specified if analytic jacobian/jvp/vjp is not
available.
# Keyword Arguments

- `autodiff`: automatic differentiation backend used to compute directional
derivatives when analytic jacobian/JVP/VJP information is unavailable.
- `c1`: Armijo sufficient-decrease coefficient.
- `c2`: curvature-condition coefficient.
- `α_init`: initial trial step length.
- `α_max`: maximum trial step length.
- `maxiters`: maximum iterations for the outer bracketing loop.
- `zoom_maxiters`: maximum iterations for the inner zoom loop.

`maxiters` bounds the outer bracketing loop (Alg. 3.5). `zoom_maxiters` bounds
the inner zoom loop (Alg. 3.6) independently.

# Examples

```julia
using ADTypes
using LineSearch

alg = StrongWolfeLineSearch(autodiff = AutoForwardDiff(), c1 = 1e-4, c2 = 0.9)
```
"""
@kwdef @concrete struct StrongWolfeLineSearch <: AbstractLineSearchAlgorithm
autodiff = nothing
Expand Down
30 changes: 30 additions & 0 deletions test/qa/public_api_docs.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
using LineSearch
using Test

@testset "public API documentation" begin
public_names = filter(!=(:LineSearch), names(LineSearch; all = false, imported = false))
expected_names = Set(
[
:BackTracking,
:GoldenSection,
:LiFukushimaLineSearch,
:LineSearchSolution,
:LineSearchesJL,
:NoLineSearch,
:RobustNonMonotoneLineSearch,
:StrongWolfeLineSearch,
]
)
@test Set(public_names) == expected_names

for name in public_names
binding = Docs.Binding(LineSearch, name)
@test Docs.hasdoc(binding)
end

docs_text = read(joinpath(pkgdir(LineSearch), "docs", "src", "api", "native.md"), String) *
read(joinpath(pkgdir(LineSearch), "docs", "src", "api", "line_searches.md"), String)
for name in public_names
@test occursin(string(name), docs_text)
end
end
2 changes: 2 additions & 0 deletions test/qa/qa.jl
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
using SciMLTesting, LineSearch, Test

include("public_api_docs.jl")

run_qa(LineSearch; explicit_imports = true)
Loading