Skip to content

Backtrack along the Cauchy step once the Newton step is cut back below it - #1635

Merged
NAThompson merged 1 commit into
developfrom
multivariate-roots-gradient
Nov 7, 2026
Merged

NAThompson merged 1 commit into
developfrom
multivariate-roots-gradient

Conversation

@NAThompson

@NAThompson NAThompson commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Near a singular J, Newton and Halley crept to max_iter; they now reach the local minimum of |F| and report it.
Closes #1628

Once the backtracked Newton step t·a is shorter than the Cauchy step −(‖g‖²/‖Jg‖²)g, g = JᵀF, the line search restarts along the Cauchy step (the dogleg rule, with t‖a‖ as the radius), and the Armijo test and quadratic interpolation use the actual slope gᵀp. Acceptance is now strict, so a step too short to change ‖F‖ is not accepted.

A test on the angle between the Newton step and −g, as suggested in the issue, breaks Powell's badly scaled problem, where that angle is ~1e-8 throughout but Newton is right.

Calls of f on Freudenstein–Roth. Bold: ran to max_iter = 1000. Every other run ends at the non-root minimum, ‖F‖ = 6.9989, with evaluation_error.

start Newton, develop Newton, PR Halley, develop Halley, PR
(15, −2) 96 84 6098 127
(8.424166, −0.933377) 6249 121 6249 121
(9.15, −0.8964) 7001 123 7001 123
(0.5, −2) 96 84 5700 208

On the existing test problems (circle_line, rosenbrock, broyden10, powell, squares) the iterations and calls are unchanged, except Newton on squares from (1/1024, 0.5): 11 → 17 calls.

Freudenstein–Roth from 400×300 starts, x1 ∈ [−5, 20], x2 ∈ [−4, 6], max_iter = 1000:

root evaluation_error max_iter total calls
develop Newton 45200 71278 3522 43.3M
develop Halley 45200 67544 7256 65.4M
PR Newton 45684 74316 0 8.32M
PR Halley 45200 74800 0 8.60M

Calls of f from each start, develop vs this PR

🤖 Generated with Claude Code

…w it

Near a singular J, Newton and Halley crept to max_iter; they now reach the local minimum of |F| and report it.
Closes #1628
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NAThompson
NAThompson merged commit f7b7193 into develop Nov 7, 2026
51 checks passed
@NAThompson
NAThompson deleted the multivariate-roots-gradient branch November 7, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

multivariate_newton and multivariate_halley creep to max_iter near a singular Jacobian

1 participant