Solvers¶
A Solver is anything callable with the signature
(Problem, Hyperparam, *, x0=None, tol=None) -> SolverResult. It is the
boundary between sparho’s bilevel machinery and a concrete inner-problem
fitter (sklearn, celer, or a user callable).
- class sparho.Solver[source]¶
Bases:
ProtocolAnything callable with the signature
(Problem, Hyperparam, *, x0=None) -> SolverResult.The
Solveris the boundary between sparho’s bilevel machinery and the underlying inner-problem solver (sklearn, celer, or a user callable). It has no awareness of the outer loop — it just returns a converged inner solution at the given hyperparameter, alongside the active set and dual gap needed by the hypergradient.Warm-start:
x0is an optional initial guess for the coefficients (e.g.β*from the previous outer iteration at a nearby hyperparameter). Solvers may use it to seed the inner iteration and converge faster; those that can’t are free to ignore it. The criterion / outer loop owns the decision of when to warm-start; the solver just honorsx0if given.Tolerance override:
tolis an optional inner-solver tolerance that supersedes whatever default the adapter holds (e.g.SklearnLasso.tol). HOAG-style outer loops (sparho.search.hoag_search()) use this to shrink the inner tolerance across iterations — loose early when only the gradient direction matters, tight late when the criterion value resolution drives convergence. Adapters that have no meaningful tolerance setting may ignore it.- __init__(*args, **kwargs)¶
sklearn adapters¶
- class sparho.adapters.SklearnLasso[source]¶
Bases:
objectAdapter for
Problem(SquaredLoss, L1, X, y)viasklearn.linear_model.Lasso.When the criterion uses warm-start, prefer
tol ≤ 1e-8: sklearn’s convergence check isdual_gap < tol · ||y||², so on small-||y||problems a loosetollets the warm coef pass the check immediately and the inner solver returns it unchanged — which makes nearby CV evaluations indistinguishable and stalls outer line searches.
- class sparho.adapters.SklearnElasticNet[source]¶
Bases:
objectAdapter for
Problem(SquaredLoss, ElasticNet(rho), X, y)via sklearn.
- class sparho.adapters.SklearnWeightedLasso[source]¶
Bases:
objectAdapter for
Problem(SquaredLoss, WeightedL1, X, y)via column-rescaling.sklearn’s
Lassoonly supports a scalarα. We solve the equivalent problem with rescaled designX' = X · diag(1/α_vec)and unitα=1; the recoveredβ'is rescaled back asβ = β' / α_vec. Coefficients whereα_j = 0(no regularization) are not supported in this adapter — sparse-ho’s behavior was the same.
- class sparho.adapters.SklearnLogisticRegression[source]¶
Bases:
objectAdapter for
Problem(LogisticLoss, L1, X, y)viaLogisticRegression(penalty='l1').Assumes binary
y ∈ {−1, +1}. sklearn relabels internally. The dual gap is not exposed by sklearn for logistic regression; we report a stationarity proxy||X_A^T (σ(Xβ) − y₀₁) + α sign(β_A)||_∞instead, which is zero at a KKT-optimal point.x0is accepted for protocol conformance but ignored — sklearn’sLogisticRegression(solver='liblinear')does not support warm-start.
celer adapters¶
celer adapters are available behind the [celer] extra. The
celer package is imported lazily so the module loads even when the
extra is not installed.
Wrapping an arbitrary callable¶
- sparho.adapters.as_solver(fn, *, name='<callable>')[source]¶
Wrap
fnas a Solver. The wrapper is frozen so it can be hashed and reprd.Optional Solver-protocol kwargs (
x0for warm-start,tolfor an inner-tolerance override) are forwarded only whenfndeclares them in its signature. A plain(problem, hp) -> SolverResultcallable keeps working — the wrapper silently drops kwargs the function doesn’t accept.- Parameters:
fn (Callable[[...], SolverResult])
name (str)
- Return type:
_CallableSolver