Criteria¶
A Criterion is the outer-loop validation oracle: it slices the full
Problem into a training subproblem, drives the inner solver, and
returns the criterion value (and, when asked, the hypergradient chained
through implicit_forward).
- class sparho.Criterion[source]¶
Bases:
ProtocolOuter-loop validation oracle.
Implementations:
HeldOutMSE,HeldOutLogistic,CrossVal.x0is an optional warm-start coefficient guess threaded through to the inner solver. Single-split criteria forward it directly;CrossValignores caller-suppliedx0because it manages its own per-fold cache.tolis an optional inner-solver tolerance that overrides the adapter’s default. Threaded through toSolver.__call__(tol=...). Used by HOAG-style outer loops to schedule inner accuracy across iterations.- __init__(*args, **kwargs)¶
- class sparho.CriterionResult[source]¶
Bases:
objectOutcome of
Criterion.value_and_hypergrad().coefandactive_setare reported from the last (or only) inner solve; forCrossValthey come from the final fold and are diagnostic only — the user is expected to refit on the full data atbest_hyperparamif a single final β is needed.inner_dual_gapis the maximum over contributing inner solves (worst-fold convergence forCrossVal, max of the two solves forSure) — the outer loop reads it viaIterationRecord.extrasfor live diagnostics.Noneif the criterion didn’t surface a gap.- __init__(value, hypergrad, coef, active_set, inner_dual_gap=None)¶
- class sparho.HeldOutMSE[source]¶
Bases:
objectHeld-out mean-squared-error.
C(β) = (1/|val|) Σ_{i ∈ val} (yᵢ − Xᵢ β)²— matches sklearn’smean_squared_error(no1/2). The gradient∂C/∂βcarries the factor of2.
- class sparho.HeldOutLogistic[source]¶
Bases:
objectHeld-out logistic loss:
C(β) = (1/|val|) Σᵢ log(1 + exp(−yᵢ Xᵢβ)).Labels assumed in
{−1, +1}(sparho’sLogisticLossconvention).
- class sparho.CrossVal[source]¶
Bases:
objectK-fold cross-validation aggregator.
Wraps a single-split base criterion class (typically
HeldOutMSE) over a tuple of(train_idx, val_idx)pairs. Both value and hypergradient are means across folds.Build via
kfold():cv = CrossVal.kfold(problem.n_samples, k=5)
For classification, pass
base=HeldOutLogistictokfold.Warm-start: with
warm_start=True, each fold’s previous-iterationβ*seeds the next inner solve at the same fold. Big wins when the inner solver dominates (sparse-X, small α, large active set); converges to the same answer aswarm_start=Falsebecause Lasso is convex. The cache is mutable but excluded from equality / hash so the dataclass remains a well-behaved value object.See also
sparho.implicit_forwardHypergradient called once per fold via
hypergrad_fn.
Notes
The per-fold chain rule
dC_k/dα = (∂C_k/∂β)ᵀ dβ*/dαand the linearity-of-expectation argument behindCrossVal’s aggregation are covered in Criteria and the outer chain rule.- folds: tuple[tuple[ndarray[tuple[Any, ...], dtype[int32]], ndarray[tuple[Any, ...], dtype[int32]]], ...]¶
- base: Callable[[ndarray[tuple[Any, ...], dtype[int32]], ndarray[tuple[Any, ...], dtype[int32]]], Criterion]¶
- classmethod kfold(n_samples, k=5, *, shuffle=True, random_state=0, base=<class 'sparho.criteria.HeldOutMSE'>, warm_start=False)[source]¶
Build a
CrossValfromsklearn.model_selection.KFold.
- __init__(folds, base=<class 'sparho.criteria.HeldOutMSE'>, warm_start=False, _cache=<factory>)¶
- Parameters:
- Return type:
None
- class sparho.Sure[source]¶
Bases:
objectStein’s Unbiased Risk Estimator via Finite-Difference Monte Carlo (FDMC).
Estimates the expected prediction-error MSE without a held-out set, for
SquaredLossproblems with i.i.d. Gaussian observation noise of known standard deviationsigma:SURE(α) = (1/n) ‖y − Xβ̂(α; y)‖² − σ² + (2σ²/(n·ε)) · δᵀ X (β̂(α; y+εδ) − β̂(α; y))where δ ~ 𝒩(0, I_n) is a single random probe and ε is the finite-difference step. The probe and step are fixed for the lifetime of the instance so the criterion is a deterministic function of α (this is required for line-search monotonicity and FD gradient checks). Two inner solves per evaluation; value_and_hypergrad makes two hypergrad_fn calls and sums their results.
The default ε follows Deledalle et al. 2014 (SUGAR):
ε = 2σ / n^{0.3}, which trades MC variance against bias from the finite-difference truncation.SURE is the cleanest tuning signal when no held-out set exists (denoising, signal recovery, single-fold) — its minimizer is an unbiased estimate of the held-out-MSE minimizer in expectation.
- Parameters:
sigma (float) – Noise standard deviation. Must be supplied; SURE has no way to estimate
σfrom the data within its own pipeline.epsilon (float | None) – Finite-difference step.
None(default) uses the Deledalle heuristic.random_state (int | None) – Seed for the probe
δ. Fixed seed → fixed probe → reproducible SURE.warm_start (bool) – If
True, the two inner solves at the next outer iter are seeded from the previous iter’sβ̂₁andβ̂₂. MirrorsCrossVal.
References
Deledalle, Vaiter, Fadili & Peyré, Stein Unbiased GrAdient estimator of the Risk (SUGAR) for multiple parameter selection, SIAM J. Imaging Sci. 7(4), 2014.
See also
sparho.implicit_forwardHypergradient called twice per
value_and_hypergrad— once per inner solve.
Notes
Full SURE / SUGAR derivation, the Stein identity that justifies the formula above, and the FDMC trade-off between MC variance and bias of the finite-difference step are in Criteria and the outer chain rule.
- __init__(sigma, epsilon=None, random_state=42, warm_start=False, _probe=<factory>, _cache=<factory>)¶