.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples_built/plot_migration_from_sparse_ho.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_built_plot_migration_from_sparse_ho.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_built_plot_migration_from_sparse_ho.py:


Migrating from sparse-ho to sparho
==================================

Runnable companion to ``docs/migration_from_sparse_ho.md``. We don't
import ``sparse_ho`` (it's optional, not on PyPI) — instead each section
shows the sparse-ho idiom in a comment and the sparho equivalent in
runnable code. The prose guide has the full translation table.

The problem: tune the Lasso α on a 200-sample / 50-feature regression
with a held-out validation split, compute a hypergradient by implicit
differentiation, drive the outer search with HOAG, and refit on the full
problem at the chosen α*.

.. GENERATED FROM PYTHON SOURCE LINES 15-28

.. code-block:: Python


    import numpy as np
    from sklearn.datasets import make_regression
    from sklearn.metrics import mean_squared_error
    from sparho import (
        L1,
        HeldOutMSE,
        Problem,
        SquaredLoss,
        hoag_search,
    )
    from sparho.adapters import SklearnLasso








.. GENERATED FROM PYTHON SOURCE LINES 29-32

Data + split: 200 samples × 50 features, 150 train / 50 val. The split
is what `HeldOutMSE` reads — the inner solver sees only the train slice;
the criterion evaluates on the val slice.

.. GENERATED FROM PYTHON SOURCE LINES 32-42

.. code-block:: Python

    X, y = make_regression(
        n_samples=200,
        n_features=50,
        n_informative=8,
        noise=1.0,
        random_state=0,
    )
    idx_train = np.arange(150, dtype=np.int32)
    idx_val = np.arange(150, 200, dtype=np.int32)








.. GENERATED FROM PYTHON SOURCE LINES 43-54

Step 1 — pick the **model**.

sparse-ho:

.. code-block:: python

    model = sparse_ho.Lasso(X, y, estimator=None)

sparho splits "what's being optimized" from "how it's solved": a
``Problem`` (the bilevel inner problem in math) plus a ``Solver`` (an
adapter wrapping the actual numerical fitter).

.. GENERATED FROM PYTHON SOURCE LINES 54-57

.. code-block:: Python

    problem = Problem(SquaredLoss(), L1(), X, y)
    solver = SklearnLasso(tol=1e-8)








.. GENERATED FROM PYTHON SOURCE LINES 58-69

Step 2 — pick the **criterion**.

sparse-ho:

.. code-block:: python

    criterion = sparse_ho.HeldOutMSE(idx_train, idx_val)

Same name, same idea, same int32-index convention. ``CrossVal`` and
``HeldOutLogistic`` carry over too — see the translation table in
``migration_from_sparse_ho.md``.

.. GENERATED FROM PYTHON SOURCE LINES 69-71

.. code-block:: Python

    criterion = HeldOutMSE(idx_train=idx_train, idx_val=idx_val)








.. GENERATED FROM PYTHON SOURCE LINES 72-87

Step 3 — pick the **hypergradient algorithm**.

sparse-ho:

.. code-block:: python

    algo = sparse_ho.ImplicitForward(tol_jac=1e-8, n_iter_jac=200)

sparho ships ``implicit_forward`` only at v0.x and uses it by default —
nothing to pass unless you want to override the CG tolerance:

.. code-block:: python

    from sparho import implicit_forward
    hoag_search(..., hypergrad=implicit_forward)

.. GENERATED FROM PYTHON SOURCE LINES 89-103

Step 4 — pick the **outer optimizer**, and run.

sparse-ho:

.. code-block:: python

    optimizer = sparse_ho.LineSearch(n_outer=20)
    monitor = sparse_ho.Monitor()
    sparse_ho.grad_search(algo, criterion, model, optimizer, X, y, alpha0,
                          monitor=monitor)

sparho rolls ``algo`` + ``optimizer`` + the outer loop into a single
call. The ``LineSearch`` outer becomes ``hoag_search``; the ``Monitor``
becomes ``SearchResult.history`` (an immutable tuple).

.. GENERATED FROM PYTHON SOURCE LINES 103-114

.. code-block:: Python

    result = hoag_search(
        problem,
        hp0=1e-2,
        solver=solver,
        criterion=criterion,
        n_iter=20,
        inner_tol=1e-7,
    )
    print(f"α* = {result.best_hyperparam:.4g}   converged: {result.converged}")
    print(f"outer iterations: {result.n_iter}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    α* = 0.09959   converged: False
    outer iterations: 20




.. GENERATED FROM PYTHON SOURCE LINES 115-123

Step 5 — inspect the trajectory.

sparse-ho stored the per-iter α / value / time in ``monitor.alphas``,
``monitor.objs``, ``monitor.times``. sparho returns the immutable
``history`` tuple of :class:`IterationRecord`. Each record carries
``iteration``, ``hyperparam``, ``value``, ``grad_norm``,
``n_inner_iter``, and an ``extras`` mapping (HOAG records also include
``step_size`` / ``L_estimate``; see ``docs/stability.md``).

.. GENERATED FROM PYTHON SOURCE LINES 123-131

.. code-block:: Python

    for rec in result.history[:5]:
        print(
            f"iter {rec.iteration:2d}: "
            f"α={float(rec.hyperparam):.4g}  "
            f"value={rec.value:.4g}  "
            f"|∇θ|={rec.grad_norm:.3g}"
        )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    iter  0: α=0.01  value=1.456  |∇θ|=0.111
    iter  1: α=0.02718  value=1.305  |∇θ|=0.142
    iter  2: α=0.07389  value=1.162  |∇θ|=0.139
    iter  3: α=0.2009  value=1.437  |∇θ|=1
    iter  4: α=0.2009  value=1.437  |∇θ|=1




.. GENERATED FROM PYTHON SOURCE LINES 132-136

Step 6 — use the result.

Refit on the **full** problem at α* — by default the search returns
``best_coef`` already refitted, so no extra step is needed.

.. GENERATED FROM PYTHON SOURCE LINES 136-140

.. code-block:: Python

    yhat_val = X[idx_val] @ result.best_coef
    mse_val = mean_squared_error(y[idx_val], yhat_val)
    print(f"held-out MSE at α*: {mse_val:.4g}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    held-out MSE at α*: 0.9847




.. GENERATED FROM PYTHON SOURCE LINES 141-145

That's the whole story. The detailed translation table for every
sparse-ho symbol (Models, Criteria, Algorithms, Optimizers, Monitor)
lives at ``docs/migration_from_sparse_ho.md``. The Sphinx-rendered
version is on Read the Docs.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.184 seconds)


.. _sphx_glr_download_examples_built_plot_migration_from_sparse_ho.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_migration_from_sparse_ho.ipynb <plot_migration_from_sparse_ho.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_migration_from_sparse_ho.py <plot_migration_from_sparse_ho.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_migration_from_sparse_ho.zip <plot_migration_from_sparse_ho.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_