.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples_built/plot_cv_lasso.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_built_plot_cv_lasso.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_built_plot_cv_lasso.py:


Cross-validated Lasso
=====================

K-fold cross-validation is wrapped by ``CrossVal``, which averages the
value and the hypergradient across folds. With ``warm_start=True`` each
fold reuses its previous-iteration ``β*`` as the next inner solve's
starting point — a substantial speed-up when the inner solver dominates.

We compare against ``sklearn.LassoCV`` on the same K-fold split.

.. GENERATED FROM PYTHON SOURCE LINES 12-23

.. code-block:: Python


    import matplotlib.pyplot as plt
    import numpy as np
    from sklearn.datasets import make_regression
    from sklearn.linear_model import LassoCV
    from sklearn.metrics import mean_squared_error
    from sklearn.model_selection import KFold

    from sparho import CrossVal, L1, Problem, SquaredLoss, hoag_search
    from sparho.adapters import SklearnLasso








.. GENERATED FROM PYTHON SOURCE LINES 24-25

Synthetic data — 300 samples × 80 features, 10 informative.

.. GENERATED FROM PYTHON SOURCE LINES 25-30

.. code-block:: Python

    X, y = make_regression(
        n_samples=300, n_features=80, n_informative=10,
        noise=1.0, random_state=0,
    )








.. GENERATED FROM PYTHON SOURCE LINES 31-33

Build the 5-fold splitter once, then feed the same folds to both pipelines
so the comparison is apples-to-apples.

.. GENERATED FROM PYTHON SOURCE LINES 33-51

.. code-block:: Python

    kf = KFold(n_splits=5, shuffle=True, random_state=0)
    folds = tuple(
        (np.asarray(tr, dtype=np.int32), np.asarray(val, dtype=np.int32))
        for tr, val in kf.split(np.arange(X.shape[0]))
    )

    problem = Problem(SquaredLoss(), L1(), X, y)
    result = hoag_search(
        problem,
        hp0=1e-2,
        solver=SklearnLasso(tol=1e-8),
        criterion=CrossVal(folds=folds, warm_start=True),
        n_iter=20,
    )
    best_alpha = float(result.best_hyperparam)
    print(f"sparho: α = {best_alpha:.4g}   mean CV MSE = "
          f"{min(r.value for r in result.history):.4f}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    sparho: α = 0.05805   mean CV MSE = 0.9962




.. GENERATED FROM PYTHON SOURCE LINES 52-53

LassoCV baseline on a 30-point grid using the same folds.

.. GENERATED FROM PYTHON SOURCE LINES 53-62

.. code-block:: Python

    alphas_grid = np.logspace(-4, 1, 30)
    cv = LassoCV(
        alphas=alphas_grid, cv=list(folds), fit_intercept=False,
        tol=1e-8, max_iter=10_000,
    )
    cv.fit(X, y)
    cv_mse = float(cv.mse_path_.mean(axis=1).min())
    print(f"LassoCV: α = {cv.alpha_:.4g}   mean CV MSE = {cv_mse:.4f}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    LassoCV: α = 0.05736   mean CV MSE = 0.9961




.. GENERATED FROM PYTHON SOURCE LINES 63-64

Refit metrics on the full data at each method's best α.

.. GENERATED FROM PYTHON SOURCE LINES 64-83

.. code-block:: Python

    fit_full = mean_squared_error(y, X @ result.best_coef)
    fit_cv = mean_squared_error(y, X @ cv.coef_)

    fig, ax = plt.subplots(figsize=(6, 4))
    xs = [float(r.hyperparam) for r in result.history]
    ys = [r.value for r in result.history]
    ax.plot(xs, ys, "o-", color="C0", label="sparho HOAG (5-fold CV)")
    ax.axvline(best_alpha, color="C0", linestyle="--", alpha=0.5)
    ax.axvline(cv.alpha_, color="C1", linestyle="--", alpha=0.5,
               label=f"LassoCV α = {cv.alpha_:.2g}")
    ax.set_xscale("log")
    ax.set_xlabel(r"$\alpha$")
    ax.set_ylabel("mean CV MSE")
    ax.set_title("Cross-validated Lasso: gradient-based vs grid")
    ax.legend()
    fig.tight_layout()
    plt.show()

    print(f"\nfull-data refit MSE — sparho: {fit_full:.4f}   LassoCV: {fit_cv:.4f}")



.. image-sg:: /examples_built/images/sphx_glr_plot_cv_lasso_001.png
   :alt: Cross-validated Lasso: gradient-based vs grid
   :srcset: /examples_built/images/sphx_glr_plot_cv_lasso_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    full-data refit MSE — sparho: 0.7984   LassoCV: 0.7956





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.217 seconds)


.. _sphx_glr_download_examples_built_plot_cv_lasso.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_cv_lasso.ipynb <plot_cv_lasso.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_cv_lasso.py <plot_cv_lasso.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_cv_lasso.zip <plot_cv_lasso.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_