Vidya Muthukumar - Surprises in overparameterized linear classification

From Katie Gentilello November 1st, 2021

11 plays 0 comments

Seemingly counter-intuitive phenomena in deep neural networks and kernel methods have prompted a recent re-investigation of classical machine learning methods, like linear models. Of particular focus is sufficiently high-dimensional setups in which interpolation of training data is possible. In this talk, we will first briefly review recent works showing that zero regularization, or fitting of noise, need not be harmful in regression tasks. Then, we will use this insight to uncover two new surprises for high-dimensional linear classification:

least-2-norm interpolation can classify consistently even when the corresponding regression task fails, and
the support-vector-machine and least-2-norm interpolation solutions exactly coincide in sufficiently high-dimensional linear model.

These findings taken together imply that the linear SVM can generalize well in settings beyond those predicted by training-data-dependent complexity measures.

This is joint work with Misha Belkin, Daniel Hsu, Adhyyan Narang, Anant Sahai, Vignesh Subramanian, Christos Thrampoulidis, Ke Wang and Ji Xu.

Tags: algorithms and randomness centerarc colloquiumgeorgia techrecorded by gtlibraryfall 2021

name: Vidya Muthukumar
Date: October 25th, 2021
Appears In: Open Scholarship

Comments
Related Media

Loading…

Vidya Muthukumar - Surprises in overparameterized linear classification

Related Media