Séminaire Images Optimisation et Probabilités

A définir

The behavior of the random feature model in a high-dimensional framework has recently become a popular topic of interest in the machine learning literature. This model is generally considered for feature vectors composed of independent and identically distributed (iid) entries. We move beyond this specific assumption, which may be restrictive in various applications. To this end, we propose studying the performance of the random feature model with non-iid data by introducing a variance profile to the feature matrix. The performance of this model is linked to the spectrum of the random feature matrix, which turns out to be a nonlinear mixture of random variance profiled matrices. We have computed the limiting traffic distribution of such matrices using an extension of the method of moments. Knowledge of this distribution allowed us to introduce a new random matrix, which we call the « linear plus chaos » matrix, and which shares the same limiting spectrum as the random feature matrix. This linear plus chaos model proves to be simpler to study and has enabled us to derive deterministic equivalents that describe the asymptotic behavior of the performance of the random feature model.

À préciser

A définir

 A définir