Search

In singular models, the optimal set of parameters forms an analytic set with singularities, and a classical statistical inference cannot be applied to such models. This is significant for deep learning as neural networks are singular, and thus, “dividing” by the determinant of the Hessian or employing the Laplace approximation is not appropriate. Despite its potential for addressing fundamental issues in deep learning, a singular learning theory appears to have made little inroads into the developing canon of a deep learning theory. Via a mix of theory and experiment, we present an invitation to the singular learning theory as a vehicle for understanding deep learning and suggest an important future work to make the singular learning theory directly applicable to how deep learning is performed in practice.

AbstractWe study cocoverings of triangulated categories, in the sense of Rouquier, and prove that for any regular cardinal α the condition of α-compactness, in the sense of Neeman, is local with respect to such cocoverings. This was established for ordinary compactness by Rouquier. Our result yields a new technique for proving that a given triangulated category is well-generated. As an application we describe the α-compact objects in the unbounded derived category of a quasi-compact and semi-separated scheme.