Search

We provide high-probability bounds on the condition number of random feature matrices. In particular, we show that if the complexity ratio |$N/m$|⁠ , where |$N$| is the number of neurons and |$m$| is the number of data samples, scales like |$\log ^{-1}(N)$| or |$\log (m)$|⁠ , then the random feature matrix is well-conditioned. This result holds without the need of regularization and relies on establishing various concentration bounds between dependent components of the random feature matrix. Additionally, we derive bounds on the restricted isometry constant of the random feature matrix. We also derive an upper bound for the risk associated with regression problems using a random feature matrix. This upper bound exhibits the double descent phenomenon and indicates that this is an effect of the double descent behaviour of the condition number. The risk bounds include the underparameterized setting using the least squares problem and the overparameterized setting where using either the minimum norm interpolation problem or a sparse regression problem. For the noiseless least squares or sparse regression cases, we show that the risk decreases as |$m$| and |$N$| increase. The risk bound matches the optimal scaling in the literature and the constants in our results are explicit and independent of the dimension of the data. [ABSTRACT FROM AUTHOR]

This article proposes a continuous-time neurodynamic approach for solving the rank minimization under affine constraints. As opposed to the traditional neurodynamic approach, the proposed neurodynamic approach extends the form of the variables from the vector form to the matrix form. First, a continuous-time neurodynamic approach with variables in matrix form is developed by combining the optimal rank r projection and the gradient. Then, the optimality of the proposed neurodynamic approach is rigorously analyzed by demonstrating that the objective function satisfies the functional property which is called as (2 r , 4 r) -restricted strong convexity and smoothness ((2 r , 4 r) -RSCS). Furthermore, the convergence and stability analysis of the proposed neurodynamic approach is rigorously conducted by establishing appropriate Lyapunov functions and considering the relevant restricted isometry property (RIP) condition associated with the affine transformation. Finally, through experiments involving low-rank matrix recovery under affine transformations and the completion of low-rank real image, the effectiveness of this approach has been demonstrated, along with its superiority compared to the vector-based approach. [ABSTRACT FROM AUTHOR]