Search

Neural fields are an emerging paradigm that represent data as continuous functions parameterized by neural networks. Despite many advantages, neural fields often have a high training cost, which prevents a broader adoption. In this paper, we focus on a popular family of neural fields, called sinusoidal neural fields (SNFs), and study how it should be initialized to maximize the training speed. We find that the standard initialization scheme for SNFs -- designed based on the signal propagation principle -- is suboptimal. In particular, we show that by simply multiplying each weight (except for the last layer) by a constant, we can accelerate SNF training by 10$\times$. This method, coined $\textit{weight scaling}$, consistently provides a significant speedup over various data domains, allowing the SNFs to train faster than more recently proposed architectures. To understand why the weight scaling works well, we conduct extensive theoretical and empirical analyses which reveal that the weight scaling not only resolves the spectral bias quite effectively but also enjoys a well-conditioned optimization trajectory.

Neural field is an emerging paradigm in data representation that trains a neural network to approximate the given signal. A key obstacle that prevents its widespread adoption is the encoding speed-generating neural fields requires an overfitting of a neural network, which can take a significant number of SGD steps to reach the desired fidelity level. In this paper, we delve into the impacts of data transformations on the speed of neural field training, specifically focusing on how permuting pixel locations affect the convergence speed of SGD. Counterintuitively, we find that randomly permuting the pixel locations can considerably accelerate the training. To explain this phenomenon, we examine the neural field training through the lens of PSNR curves, loss landscapes, and error patterns. Our analyses suggest that the random pixel permutations remove the easy-to-fit patterns, which facilitate easy optimization in the early stage but hinder capturing fine details of the signal., Comment: CVPR 2024

Metamaterials are effectively homogeneous materials that display extraordinary dispersion. Negative index metamaterials, zero index metamaterials and extremely anisotropic metamaterials are just a few examples. Instead of using locally resonating elements that may cause undesirable absorption, there are huge efforts to seek alternative routes to obtain these unusual properties. Here, we experimentally demonstrate an alternative approach for constructing metamaterials with extreme dispersion by simply coiling up space with curled channels. Such a geometric approach has the advantage that the ratio between the wavelength and the lattice constant in achieving a negative or zero index is easily tunable. It allows us to construct for the first time an acoustic metamaterial with conical dispersion, leading to a clear demonstration of negative refraction from an acoustic metamaterial with airborne sound. We also design and realize a double-negative metamaterial for microwaves under the same principle., Comment: 10 pages, 4 figures