Search

Image-based reinforcement learning (RL) faces significant challenges in generalization when the visual environment undergoes substantial changes between training and deployment. Under such circumstances, learned policies may not perform well leading to degraded results. Previous approaches to this problem have largely focused on broadening the training observation distribution, employing techniques like data augmentation and domain randomization. However, given the sequential nature of the RL decision-making problem, it is often the case that residual errors are propagated by the learned policy model and accumulate throughout the trajectory, resulting in highly degraded performance. In this paper, we leverage the observation that predicted rewards under domain shift, even though imperfect, can still be a useful signal to guide fine-tuning. We exploit this property to fine-tune a policy using reward prediction in the target domain. We have found that, even under significant domain shift, the predicted reward can still provide meaningful signal and fine-tuning substantially improves the original policy. Our approach, termed Predicted Reward Fine-tuning (PRFT), improves performance across diverse tasks in both simulated benchmarks and real-world experiments. More information is available at project web page: https://sites.google.com/view/prft., Comment: L4DC 2024

Spiking Neural Networks (SNNs) are developed as a promising alternative to Artificial Neural networks (ANNs) due to their more realistic brain-inspired computing models. SNNs have sparse neuron firing over time, i.e., spatio-temporal sparsity; thus, they are useful to enable energy-efficient hardware inference. However, exploiting spatio-temporal sparsity of SNNs in hardware leads to unpredictable and unbalanced workloads, degrading the energy efficiency. In this work, we propose an FPGA-based convolutional SNN accelerator called Skydiver that exploits spatio-temporal workload balance. We propose the Approximate Proportional Relation Construction (APRC) method that can predict the relative workload channel-wisely and a Channel-Balanced Workload Schedule (CBWS) method to increase the hardware workload balance ratio to over 90%. Skydiver was implemented on a Xilinx XC7Z045 FPGA and verified on image segmentation and MNIST classification tasks. Results show improved throughput by 1.4X and 1.2X for the two tasks. Skydiver achieved 22.6 KFPS throughput, and 42.4 uJ/Image prediction energy on the classification task with 98.5% accuracy., Comment: Accepted to be published in the IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2022

The ability to engineer nonlinear optical processes in all-dielectric nanostructures is both of fundamental interest and highly desirable for high-performance, robust, and miniaturized nonlinear optical devices. Herein, we propose a novel paradigm for the efficient tuning of second-harmonic generation (SHG) process in dielectric nanoantennas by integrating with chalcogenide phase change material. In a design with Ge$_{2}$Sb$_{2}$Te$_{5}$ (GST) film sandwiched between the AlGaAs nanoantennas and AlO$_{x}$ substrate, the nonlinear SHG signal from the AlGaAs nanoantennas can be boosted via the resonantly localized field induced by the optically-induced Mie-type resonances, and further modulated by exploiting the GST amorphous-to-crystalline phase change in a non-volatile, multi-level manner. The tuning strategy originates from the modulation of resonant conditions by changes in the refractive index of GST. With a thorough examination of tuning performances for different nanoantenna radii, a maximum modulation depth as high as 540$\%$ is numerically demonstrated. This work not only reveals out the potential of GST in optical nonlinearity control, but also provides promising strategy in smart designing tunable and reconfigurable nonlinear optical devices, e.g., light emitters, modulators, and sensors.

Rotation is a common motional form in nature, existing from atoms and molecules, industrial turbines to astronomical objects. However, it still lacks an efficient and reliable method for real-time image processing of a fast-rotating object. Since the Fourier spectrum of a rotating object changes rapidly, the traditional Fourier transform (FT) techniques become extremely complicated and time consuming. Here, we propose a Laguerre-Gaussian (LG) transform to analyze the rotating object with LG-mode basis. The rotation operation provides a feasible way to acquire LG spectrum, which is similar to the function of lens in FT. Particularly, the obtained LG spectrum does not change even the object working at a high rotating speed. By analyzing the LG spectrum, one can perform image processing such as reconstruction, edge enhancement, and pattern replication. Such LG transform provides an efficient and convenient way to real-time monitor and analyze a fast-rotating object in scientific research and industry.

In device-to-device (D2D)-enabled caching cellular networks, the user terminals (UTs) collaboratively store and share a large volume of popular contents from the base station (BS) for traffic offloading and delivery delay reduction. In this article, the multi-winner auction based caching placement in D2D-enabled caching cellular networks is investigated for UT edge caching incentive and content caching redundancy reduction. Firstly, a multi-winner once auction for UT edge caching is modeled which auctions multiple contents for multiple UTs. Then the optimization problem for content caching revenue maximization is formulated. Specifically, the "cache conflict" restriction relationship among UTs is used as one of the constraints in the problem to reduce the content caching redundancy in a UT movement scenario. The problem is solved by semidefinite programming (SDP) relaxation to obtain an approximate optimal caching placement. Moreover, the payment strategy of the auction is developed as a Nash bargaining game for personal profit fairness among the UTs who win the auction for content caching. Subsequently, a multi-winner once auction based caching (MOAC) placement algorithm is proposed. In addition, due to the high complexity of MOAC, we further propose a heuristic multi-winner repeated auction based caching placement (MRAC) algorithm, which can greatly reduce the complexity with only tiny performance loss. Simulation results show that the proposed algorithms can reduce the traffic load and average content access delay effectively compared with the existing caching placement algorithms., Comment: 14 pages

Optical orbital angular momentum (OAM) provides an additional dimension for photons to carry information in high-capacity optical communication. Although the practical needs have intrigued the generations of miniaturized devices to manipulate the OAM modes in various integrated platforms, the on-chip OAM detection is still challenging to match the newly-developed compact OAM emitter and OAM transmission fiber. Here, we demonstrate an ultra-compact device, i.e., a single plasmonic nanohole, to efficiently measure an optical beam's OAM state in a nondestructive way. The device size is reduced down to a few hundreds of nanometers, which can be easily fabricated and installed in the current OAM devices. It is a flexible and robust way for in-situ OAM monitoring and detection in optical fiber networks and long-distance optical communication systems. With proper optimization of the nanohole parameters, this approach could be further extended to discriminate the OAM information multiplexed in multiple wavelengths and polarizations., Comment: 14 pages, 8 figures

We experimentally demonstrate the orbital angular momentum (OAM) conversion by the coupled nonlinear optical processes in a quasi-periodically poled LiTaO3 crystal. In such crystal, third-harmonic generation (THG) is realized by the coupled second-harmonic generation (SHG) and sum-frequency generation (SFG) processes, i.e., SHG is dependent on SFG and vice versa. The OAMs of the interacting waves are proved to be conserved in such coupled nonlinear optical processes. As increasing the input OAM in the experiment, the conversion efficiency decreases because of the reduced fundamental power intensity. Our results provide better understanding for the OAM conversions, which can be used to efficiently produce an optical OAM state at a short wavelength.