Search

The aim of continual learning is to learn new tasks continuously (i.e., plasticity) without forgetting previously learned knowledge from old tasks (i.e., stability). In the scenario of online continual learning, wherein data comes strictly in a streaming manner, the plasticity of online continual learning is more vulnerable than offline continual learning because the training signal that can be obtained from a single data point is limited. To overcome the stability-plasticity dilemma in online continual learning, we propose an online continual learning framework named multi-scale feature adaptation network (MuFAN) that utilizes a richer context encoding extracted from different levels of a pre-trained network. Additionally, we introduce a novel structure-wise distillation loss and replace the commonly used batch normalization layer with a newly proposed stability-plasticity normalization module to train MuFAN that simultaneously maintains high plasticity and stability. MuFAN outperforms other state-of-the-art continual learning methods on the SVHN, CIFAR100, miniImageNet, and CORe50 datasets. Extensive experiments and ablation studies validate the significance and scalability of each proposed component: 1) multi-scale feature maps from a pre-trained encoder, 2) the structure-wise distillation loss, and 3) the stability-plasticity normalization module in MuFAN. Code is publicly available at https://github.com/whitesnowdrop/MuFAN., Comment: Accepted to ICLR2023

Additional file 1: Supplementary Fig. 1. Comparison of proliferation, viability and cytotoxicity according to NK92 cell density cultured in two-dimensions. (a) Cell number of NK92 cell according to density. The higher the cell density, the higher the rate of increase. (b) Cell proliferation folds of NK92 cell. (c) Cell apoptosis and death on the day 5. (d) NK92 cell lytic activity based on Calcein-AM cytotoxicity assay on day 3 and 5 at and E:T ratio of 2:1 and 1:1. Supplementary Fig. 2. Printing conditions of ALG and ALG/GEL-2 groups related to pressure and printing feed rate. ALG groups were printed at 300 mm/min at 60 kPa of pressure and ALG/GEL-2 group were printed at 300 mm/min at 70 kPa. Scale bar = 400 µm. Supplementary Fig. 3. EDTA concentration test for cell viability and alginate degradation. (a) Representative microscopic image of formation of NK92 cell clusters and viability related to cell condition according to EDTA concentrations. NK92 cells tend not to maintain cell clusters over 8mM. Scale bar = 100 µm. (b) Degradation of 3% alginate hydrogel for 3D encapsulated NK92 cells preparation. 8 mM EDTA concentration were used for hydrogel degradation. Scale bar = 1mm. Supplementary Fig. 4. Concentration of calcium chloride for alginate crosslinking. When ALG and ALG/GEL-2 groups were crosslinked by 3% CaCl2, NK92 cells were released out by hydrogel degradation on day 5. Scale bar = 500 µm. Supplementary Fig. 5. Expression of activating receptors at 3D cultured zEGFR-CAR NK cells. Supplementary Table 1. Gene specific primer sequences for qRT-PCR.

This study considers a parallel dedicated machine scheduling problem towards minimizing the total tardiness of allocated jobs on machines. In addition, this problem comes under the category of NP-hard. Unlike classical parallel machine scheduling, a job is processed by only one of the dedicated machines according to its job type defined in advance, and a machine is able to process at most one job at a time. To obtain a high-quality schedule in terms of total tardiness for the considered scheduling problem, we suggest a machine scheduler based on double deep Q-learning. In the training phase, the considered scheduling problem is redesigned to fit into the reinforcement learning framework and suggest the concepts of state, action, and reward to understand the occurrences of setup, tardiness, and the statuses of allocated job types. The proposed scheduler, repeatedly finds better Q-values towards minimizing tardiness of allocated jobs by updating the weights in a neural network. Then, the scheduling performances of the proposed scheduler are evaluated by comparing it with the conventional ones. The results show that the proposed scheduler outperforms the conventional ones. In particular, for two datasets presenting extra-large scheduling problems, our model performs better compared to existing genetic algorithm by 12.32% and 29.69%.

Spiking neural networks (SNNs) have been gaining interest as energy-efficient alternatives of conventional artificial neural networks (ANNs) due to their event-driven computation. Considering the future deployment of SNN models to constrained neuromorphic devices, many studies have applied techniques originally used for ANN model compression, such as network quantization, pruning, and knowledge distillation, to SNNs. Among them, existing works on knowledge distillation reported accuracy improvements of student SNN model. However, analysis on energy efficiency, which is also an important feature of SNN, was absent. In this paper, we thoroughly analyze the performance of the distilled SNN model in terms of accuracy and energy efficiency. In the process, we observe a substantial increase in the number of spikes, leading to energy inefficiency, when using the conventional knowledge distillation methods. Based on this analysis, to achieve energy efficiency, we propose a novel knowledge distillation method with heterogeneous temperature parameters. We evaluate our method on two different datasets and show that the resulting SNN student satisfies both accuracy improvement and reduction of the number of spikes. On MNIST dataset, our proposed student SNN achieves up to 0.09% higher accuracy and produces 65% less spikes compared to the student SNN trained with conventional knowledge distillation method. We also compare the results with other SNN compression techniques and training methods., Comment: The manuscript was withdrawn because it contains inappropriate content for posting

Recently, a method of engineering the quantum states with a nonlinear interferometer was proposed to achieve precise state engineering for near-ideal single-mode operation and near-unity efficiency (L. Cui et al., Phys. Rev. A 102, 033718 (2020)), and the high-purity bi-photon states can be created without degrading brightness and collection efficiency. Here, we study the coarse or fine tunability of the nonlinear interference method to match constructive interference patterns into a transmission window of standard 100-GHz DWDM channels. The joint spectral intensity spectrum is measured for various conditions of the nonlinear interference effects. We show that the method has coarse- and fine-tuning ability while maintaining its high spectral purity. We expect that our results expand the usefulness of the nonlinear interference method. The photon-pair generation engineered via this method will be an excellent practical source of the quantum information process.

Predictive Maintenance (PdM) can only be implemented when the online knowledge of system condition is available, and this has become available with deployment of on-equipment sensors. To date, most studies on predicting the remaining useful lifetime of a system have been focusing on either single-component systems or systems with deterministic reliability structures. This assumption is not applicable on some realistic problems, where there exist uncertainties in reliability structures of complex systems. In this paper, a PdM scheme is developed by employing a Discrete Time Markov Chain (DTMC) for forecasting the health of monitored components and a Bayesian Network (BN) for modeling the multi-component system reliability. Therefore, probabilistic inferences on both the system and its components status can be made and PdM can be scheduled on both levels., Comment: 7 pages, This is a Conference paper submitted to Industrial and Systems Engineering Research Conference 2016 (ISERC)