Search

With the rise of neural networks in various domains, multi-task learning (MTL) gained significant relevance. A key challenge in MTL is balancing individual task losses during neural network training to improve performance and efficiency through knowledge sharing across tasks. To address these challenges, we propose a novel task-weighting method by building on the most prevalent approach of Uncertainty Weighting and computing analytically optimal uncertainty-based weights, normalized by a softmax function with tunable temperature. Our approach yields comparable results to the combinatorially prohibitive, brute-force approach of Scalarization while offering a more cost-effective yet high-performing alternative. We conduct an extensive benchmark on various datasets and architectures. Our method consistently outperforms six other common weighting methods. Furthermore, we report noteworthy experimental findings for the practical application of MTL. For example, larger networks diminish the influence of weighting methods, and tuning the weight decay has a low impact compared to the learning rate.

Generative foundation models like Stable Diffusion comprise a diverse spectrum of knowledge in computer vision with the potential for transfer learning, e.g., via generating data to train student models for downstream tasks. This could circumvent the necessity of collecting labeled real-world data, thereby presenting a form of data-free knowledge distillation. However, the resultant student models show a significant drop in accuracy compared to models trained on real data. We investigate possible causes for this drop and focus on the role of the different layers of the student model. By training these layers using either real or synthetic data, we reveal that the drop mainly stems from the model's final layers. Further, we briefly investigate other factors, such as differences in data-normalization between synthetic and real, the impact of data augmentations, texture vs.\ shape learning, and assuming oracle prompts. While we find that some of those factors can have an impact, they are not sufficient to close the gap towards real data. Building upon our insights that mainly later layers are responsible for the drop, we investigate the data-efficiency of fine-tuning a synthetically trained model with real data applied to only those last layers. Our results suggest an improved trade-off between the amount of real training data used and the model's accuracy. Our findings contribute to the understanding of the gap between synthetic and real data and indicate solutions to mitigate the scarcity of labeled real data.

While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we investigate paradigms in MTL in the context of STL: First, the impact of the choice of optimizer has only been mildly investigated in MTL. We show the pivotal role of common STL tools such as the Adam optimizer in MTL empirically in various experiments. To further investigate Adam's effectiveness, we theoretical derive a partial loss-scale invariance under mild assumptions. Second, the notion of gradient conflicts has often been phrased as a specific problem in MTL. We delve into the role of gradient conflicts in MTL and compare it to STL. For angular gradient alignment we find no evidence that this is a unique problem in MTL. We emphasize differences in gradient magnitude as the main distinguishing factor. Overall, we find surprising similarities between STL and MTL suggesting to consider methods from both fields in a broader context., Comment: Accepted for publication in German Conference for Pattern Recognition (GCPR), 2024

The Radiation Assessment Detector (RAD), onboard the Mars Science Laboratory (MSL) rover Curiosity, has been measuring the energetic charged and neutral particles and the radiation dose rate on the surface of Mars since the landing of the rover in August 2012. In contrast to charged particles, neutral particles (neutrons and gamma-rays) are measured indirectly: the energy deposition spectra produced by neutral particles are complex convolutions of the incident particle spectra with the detector response functions. An inversion technique has been developed and applied to jointly unfold the deposited energy spectra measured in two scintillators of different types (CsI for high gamma detection efficiency, and plastic for neutrons) to obtain the neutron and gamma-ray spectra. This result is important for determining the biological impact of the Martian surface radiation contributed by neutrons, which interact with materials differently from the charged particles. These first in-situ measurements on Mars provide (1) an important reference for assessing the radiation-associated health risks for future manned missions to the red planet and (2) an experimental input for validating the particle transport codes used to model the radiation environments within spacecraft or on the surface of planets. Here we present neutral particle spectra as well as the corresponding dose and dose equivalent rates derived from RAD measurement during a period (November 15, 2015 to January 15, 2016) for which the surface particle spectra have been simulated via different transport models.

To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the characteristics of images, such as edges and colors. The explanations are interpretable, suited for visualizing detailed evidence and can be tested as they are valid model inputs. We qualitatively and quantitatively evaluate our approach on a multitude of models and datasets., Comment: In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, June 2019

Deep neural networks (DNNs) are powerful tools in computer vision tasks. However, in many realistic scenarios label noise is prevalent in the training images, and overfitting to these noisy labels can significantly harm the generalization performance of DNNs. We propose a novel technique to identify data with noisy labels based on the different distributions of the predictive uncertainties from a DNN over the clean and noisy data. Additionally, the behavior of the uncertainty over the course of training helps to identify the network weights which best can be used to relabel the noisy labels. Data with noisy labels can therefore be cleaned in an iterative process. Our proposed method can be easily implemented, and shows promising performance on the task of noisy label detection on CIFAR-10 and CIFAR-100., Comment: Uncertainty and Robustness in Deep Visual Learning Workshop at CVPR 2019

Deep neural networks, and in particular recurrent networks, are promising candidates to control autonomous agents that interact in real-time with the physical world. However, this requires a seamless integration of temporal features into the network's architecture. For the training of and inference with recurrent neural networks, they are usually rolled out over time, and different rollouts exist. Conventionally during inference, the layers of a network are computed in a sequential manner resulting in sparse temporal integration of information and long response times. In this study, we present a theoretical framework to describe rollouts, the level of model-parallelization they induce, and demonstrate differences in solving specific tasks. We prove that certain rollouts, also for networks with only skip and no recurrent connections, enable earlier and more frequent responses, and show empirically that these early responses have better performance. The streaming rollout maximizes these properties and enables a fully parallel execution of the network reducing runtime on massively parallel devices. Finally, we provide an open-source toolbox to design, train, evaluate, and interact with streaming rollouts., Comment: To appear at NIPS 2018

Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly shown to work with medical image data which often poses a challenge as the decision for a class relies on different lesion areas scattered around the entire image. Using the DiaretDB1 dataset, we show that on retina images different lesion areas fundamental for diabetic retinopathy are detected on an image level with high accuracy, comparable or exceeding supervised methods. On lesion level, we achieve few false positives with high sensitivity, though, the network is solely trained on image-level labels which do not include information about existing lesions. Classifying between diseased and healthy images, we achieve an AUC of 0.954 on the DiaretDB1., Comment: Accepted in Proc. IEEE International Conference on Image Processing (ICIP), 2017

For future human missions to Mars, it is important to study the surface radiation environment during extreme and elevated conditions. In the long term, it is mainly Galactic Cosmic Rays (GCRs) modulated by solar activity that contributes to the radiation on the surface of Mars, but intense solar energetic particle (SEP) events may induce acute health effects. Such events may enhance the radiation level significantly and should be detected as immediately as possible to prevent severe damage to humans and equipment. However, the energetic particle environment on the Martian surface is significantly different from that in deep space due to the influence of the Martian atmosphere. Depending on the intensity and shape of the original solar particle spectra as well as particle types, the surface spectra may induce entirely different radiation effects. In order to give immediate and accurate alerts while avoiding unnecessary ones, it is important to model and well understand the atmospheric effect on the incoming SEPs including both protons and helium ions. In this paper, we have developed a generalized approach to quickly model the surface response of any given incoming proton/helium ion spectra and have applied it to a set of historical large solar events thus providing insights into the possible variety of surface radiation environments that may be induced during SEP events. Based on the statistical study of more than 30 significant solar events, we have obtained an empirical model for estimating the surface dose rate directly from the intensities of a power-law SEP spectra.

The Radiation Assessment Detector (RAD), on board Mars Science Laboratory's (MSL) rover Curiosity, measures the {energy spectra} of both energetic charged and neutral particles along with the radiation dose rate at the surface of Mars. With these first-ever measurements on the Martian surface, RAD observed several effects influencing the galactic cosmic ray (GCR) induced surface radiation dose concurrently: [a] short-term diurnal variations of the Martian atmospheric pressure caused by daily thermal tides, [b] long-term seasonal pressure changes in the Martian atmosphere, and [c] the modulation of the primary GCR flux by the heliospheric magnetic field, which correlates with long-term solar activity and the rotation of the Sun. The RAD surface dose measurements, along with the surface pressure data and the solar modulation factor, are analysed and fitted to empirical models which quantitatively demonstrate} how the long-term influences ([b] and [c]) are related to the measured dose rates. {Correspondingly we can estimate dose rate and dose equivalents under different solar modulations and different atmospheric conditions, thus allowing empirical predictions of the Martian surface radiation environment.

Aims: To predict the cruise radiation environment related to future human missions to Mars, the correlation between solar modulation potential and the dose rate measured by the Radiation Assessment Detector (RAD) has been analyzed and empirical models have been employed to quantify this correlation. Methods: The instrument RAD, onboard Mars Science Laboratory's (MSL) rover Curiosity, measures a broad spectrum of energetic particles along with the radiation dose rate during the 253-day cruise phase as well as on the surface of Mars. With these first ever measurements inside a spacecraft from Earth to Mars, RAD observed the impulsive enhancement of dose rate during solar particle events as well as a gradual evolution of the galactic cosmic ray (GCR) induced radiation dose rate due to the modulation of the primary GCR flux by the solar magnetic field, which correlates with long-term solar activities and heliospheric rotation. Results: We analyzed the dependence of the dose rate measured by RAD on solar modulation potentials and estimated the dose rate and dose equivalent under different solar modulation conditions. These estimations help us to have approximate predictions of the cruise radiation environment, such as the accumulated dose equivalent associated with future human missions to Mars. Conclusions: The predicted dose equivalent rate during solar maximum conditions could be as low as one-fourth of the current RAD cruise measurement. However, future measurements during solar maximum and minimum periods are essential to validate our estimations., Comment: 6 pages, 4 figures

Slow-wave sleep in mammalians is characterized by a change of large-scale cortical activity currently paraphrased as cortical Up/Down states. A recent experiment demonstrated a bistable collective behaviour in ferret slices, with the remarkable property that the Up states can be switched on and off with pulses, or excitations, of same polarity; whereby the effect of the second pulse significantly depends on the time interval between the pulses. Here we present a simple time discrete model of a neural network that exhibits this type of behaviour, as well as quantitatively reproduces the time-dependence found in the experiments., Comment: epl Europhysics Letters, accepted (2010)