Search

Estimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fully-understood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human-annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations ("strong labels") can be effectively augmented with automatically derived "weak labels" in the context of learning a low-dimensional image-computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs., Comment: Computer Graphics Forum (Eurographics 2024)

Why do we perceive illusory motion in some static images? Several accounts have been proposed based on eye movements, response latencies to different image elements, or interactions between image patterns and motion energy detectors. Recently, PredNet, a recurrent deep neural network (DNN) based on predictive coding principles, was reported to reproduce the "Rotating Snakes" illusion, suggesting a role for predictive coding in illusory motion. We replicate this finding and then use a series of "in silico psychophysics" experiments to examine whether PredNet behaves consistently with human observers for simplified variants of the illusory stimuli. We also measure response latencies to individual elements of the Rotating Snakes pattern by probing internal units in the network. A pretrained PredNet model predicted illusory motion for all subcomponents of the Rotating Snakes stimulus, consistent with human observers. However, we found no simple response delays in internal units, as found in physiological data. The PredNet model's detection of motion in gradients was based on contrast, not luminance, as it is in human perception. Finally, we tested the robustness of the illusion on 10 identical PredNets trained on the same video data; we found a large variation in the ability of the network to reproduce the illusion and predict motion for simplified variants of the illusion. Also, unlike human observers, none of the networks predicted illusory motion for greyscale variants of the pattern. Even when a DNN successfully reproduces some idiosyncrasy of human vision, a more detailed investigation can reveal inconsistencies between humans and the network and between different instances of the same network. The inconsistency of the Rotating Snakes illusion in PredNets trained from different initializations suggests that predictive coding does not reliably lead to human-like illusory motion., Comment: Published in vision research (https://doi.org/10.1016/j.visres.2023.108195)

Neural network models can now recognise images, understand text, translate languages, and play many human games at human or superhuman levels. These systems are highly abstracted, but are inspired by biological brains and use only biologically plausible computations. In the coming years, neural networks are likely to become less reliant on learning from massive labelled datasets, and more robust and generalisable in their task performance. From their successes and failures, we can learn about the computational requirements of the different tasks at which brains excel. Deep learning also provides the tools for testing cognitive theories. In order to test a theory, we need to realise the proposed information-processing system at scale, so as to be able to assess its feasibility and emergent behaviours. Deep learning allows us to scale up from principles and circuit models to end-to-end trainable models capable of performing complex tasks. There are many levels at which cognitive neuroscientists can use deep learning in their work, from inspiring theories to serving as full computational models. Ongoing advances in deep learning bring us closer to understanding how cognition and perception may be implemented in the brain -- the grand challenge at the core of cognitive neuroscience., Comment: Chapter to appear in The Cognitive Neurosciences, 6th Edition