Search

The ability to detect and analyze a human figure is key to our survival and social interactions. Efficient and robust identification of body parts can help to interpret images when bodies are partially occluded. While previous studies emphasized the configural processing of whole bodies using simplified stimuli, it still remains unclear how spatial contexts about body parts are integrated to resolve ambiguities (e.g., from occlusion) regarding the identities of or spatial relations among body parts. In a series of online experiments, we asked human observers to identify an ambiguous target body part in the presence of another “context” part. Our results showed that humans can use various amounts of spatial context to discount local ambiguities in natural images of pairs of parts, and are sensitive to low- and mid-level cues such as alignment and connectedness. Further simulations using deep convolutional neural networks (DCNNs) exhibited comparable similar sensitivity to spatial context variations, despite being trained solely on local part appearances without explicit prior knowledge of body structure. However, discrepancies between human and model performance were also observed, with humans showing greater sensitivity to spatial relations compared to the models. Our findings suggest that while both humans and models utilize low- and mid-level features for body part recognition, humans possess a stronger prior knowledge of body structure that in- fluences their perception. These results contribute to our understanding of how humans integrate spatial context to resolve ambiguities and provide insights into the computational mechanisms underlying body perception.

Perceiving 3D structure in natural images is an immense computational challenge for the visual system. While many previous studies focused on rigid 3D objects, we applied a novel method on a common set of non-rigid objects—static images of the human body in the natural world. We investigated to what extent human ability to interpret 3D poses in natural images depends on pose typicality and viewpoint informativeness. We tested subjects on matching natural pose images with synthetic body images of the same poses given viewpoint changes. We found that performance for typical poses was measurably better than atypical poses; however, we found no significant difference between informative and less informative viewpoints. Results suggested that human ability to interpret 3D poses depends on pose typicality but not viewpoint informativeness. Further comparisons of 2D and 3D pose matching models suggested that humans probably use prior knowledge of 3D pose structures.

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects. [ABSTRACT FROM AUTHOR]

We argue that the study of human vision should be aimed at determining how humans perform natural tasks on natural images. Attempts to understand the phenomenology of vision from artificial stimuli, though worthwhile as a starting point, risk leading to faulty generalizations about visual systems. In view of the enormous complexity of natural images, they are similar to trying to evaluate the performance of a soldier in battle from his ability at playing with a water pistol. Dealing with this complexity is daunting, but Bayesian inference on structured probability distributions offers the ability to design theories of vision that can deal with the complexity of natural images and which use analysis by synthesis strategies with intriguing similarities to the brain.

This paper compares the ability of human observers to detect target image curves with that of an ideal observer. The target curves are sampled from a generative model which specifies (probabilistically) the geometry and local intensity properties of the curve. The ideal observer performs Bayesian inference on the generative model using MAP estimation. Varying the probability model for the curve geometry enables us investigate whether human performance is best for target curves that obey specific shape statistics, in particular those observed on natural shapes. Experiments are performed with data on both rectangular and hexagonal lattices. Our results show that human observers' performance approaches that of the ideal observer and are, in general, closest to the ideal for conditions where the target curve tends to be straight or similar to natural statistics on curves. This suggests a bias of human observers towards straight curves and natural statistics.

The human visual system is the most complex pattern recognition device known. In ways that are yet to be fully understood, the visual cortex arrives at a simple and unambiguous interpretation of data from the retinal image that is useful for the decisions and actions of everyday life. Recent advances in Bayesian models of computer vision and in the measurement and modeling of natural image statistics are providing the tools to test and constrain theories of human object perception. In turn, these theories are having an impact on the interpretation of cortical function.

A successful vision system must solve the problem of deriving geometrical information about three-dimensional objects from two-dimensional photometric input. The human visual system solves this problem with remarkable efficiency, and one challenge in vision research is to understand how neural representations of objects are formed and what visual information is used to form these representations. Ideal observer analysis has demonstrated the advantages of studying vision from the perspective of explicit generative models and a specified visual task, which divides the causes of image variations into the separate categories of signal and noise. Classification image techniques estimate the visual information used in a task from the properties of ''noise'' images that interact most strongly with the task. Both ideal observer analysis and classification image techniques rely on the assumption of a generative model. We show here how the ability of the classification image approach to understand how an observer uses visual information can be improved by matching the type and dimensionality of the model to that of the neural representation or internal template being studied. Because image variation in real world object tasks can arise from both geometrical shape and photometric (illumination or material) changes, a realistic image generation process should model geometry as well as intensity. A simple example is used to demonstrate what we refer to as a ''classification object'' approach to studying three-dimensional object representations.

Most people with low vision rely on their remaining functional vision for mobility. Our goal is to provide tools to help design architectural spaces in which safe and effective mobility is possible by those with low vision – spaces that we refer to as visually accessible. We describe an approach that starts with a 3D CAD model of a planned space and produces labeled images indicating whether or not structures that are potential mobility hazards are visible at a particular level of low vision. There are two main parts to the analysis. The first, previously described, represents low-vision status by filtering a calibrated luminance image generated from the CAD model and associated lighting and materials information to produce a new image with unseen detail removed. The second part, described in this paper, uses both these filtered images and information about the geometry of the space obtained from the CAD model and related lighting and surface material specifications to produce a quantitative estimate of the likelihood of particular hazards being visible. We provide examples of the workflow required, a discussion of the novelty and implications of the approach, and a short discussion of needed future work. [ABSTRACT FROM AUTHOR]

A successful vision system must solve the problem of deriving geometrical information about three-dimensional objects from two-dimensional photometric input. The human visual system solves this problem with remarkable efficiency, and one challenge in vision research is to understand how neural representations of objects are formed and what visual information is used to form these representations. Ideal observer analysis has demonstrated the advantages of studying vision from the perspective of explicit generative models and a specified visual task, which divides the causes of image variations into the separate categories of signal and noise. Classification image techniques estimate the visual information used in a task from the properties of 'noise' images that interact most strongly with the task. Both ideal observer analysis and classification image techniques rely on the assumption of a generative model. We show here how the ability of the classification image approach to understand how an observer uses visual information can be improved by matching the type and dimensionality of the model to that of the neural representation or internal template being studied. Because image variation in real world object tasks can arise from both geometrical shape and photometric (illumination or material) changes, a realistic image generation process should model geometry as well as intensity. A simple example is used to demonstrate what we refer to as a 'classification object' approach to studying three-dimensional object representations. Keywords: Classification image; Ideal observer; Reverse correlation