Search

The geometric distortion in panoramic images significantly mediates the performance of saliency detection method based on traditional CNN. The strategy of dynamically expanding convolution kernel can achieve good results, but it also produces a lot of computational overhead in the process of reading the adjacency list, which decreases the computational efficiency. The appearance of graph convolution provides a new way to solve such problems. Although using graph convolution can effectively extract the structural features of the graph, it reduces the accuracy of the model resulting from ignoring the spatial features of the image signal. To this end, this paper proposes a construction method of the multi-scale graph structure of the panoramic image and a panoramic image saliency detection model composed of an image saliency feature extraction network and multi-scale saliency feature fusion network combining the image structure information and spatial information in the panoramic image. First, we establish a graph structure consisting of root and leaf nodes obtained by super-pixel segmentation at different scales and spherical Fibonacci sampling, respectively. Then, a feature extraction network composed of two graph convolution layers and two one-dimensional auto-encoders with the same parameterization is used to extract the salient features of the multi-scale graph structure. Finally, the U-Net network fuses the multi-scale saliency features to get the final saliency map. The results show that the proposed model performs better than the state-of-the-art models in terms of calculation speed and accuracy. [ABSTRACT FROM AUTHOR]

The geometric distortion of the panoramic image makes the saliency detection method based on traditional 2D convolution invalid. "Mapped Convolution" can effectively solve this problem, which accepts a task- or domain-specific mapping function in the form of an adjacency list that dictates where the convolutional filters sample the input. However, when applied to panorama saliency detection, the method results in additional computational overhead due to repeatedly sampling overlapping regions of adjacent convolution positions along the longitude. In order to solve this problem, we improved the calculation process of "Mapped Convolution". Rather than accessing adjacency list during the convolution, we first sample the panorama based on the adjacency list for only once and obtain a sampled map. This sampling process is called the decoupled sampling of "Mapped Convolution". And then the map is convoluted in traditional 2D way, thus avoiding repeatedly sampling. In this paper, an interpolation method based on the Softmax function is also proposed and applied to the interpolation calculation of decoupled sampling. Compared with common interpolation methods such as linear interpolation, this interpolation method makes our network more efficient during training. We additionally introduce a new adaptive equator bias algorithm allowing for different attention distributions at different longitudes, which is more consistent with viewer's visual behavior. Combining the U-Autoencoder network containing the decoupled sampling with the adaptive equator bias algorithm, we construct a 360-degree visual saliency detection model. We map the original panorama into a cube, and then use the the cube isometric mapping method to remap it into a panorama and input it into the network for training. Then, the crude saliency map output by the decoder is combined with the equator bias map to obtain the final saliency map. The results show that the model proposed is superior to recent state-of-the-art models in terms of computational speed and saliency-map prediction. [ABSTRACT FROM AUTHOR]

Monte Carlo path tracing is a widely used method for generating realistic rendering results in multimedia applications, but often suffers from poor convergence and heavy sampling budget. Insufficient path samples may lead to noisy results. Some noises are hidden in textures, and the human visual system cannot detect them all. Just noticeable difference (JND) quantifies this limitation as a full-reference perception threshold. In rendering, the reference is unavailable and a surrogate is required. This paper proposed a perception-JND-driven path tracing method for reducing sampling budget. We tested and verified the surrogate JND thresholds derived from current rendering results. Then, we introduced difference pooling module and shading restart module to control perceptual convergence. Further, to improve accuracy, we developed the strategy for optimizing sampling steps. Experiments showed that the proposed method outperformed the state-of-the-art method at moderately low sampling levels, offering a lightweight and efficient solution to reducing sample budget while improving visual quality. [ABSTRACT FROM AUTHOR]