Search

Classifying and segmenting natural disaster images are crucial for predicting and responding to disasters. However, current convolutional networks perform poorly in processing natural disaster images, and there are few proprietary networks for this task. To address the varying scales of the region of interest (ROI) in these images, we propose the Hierarchical TSAM-CB-ViT (HTCViT) network, which builds on the ViT network's attention mechanism to better process natural disaster images. Considering that ViT excels at extracting global context but struggles with local features, our method combines the strengths of ViT and convolution, and can capture overall contextual information within each patch using the Triple-Strip Attention Mechanism (TSAM) structure. Experiments validate that our HTCViT can improve the classification task with 3 - 4 % and the segmentation task with 1 - 2 % on natural disaster datasets compared to the vanilla ViT network. [ABSTRACT FROM AUTHOR]

In classical smoothed particle hydrodynamics (SPH) fluid simulation approaches, the smoothing length of Lagrangian particles is typically constant. One major disadvantage is the lack of adaptiveness, which may compromise accuracy in fluid regions such as splashes and surfaces. Attempts to address this problem used variable smoothing lengths. Yet the existing methods are computationally complex and non-efficient, because the smoothing length is typically calculated using iterative optimization. Here, we propose an efficient non-iterative SPH fluid simulation method with variable smoothing length (VSLSPH). VSLSPH correlates the smoothing length to the density change, and adaptively adjusts the smoothing length of particles with high accuracy and low computational cost, enabling large time steps. Our experimental results demonstrate the advantages of the VSLSPH approach in terms of its simulation accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Fast and accurate semantic analysis of natural disaster images is crucial for rational rescue plans and resource allocation. However, the scarcity of meticulously labelled datasets and the ignorance of region-of-interest scale variations of popular general-purpose methods lead to undesirable performance. In this paper, we propose a novel triple-strip attention mechanism (TSAM) to solve the generalization problem of disaster images that can be combined into general networks. Our TSAM accumulates features of three parallel-strip attentions (row strip attention, column strip attention, and channel strip attention), and the output is multiplied with original input features for further processing. Our attention mechanism can effectively overcome the defect of ignoring global features caused by the convolution and enhance the performance of the network by weighting the features from both spatial and channel aspects more comprehensively. Besides, we employ both the compression and expansion operations in the weighting operation to reduce the amount of parameters, leading to negligible computational overhead. Experiments validate that our TSAM outperforms other state-of-the-art methods on natural disaster segmentation. Due to its concise form, plug-and-play pattern, and high promotion rate, our TSAM can be combined with many existing neural networks for better performance improvement. [ABSTRACT FROM AUTHOR]

3D point cloud classification is one of the basic topics in multimedia analysis and understanding. By the construction of the discriminant model and efficient parameter optimization, point cloud classification can be achieved after the training. However, most parameter optimization methods do not guarantee the highest global classification accuracy with a high classification accuracy on smaller classes. In addition, geometric features of the point cloud are not sufficiently utilized. In this paper, we use local geometric shape features including the nearest neighbor tetrahedral volume, Gaussian curvature, the neighbourhood normal vector consistency and the neighbourhood minimum principal curvature direction consistency. We propose three discrete criteria for parameter optimization to design explicit functions, and we present concrete algorithms, in which Monte Carlo method and Probabilistic Neural Network method are employed to estimate these parameters respectively. Experimental results show that our criteria can be applied to the classification of the 3D point cloud of the scene, and can be used to improve the classification accuracy of small-scale point sets when different classes have great disparities in the number. [ABSTRACT FROM AUTHOR]

The full-image based kernel estimation strategy is usually susceptible by the smooth and fine-scale background regions impacting and it is time-consuming for large-size image deblurring. Since not all the pixels in the blurred image are informative and it is frequent to restore human-interested objects in the foreground rather than background, we propose a novel concept “SalientPatch” to denote informative regions for better blur kernel estimation without user guidance by computing three cues (objectness probability, structure richness and local contrast). Although these cues are not new, it is innovative to integrate and complement each other in motion blur restoration. Experiments demonstrate that our SalientPatch-based deblurring algorithm can significantly speed up the kernel estimation and guarantee high-quality recovery for large-size blurry images as well. [ABSTRACT FROM AUTHOR]

Shape exploration of 3D heterogeneous models is essential for special effects in 3D animation and games. As heterogeneous models have different numbers of vertices and different topological structures,the mapping between source and target model may be ambiguous for deformation transfer. We propose a new framework for heterogeneous model shape exploration based on cages, which provides a feasible and fast solution to this open problem. Using a public cage as an intermediate medium, the deformation of the source models can be denoted as the position changing of the cage. When applying the cage change to the target model, rough deformation transfer can be achieved. After an optimization and interpolation to generate the explored shape of the heterogeneous target model, animation can be acquired. Our method is not only suitable for triangle meshes, but also for quadrilateral meshes or any other type of meshes. We demonstrate the validity of our scheme by a series of shape exploration experiments with different models. [ABSTRACT FROM AUTHOR]

Point distributions with different characteristics have a crucial influence on graphics applications. Various analysis tools have been developed in recent years, mainly for blue noise sampling in Euclidean domains. In this paper, we present a new method to analyze the properties of general sampling patterns that are distributed on mesh surfaces. The core idea is to generalize to surfaces the pair correlation function (PCF) which has successfully been employed in sampling pattern analysis and synthesis in 2D and 3D. Experimental results demonstrate that the proposed approach can reveal correlations of point sets generated by a wide range of sampling algorithms. An acceleration technique is also suggested to improve the performance of the PCF. [ABSTRACT FROM AUTHOR]

This paper presents a novel GPU-based multiresolution rendering on sole-cube maps (SCMs), which is a variant of geometry images built upon spherical parameterization. Given spherical parametrization of a manifold mesh, the sphere domain is gnomonically projected to a closed cube, which constitutes the 6-chart sole-cube maps. A quadtree structure of SCMs and normal map atlas are then constructed by using the regular re-sampling. Then, by packing the quadtree nodes into the SCMs texture atlas, a new parallel multiresolution rendering is processed on the latest GPU in two rendering passes: the multiresolution node selection in fragment shader; the triangulation in vertex shader followed by the node culling operation in geometry shader. The proposed approach generates adaptive mesh surfaces dynamically, and can be fully implemented in GPU parallelization. The proposed scheme alleviates the computing load of multiresolution mesh refinement on CPU, and our GPU-based multiresolution rendering is demonstrated with a variety of examples. Our user study confirmed that the visual quality of the SCMs multiresolution rendering, in comparison with the meshes/geometry images rendering, is also highly efficient especially for complex models in large-scale virtual environment. [ABSTRACT FROM AUTHOR]

To synthesize real-time and realistic facial animation, we present an effective algorithm which combines image- and geometry-based methods for facial animation simulation. Considering the numerous motion units in the expression coding system, we present a novel simplified motion unit based on the basic facial expression, and construct the corresponding basic action for a head model. As image features are difficult to obtain using the performance driven method, we develop an automatic image feature recognition method based on statistical learning, and an expression image semi-automatic labeling method with rotation invariant face detection, which can improve the accuracy and efficiency of expression feature identification and training. After facial animation redirection, each basic action weight needs to be computed and mapped automatically. We apply the blend shape method to construct and train the corresponding expression database according to each basic action, and adopt the least squares method to compute the corresponding control parameters for facial animation. Moreover, there is a pre-integration of diffuse light distribution and specular light distribution based on the physical method, to improve the plausibility and efficiency of facial rendering. Our work provides a simplification of the facial motion unit, an optimization of the statistical training process and recognition process for facial animation, solves the expression parameters, and simulates the subsurface scattering effect in real time. Experimental results indicate that our method is effective and efficient, and suitable for computer animation and interactive applications. [ABSTRACT FROM AUTHOR]

User interfaces have traditionally followed the WIMP (window, icon, menu, pointer) paradigm. Though functional and powerful, they are usually cumbersome for a novice user to design a complex model, requiring considerable expertise and effort. This paper presents a system for designing geometric models and image deformation with sketching curves, with the use of Green coordinates. In 3D modeling, the user first creates a 3D model by using a sketching interface, where a given 2D curve is interpreted as the projection of the 3D curve. The user can add, remove, and deform these control curves easily, as if working with a 2D line drawing. For a given set of curves, the system automatically identifies the topology and face embedding by applying graph rotation system. Green coordinates are then used to deform the generated models with detail-preserving property. Also, we have developed a sketch-based image-editing interface to deform image regions using Green coordinates. Hardware-assisted schemes are provided for both control shape deformation and the subsequent surface optimization, the experimental results demonstrate that 3D/2D deformations can be achieved in realtime. [ABSTRACT FROM AUTHOR]

Image coloring has always been a challenging problem. Current coloring methods cannot eliminate the ambiguity of bright material surfaces from specular highlight images, as the presence of specular highlights introduces complex brightness and color changes. To tackle this issue, we propose a novel multi-stage colorization network (MSC-Net) to identify and eliminate color ambiguity caused by specular highlights accurately. Firstly, we design a new method for detecting and removing specular highlights in images, including a specular highlight segmentation network(SHSNet) and a highlight-eliminating module, in order to accurately detect and eliminate specular highlight areas for subsequent coloring. Subsequently, we give a deep learning-based coloring network(ColorNet) and integrated highlight features to color the image. After coloring, the completed image will be combined with the original specular highlight information to achieve a comprehensive restoration of the highlighted area. Experiments on multiple public datasets are conducted, validating that our MSC-Net achieves favorable results in specular highlight segmentation and performs excellently in coloring images with specular highlights. Our MSC-Net provides a mature solution for image coloring with specular highlights, which has significant theoretical and practical implications. Our code is available at: https://github.com/ymm0304/MSC-Net. [ABSTRACT FROM AUTHOR]

Traditional multi-object tracking is limited to tracking a predefined set of categories, whereas open-vocabulary tracking expands its capabilities to track novel categories. In this paper, we propose ROMOT (referring-expression-comprehension open-set multi-object tracking), which not only tracks objects from novel categories not included in the training data, but also enables tracking based on referring expression comprehension (REC). REC describes targets solely by their attributes, such as “the person running at the front” or “the bird flying in the air rather than on the ground,” making it particularly relevant for real-world multi-object tracking scenarios. Our ROMOT achieves this by harnessing the exceptional capabilities of a visual language model and employing multi-stage cross-modal attention to handle tracking for novel categories and REC tasks. Integrating RSM (reconstruction similarity metric) and OCM (observation-centric momentum) in our ROMOT eliminates the need for task-specific training, addressing the challenge of insufficient datasets. Our ROMOT enhances efficiency and adaptability in handling tracking requirements without relying on extensive tracking training data. [ABSTRACT FROM AUTHOR]

Tracking soccer players serves as a fundamental prerequisite for the automated analysis of soccer videos. The strides in computer vision technology have paved the way for consistently precise player tracking in soccer videos. However, it’s crucial to acknowledge that real-world soccer videos often come with inherent limitations. In response to the practical demand for soccer player tracking, we propose a comprehensive pipeline. For practicality and validity of data analysis, our pipeline includes the main method and the preprocessing process; The main method is to get valid tracking data and to further apply the data, including tracking soccer players based on the attention mechanism, which we call Soccer Player TrackFormer, correction and two-dimensional mapping. The preprocessing process is to handle video concatenation and dataset creation for the sake of making up for the lack of a full-field soccer video dataset. Experiments validate that our method can reliably obtain precise tracking data, significantly augmenting the value of soccer video analysis. [ABSTRACT FROM AUTHOR]