Search

Soccer is a popular sport, and there is a growing need for automated analysis of soccer videos, while the detection and tracking of the players is the indispensable prerequisite. In this paper, we first introduce and classify multi-object tracking and then present two mostly used multi-object tracking methods, DeepSort and TrackFormer. When multi-object tracking is applied to soccer scenarios, some preprocessing and post-processing are generally performed, with preprocessing including processing of the video, such as splicing and background removing, and post-processing including further applications, such as player mapping for a 2D stadium. By directly employing the two methods above, we test the real scene and train TrackFormer to get further results. Meanwhile, in order to facilitate researchers who are interested in multi-object tracking as well as in the direction of player tracking, recent advances in preprocessing and processing methods for soccer player tracking are given and future research directions are suggested. [ABSTRACT FROM AUTHOR]

To monitor and assess social dynamics and risks at large gatherings, we propose "SocialVis," a comprehensive monitoring system based on multi‐object tracking and graph analysis techniques. Our SocialVis includes a camera detection system that operates in two modes: a real‐time mode, which enables participants to track and identify close contacts instantly, and an offline mode that allows for more comprehensive post‐event analysis. The dual functionality not only aids in preventing mass gatherings or overcrowding by enabling the issuance of alerts and recommendations to organizers, but also allows for the generation of proximity‐based graphs that map participant interactions, thereby enhancing the understanding of social dynamics and identifying potential high‐risk areas. It also provides tools for analyzing pedestrian flow statistics and visualizing paths, offering valuable insights into crowd density and interaction patterns. To enhance system performance, we designed the SocialDetect algorithm in conjunction with the BYTE tracking algorithm. This combination is specifically engineered to improve detection accuracy and minimize ID switches among tracked objects, leveraging the strengths of both algorithms. Experiments on both public and real‐world datasets validate that our SocialVis outperforms existing methods, showing 1.2%$$ 1.2\% $$ improvement in detection accuracy and 45.4%$$ 45.4\% $$ reduction in ID switches in dense pedestrian scenarios. [ABSTRACT FROM AUTHOR]

Parametric human modeling are limited to either single‐view frameworks or simple multi‐view frameworks, failing to fully leverage the advantages of easily trainable single‐view networks and the occlusion‐resistant capabilities of multi‐view images. The prevalent presence of object occlusion and self‐occlusion in real‐world scenarios leads to issues of robustness and accuracy in predicting human body parameters. Additionally, many methods overlook the spatial connectivity of human joints in the global estimation of model pose parameters, resulting in cumulative errors in continuous joint parameters.To address these challenges, we propose a flexible and efficient iterative decoding strategy. By extending from single‐view images to multi‐view video inputs, we achieve local‐to‐global optimization. We utilize attention mechanisms to capture the rotational dependencies between any node in the human body and all its ancestor nodes, thereby enhancing pose decoding capability. We employ a parameter‐level iterative fusion of multi‐view image data to achieve flexible integration of global pose information, rapidly obtaining appropriate projection features from different viewpoints, ultimately resulting in precise parameter estimation. Through experiments, we validate the effectiveness of the HIDE method on the Human3.6M and 3DPW datasets, demonstrating significantly improved visualization results compared to previous methods. [ABSTRACT FROM AUTHOR]

Debris flow is a highly destructive natural disaster, necessitating accurate simulation and prediction. Existing simulation methods tend to be overly simplified, neglecting the three‐dimensional complexity and multiphase fluid interactions, and they also lack comprehensive consideration of soil conditions. We propose a novel two‐particle debris flow simulation method based on smoothed particle hydrodynamics (SPH) for enhanced accuracy. Our method employs a sophisticated two‐particle model coupling debris flow dynamics with SPH to simulate fluid‐solid interaction effectively, which considers various soil factors, dividing terrain into variable and fixed areas, incorporating soil impact factors for realistic simulation. By dynamically updating positions and reconstructing surfaces, and employing GPU and hash lookup acceleration methods, we achieve accurate simulation with significantly efficiency. Experimental results validate the effectiveness of our method across different conditions, making it valuable for debris flow risk assessment in natural disaster management. [ABSTRACT FROM AUTHOR]

We propose a comprehensive soccer match video analysis pipeline tailored for broadcast footage, which encompasses three pivotal stages: soccer field localization, player tracking, and soccer ball detection. Firstly, we introduce sports camera calibration to seamlessly map soccer field images from match videos onto a standardized two‐dimensional soccer field template. This addresses the challenge of consistent analysis across video frames amid continuous camera angle changes. Secondly, given challenges such as occlusions, high‐speed movements, and dynamic camera perspectives, obtaining accurate position data for players and the soccer ball is non‐trivial. To mitigate this, we curate a large‐scale, high‐precision soccer ball detection dataset and devise a robust detection model, which achieved the mAP50−95$$ mA{P}_{50-95} $$ of 80.9%. Additionally, we develop a high‐speed, efficient, and lightweight tracking model to ensure precise player tracking. Through the integration of these modules, our pipeline focuses on real‐time analysis of the current camera lens content during matches, facilitating rapid and accurate computation and analysis while offering intuitive visualizations. [ABSTRACT FROM AUTHOR]

As an amalgamation of landscape design and ichthyology, aquascape endeavors to create visually captivating aquatic environments imbued with artistic allure. Traditional methodologies in aquascape, governed by rigid principles such as composition and color coordination, may inadvertently curtail the aesthetic potential of the landscapes. In this paper, we propose Aquascape Generation based on Stable Diffusion Models (AG‐SDM), prioritizing aesthetic principles and color coordination to offer guiding principles for real artists in Aquascape creation. We meticulously curated and annotated three aquascape datasets with varying aspect ratios to accommodate diverse landscape design requirements regarding dimensions and proportions. Leveraging the Fréchet Inception Distance (FID) metric, we trained AGFID for quality assessment. Extensive experiments validate that our AG‐SDM excels in generating hyper‐realistic underwater landscape images, closely resembling real flora, and achieves state‐of‐the‐art performance in aquascape image generation. [ABSTRACT FROM AUTHOR]

Neural Radiance Field (NeRF) can render complex 3D scenes with viewpoint‐dependent effects. However, less work has been devoted to exploring its limitations in high‐resolution environments, especially when upscaled to ultra‐high resolution (e.g., 4k). Specifically, existing NeRF‐based methods face severe limitations in reconstructing high‐resolution real scenes, for example, a large number of parameters, misalignment of the input data, and over‐smoothing of details. In this paper, we present a novel and effective framework, called De‐NeRF, based on NeRF and deformable convolutional network, to achieve high‐fidelity view synthesis in ultra‐high resolution scenes: (1) marrying the deformable convolution unit which can solve the problem of misaligned input of the high‐resolution data. (2) Presenting a density sparse voxel‐based approach which can greatly reduce the training time while rendering results with higher accuracy. Compared to existing high‐resolution NeRF methods, our approach improves the rendering quality of high‐frequency details and achieves better visual effects in 4K high‐resolution scenes. [ABSTRACT FROM AUTHOR]

Flight simulation of catkins using computer technology helps their prevention and control. However, this is a challenging task due to the complex characteristics, and irregular shapes of catkins, while existing methods mainly focus on rain and snow, which are not suitable for catkins. In this paper, we propose a physics-based algorithm for the dynamic simulation of fluttering catkins. Our approach includes an L-system based 3D modeling method for simulating the natural phenomena of the catkin. We consider the motion of wind, free fall of catkins, and the dynamics of catkins under the joint action of attraction between them, while adhering to the physical motion law of catkins. To provide wind force, we first establish a three-dimensional wind field based on Boltzmann's equation. We then use the kernel function idea to calculate the attraction force between catkins and finally update the position of the catkin. We incorporate the phenomena of collision and adhesion, attraction, and accumulation of catkins while simulating motion states depending on the adjusted wall height and ground humidity parameters. Our approach overcomes limitations of previous models by achieving good simulation while using relatively less code to simulate various realistic motion states. According to our users' study, more than 71% of users found the simulation results to be acceptable, authentic, and realistic, confirming the authenticity of our simulation. Our method can generate highly realistic effects, significantly improving efficiency by several orders of magnitude compared to manual modeling. In addition, it can effectively simulate the dynamics of catkins in different scales, providing a decision-making reference for catkin control. [ABSTRACT FROM AUTHOR]

High spatial resolution (HSR) remote sensing images contain complex foreground-background relationships, which makes the remote sensing land cover segmentation a special semantic segmentation task. The main challenges come from the large-scale variation, complex background samples and imbalanced foreground-background distribution. These issues make recent context modeling methods sub-optimal due to the lack of foreground saliency modeling. To handle these problems, we propose a Remote Sensing Segmentation framework (RSSFormer), including Adaptive TransFormer Fusion Module, Detail-aware Attention Layer and Foreground Saliency Guided Loss. Specifically, from the perspective of relation-based foreground saliency modeling, our Adaptive Transformer Fusion Module can adaptively suppress background noise and enhance object saliency when fusing multi-scale features. Then our Detail-aware Attention Layer extracts the detail and foreground-related information via the interplay of spatial attention and channel attention, which further enhances the foreground saliency. From the perspective of optimization-based foreground saliency modeling, our Foreground Saliency Guided Loss can guide the network to focus on hard samples with low foreground saliency responses to achieve balanced optimization. Experimental results on LoveDA datasets, Vaihingen datasets, Potsdam datasets and iSAID datasets validate that our method outperforms existing general semantic segmentation methods and remote sensing segmentation methods, and achieves a good compromise between computational overhead and accuracy. Our code is available at https://github.com/Rongtao-Xu/RepresentationLearning/tree/main/RSSFormer-TIP2023. [ABSTRACT FROM AUTHOR]

Existing architecture semantic modeling methods in 3D complex urban scenes continue facing difficulties, such as limited training data, lack of semantic information, and inflexible model processing. Focusing on extracting and adopting accurate semantic information into a modeling process, this work presents a framework for lightweight modeling of buildings that joints point clouds semantic segmentation and 3D feature line detection constrained by geometric and photometric consistency. The main steps are: (1) Extraction of single buildings from point clouds using 2D-3D semi-supervised semantic segmentation under photometric and geometric constraints. (2) Generation of lightweight building models by using 3D plane-constrained multi-view feature line extraction and optimization. (3) Introduction of detailed semantics of building elements into independent 3D building models by using fine-grained segmentation of multi-view images to achieve high-accuracy architecture lightweight modeling with fine-grained semantic information. Experimental results demonstrate that it can perform independent lightweight modeling of each building on point cloud at various scales and scenes, with accurate geometric appearance details and realistic textures. It also enables independent processing and analysis of each building in the scenario, making them more useful in practical applications. [ABSTRACT FROM AUTHOR]

Accurate depth estimation from images is a fundamental problem in computer vision. In this paper, we propose an unsupervised learning based method to predict high-quality depth map from multiple images. A novel multi-view constrained DenseDepthNet is designed for this task. Our DenseDepthNet can effectively leverage both the low-level and high-level features of input images and generate appealing results, especially with sharp details. We employ the public datasets KITTI and Cityscapes for training in an end-to-end unsupervised fashion. A novel depth consistency loss based on multi-view geometry constraint is also applied to the corresponding points across pairwise images, which helps to improve the quality of predicted depth maps significantly. We conduct comprehensive evaluations on our DenseDepthNet and our depth consistency loss function. Experiments validate that our method outperforms the state-of-the-art unsupervised methods and produce comparable results with supervised methods. [ABSTRACT FROM AUTHOR]

The full-image based kernel estimation strategy is usually susceptible by the smooth and fine-scale background regions impacting and it is time-consuming for large-size image deblurring. Since not all the pixels in the blurred image are informative and it is frequent to restore human-interested objects in the foreground rather than background, we propose a novel concept “SalientPatch” to denote informative regions for better blur kernel estimation without user guidance by computing three cues (objectness probability, structure richness and local contrast). Although these cues are not new, it is innovative to integrate and complement each other in motion blur restoration. Experiments demonstrate that our SalientPatch-based deblurring algorithm can significantly speed up the kernel estimation and guarantee high-quality recovery for large-size blurry images as well. [ABSTRACT FROM AUTHOR]

We present a level of detail (LOD) method designed for tree branches. It can be combined with methods for processing tree foliage to facilitate navigation through large virtual forests. Starting from a skeletal representation of a tree, we fit polygon meshes of various densities to the skeleton while the mesh density is adjusted according to the required visual fidelity. For distant models, these branch meshes are gradually replaced with semi-transparent lines until the tree recedes to a few lines. Construction of these complete LOD models is guided by error metrics to ensure smooth transitions between adjacent LOD models. We then present an instancing technique for discrete LOD branch models, consisting of polygon meshes plus semi-transparent lines. Line models with different transparencies are instanced on the GPU by merging multiple tree samples into a single model. Our technique reduces the number of draw calls in GPU and increases rendering performance. Our experiments demonstrate that large-scale forest scenes can be rendered with excellent detail and shadows in real time. [ABSTRACT FROM AUTHOR]

Shape exploration of 3D heterogeneous models is essential for special effects in 3D animation and games. As heterogeneous models have different numbers of vertices and different topological structures,the mapping between source and target model may be ambiguous for deformation transfer. We propose a new framework for heterogeneous model shape exploration based on cages, which provides a feasible and fast solution to this open problem. Using a public cage as an intermediate medium, the deformation of the source models can be denoted as the position changing of the cage. When applying the cage change to the target model, rough deformation transfer can be achieved. After an optimization and interpolation to generate the explored shape of the heterogeneous target model, animation can be acquired. Our method is not only suitable for triangle meshes, but also for quadrilateral meshes or any other type of meshes. We demonstrate the validity of our scheme by a series of shape exploration experiments with different models. [ABSTRACT FROM AUTHOR]

The visualization of tomato growth can be used in 3D computer games and virtual gardens. Based on the growth theory involving the respiration theory, the photosynthesis, and dry matter partition, a visual system is developed. The tomato growth visual simulation system is light-and-temperature-dependent and shows plausible visual effects in consideration of the continuous growth, texture map, gravity influence, and collision detection. In addition, the virtual tomato plant information, such as the plant height, leaf area index, fruit weight, and dry matter, can be updated and output in real time. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel two-step building extraction method from remote sensing images by integrating saliency cue. We first utilize classical features such as shadow, color, and shape to find out initial building candidates. A fully connected conditional random field model is introduced in this step to ensure that most of the buildings are incorporated. While it is hard to further remove the mislabled rooftops from the building candidates by only using classical features, we adopt saliency cue as a new feature to determine whether there is a rooftop in each segmentation patch obtained from previous step. The basic idea behind the use of saliency information is that rooftops are more likely to attract visual attention than surrounding objects. Based on a specifically designed saliency estimation algorithm for building object, we extract saliency cue in the local region of each building candidate, which is integrated into a probabilistic model to get the final building extraction result. We show that the saliency cue can provide an efficient probabilistic indication of the presence of rooftops, which helps to reduce false positives while without increasing false negatives at the same time. Experimental results on two benchmark datasets highlight the advantages of the integration of saliency cue and demonstrate that the proposed method outperforms the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

In this paper, we present an information-theoretic framework to compute the shape similarity between 3D polygonal models. Given a 3D model, an information channel between a sphere of viewpoints around the model and its polygonal mesh is defined to compute the specific information associated with each viewpoint. The obtained information sphere can be seen as a shape descriptor of the model. Then, given two models, their similarity is obtained by performing a registration process between the corresponding information spheres. The distance between the information histograms is also defined as a coarse measure of similarity, as well as the scalar value given by the mutual information of the channel. The performance of all these measures is tested using the Princeton Shape Benchmark database. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]