Search

Abstract Online detection of action start is a significant and challenging task that requires prompt identification of action start positions and corresponding categories within streaming videos. This task presents challenges due to data imbalance, similarity in boundary content, and real‐time detection requirements. Here, a novel Time‐Attentive Fusion Network is introduced to address the requirements of improved action detection accuracy and operational efficiency. The time‐attentive fusion module is proposed, which consists of long‐term memory attention and the fusion feature learning mechanism, to improve spatial‐temporal feature learning. The temporal memory attention mechanism captures more effective temporal dependencies by employing weighted linear attention. The fusion feature learning mechanism facilitates the incorporation of current moment action information with historical data, thus enhancing the representation. The proposed method exhibits linear complexity and parallelism, enabling rapid training and inference speed. This method is evaluated on two challenging datasets: THUMOS’14 and ActivityNet v1.3. The experimental results demonstrate that the proposed method significantly outperforms existing state‐of‐the‐art methods in terms of both detection accuracy and inference speed.

Cell-free massive multi-input-multi-output (MIMO) systems comprise a large number of distributed access points (APs) to serve a small number of user equipments (UEs). In this paper, compute-and-forward (CF) is investigated for uplink in cell-free massive MIMO. We propose a novel reverse CF precoding scheme for downlink transmission for cell-free massive MIMO. The APs send the linear equations of messages over a multiple access channel, and the UEs recover messages without interferences. The max–min power allocation is investigated to provide uniformly good service to all UEs. Numeral results demonstrate the performance improvement provided by the proposed reverse CF precoder against zero-forcing, conjugate beamforming, and linear minimum mean-square error precoder.

Railway track detection, which is crucial for train operational safety, faces numerous challenges such as the curved track, obstacle occlusion, and vibrations during the train’s operation. Most existing methods for railway track detection use a camera or LiDAR. However, the vision-based approach lacks essential 3D environmental information about the train, while the LiDAR-based approach tends to detect tracks of insufficient length due to the inherent limitations of LiDAR. In this study, we propose a real-time method for railway track detection and 3D fitting based on camera and LiDAR fusion sensing. Semantic segmentation of the railway track in the image is performed, followed by inverse projection to obtain 3D information of the distant railway track. Then, 3D fitting is applied to the inverse projection of the railway track for track vectorization and LiDAR railway track point segmentation. The extrinsic parameters necessary for inverse projection are continuously optimized to ensure robustness against variations in extrinsic parameters during the train’s operation. Experimental results show that the proposed method achieves desirable accuracy for railway track detection and 3D fitting with acceptable computational efficiency, and outperforms existing approaches based on LiDAR, camera, and camera–LiDAR fusion. To the best of our knowledge, our approach represents the first successful attempt to fuse camera and LiDAR data for real-time railway track detection and 3D fitting.

Detecting obstacles in the rail track area is crucial for ensuring the safe operation of trains. However, this task presents numerous challenges, including the diverse nature of intrusions, and the complexity of the driving environment. This paper presents a multimodal fusion rail-obstacle detection approach by key points processing and rail track topology reconstruction. The core idea is to leverage the rich semantic information provided by images to design algorithms for reconstructing the topological structure of railway tracks. Additionally, it combines the effective geometric information provided by LiDAR to accurately locate the railway tracks in space and to filter out intrusions within the track area. Experimental results demonstrate that our method outperforms other approaches with a longer effective working distance and superior accuracy. Furthermore, our post-processing method exhibits robustness even under extreme weather conditions.

Video temporal grounding (VTG) aims to locate specific temporal segments from an untrimmed video based on a linguistic query. Most existing VTG models are trained on extensive annotated video-text pairs, a process that not only introduces human biases from the queries but also incurs significant computational costs. To tackle these challenges, we propose VTG-GPT, a GPT-based method for zero-shot VTG without training or fine-tuning. To reduce prejudice in the original query, we employ Baichuan2 to generate debiased queries. To lessen redundant information in videos, we apply MiniGPT-v2 to transform visual content into more precise captions. Finally, we devise the proposal generator and post-processing to produce accurate segments from debiased queries and image captions. Extensive experiments demonstrate that VTG-GPT significantly outperforms SOTA methods in zero-shot settings and surpasses unsupervised approaches. More notably, it achieves competitive performance comparable to supervised methods. The code is available on GitHub.

Estimating 360∘ depth from multiple cameras has been a challenging problem. However, existing methods often adopt a fixed-step spherical sweeping approach with densely sampled spheres and use numerous 3D convolutions in networks, which limits the speed of algorithms in practice. Additionally, obtaining high-precision depth maps of real scenes poses a challenge for the existing algorithms. In this paper, we design a cascade architecture using a dynamic spherical sweeping method that progressively refines the depth estimation from coarse to fine over multiple stages. The proposed method adaptively adjusts sweeping intervals and ranges based on the predicted depth and the uncertainty from the previous stage, resulting in a more efficient cost aggregation performance. The experimental results demonstrated that our method achieved state-of-the-art accuracy with reduced GPU memory usage and time consumption compared to the other methods. Furthermore, we illustrate that our method achieved satisfactory performance on real-world data, despite being trained on synthetic data, indicating its generalization potential and practical applicability.

Highlight removal is a fundamental and challenging task that has been an active field for decades. Although several methods have recently been improved for facial images, they are typically designed for a single image. This paper presents a lightweight optimization method for removing the specular highlight reflections of multi-view facial images. This is achieved by taking full advantage of the Lambertian consistency, which states that the diffuse component does not vary with the change in the viewing angle, while the specular component changes the behavior. We provide non-negative constraints on light and shading in all directions, rather than normal directions contained in the face, to obtain physically reliable properties. The removal of highlights is further facilitated through the estimation of illumination chromaticity, which is done by employing orthogonal subspace projection. An important practical feature of the proposed method does not require face reflectance priors. A dataset with ground truth for highlight removal of multi-view facial images is captured to quantitatively evaluate the performance of our method. We demonstrate the robustness and accuracy of our method through comparisons to existing methods for removing specular highlights and improvement in applications such as reconstruction.

Suspension bridges are supported by main cables that continue beyond the pillars to deck-level supports and must be anchored at each end of the bridge. The dynamic characteristics of the main cables are key indicators used to assess the structural health status of a bridge. In situ real-time health monitoring is an effective way to assess the dynamic characteristics. This paper presents a case study using vibration-based wireless smart sensors deployed on the main cables of a large-scale three-pylon suspension bridge to obtain its dynamic features. The methods of anti-aliasing filtering, statistical analysis and main cable tension force estimation were proposed and embedded into wireless smart sensors to provide autonomous data processing. According to the analysis of the vibration data from the main cables, the results demonstrate that the main cables have been in a stable state over time, and wireless smart sensors are promising for autonomous main-cable monitoring of large-scale three-pylon suspension bridges.

In this paper, a general function expansion bilinear (FEB) filter with a 3-D diagonal structure is proposed to deal with the problem of saturation nonlinearity in active noise control (ANC) systems. In an ANC system, the reference microphone and/or loudspeaker may be saturated when the acoustic noise and/or the controller output exceeds the dynamic limits of electronic devices. Such saturation nonlinearity degrades the control performance of the linear and some nonlinear control filters equipped with a filtered-x least mean square (FXLMS) algorithm. In order to tackle the problem of signal saturation, we use a new nonlinear saturation ANC (NSANC) model and derive a function expansion diagonal-structure bilinear FXLMS (FEDBFXLMS) algorithm. The performance of the proposed filters equipped with the associated FEDBFXLMS algorithm is validated through the analysis of computational complexity and simulations of various nonlinearities for NSANC systems. Computer simulation results demonstrate that the proposed FEB filter can achieve significant performance improvement in reducing saturation effects in comparison with the diagonal-channel bilinear filter and the recursive second-order Volterra filter based on the FXLMS algorithm, which often outperform the conventional nonlinear Volterra and functional link artificial neural network (FLANN) filters.

Cerebral micro-bleed (CMB) is small perivascular hemosiderin deposits from leakage through cerebral small vessels. They can result from cerebra-vascular disease, dementia, or simply from normal aging. It can be visualized via the susceptibility weighted imaging (SWI). Based on the SWI, we propose to use different structures of the CNN with rank-based average pooling to detect the CMB, and compare this method used in this paper to the current state-of-the-art methods. We can find that the CNN with five layers obtains the best performance, with a sensitivity of 96.94%, a specificity of 97.18%, and an accuracy of 97.18%.

Photoacoustic (PA) techniques provide optical absorption contrast and spatial information at an ultrasound resolution in deep biological tissues. Among the greatest challenges encountered in the PA examination of bone is the analysis of trabecular bone, which holds key chemical and physical information required for bone health assessments. Ultrasound detection is naturally registered with PA detection; therefore, in this study, we propose ultrasound guidance for the PA detection of trabecular bone. We perform both numerical simulations and an in vivo experiment on a human subject to investigate the possibility of ultrasound-guided detection and segmentation of photoacoustic signals from bone tissue in vivo in a non-invasive manner. The results obtained from the simulation and in vivo experiment suggest that the ultrasound-guided PA method can distinguish PA signals from trabecular and cortical bones as well as from the overlying soft tissue. Considering that the PA technique is non-ionizing and non-invasive, it holds potential for clinical bone health assessment.

We present a real-time Truncated Signed Distance Field (TSDF)-based three-dimensional (3D) semantic reconstruction for LiDAR point cloud, which achieves incremental surface reconstruction and highly accurate semantic segmentation. The high-precise 3D semantic reconstruction in real time on LiDAR data is important but challenging. Lighting Detection and Ranging (LiDAR) data with high accuracy is massive for 3D reconstruction. We so propose a line-of-sight algorithm to update implicit surface incrementally. Meanwhile, in order to use more semantic information effectively, an online attention-based spatial and temporal feature fusion method is proposed, which is well integrated into the reconstruction system. We implement parallel computation in the reconstruction and semantic fusion process, which achieves real-time performance. We demonstrate our approach on the CARLA dataset, Apollo dataset, and our dataset. When compared with the state-of-art mapping methods, our method has a great advantage in terms of both quality and speed, which meets the needs of robotic mapping and navigation.

ABSTRACT Axonal transport of mitochondria is critical for neuronal survival and function. Automatically quantifying and analyzing mitochondrial movement in a large quantity remain challenging. Here, we report an efficient method for imaging and quantifying axonal mitochondrial transport using microfluidic-chamber-cultured neurons together with a newly developed analysis package named “MitoQuant”. This tool-kit consists of an automated program for tracking mitochondrial movement inside live neuronal axons and a transient-velocity analysis program for analyzing dynamic movement patterns of mitochondria. Using this method, we examined axonal mitochondrial movement both in cultured mammalian neurons and in motor neuron axons of Drosophila in vivo. In 3 different paradigms (temperature changes, drug treatment and genetic manipulation) that affect mitochondria, we have shown that this new method is highly efficient and sensitive for detecting changes in mitochondrial movement. The method significantly enhanced our ability to quantitatively analyze axonal mitochondrial movement and allowed us to detect dynamic changes in axonal mitochondrial transport that were not detected by traditional kymographic analyses.

Photoacoustic imaging, with the capability to provide simultaneous structural, functional, and molecular information, is one of the fastest growing biomedical imaging modalities of recent times. As a hybrid modality, it not only provides greater penetration depth than the purely optical imaging techniques, but also provides optical contrast of molecular components in the living tissue. Conventionally, photoacoustic imaging systems utilize bulky and expensive class IV lasers, which is one of the key factors hindering the clinical translation of this promising modality. Use of LEDs which are portable and affordable offers a unique opportunity to accelerate the clinical translation of photoacoustics. In this paper, we first review the development history of LED as an illumination source in biomedical photoacoustic imaging. Key developments in this area, from point-source measurements to development of high-power LED arrays, are briefly discussed. Finally, we thoroughly review multiple phantom, ex-vivo, animal in-vivo, human in-vivo, and clinical pilot studies and demonstrate the unprecedented preclinical and clinical potential of LED-based photoacoustic imaging.

Accurate motion capture plays an important role in sports analysis, the medical field and virtual reality. Current methods for motion capture often suffer from occlusions, which limits the accuracy of their pose estimation. In this paper, we propose a complete system to measure the pose parameters of the human body accurately. Different from previous monocular depth camera systems, we leverage two Kinect sensors to acquire more information about human movements, which ensures that we can still get an accurate estimation even when significant occlusion occurs. Because human motion is temporally constant, we adopt a learning analysis to mine the temporal information across the posture variations. Using this information, we estimate human pose parameters accurately, regardless of rapid movement. Our experimental results show that our system can perform an accurate pose estimation of the human body with the constraint of information from the temporal domain.

Published methods for imaging and quantitatively analyzing morphological changes in neuronal axons have serious limitations because of their small sample sizes, and their time-consuming and nonobjective nature. Here we present an improved microfluidic chamber design suitable for fast and high-throughput imaging of neuronal axons. We developed the AxonQuant algorithm, which is suitable for automatic processing of axonal imaging data. This microfluidic chamber-coupled algorithm allows calculation of an ‘axonal continuity index' that quantitatively measures axonal health status in a manner independent of neuronal or axonal density. This method allows quantitative analysis of axonal morphology in an automatic and nonbiased manner. Our method will facilitate large-scale high-throughput screening for genes or therapeutic compounds for neurodegenerative diseases involving axonal damage. When combined with imaging technologies utilizing different gene markers, this method will provide new insights into the mechanistic basis for axon degeneration. Our microfluidic chamber culture-coupled AxonQuant algorithm will be widely useful for studying axonal biology and neurodegenerative disorders. © 2014 S. Karger AG, Basel

Non-contact human body measurement plays an important role in surveillance, physical healthcare, on-line business and virtual fitting. Current methods for measuring the human body without physical contact usually cannot handle humans wearing clothes, which limits their applicability in public environments. In this paper, we propose an effective solution that can measure accurate parameters of the human body with large-scale motion from a Kinect sensor, assuming that the people are wearing clothes. Because motion can drive clothes attached to the human body loosely or tightly, we adopt a space-time analysis to mine the information across the posture variations. Using this information, we recover the human body, regardless of the effect of clothes, and measure the human body parameters accurately. Experimental results show that our system can perform more accurate parameter estimation on the human body than state-of-the-art methods.

(Aim) Sensorineural hearing loss (SNHL) is correlated to many neurodegenerative disease. Now more and more computer vision based methods are using to detect it in an automatic way. (Materials) We have in total 49 subjects, scanned by 3.0T MRI (Siemens Medical Solutions, Erlangen, Germany). The subjects contain 14 patients with right-sided hearing loss (RHL), 15 patients with left-sided hearing loss (LHL), and 20 healthy controls (HC). (Method) We treat this as a three-class classification problem: RHL, LHL, and HC. Wavelet entropy (WE) was selected from the magnetic resonance images of each subjects, and then submitted to a directed acyclic graph support vector machine (DAG-SVM). (Results) The 10 repetition results of 10-fold cross validation shows 3-level decomposition will yield an overall accuracy of 95.10% for this three-class classification problem, higher than feedforward neural network, decision tree, and naive Bayesian classifier. (Conclusions) This computer-aided diagnosis system is promising. We hope this study can attract more computer vision method for detecting hearing loss.

Dendritic spines are described as neuronal protrusions. The morphology of dendritic spines and dendrites has a strong relationship to its function, as well as playing an important role in understanding brain function. Quantitative analysis of dendrites and dendritic spines is essential to an understanding of the formation and function of the nervous system. However, highly efficient tools for the quantitative analysis of dendrites and dendritic spines are currently undeveloped. In this paper we propose a novel three-step cascaded algorithm–RTSVM— which is composed of ridge detection as the curvature structure identifier for backbone extraction, boundary location based on differences in density, the Hu moment as features and Twin Support Vector Machine (TSVM) classifiers for spine classification. Our data demonstrates that this newly developed algorithm has performed better than other available techniques used to detect accuracy and false alarm rates. This algorithm will be used effectively in neuroscience research.

In one-shot color structured light systems, the color of stripe patterns are typically distorted with respect to color crosstalk, ambient light and the albedo of the scanned objects, leading to mismatch in the correspondence of color stripes between the projected and captured images. In this paper, an adaptive color calibration and Discrete Trend Transform algorithm are presented to achieve high-resolution 3D reconstructions. The adaptive color calibration, according to the relative albedo in RGB channels, can improve the accuracy of labeling stripe by alleviating the effect of albedo and ambient light while decoding the color. Furthermore, the Discrete Trend Transform in the M channel makes the color calibration an effective method for detecting weak stripes due to the uneven surfaces or reflectance characteristics of the scanned objects. With this approach, the presented system is suitable for scanning moving objects and generating high-resolution 3D reconstructions without the need of dark laboratory environments.

This paper focuses on developing a novel technique based on machine vision for detection of foreign substances in injections. Mechanical control yields spin/stop movement of injections which helps to cause relative movement between foreign substances in liquid and an ampoule bottle. Foreign substances are classified into two categories: subsiding-slowly object and subsiding-fast object. A sequence of frames are captured by a camera and used to recognize foreign substances. After image preprocessing like noise reduction and motion detection, two different methods, Moving-object Clustering (MC) and Frame Difference, are proposed to detect the two categories respectively. MC is operated to cluster subsiding-slowly foreign substances, based on the invariant features of those objects. Frame Difference is defined to calculate the difference between two frames due to the change of subsiding-fast objects. 200 ampoule samples filled with injection are tested and the experimental result indicates that the approach can detect the visible foreign substances effectively.

FUS-proteinopathies, a group of heterogeneous disorders including ALS-FUS and FTLD-FUS, are characterized by the formation of inclusion bodies containing the nuclear protein FUS in the affected patients. However, the underlying molecular and cellular defects remain unclear. Here we provide evidence for mitochondrial localization of FUS and its induction of mitochondrial damage. Remarkably, FTLD-FUS brain samples show increased FUS expression and mitochondrial defects. Biochemical and genetic data demonstrate that FUS interacts with a mitochondrial chaperonin, HSP60, and that FUS translocation to mitochondria is, at least in part, mediated by HSP60. Down-regulating HSP60 reduces mitochondrially localized FUS and partially rescues mitochondrial defects and neurodegenerative phenotypes caused by FUS expression in transgenic flies. This is the first report of direct mitochondrial targeting by a nuclear protein associated with neurodegeneration, suggesting that mitochondrial impairment may represent a critical event in different forms of FUS-proteinopathies and a common pathological feature for both ALS-FUS and FTLD-FUS. Our study offers a potential explanation for the highly heterogeneous nature and complex genetic presentation of different forms of FUS-proteinopathies. Our data also suggest that mitochondrial damage may be a target in future development of diagnostic and therapeutic tools for FUS-proteinopathies, a group of devastating neurodegenerative diseases.

We address the problem of classifying 3D point clouds: given 3D urban street scenes gathered by a lidar sensor, we wish to assign a class label to every point. This work is a key step toward realizing applications in robots and cars, for example. In this paper, we present a novel approach to the classification of 3D urban scenes based on super-segments, which are generated from point clouds by two stages of segmentation: a clustering stage and a grouping stage. Then, six effective normal and dimension features that vary with object class are extracted at the super-segment level for training some general classifiers. We evaluate our method both quantitatively and qualitatively using the challenging Velodyne lidar data set. The results show that by only using normal and dimension features we can achieve better recognition than can be achieved with high-dimensional shape descriptors. We also evaluate the adopting of the MRF framework in our approach, but the experimental results indicate that thisbarely improved the accuracy of the classified results due to the sparse property of the super-segments.

Recently manifold learning has received extensive interest in the community of pattern recognition. Despite their appealing properties, most manifold learning algorithms are not robust in practical applications. In this paper, we address this problem in the context of the Hessian locally linear embedding (HLLE) algorithm and propose a more robust method, called RHLLE, which aims to be robust against both outliers and noise in the data. Specifically, we first propose a fast outlier detection method for high-dimensional datasets. Then, we employ a local smoothing method to reduce noise. Furthermore, we reformulate the original HLLE algorithm by using the truncation function from differentiable manifolds. In the reformulated framework, we explicitly introduce a weighted global functional to further reduce the undesirable effect of outliers and noise on the embedding result. Experiments on synthetic as well as real datasets demonstrate the effectiveness of our proposed algorithm.

In this paper, we present a novel automatic pipeline to build personalized parametric models of dynamic people using a single RGB camera. Compared to previous approaches that use monocular RGB images, our system can model a 3D human body automatically and incrementally, taking advantage of human motion. Based on coarse 2D and 3D poses estimated from image sequences, we first perform a kinematic classification of human body parts to refine the poses and obtain reconstructed body parts. Next, a personalized parametric human model is generated by driving a general template to fit the body parts and calculating the non-rigid deformation. Experimental results show that our shape estimation method achieves comparable accuracy with reconstructed models using depth cameras, yet requires neither user interaction nor any dedicated devices, leading to the feasibility of using this method on widely available smart phones.