Search

Research on mobile robots began in the late 1960s. Mobile robots are a typical autonomous intelligent system and a hot spot in the high-tech field. They are the intersection of multiple technical disciplines such as computer artificial intelligence, robotics, control theory and electronic technology. The product not only has potentially very attractive application value and commercial value, but the research on it is also a challenge to intelligent technology. The development of mobile robots provides excellent research for various intelligent technologies and solutions. This dissertation aims to study the research of multi-sensor information fusion and intelligent optimization methods and the methods of applying them to mobile robot related technologies, and in-depth study of the construction of mobile robot maps from the perspective of multi-sensor information fusion. And, in order to achieve this function, combined with autonomous exploration and other related theories and algorithms, combined with the Robot Operating System (ROS). This paper proposes the area equalization method, equalization method, fuzzy neural network and other methods to promote the realization of related technologies. At the same time, this paper conducts simulation research based on the SLAM comprehensive experiment of the JNPF-4WD square mobile robot. On this basis, the high precision and high reliability of robot positioning are further realized. The experimental results in this paper show that the maximum error of the X-axis and Y-axis, FastSLAM algorithm is smaller than EKF algorithm, and the improved FASTSALM algorithm error is further reduced compared with the original FastSLAM algorithm, the value is less than 0.1.

The current navigation systems used in many autonomous mobile robotic applications, like unmanned vehicles, are always equipped with various sensors to get accurate navigation results. The key point is to fuse the information from different sensors efficiently. However, different sensors provide asynchronous measurements, some of which even appear to be nonlinear. Moreover, some sensors are vulnerable in specific environments, e.g., GPS signal is likely to work poorly in interior space, underground, and tall buildings. We propose a multi-sensor information fusion method based on a factor graph to fuse all available asynchronous sensor information and efficiently and accurately calculate a navigation solution. Assuming the sensor measurements and navigation states in a navigation system as factor nodes and variable nodes in a factor graph, respectively, the update of the states can be implemented in the framework of the factor graph. The proposed method is experimentally validated using two different datasets. A comparison with Federated Filter, which has been widely used in integrated navigation systems, demonstrates the proposed method's effectiveness. Additionally, analyzing the navigation results with data loss verifies that the proposed method could achieve sensor plug and play in software.

We propose a Recurrent Trend Predictive Neural Network (rTPNN) for multi-sensor fire detection based on the trend as well as level prediction and fusion of sensor readings. The rTPNN model significantly differs from the existing methods due to recurrent sensor data processing employed in its architecture. rTPNN performs trend prediction and level prediction for the time series of each sensor reading and captures trends on multivariate time series data produced by multi-sensor detector. We compare the performance of the rTPNN model with that of each of the Linear Regression (LR), Nonlinear Perceptron (NP), Multi-Layer Perceptron (MLP), Kendall- $\tau $ combined with MLP, Probabilistic Bayesian Neural Network (PBNN), Long-Short Term Memory (LSTM), and Support Vector Machine (SVM) on a publicly available fire data set. Our results show that rTPNN model significantly outperforms all of the other models (with 96% accuracy) while it is the only model that achieves high True Positive and True Negative rates (both above 92%) at the same time. rTPNN also triggers an alarm in only 11 s from the start of the fire, where this duration is 22 s for the second-best model. Moreover, we present that the execution time of rTPNN is acceptable for real-time applications.

Recently, development in intelligent transportation systems (ITS) requires the input of various kinds of data in real-time and from multiple sources, which imposes additional research and application challenges. Ongoing studies on Data Fusion (DF) have produced significant improvement in ITS and manifested an enormous impact on its growth. This paper reviews the implementation of DF methods in ITS to facilitate traffic flow analysis (TFA) and solutions that entail the prediction of various traffic variables such as driving behavior, travel time, speed, density, incident, and traffic flow. It attempts to identify and discuss real-time and multi-sensor data sources that are used for various traffic domains, including road/highway management, traffic states estimation, and traffic controller optimization. Moreover, it attempts to associate abstractions of data level fusion, feature level fusion, and decision level fusion on DF methods to better understand the role of DF in TFA and ITS. Consequently, the main objective of this paper is to review DF methods used for real-time and multi-sensor (heterogeneous) TFA studies. The review outcomes are (i) a guideline of constructing DF methods which involve preprocessing, filtering, decision, and evaluation as core steps, (ii) a description of the recent DF algorithms or methods that adopt real-time and multi-sensor sources data and the impact of these data sources on the improvement of TFA, (iii) an examination of the testing and evaluation methodologies and the popular datasets and (iv) an identification of several research gaps, some current challenges, and new research trends.

Use of a multi-sensor approach can provide citizens with holistic insights into the air quality of their immediate surroundings and their personal exposure to urban stressors. Our work, as part of the ICARUS H2020 project, which included over 600 participants from seven European cities, discusses the data fusion and harmonization of a diverse set of multi-sensor data streams to provide a comprehensive and understandable report for participants. Harmonizing the data streams identified issues with the sensor devices and protocols, such as non-uniform timestamps, data gaps, difficult data retrieval from commercial devices, and coarse activity data logging. Our process of data fusion and harmonization allowed us to automate visualizations and reports, and consequently provide each participant with a detailed individualized report. Results showed that a key solution was to streamline the code and speed up the process, which necessitated certain compromises in visualizing the data. A thought-out process of data fusion and harmonization of a diverse set of multi-sensor data streams considerably improved the quality and quantity of distilled data that a research participant received. Though automation considerably accelerated the production of the reports, manual and structured double checks are strongly recommended. This work has received funding from the European Union’s Horizon 2020 Programme for Research, Technological Development, and Demonstration, under grant agreement No. 690105 (Integrated Climate forcing and Air Pollution Reduction in Urban Systems (ICARUS)). This work reflects only the authors’ views, and the European Commission is not responsible for any use that may be made of the information it contains. Funding was received from the Young Researchers Program and the P1-0143 program “Cycling of substances in the environment, mass balances, modelling of environmental processes and risk assessment“, both funded by the Slovenian Research Agency. The authors thank RECETOX Research Infrastructure (No. LM2018121) and ACTRIS-CZ Research Infrastructure (No. LM2018122) financed by the Ministry of Education, Youth and Sports and the Operational Programme Research, Development and Innovation (CZ.02.1.01/0.0/0.0/16_013/0001315 and CZ.02.1.01/0.0/0.0/16_013/0001761), as well as the CETOCOEN Excellence Project, supported by the Teaming Action of the EU Horizon 2020 programme (857560) and the Operational Programme Research, Development and Innovation (No. CZ.02.1.01/0.0/0.0/17_043/0009632). Sí

Gait is a significant factor that affects human health, and monitoring a person's gait with sensing devices during daily life can detect abnormal gait events that affect numerous physical health problems. In particular, flat feet can cause changes in alignment conditions of the foot, ankle, leg, pelvis and spine. The primary problem with previous studies of wearable devices for measuring gait have focused on quantitatively monitoring the degree of gait rather than the limited gait ability. The existing method of feeding back the degree of gait or activity does not consider the severity of the subject and is insufficient for qualitative evaluation or training of gait. The significance of this study is development of convenient detecting and long-term tracking tools that can be used by both patients and clinicians for prescreening flat feet and monitoring the progress of flat feet treatment. For wearable devices for flatfoot detection to be most effective, detection systems and algorithms must be accurate, robust, reliable and computationally-efficient. In this paper, we developed an integrated smart wearable gait-monitoring device comprised of three sensors: front force, rear force, and an ankle flex sensor. We propose a new flat feet detection methodology based on a dynamic sensing window and a deep neural network with scaled principal component analysis (PCA). We tested 24 subjects, including both those with healthy gait and flat-feet-affected gait. Our study shows that the proposed sensing devices could be worn comfortably. The proposed deep neural network (DNN) model outperformed the other five classifier algorithms considered, and the area under the curve (AUC) value of the method was 87.1%. This wearable device can thus be easily and simply used both by patients and doctors to monitor the progress of flat feet and prescreen for possible gait problems in daily life.

Fusion technologies have rapidly evolved. These technologies are normally customized according to the needs of domains. Despite a large number of publications on intelligence fusion applications for various domains, they are scattered. The aim of this review is to present the state of the art for intelligence fusion applications within a specific domain. We identified three major domains for the purpose, namely robotics, military, and healthcare, during the initial process of the systematic review. These three domains are always in need of superior intelligence. Articles were searched mainly in IEEE Xplore. We limit the range of publications to the year 2014 to 2019, to focus on the most recent publications. We adopt the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol to screen, filter and evaluate qualities of each retrieved article. As a result, we retrieved 675 articles at the initial stage of the search, we conducted screening and filtering process and reviewed 153 articles potential articles, and finally, we excluded 36 articles as they do not comply with our quality assessment criteria. Only 117 articles are included. The results of this study are a list of classified applications within the domains and a number of relevant techniques or approaches used in each classified application. The finding of this review showed that the most published works for the use of intelligence fusion are mainly applications in the robotics domain, where mostly used techniques are Kalman Filter and its variants. Outcomes of this study can be a guideline or an insight for researchers to further develop and implement in this field.

Sleep recognition refers to detection or identification of sleep posture, state or stage, which can provide critical information for the diagnosis of sleep diseases. Most of sleep recognition methods are limited to single-task recognition, which only involves single-modal sleep data, and there is no generalized model for multi-task recognition on multi-sensor sleep data. Moreover, the shortage and imbalance of sleep samples also limits the expansion of the existing machine learning methods like support vector machine, decision tree and convolutional neural network, which lead to the decline of the learning ability and over-fitting. Self-supervised learning technologies have shown their capabilities to learn significant feature representations. In this paper, a novel self-supervised learning model is proposed for sleep recognition, which is composed of an upstream self-supervised pre-training task and a downstream recognition task. The upstream task is conducted to increase the data capacity, and the information of frequency domain and the rotation view are used to learn the multi-dimensional sleep feature representations. The downstream task is undertaken to fuse bidirectional long-short term memory and conditional random field as the sequential data recognizer to produce the sleep labels. Our experiments shows that our proposed algorithm provide promising results in sleep identification and can further be applied in clinical and smart home environments as a diagnostic tool. The source code is provided at: “https://github.com/zhaoaite/SSRM”.

The cueing model for sensor cueing phased array radar is built in this paper to research the mutual cueing of multi-sensor in multi-platform for target searching. For the purpose of establishing the distribution probability density model of targets in cueing airspace, cueing error of multi-platform is derived with error transfer method. The successful cueing probability and false hand-off probability between platforms are analyzed, meanwhile the requirement of timeliness is defined. An optimized target searching strategy of cueing radar which adopts distance search through using the cueing center as starting point and controlling false hand-off probability of a single wave position is put forward. The simulation result illustrates that the searching strategy can significantly save radar search resources to capture target successfully, simultaneously decrease the false hand-off probability. The strategy can be applied to the design of dynamic programming software of multi-sensor in multi-platform for collaborative search, which requires accuracy and real-time capability.

The research activities presented in this manuscript are focused on the documentation of a valuable built heritage asset: the Santa Marta bell tower (1769-1772), designed by the Italian architect Bernardo Antonio Vittone and located in the municipality of Montanaro (30 km North-East of Turin, Italy). The documentation of this complex palimpsest was designed to meet the requirements of the decay analysis and to provide a reference for the future restoration and valorisation project. To achieve these objectives a multi-scale and multi-sensor survey was designed and carried out exploiting several geomatics techniques (both range and image based). The fieldwork activities were firstly dedicated to the creation and measurement of a reference topographic network to be used as common local reference system for all the acquisitions as well as a series of control points (both inside and outside the bell towe) to be used for data orientation and accuracy assessment. Secondly, the exterior of the bell tower and its surroundings were imaged by means of an Unmanned Aerial Vehicle (UAV) and a set of Terrestrial Laser Scanning (TLS) acquisitions. The interior of the bell tower was sensed with two different rapid mapping approaches: using a handheld laser scanner based on the Simultaneous Localisation And Mapping (SLAM) technology and two different 360° cameras. All the acquired data were processed following both consolidated and experimental approaches and then integrated to generate traditional 2D architectural drawings supported by added value metric products. Of particular interest are the tests connected with the processing of the SLAM data and 360° images using a spherical photogrammetric approach that delivered preliminar encouraging results.

Human pose estimation and tracking in real-time from multi-sensor systems is essential for many applications. Combining multiple heterogeneous sensors increases opportunities to improve human motion tracking. Using only a single sensor type, e.g., inertial sensors, human pose estimation accuracy is affected by sensor drift over longer periods. This paper proposes a human motion tracking system using lidar and inertial sensors to estimate 3D human pose in real-time. Human motion tracking includes human detection and estimation of height, skeletal parameters, position, and orientation by fusing lidar and inertial sensor data. Finally, the estimated data are reconstructed on a virtual 3D avatar. The proposed human pose tracking system was developed using open-source platform APIs. Experimental results verified the proposed human position tracking accuracy in real-time and were in good agreement with current multi-sensor systems.

Accurate short-term small-area meteorological forecasts are essential to ensure the safety of operations and equipment operations in the Antarctic interior. This study proposes a deep learning-based multi-input neural network model to address this problem. The newly proposed model is predicted by combining a stacked autoencoder and a long- and short-term memory network. The self-stacking autoencoder maximises the features and removes redundancy from the target weather station&rsquo, s sensor data and extracts temporal features from the sensor data using a long- and short-term memory network. The proposed new model evaluates the prediction performance and generalisation capability at four observation sites at different East Antarctic latitudes (including the Antarctic maximum and the coastal region). The performance of five deep learning networks is compared through five evaluation metrics, and the optimal form of input combination is discussed. The results show that the prediction capability of the model outperforms the other models. It provides a new method for short-term meteorological prediction in a small inland Antarctic region.

Based on depth perception of individual stereo cameras, spatial structures can be derived as point clouds. Thequality of such three-dimensional data is technically restricted by sensor limitations, latency of recording, andinsufficient object reconstructions caused by surface illustration. Additionally external physical effects likelighting conditions, material properties, and reflections can lead to deviations between real and virtual objectperception. Such physical influences can be seen in rendered point clouds as geometrical imaging errors onsurfaces and edges. We propose the simultaneous use of multiple and dynamically arranged cameras. Theincreased information density leads to more details in surrounding detection and object illustration. During apre-processing phase the collected data are merged and prepared. Subsequently, a logical analysis part examinesand allocates the captured images to three-dimensional space. For this purpose, it is necessary to create a newmetadata set consisting of image and localisation data. The post-processing reworks and matches the locallyassigned images. As a result, the dynamic moving images become comparable so that a more accurate point cloudcan be generated. For evaluation and better comparability we decided to use synthetically generated data sets. Ourapproach builds the foundation for dynamic and real-time based generation of digital twins with the aid of realsensor data.

International audience; In recent years, due to advances in sensor technology, multi-modal measurement of process and products properties has become easier. However, multi-modal measurements are only of use if the data from adding new sensors is worthwhile, especially in the case of industrial applications where financial justification is needed for new sensor purchase and integration, and if the multi-modal data generated can be properly utilised. Several multi-block methods have been developed to do this; however, their use is largely limited to chemometricians, and non-experts have little experience with such methods. To deal with this, we present the first version of a MATLAB-based graphical user interface (GUI) for multi-block data analysis (MBA), capable of performing data visualisation, regression, classification and variable selection for up to 4 different sensors. The MBA-GUI can also be used to implement a recent technique called sequential pre-processing through orthogonalization (SPORT). Data sets are supplied to demonstrate how to use the MBA-GUI. In summary, the developed GUI makes the implementation of multi-block data analysis easier, so that it could be used also by practitioners with no programming skills or unfamiliar with the MATLAB environment. The fully functional GUI can be downloaded from (https://github.com/puneetmishra2/Multi-block.git) and can be either installed to run in the MATLAB environment or as a standalone executable program. The GUI can also be used for analysis of a single block of data (standard chemometrics).

ImaACor is a tool for the simultaneous correction of remotely sensed data from atmospheric and topographic effects, including second-order corrections, such as adjacency effects. The implemented approach is physically based and uses MODTRAN for computation of the main radiometric quantities. A user-friendly, comprehensive, and exhaustive graphic interface allows the user to choose from different correction levels. The various panels allow one to set all the parameters to correctly characterize the atmosphere and define the acquisition and illumination geometries. The tool provides a number of facilities to easily manage the correction process for a wide range of sensor data, including the ability to process multiple data in batch mode, which is very useful for dealing with temporal series. Under the inclusion of topographic correction, this tool allows the user to select a digital elevation model that is automatically resampled to the image resolution and processed to obtain the parameters for radiometric transfer modeling and the subsequent harmonization of the surface with the model inversion. This tool also includes utilities for the pre-processing of PRISMA data.

Mobile robot is a very important branch of robotics. In practice, the performance of mobile robot is required to be higher and higher. It is required that mobile robot can adapt to different complex environments through its own intelligent system to achieve the established functional goals. Compared with traditional path planning with single sensor information, Kalman filter is used to fuse multi-sensor information, and path planning method based on improved dynamic artificial potential field method is studied. Using the improved dynamic artificial potential field method, the robot can also achieve optimal path planning and obstacle avoidance in complex dynamic environment. The simulation results show that the proposed algorithm is feasible.

Brain source imaging and time frequency mapping (TFM) are commonly used in magneto/electro encephalography (M/EEG) imaging. However, these methods suffer from important limitations. Source imaging is based on an ill-posed inverse problem leading to instability of source localization solutions, has a limited capacity to localize high frequency oscillations and loses its robustness for induced responses (ill-defined trigger). The drawback of TFM is that it involves independent analysis of signals from a number of frequency bands, and from co-localized sensors. In the present article, a regression-based multi-sensor space&ndash, time&ndash, frequency analysis (MSA) approach, which integrates co-localized sensors and/or multi-frequency information, is proposed. To estimate task-specific brain activations, MSA uses cross-validated, shifted, multiple Pearson correlation, calculated from the time&ndash, frequency transformed brain signal and the binary signal of stimuli. The results are projected from the sensor space onto the cortical surface. To assess MSA performance, the proposed method was compared to the weighted minimum norm estimate (wMNE) source imaging method, in terms of spatial selectivity and robustness against an ill-defined trigger. Magnetoencephalography (MEG) recordings were performed in fourteen subjects during two motor tasks: finger tapping and elbow flexion/extension. In particular, our results show that the MSA approach provides good localization performance when compared to wMNE and statistically significant improvement of robustness against ill-defined trigger.

As a result of the dependence worldwide on satellite technology, it is now necessary to use advanced multi-target tracking algorithms for space debris tracking systems to maintain custody of space objects around the earth. One principal challenge is the correct association of observations with objects. This paper presents a multi-sensor, space-debris tracking algorithm using δ-generalized labeled multi-Bernoulli (δ-GLMB) filtering. The algorithm provides a solution to the key challenges (e.g., detection uncertainty, data association uncertainty, and clutter) in multiple object tracking. An efficient implementation of the multi-sensor δ-GLMB filter is proposed. In order to avoid exhaustively computing all the terms, we propose to use the ranked assignment algorithm with an extended assignment matrix for multiple sensors to determine the most significant terms. A measurement-based birth model is used to identify the previously unknown space objects. Sensors can have the same or different observation volumes. The expectation-maximization (EM) algorithm is used to approximate densities across observation volumes. The performance is demonstrated using the simulation results.

Internet of Moving Things are connected to a variety of different types of sensors to form a world of moving things, including people, animals, vehicles, drones, and boats, etc. As the data of collectible moving things continue to increase, anomaly detection of moving things has become an increasingly popular data mining task. Traditional trajectory outlier detection algorithms can detect common anomalies effectively, but it is hard to detect generalized anomalies, such as viewable direction anomalies, gravity anomalies, and magnetic field anomalies which can be collected by the accelerometer, gyroscope, magnetometer, and RPM sensor, etc. For this, we proposed a generalized approach for anomaly detection from the Internet of Moving Things, called the moving things outlier detection algorithm (MTOD). We propose the distance of moving things, which is equal to the weighted sum of the location distance and the multi-sensor distance, and then use the multi-sensor data generalization and moving things partitioning and anomaly detection three-step framework to detect the generalized anomaly. The experimental results show that our MTOD algorithm can detect moving things anomaly efficiency and accurately.

This paper presents an effective multi-sensor approach for improving the accuracy of laser scanner by using a tactile probe to perform rapid and accurate reverse engineering of complex objects. The proposed system is unique in that it includes not only the physical integration of the two digitizers but also their combination at the measurement information level. With the coordinate data acquired using the optical scanner, intelligent data segmentation and feature recognition algorithm are proposed to divide the original point set into geometric elements and free-form surfaces. The tactile probe is guided to re-measure each feature with a small number of sampling points. Then different data fusion algorithms are proposed to compensate the scanned data patches of geometric features and free-form features with the accurate tactile probing data. Finally, the compensated point data can be exploited for accurate reverse engineering of a CAD model. Through the fusion of different sensor's information, both sensors complement each other with their advantages. Experimental results show that for the measurement of geometric surface, the proposed method can improve the accuracy of optical measurement to the accuracy of contact measurement. For free-form surface measurement, the proposed method reduces the optical measurement error from +0.032/-0.019 mm to ±0.016 mm, and the standard deviation from 0.012 mm to 0.004 mm. The measurement efficiency of proposed method is 4.5 times higher than that of contact measurement alone.

Tool wear monitoring is a typical multi-sensor information fusion task. The handcrafted features may be a suboptimal choice that will lower the monitoring accuracy and require significant computational costs that hinder the real-time applications. In order to solve these problems, this paper proposed a new multisensory data-driven tool wear predicting method based on reshaped time series convolutional neural network (RTSCNN). In this method, the reshaped time series layer is introduced to represent the multisensory raw signals, the alternately convolutional and pooling layers is employed to adaptively learn distinctive characteristics of tool wear directly from multisensory raw signals while the multi-layer perceptron with regression layer performs automatic tool wear prediction. In addition, three tool run-to-failure datasets measured from three-flute ball nose tungsten carbide cutter of high-speed CNC machine under milling operations are used to experimentally demonstrate the performance of the proposed RTSCNN-based multisensory data-driven tool wear predicting method. The experimental results show that the prediction error of the RTSCNN-based data-driven method is observably lower than other state-of-art methods.

Virgin polymers based on petrochemical feedstock are mainly preferred by most plastic goods manufacturers instead of recycled plastic feedstock. Major reason for this is the lack of reliable information about the quality, suitability, and availability of recycled plastics, which is partly due to lack of proper segregation techniques. In this paper, we present our ongoing efforts to segregate plastics based on its types and improve the reliability of information about recycled plastics using the first-of-its-kind blockchain smart contracts powered by multi-sensor data-fusion algorithms using artificial intelligence. We have demonstrated how different data-fusion modes can be employed to retrieve various physico-chemical parameters of plastic waste for accurate segregation. We have discussed how these smart tools help in efficiently segregating commingled plastics and can be reliably used in the circular economy of plastic. Using these tools, segregators, recyclers, and manufacturers can reliably share data, plan the supply chain, execute purchase orders, and hence, finally increase the use of recycled plastic feedstock.

Human body scanning is an important means to build a digital 3D model of the human body, which is the basis for intelligent clothing production, human obesity analysis, and medical plastic surgery applications, etc. Comparing to commonly used optical scanning technologies such as laser scanning and fringe structured light, infrared laser speckle projection-based 3D scanning technology has the advantages of single-shot, simple control, and avoiding light stimulation to human eyes. In this paper, a multi-sensor collaborative digital human body scanning system based on near-infrared laser speckle projection is proposed, which occupies less than 2 m2 and has a scanning period of about 60 s. Additionally, the system calibration method and control scheme are proposed for the scanning system, and the serial-parallel computing strategy is developed based on the unified computing equipment architecture (CUDA), so as to realize the rapid calculation and automatic registration of local point cloud data. Finally, the effectiveness and time efficiency of the system are evaluated through anthropometric experiments.

As the world population increases and the need for food monoculture farms are using more and more agrochemicals, there is also an increase in the possibility of theft, misuse, environmental damage, piracy of products, and health problems. This article addresses these issues by introducing the agrochemical pervasive traceability model (APTM), which integrates machine learning, sensors, microcontrollers, gamification, and two blockchains. It contributes in two dimensions: (I) the study of the environmental, product piracy and regulatory of agrochemical control, (II) the technological dimension: application of an adequate set of sensors collecting multiple data, modeling and implementation of a system via machine learning for analyzing and predicting the behavior and use of agrochemicals, development of a scoring system via gamification for reverse use of agrochemicals, and presenting a record of transactions in a consortium of two blockchains, simultaneously. Its main advantage is to be a flexible, adaptable, and expansive model. Results indicated that the model has positive aspects, from detecting the agrochemical, its handling, and disposal, recording of transactions, and data visualization along the reverse supply chain. This study obtained a round trip time of 0.510 ms on average, data transfers between layer one and its persistence in the database were between 4 to 5 s. Thus, blockchain nodes consumed only 34 to 38% of CPU and recorded transactions between 2 to 4 s. These results point to a horizon of applicability in real situations within agricultural farms.

Forest border crossing animals creates major societal related issues, in addition to endangering their own lives. This is the objective focused in this paper targeting the species “The Elephant”, incorporating with technical methodologies namely, multi-sensor data fusion, cognition theories and computational intelligence techniques. Multi-sensor data fusion provides three level detection of target, along with its related outputs, which improves performance metrics. Cognition theory resulted in obtaining other interesting features about the target. Computational intelligence techniques integrate and conclude the presence of the target in the pseudo-boundary. The technical combination enhances the novelty of the research work, resulting in achieving remarkable accuracy and minimized false alert. An IoT kit was designed and deployed in the real time wild environment in Hosur forest region for collecting the data of Elephant. Further, the notification is sent to the registered mobile of the forest authority, as an early warning for chasing the pachyderm back to the forest.

Usage of Unmanned Aerial Vehicles (UAVs) is growing rapidly in a wide range of consumer applications, as they prove to be both autonomous and flexible in a variety of environments and tasks. However, this versatility and ease of use also brings a rapid evolution of threats by malicious actors that can use UAVs for criminal activities, converting them to passive or active threats. The need to protect critical infrastructures and important events from such threats has brought advances in counter UAV (c-UAV) applications. Nowadays, c-UAV applications offer systems that comprise a multi-sensory arsenal often including electro-optical, thermal, acoustic, radar and radio frequency sensors, whose information can be fused to increase the confidence of threat’s identification. Nevertheless, real-time surveillance is a cumbersome process, but it is absolutely essential to detect promptly the occurrence of adverse events or conditions. To that end, many challenging tasks arise such as object detection, classification, multi-object tracking and multi-sensor information fusion. In recent years, researchers have utilized deep learning based methodologies to tackle these tasks for generic objects and made noteworthy progress, yet applying deep learning for UAV detection and classification is considered a novel concept. Therefore, the need to present a complete overview of deep learning technologies applied to c-UAV related tasks on multi-sensor data has emerged. The aim of this paper is to describe deep learning advances on c-UAV related tasks when applied to data originating from many different sensors as well as multi-sensor information fusion. This survey may help in making recommendations and improvements of c-UAV applications for the future.

To release the strong dependence of the conventional inertial navigation mechanization on the a priori low-cost inertial measurement unit (IMU) error model, this research applies an unconventional multi-sensor integration strategy to integrate multiple low-cost IMUs and a global positioning system (GPS) for mass-market automotive applications. The unconventional integration strategy utilizes a basic three-dimensional (3D) kinematic trajectory model as the system model to directly estimate navigational parameters, and it allows the measurements from all of the sensors independently participating in measurement updates. However, the less complex kinematic model cannot realize smooth transitions between different motion statuses for the road vehicle with acceleration maneuvers. In this manuscript, we establish a more practical 3D kinematic trajectory model based on a &ldquo, current&rdquo, statistical Singer acceleration model to realize smooth transitions for the maneuvering vehicle. In addition, taking advantage of the unconventional strategy, we individually model the systematic errors of each IMU and the measurements of all sensors, in contrast to most existing approaches that adopt the common-mode errors for different sensors of the same design. A real dataset involving a GPS and multiple IMUs is processed to validate the success of the proposed algorithm model under the unconventional integration strategy.

Sericulture is traditionally a labor-intensive rural-based industry. In modern contexts, the development of process automation faces new challenges related to quality and efficiency. During the silkworm farming life cycle, a common issue is represented by the gender classification of the cocoons. Improper cocoon separation negatively affects quantity and quality of the yield resulting in disruptive bottlenecks for the productivity. To tackle this issue, this paper proposes a multi sensor system for silkworm cocoons gender classification and separation. Utilizing a load sensor and a digital camera, the system acquires weight and digital images from individual silkworm cocoons. An image processing procedure is then applied to extract significant shape-related features from each image instance, which, combined with the weight data, are provided as inputs to train a Support Vector Machine-based pattern classifier for gender classification. Subsequently, an air blower mechanism and a conveyor system sort the cocoons into their respective bins. The developed system was trained and tested on two different types of silkworm cocoons breeds, respectively CSR2 and Pure Mysore. The system performances are finally discussed in terms of accuracy, robustness and computation time.

With the ability to provide long range, highly accurate 3D surrounding measurements, while lowering the device cost, non-repetitive scanning Livox lidars have attracted considerable interest in the last few years. They have seen a huge growth in use in the fields of robotics and autonomous vehicles. In virtue of their restricted FoV, they are prone to degeneration in feature-poor scenes and have difficulty detecting the loop. In this paper, we present a robust multi-lidar fusion framework for self-localization and mapping problems, allowing different numbers of Livox lidars and suitable for various platforms. First, an automatic calibration procedure is introduced for multiple lidars. Based on the assumption of rigidity of geometric structure, the transformation between two lidars can be configured through map alignment. Second, the raw data from different lidars are time-synchronized and sent to respective feature extraction processes. Instead of sending all the feature candidates for estimating lidar odometry, only the most informative features are selected to perform scan registration. The dynamic objects are removed in the meantime, and a novel place descriptor is integrated for enhanced loop detection. The results show that our proposed system achieved better results than single Livox lidar methods. In addition, our method outperformed novel mechanical lidar methods in challenging scenarios. Moreover, the performance in feature-less and large motion scenarios has also been verified, both with approvable accuracy.

Investigating the model ship dynamic positioning system by simulating the actual sea conditions in the laboratory can not only avoid the risks caused by the directly experiments on a true ship, but also reduce the costs. With the purpose of realizing the high accuracy control of the dynamic positioning, besides a high accuracy mathematical model of the ship, an important condition is that the position information provided by the position detection system must be accurate, reliable and continuous. The global positioning system (GPS) signal is restricted when the model ship dynamic positioning system is set indoors. This paper describes a novel scheme for ship target tracking based on the multi-sensor data fusion techniques. To improve the accuracy of indoor positioning and ship target tracking, the characteristics of many sensors are systematically analyzed, such as radar, difference global positioning system (DGPS) and ultrasonic sensors. Other important factors, including the indoor temperature, position and environment, are also taken into account to further optimize the performance. Combining the Kalman filter method, the time alignment method, the coordinate transformation method and the optimal fusion criterion method, the core algorithm of our framework employs the track correlation as the performance index of the optimal fusion. The experimental results indicate that our method outperforms the methods based on a single ultrasonic sensor. The maximum error between the estimated location and the real location is only 1.32 cm, which meets the standard for engineering applications.

This paper presents a project of recording and modelling tunnels, traffic circles and roads from multiple sensors. The aim is the representation and the accurate 3D modelling of a selection of road infrastructures as dense point clouds in order to extract profiles and metrics from it. Indeed, these models will be used for the sizing of infrastructures in order to simulate exceptional convoy truck routes. The objective is to extract directly from the point clouds the heights, widths and lengths of bridges and tunnels, the diameter of gyrating and to highlight potential obstacles for a convoy. Light, mobile and fast acquisition approaches based on images and videos from a set of synchronized sensors have been tested in order to obtain useable point clouds. The presented solution is based on a combination of multiple low-cost cameras designed on an on-boarded device allowing dynamic captures. The experimental device containing GoPro Hero4 cameras has been set up and used for tests in static or mobile acquisitions. That way, various configurations have been tested by using multiple synchronized cameras. These configurations are discussed in order to highlight the best operational configuration according to the shape of the acquired objects. As the precise calibration of each sensor and its optics are major factors in the process of creation of accurate dense point clouds, and in order to reach the best quality available from such cameras, the estimation of the internal parameters of fisheye lenses of the cameras has been processed. Reference measures were also realized by using a 3D TLS (Faro Focus 3D) to allow the accuracy assessment.

Data labeling has become a major problem in industries aiming to create and use ground truth labels from massive multi-sensor archives to feed into Artificial Intelligence (AI) applications. Annotation of multi-sensor set-ups with multiple cameras and LIDAR is now particularly relevant for the automotive industry aiming to build Autonomous Driving (AD) functions. In this paper, we present the Video Content Description (VCD), as the first open source metadata structure and set of tools, able to structure annotations for such complex scenes, including unprecedented flexibility to label 2D and 3D objects, pixel-wise labels, actions, events, contexts, semantic relations, odometry, and calibration. Several example cases are reported to demonstrate the flexibility of the VCD.

Internet of things (IoT) cloud-based applications deliver advanced solutions for smart cities to decrease traffic accidents caused by driver fatigue while driving on the road. Environmental conditions or driver behavior can ultimately lead to serious roadside accidents. In recent years, the authors have developed many low-cost, computerized, driver fatigue detection systems (DFDs) to help drivers, by using multi-sensors, and mobile and cloud-based computing architecture. To promote safe driving, these are the most current emerging platforms that were introduced in the past. In this paper, we reviewed state-of-the-art approaches for predicting unsafe driving styles using three common IoT-based architectures. The novelty of this article is to show major differences among multi-sensors, smartphone-based, and cloud-based architectures in multimodal feature processing. We discussed all of the problems that machine learning techniques faced in recent years, particularly the deep learning (DL) model, to predict driver hypovigilance, especially in terms of these three IoT-based architectures. Moreover, we performed state-of-the-art comparisons by using driving simulators to incorporate multimodal features of the driver. We also mention online data sources in this article to test and train network architecture in the field of DFDs on public available multimodal datasets. These comparisons assist other authors to continue future research in this domain. To evaluate the performance, we mention the major problems in these three architectures to help researchers use the best IoT-based architecture for detecting DFDs in a real-time environment. Moreover, the important factors of Multi-Access Edge Computing (MEC) and 5th generation (5G) networks are analyzed in the context of deep learning architecture to improve the response time of DFD systems. Lastly, it is concluded that there is a research gap when it comes to implementing the DFD systems on MEC and 5G technologies by using multimodal features and DL architecture.

Three-dimensional (3D) object detection has important applications in robotics, automatic loading, automatic driving and other scenarios. With the improvement of devices, people can collect multi-sensor/multimodal data from a variety of sensors such as Lidar and cameras. In order to make full use of various information advantages and improve the performance of object detection, we proposed a Complex-Retina network, a convolution neural network for 3D object detection based on multi-sensor data fusion. Firstly, a unified architecture with two feature extraction networks was designed, and the feature extraction of point clouds and images from different sensors realized synchronously. Then, we set a series of 3D anchors and projected them to the feature maps, which were cropped into 2D anchors with the same size and fused together. Finally, the object classification and 3D bounding box regression were carried out on the multipath of fully connected layers. The proposed network is a one-stage convolution neural network, which achieves the balance between the accuracy and speed of object detection. The experiments on KITTI datasets show that the proposed network is superior to the contrast algorithms in average precision (AP) and time consumption, which shows the effectiveness of the proposed network.

The fast progress in research and development of multifunctional, distributed sensor networks has brought challenges in processing data from a large number of sensors. Using deep learning methods such as convolutional neural networks (CNN), it is possible to build smarter systems to forecasting future situations as well as precisely classify large amounts of data from sensors. Multi-sensor data from atmospheric pollutants measurements that involves five criteria, with the underlying analytic model unknown, need to be categorized, so do the Diabetic Retinopathy (DR) fundus images dataset. In this work, we created automatic classifiers based on a deep convolutional neural network (CNN) with two models, a simpler feedforward model with dual modules and an Inception Resnet v2 model, and various structural tweaks for classifying the data from the two tasks. For segregating multi-sensor data, we trained a deep CNN-based classifier on an image dataset extracted from the data by a novel image generating method. We created two deepened and one reductive feedforward network for DR phase classification. The validation accuracies and visualization results show that increasing deep CNN structure depth or kernels number in convolutional layers will not indefinitely improve the classification quality and that a more sophisticated model does not necessarily achieve higher performance when training datasets are quantitatively limited, while increasing training image resolution can induce higher classification accuracies for trained CNNs. The methodology aims at providing support for devising classification networks powering intelligent sensors.

Freeform parameterizations to reproduce structure deformation are increasingly important topics in laser-scanner-based deformation analyses. High-accuracy assurance of free-form surface approximati...

Individuals with mobility impairments related to age, injury, or disease, often require the help of an assistive device (AD) such as a cane to ambulate, increase safety, and improve overall stability. Instrumenting these devices has been proposed as a non-invasive way to proactively monitor an individual&rsquo, s reliance on the AD while also obtaining information about behaviors and changes in gait. A critical first step in the analysis of these data, however, is the accurate processing and segmentation of the sensor data to extract relevant gait information. In this paper, we present a highly accurate multi-sensor-based gait segmentation algorithm that is robust to a variety of walking conditions using an AD. A matched filtering approach based on loading information is used in conjunction with an angular rate reversal and peak detection technique, to identify important gait events. The algorithm is tested over a variety of terrains using a hybrid sensorized cane, capable of measuring loading, mobility, and stability information. The reliability and accuracy of the proposed multi-sensor matched filter (MSMF) algorithm is compared with variations of the commonly employed gyroscope peak detection (GPD) algorithm. Results of an experiment with a group of 30 healthy participants walking over various terrains demonstrated the ability of the proposed segmentation algorithm to reliably and accurately segment gait events.

Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quarantined by mapping (e.g., GPS) or via beacons that delineate a no-entry area. We propose a delineation method where the industrial vehicle utilizes a LiDAR (Light Detection and Ranging) and a single color camera to detect passive beacons and model-predictive control to stop the vehicle from entering a restricted space. The beacons are standard orange traffic cones with a highly reflective vertical pole attached. The LiDAR can readily detect these beacons, but suffers from false positives due to other reflective surfaces such as worker safety vests. Herein, we put forth a method for reducing false positive detection from the LiDAR by projecting the beacons in the camera imagery via a deep learning method and validating the detection using a neural network-learned projection from the camera to the LiDAR space. Experimental data collected at Mississippi State University&rsquo, s Center for Advanced Vehicular Systems (CAVS) shows the effectiveness of the proposed system in keeping the true detection while mitigating false positives.

Registration is a critical step in multi-sensor dimensional measurement. As the accuracy of registration directly impacts the quality of final results, a reference sphere as a common standard is problematic in high-precision registration. In this paper, a novel method based on a composite standard is proposed to fuse the multiple heterogeneous sensors in high-precision coordinate measuring machines (CMMs), which will void the drawbacks of a reference sphere. The composite standard consists of a cone and cylinder, which share a same central axis. To ensure high precision in the submicron range, or better, the standard is manufactured by an ultra-precision machine. Three features of the composite standard are inspected by three sensors: a video camera (VC), a tactile probe (TP), and a chromatic confocal displacement sensor (CC). All features will concentrate on a common point through which the relation between the three sensors will be obtained. The errors of each measurement were analyzed theoretically, and simulations and real experiments were carried out to verify the composite standard. This study demonstrates that the proposed registration method is stable and that the standard has potential use for the registration of multiple sensors in high-precision dimensional measurement.

This paper presents a resolution-reconfigurable wide-range resistive sensor readout interface for wireless multi-gas monitoring applications that displays results on a smartphone. Three types of sensing resolutions were selected to minimize processing power consumption, and a dual-mode front-end structure was proposed to support the detection of a variety of hazardous gases with wide range of characteristic resistance. The readout integrated circuit (ROIC) was fabricated in a 0.18 μm CMOS process to provide three reconfigurable data conversions that correspond to a low-power resistance-to-digital converter (RDC), a 12-bit successive approximation register (SAR) analog-to-digital converter (ADC), and a 16-bit delta-sigma modulator. For functional feasibility, a wireless sensor system prototype that included in-house microelectromechanical (MEMS) sensing devices and commercial device products was manufactured and experimentally verified to detect a variety of hazardous gases.

As the number of space-borne SAR sensors increases, a rising number of different SAR acquisition modes is in use, resulting in a higher variation within the image products. This variability in acquisition geometry, radiometry, and last but not least polarimetry raises the need for a consistent SAR image description incorporating all available sensors and acquisition modes. This paper therefore introduces the framework of the Kennaugh elements to comparably represent all kinds of multi-scale, multi-temporal, multi-polarized, multi-frequency, and hence, multi-sensor data in a consistent mathematical framework. Furthermore, a novel noise model is introduced that estimates the significance and thus the (polarimetric) information content of the Kennaugh elements. This facilitates an advanced filtering approach, called multi-scale multi-looking, which is shown to improve the radiometric accuracy while preserving the geometric resolution of SAR images. The proposed methodology is finally demonstrated using sample applications that include TerraSAR-X (X-band), Envisat-ASAR, RADARSAT-2 (C-band) and ALOS-PALSAR (L-band) data as well as the combination of all three frequencies. Thus the suitability of the Kennaugh element framework for practical use in proved for advanced SAR remote sensing.

In this paper, we focus on UAV-borne laser scanning with the objective of densely sampling object surfaces in the local surrounding of the UAV. In this regard, using a line scanner which scans along the vertical direction and perpendicular to the flight direction results in a point cloud with low point density if the UAV moves fast. Using a line scanner which scans along the horizontal direction only delivers data corresponding to the altitude of the UAV and thus a low scene coverage. For these reasons, we present a concept and a system for UAV-borne laser scanning using multiple line scanners. Our system consists of a quadcopter equipped with horizontally and vertically oriented line scanners. We demonstrate the capabilities of our system by presenting first results obtained for a flight within an outdoor scene. Thereby, we use a downsampling of the original point cloud and different neighborhood types to extract fundamental geometric features which in turn can be used for scene interpretation with respect to linear, planar or volumetric structures.

Observation of the climate and ecosystem of ice covered polar seas is a timely task for the scientific community. The goal is to assess the drastic and imminent changes of the polar sea ice cover induced by climate change. Retreating and thinning sea ice affects the planets energy budget, atmospheric, and oceanic circulation patterns as well as the ecosystem associated with this unique habitat. To increase the observational capabilities of sea ice scientists, we equipped a remotely operated vehicle (ROV) as sensor platform for interdisciplinary research at the ice water interface. Here, we present the technical details and operation scheme of the new vehicle and provide data examples from a first campaign in the Arctic in autumn 2016 to demonstrate the vehicle's capabilities. The vehicle is designed for efficient operations in the harsh polar conditions. Redundant modular design allows operation by three scientists simultaneously operating a wide variety of sensors. Sensors from physical, chemical, and biological oceanography are combined with optical and acoustic sea ice sensors to provide a comprehensive picture of the underside of sea ice. The sensor suite provides comprehensive capabilities and can be further extended as additional ports for power and communication are available. The vehicle provides full six degrees of freedom in navigation, enabling intervention, and manipulation skills despite its simple one function manipulator arm.

An object-based method is proposed in this paper for change detection in urban areas with multi-sensor multispectral (MS) images. The co-registered bi-temporal images are resampled to match each other. By mapping the segmentation of one image to the other, a change map is generated by characterizing the change probability of image objects based on the proposed change feature analysis. The map is then used to separate the changes from unchanged areas by two threshold selection methods and k-means clustering (k = 2). In order to consider the multi-scale characteristics of ground objects, multi-scale fusion is implemented. The experimental results obtained with QuickBird and IKONOS images show the superiority of the proposed method in detecting urban changes in multi-sensor MS images.

Measurement of water content in oil-water mixing flow was restricted by special problems such as narrow measuring range and low accuracy. A simulated multi-sensor measurement system in the laboratory was established, and the influence of multi-factor such as temperature, and salinity content on the measurement was investigated by numerical simulation combined with experimental test. A soft measurement model based on rough set-support vector machine (RS-SVM) classifier and genetic algorithm-neural network (GA-NN) predictors was reported in this paper. Investigation results indicate that RS-SVM classifier effectively realized the pattern identification for water holdup states via fuzzy reasoning and self-learning, and GA-NN predictors are capable of subsection forecasting water content in the different water holdup patterns, as well as adjusting the model parameters adaptively in terms of online measuring range. Compared with the actual laboratory analyzed results, the soft model proposed can be effectively used for estimating the water content in oil-water mixture in all-round measuring range

In order to perform an optical assembly accurately, a multi-sensor control strategy is developed which includes an attitude measurement system, a vision system, a loss measurement system and a force sensor. A 3-DOF attitude measuring method using linear variable differential transformers (LVDT) is designed to adjust the relation of position and attitude between the spherical mirror and the resonator. A micro vision feedback system is set up to extract the light beam and the diaphragm, which can achieve the coarse positioning of the spherical mirror in the optical assembly process. A rapid self-correlation method is presented to analyze the spectrum signal for the fine positioning. In order to prevent the damage of the optical components and realize sealing of the resonator, a hybrid force-position control is constructed to control the contact force of the optical components. The experimental results show that the proposed multi-sensor control strategy succeeds in accomplishing the precise assembly of the optical components, which consists of parallel adjustment, macro coarse adjustment, macro approach, micro fine adjustment, micro approach and optical contact. Therefore, the results validate the multi-sensor control strategy.

The work reported in this paper aims at utilizing the global geometrical relationship and local shape feature to register multi-spectral images for fusion-based face recognition. We first propose a multi-spectral face images registration method based on both global and local structures of feature point sets. In order to combine the global geometrical relationship and local shape feature in a new Student&rsquo, s t Mixture probabilistic model framework. On the one hand, we use inner-distance shape context as the local shape descriptors of feature point sets. On the other hand, we formulate the feature point sets registration of the multi-spectral face images as the Student&rsquo, s t Mixture probabilistic model estimation, and local shape descriptors are used to replace the mixing proportions of the prior Student&rsquo, s t Mixture Model. Furthermore, in order to improve the anti-interference performance of face recognition techniques, a guided filtering and gradient preserving image fusion strategy is used to fuse the registered multi-spectral face image. It can make the multi-spectral fusion image hold more apparent details of the visible image and thermal radiation information of the infrared image. Subjective and objective registration experiments are conducted with manual selected landmarks and real multi-spectral face images. The qualitative and quantitative comparisons with the state-of-the-art methods demonstrate the accuracy and robustness of our proposed method in solving the multi-spectral face image registration problem.

A major research area in remote sensing is the problem of multi-sensor data fusion. Especially the combination of images acquired by different sensor types, e.g. active and passive, is a difficult task. Over the last years deep learning methods have proven their high potential for remote sensing applications. In this paper we will show how a deep learning method can be valuable for the problem of optical and SAR image matching. We investigate the possible of conditional generative adversarial networks (cGANs) for the generation of artificial templates. Contrary to common template generation approaches for image matching, the generation of templates using cGANs does not require the extraction of features. Our results show the possibility of realistic SAR-like template generation from optical images through cGANs and the potential of these templates for enhancing the matching of optical and SAR images by means of reliability and accuracy.

Optical Multi-Sensor-Measurements in the shop are often affected by external influences which dominate the quality of the measurements and disable low uncertainties. In this approach simulations are carried out to estimate the uncertainty contributions when the gravity point is set on external influences as they dominate production environments. The results are gained by different simulation types and are validated on two applications. The systematic contributions on the measurement uncertainty can be partly compensated a priori and the needed test time as well as efforts for modifications of the prototypic measurement systems can be reduced. So, the presented method leads to reduced costs for designing and implementing optical multi-sensor-metrology for efficient production.

The cementing manufacturing process of ferrite phase shifters has the defect that cementing strength is insufficient and fractures always appear. A detection method of these defects was studied utilizing the multi-sensors Prognostic and Health Management (PHM) theory. Aiming at these process defects, the reasons that lead to defects are analyzed in this paper. In the meanwhile, the key process parameters were determined and Differential Scanning Calorimetry (DSC) tests during the cure process of resin cementing were carried out. At the same time, in order to get data on changing cementing strength, multiple-group cementing process tests of different key process parameters were designed and conducted. A relational model of cementing strength and cure temperature, time and pressure was established, by combining data of DSC and process tests as well as based on the Avrami formula. Through sensitivity analysis for three process parameters, the on-line detection decision criterion and the process parameters which have obvious impact on cementing strength were determined. A PHM system with multiple temperature and pressure sensors was established on this basis, and then, on-line detection, diagnosis and control for ferrite phase shifter cementing process defects were realized. It was verified by subsequent process that the on-line detection system improved the reliability of the ferrite phase shifter cementing process and reduced the incidence of insufficient cementing strength defects.