Search

Real-time 10,000 km transmission over a straight-line link is achieved using a software-defined multi-modulation format receiver implemented on a commercial off-the-shelf general-purpose graphics processing unit (GPU). Minimum phase 1 GBaud 4-ary quadrature amplitude modulation (QAM) signals are transmitted over 10,000 km and successfully received after detection with a Kramers-Kronig (KK) coherent receiver. 8-, 16-, 32-, and 64-QAM are successfully transmitted over 7600, 5600, 3600, and 1600 km, respectively., Accepted for publication at Photonics Technology Letters

The escalating transmission intensity of COVID-19 pandemic is straining the healthcare systems worldwide. Due to the unavailability of effective pharmaceutical treatment and vaccines, monitoring social distancing is the only viable tool to strive against asymptomatic transmission. Pertaining to the need of monitoring the social distancing at populated areas, a novel bird eye view computer vision-based framework implementing deep learning and utilizing surveillance video is proposed. This proposed method employs YOLO v3 object detection model and uses key point regressor to detect the key feature points. Additionally, as the massive crowd is detected, the bounding boxes on objects are received, and red boxes are also visible if social distancing is violated. When empirically tested over real-time data, proposed method is established to be efficacious than the existing approaches in terms of inference time and frame rate.

International audience; Rendering participating media is important to the creation of photorealistic images. Participating media has a translucent aspect that comes from light being scattered inside the material. For materials with a small mean-free-path (mfp), multiple scattering effects dominate. Simulating these effects is computationally intensive, as it requires tracking a large number of scattering events inside the material. Existing approaches precompute multiple scattering events inside the material and store the results in a table. During rendering time, this table is used to compute the scattering effects. While these methods are faster than explicit scattering computation, they incur higher storage costs. In this paper, we present a new representation for double and multiple scattering effects that uses a neural network model. The scattering response from all homogeneous participating media is encoded into a neural network in a preprocessing step. At run time, the neural network is then used to predict the double and multiple scattering effects. We demonstrate the effects combined with Virtual Ray Lights (VRL), although our approach can be integrated with other rendering algorithms. Our algorithm is implemented on GPU. Double and multiple scattering effects for the entire participating media space are encoded using only 23.6 KB of memory. Our method achieves 50 ms per frame in typical scenes and provides results almost identical to the reference.

International audience; Historically, the study of mountain hydrology and the water cycle has been largely observational, with meteorological forcing and hydrological variables extrapolated from a few infrequent manual measurements. Recent developments in Internet of Things (IoT) technology are revolutionizing the field of mountain hydrology. Low-power wireless sensor networks can now generate denser data in real-time and for a fraction of the cost of labor-intensive manual measurement campaigns.The American River Hydrological Observatory (ARHO) project has deployed thirteen low-power wireless IoT networks throughout the American River basin to monitor California’s snowpack. The networks feature a total of 945 environmental sensors, each reporting a reading every 15 minutes. The data reported is made available to the scientific community minutes after it is generated.This paper provides an in-depth technical description of the ARHO project. It details the requirements and different technical options, describes the technology deployed today, and discusses the challenges associated with large-scale environmental monitoring in extreme conditions.

In the past decade, single image super-resolution (SISR) based on convolutional neural networks (CNNs) has been represented remarkable performance. Powerful characterization of CNN is important for recent methods to learn an intricate non-linear mapping between high-resolution and corresponding low-resolution images. However, a deeper and wider network structure brings superior performance while increasing the number of network parameters and calculations so that it is difficult to handle the real-time information. Hence, it can be embedded in mobile devices with difficulty. Inspired by the above motivation, a lightweight network for the real-time SISR is proposed by stacking efficient cascading residual blocks, which consist of several concatenate effective modules with wide activation. To further improve the network performance, with the increase of a slight number of parameters, the proposed network cooperates with a lightweight residual efficient channel attention module to capture feature interaction between channels. Extensive experiments provide significant demonstrations that the proposed network obtains the superior trade-off between performance and parameters compared with other current methods. The lightweight trait of our method allows it to implement real-time image processing and can be embedded in mobile devices.

Fuel is an important aspect in the operation of ships that require high costs. The high cost of fuel is not followed by an automatic fuel monitoring process. By not using the fuel consumption monitoring method that works automatically, the shipping management does not know for sure the ship's fuel consumption is in accordance with the shipping mileage, thus triggering fraud committed by the ship's crew against the ship's fuel. Fuel consumption monitoring is carried out primarily to identify opportunities to improve energy efficiency and reduce costs. By following technological developments, Internet of Things (IoT) technology has begun to be applied in various industrial fields because it can transmit data in real-time via the internet network without human-to-human or human-to-computer interaction. In this research, the design of models and experiments related to a monitoring system for fuel consumption was carried out using sensors and microcontrollers integrated with the internet to obtain accurate and real-time data. The test results show that the volume of fuel available in the tank, the volume of fuel discharged, the flow rate of fuel, and the location of the system can be known by the user in real-time via the IoT website. Based on the results of measurements using an ultrasonic level sensor, it is known that the measurement results are quite accurate with a deviation of ± 0.5 cm. Meanwhile, the measurement results by the flow sensor are less accurate because the fuel flow only relies on the force of gravity.

Triclustering algorithms group sets of coordinates of 3-dimensional datasets. In this paper, a new triclustering approach for data streams is introduced. It follows a streaming scheme of learning in two steps: offline and online phases. First, the offline phase provides a sum mary model with the components of the triclusters. Then, the second stage is the online phase to deal with data in streaming. This online phase consists in using the summary model obtained in the offline stage to update the triclusters as fast as possible with genetic operators. Results using three types of synthetic datasets and a real-world environmental sensor dataset are reported. The performance of the proposed triclustering streaming algo rithm is compared to a batch triclustering algorithm, showing an accurate performance both in terms of quality and running times Ministerio de Ciencia, Innovación y Universidades TIN2017-88209-C2

Predicting the rate of penetration (ROP) plays a key role in the success of the drilling operation. It is not an easy task to predict the ROP with high accuracy as it depends on several factors such as; drilling parameters, drilling fluid properties, and drilled formation characteristics. The objective of this paper is to develop a new empirical equation for predicting the ROP in real-time using different artificial intelligence (AI) techniques such as artificial neural networks (ANN), adaptive neuro-fuzzy inference system (ANFIS), and support vector machine (SVM). For the first time, poly diamond crystalline (PDC) bit design parameters, total flow area, in addition to mud density (MWin), gamma ray (GR), and drilling parameters were used to build the AI models. Actual field data was used to build the AI models (1000 data points from Well A) and another 972 data points from Well B were used for validating the developed AI models. The obtained results confirmed that the three AI techniques could be used to predict the ROP for complex lithologies with high accuracy. The ANN outperformed the SVM and ANFIS for predicting the ROP for the unseen data (972 data points of validation). The developed ROP-ANN model could be used to predict the ROP with high accuracy (the root mean square error (RMSE) was less than 0.659 for the available two wells). The developed empirical correlation was able to predict the ROP with high accuracy, RMSE was 0.66. The new ROP equation can be used without the need for the ANN Matlab code or special software.

In this paper our group proposes and designs a lightweight convolutional neural network (CNN) for detecting facial emotions in real-time and in bulk to achieve a better classification effect. We verify whether our model is effective by creating a real-time vision system. This system employs multi-task cascaded convolutional networks (MTCNN) to complete face detection and transmit the obtained face coordinates to the facial emotions classification model we designed firstly. Then it accomplishes the task of emotion classification. Multi-task cascaded convolutional networks have a cascade detection feature, one of which can be used alone, thereby reducing the occupation of memory resources. Our expression classification model employs Global Average Pooling to replace the fully connected layer in the traditional deep convolution neural network model. Each channel of the feature map is associated with the corresponding category, eliminating the black box characteristics of the fully connected layer to a certain extent. At the same time, our model marries the residual modules and depth-wise separable convolutions, reducing large quantities of parameters and making the model more portable. Finally, our model is tested on the FER-2013 dataset. It only takes 3.1% of the 16GB memory, that is, only 0.496GB memory is needed to complete the task of classifying facial expressions. Not only can our model be stored in an 872.9 kilobytes file, but also its accuracy has reached 67% on the FER-2013 dataset. And it has good detection and recognition effects on those figures which are out of the dataset.

Precision time protocol (PTP) is one of the most widely used protocols for clock synchronization in packet-switched networks, on which, among others, the transaction synchronization of the stock markets relies. PTP was not standardized with security as a core requirement and is therefore vulnerable and attractive to manifold kinds of malicious attacks, such as time-delay attacks (TDAs). TDAs, in short, corrupt the exchange of timestamped messages and thus cause an incorrect synchronization process. The annex P of the IEEE 1588-2019 standard has defined a number of security mechanisms for clock synchronization, but, however, none of these can protect a PTP-based system completely against TDAs. In this work, we enhance existing approaches by introducing a so-called observation task and analytically deriving attack parameters of an ongoing TDA. Following the recommendation of the annex P of the IEEE 1588-2019 standard, these attack parameters can serve as an additional input for intrusion detection systems and allow for a more reliable and sensitive detection of TDAs. In a comprehensive evaluation, we experimentally investigate the impact different attack parameter combinations can have on a system.

This paper presents the first measured cyclic-coupled ring oscillator (CCRO) time-to-digital converter (TDC). The CCRO realizes a robust true time-domain delay interpolation with sub-gate-delay resolution. The architecture employs real-time quantization to reduce conversion time and hence maximize bandwidth. Furthermore, the CCRO phase progression is encoded with a bubble error suppression logic, thereby building resilience to delay mismatches from circuit/layout imperfections. The prototype circuit implemented in a 28 nm CMOS process demonstrates a combination of high resolution and high sample rate over wide range of sample rates. The TDC achieves its peak figure-of-merit (FoM) of 0.051 pJ/conv.-step at 100 MS/s while delivering 8.38-bit linear resolution and 15.4 ps time resolution, operating from a 0.55 V supply. The TDC demonstrates the highest reported linear resolution of 9.29 bits among converters operating above 100 MS/s, at 125 MS/s and 0.9 V supply, while achieving 4.4 ps time resolution and 0.16 pJ/conv.-step FoM. Further, the real-time quantizing architecture allows fast operation up to 750 MS/s, where the TDC delivers 6-bit linear resolution and 0.48 pJ/conv.-step FoM operating from 0.9 V supply.

In recent years, many large-scale information systems in the Internet of Things (IoT) can be converted into interdependent sensor networks, such as smart cities, smart medical systems, and industrial Internet systems. The successful application of edge computing in the IoT will make our algorithms faster, more convenient, lower overall costs, providing better business practices, and enhance sustainability. Facial action unit (AU) detection recognizes facial expressions by analyzing cues about the movement of certain atomic muscles in the local facial area. According to the detected facial feature points, we could calculate the values of AU, and then use classification algorithms for emotion recognition. In edge devices, using optimized and custom algorithms to directly process the raw image data from each camera, the detected emotions can be more easily transmitted to the end-user. Due to the tremendous network overhead of transferring the facial action unit feature data, it poses challenges of a real-time facial expression recognition system being deployed in a distributed manner while running in production. Therefore, we designed a lightweight edge computing-based distributed system using Raspberry Pi tailed for this need, and we optimized the data transfer and components deployment. In the vicinity, the front-end and back-end processing modes are separated to reduce round-trip delay, thereby completing complex computing tasks and providing high-reliability, large-scale connection services. For IoT or smart city applications and services, they can be made into smart sensing systems that can be deployed anywhere with network connections.

This paper aims to apply real-time light-weight high-precision 3D detection for autonomous driving. We propose LIDAR-based 3D object detection based on foreground segmentation using a fully sparse convolutional network (FS23D). We design a sparse convolutional backbone network and a sparse convolutional detection head to efficiently use the computing and memory resources and accelerate the inference. Instead of using the anchor-based method, we convert the detection problem into a foreground segmentation problem on a bird’s-eye view. The sparse convolutional detection head predicts the objectness and bounding box on each active point on the sparse feature map. We design a new oriented bounding box coding method and corresponding loss functions. We predict the endpoints of two mutually perpendicular lines that pass through the foreground active points and indirectly predict the objects’ oriented bounding box from these four endpoints. We use the indirectly calculated object center, size, and orientation as inputs of loss functions in the training step. Experiments on the KITTI dataset show that the sparse backbone network we designed is 2.2 times faster and 18.4 times fewer FLOPs than the dense backbone network. The average improvement of the loss functions based on the bounding box code is 1.1% and 0.8% on the BEV and 3D detection, respectively, compared to no addition of these losses. Moreover, FS23D outperforms the state-of-the-art LIDAR-based method in speed and precision for both cars and cyclists.

Recent advancements in network technology and associated services have led to a rapid increase in the amount of data traffic. However, the detrimental effects caused by cyber-attacks have also significantly increased. Network attacks are evolving in various forms. Two primary approaches exist for addressing such threats: signature-based detection and anomaly detection. Although the aforementioned approaches can be effective, they also have certain drawbacks. Signature-based detection is vulnerable to variant attacks, while anomaly detection cannot be used for real-time data traffic. For resolving such issues, this paper proposes a two-level classifier that can simultaneously achieve high performance and real-time classification. It employs level 1 and 2 classifiers internally. The level 1 classifier initially performs real-time detection with moderate accuracy for incoming data traffic. If the data cannot be classified with high probability by the classifier, the classification is delayed until the traffic flow terminates. The level 2 classifier then collects the statistical features of the traffic flow for performing precise classification. Compared to existing techniques, the proposed two-level classification method can achieve superior performance in terms of accuracy and detection time.

Direction-of-arrival (DOA) estimation algorithm for uniform planar arrays has been applied in many fields. The multiple signal classification (MUSIC) algorithm has obvious advantage in high-resolution signal source estimation scenarios. However, the MUSIC algorithm has high computational costs, therefore it is hard to be used in real-time scenes. Many studies are dedicated to accelerating MUSIC algorithm by parallel hardware, especially by Graphics Processing Units (GPU). MUSIC algorithm based on Central Processing Unit (CPU) -GPU architecture acceleration is rarely investigated in previous literatures, and how well MUSIC Algorithm with CPU-GPU architecture could perform remains unknown. In this paper, we present and evaluate a model of search parallel MUSIC algorithm with CPU-GPU architecture. In the proposed model, the steering vector of each candidate incident signal and the corresponding value of 2D spatial pseudo-spectrum (SPS) function are sequentially calculated in a single core of the GPU, and the subsequent calculation of each elevation or azimuth is parallel in batches. Furthermore, in order to improve the peak search speed, we propose a new Coarse and Fine Traversal (CFT) peak search algorithm via CPU and a new parallel peak search algorithm based on GPU acceleration. Across strategy comparison, utilizing CPU-GPU architecture for processing, a 150-160x performance gain is achieved compared to using CPU only. Besides, the resolution of uniform planar arrays is also analyzed.

The scene rigidity is a strong assumption in typical visual Simultaneous Localization and Mapping (vSLAM) algorithms. Such strong assumption limits the usage of most vSLAM in dynamic real-world environments, which are the target of several relevant applications such as augmented reality, semantic mapping, unmanned autonomous vehicles, and service robotics. Many solutions are proposed that use different kinds of semantic segmentation methods (e.g., Mask R-CNN, SegNet) to detect dynamic objects and remove outliers. However, as far as we know, such kind of methods wait for the semantic results in the tracking thread in their architecture, and the processing time depends on the segmentation methods used. In this paper, we present RDS-SLAM, a real-time visual dynamic SLAM algorithm that is built on ORB-SLAM3 and adds a semantic thread and a semantic-based optimization thread for robust tracking and mapping in dynamic environments in real-time. These novel threads run in parallel with the others, and therefore the tracking thread does not need to wait for the semantic information anymore. Besides, we propose an algorithm to obtain as the latest semantic information as possible, thereby making it possible to use segmentation methods with different speeds in a uniform way. We update and propagate semantic information using the moving probability, which is saved in the map and used to remove outliers from tracking using a data association algorithm. Finally, we evaluate the tracking accuracy and real-time performance using the public TUM RGB-D datasets and Kinect camera in dynamic indoor scenarios. Source code and demo: https://github.com/yubaoliu/RDS-SLAM.git

Using sensor-rich smartphones to sense various contexts attracts much attention, such as transportation mode recognition. Local solutions make efforts to achieve trade-offs among detection accuracy, delay, and battery usage. We propose a real-time recognition model consisting of two long short-term memory classifiers with different sequence lengths. The shorter one is a binary classifier distinguishing elevator scene and the longer one implements a finer classification among bus, subway, high-speed railway, and others. Light-weighted sensors are employed with a much smaller sampling rate (10Hz) compared with previous works. A two-stage setting makes it robust to scenes with different duration and therefore reduces the latency of recognition greatly. Further, the real-time system refines the classification results and attains smoothed predictions. We present experiments on accuracy and resource usage and prove that our system realizes a latency-low and power-efficient scene recognition approach by trading off a reasonable performance loss (averaged recall of 92.22%).

Real-time emulation of 5G networks is highly beneficial for several purposes, such as prototyping or performance evaluation of distributed applications meant to run on 5G networks, research demonstration, evaluation of other technologies (e.g., Multi-access Edge Computing) meant to interoperate with 5G access. In this work, we describe how to use Simu5G, a new end-to-end simulator of 5G networks based on OMNeT++, as a real-time emulator. We describe in detail the modeling choices that allow emulation to scale up without compromising accuracy. We present a thorough evaluation of the Simu5G’s emulation capabilities, showing that networks with hundreds of simulated users and tens of cells can be emulated on a single desktop machine.

Volumetric videos allow for a true three-dimensional experience where users can freely choose their viewing angles and be actually immersed in a video clip. High quality video productions are gaining attention and first volumetric video recordings are commercially provided at select places. Unfortunately, the production process is very time, labour, and technology-resource intensive, which requires specialist hardware, software, and production experts. A “youtube-like” production, distribution, and experience system would be desirable. Here we present an approach which allows for the creation and interactive replay of three-dimensional videoclips using a novel voxel-based platform—voxelvideos. We can show that our voxelvideos experienced in virtual and augmented reality are effective, enjoyable, and perceived as useful. We hope that our approach and findings will encourage researchers, media experts, and hobbyists to experiment with voxelvideos as a new form of affordable media production and experience.

The existing real-time object detection algorithm often omits the objects in the object detection. So an improved Tiny YOLOv3 (you look only once) algorithm is proposed with both lightweight and high accuracy of object detection. The improved Tiny YOLOv3 uses K-means clustering to estimate the size of the anchor boxes for dataset. The pooling and convolution layers are added in the network to strengthen feature fusion and reduce parameters. The network structure increases upsampling and downsampling to enhance multi-scale fusion. The complete intersection over union is added in the loss function, which effectively improves the detection results. In addition, the proposed method has the lightweight module size and can be trained in the CPU. The experimental results show that the proposed method can meet the requirements of the detection speed and accuracy.

With the advantages of high positioning accuracy and low cost, visible light positioning (VLP) is becoming a promising solution for practical indoor positioning system. However, most of the VLP systems require at least two VLP LED lamps for accurate position calculation. Therefore, the application of VLP in practical scenarios may be restricted due to this limitation. In this paper, we propose a fast and high-accuracy single-LED based VLP system. Firstly, an unbalanced single-LED VLP algorithm is proposed to increase the positioning accuracy and reduce the computational complexity. Secondly, a fast beacon searching algorithm is proposed to further reduce the processing time for each captured image. Finally, since the proposed algorithms have the advantages of high accuracy and low complexity, the proposed system can also be implemented on a low-end hardware platform. Experimental results show that the average positioning error of the proposed system is decreased to 2.26 cm at the height of 3 m, and the average positioning time is reduced to 6.3 ms on a laptop and 60ms on a low-end embedded platform.

The strict requirements on the timing correctness biased the modeling and analysis of real-time systems toward the worst-case performances. Such focus on the worst-case, however, does not provide enough information to effectively steer the resource/energy optimization. In this article, we integrate a probabilistic-based energy prediction strategy with the precise scheduling of mixed-criticality tasks, where the timing correctness must be met for all tasks at all scenarios. The dynamic voltage and frequency scaling (DVFS) is applied to this precise scheduling policy to enable energy minimization. We propose a probabilistic technique to derive an energy-efficient speed (for the processor) that minimizes the average energy consumption, while guaranteeing the (worst-case) timing correctness for all tasks, including LO-criticality ones, under any execution condition. We present a response time analysis for such systems under the nonpreemptive fixed-priority scheduling policy. Finally, we conduct an extensive simulation campaign based on randomly generated task sets to verify the effectiveness of our algorithm (with respect to energy savings) and it reports up to 46% energy-saving.

The Internet-of-Things (IoT) has emerged as an alternative to communicate different pieces of technology to foster the distributed data collection. The measurement projects and the Real-time data processing are articulated to take advantage of this environment, fostering a sustainable data-driven decision making. The Data Stream Processing Strategy (DSPS) is a Stream Processing Engine focused on measurement projects, where each concept is previously agreed through a measurement framework. The Measurement Adapter (MA) is a component whose responsibility is to pair each metric’s definition from the measurement project with data sensors to transmit data (i.e., measures) with metadata (i.e., tags indicating the data meaning) together. The Gathering Function (GF) receives and derivates data for its processing from each MA, while it implements load-shedding (LS) techniques based on Metadata to avoid a processing collapse when all MAs informs jointly and frequently. Here, a Metadata and Z-score based load-shedding technique implemented locally in the MA is proposed. Thus, the load-shedding is located at the same data source to avoid data transmission and saving resources. Also, an incremental estimation of average, deviations, covariance, and correlations are implemented and employed to calculate the Z-scores and to data retain/discard selectively. Four simulations discrete were designed and performed to analyze the proposal. Results indicate that the local LS required only 24% of the original data transmissions, a minimum of 18.61 ms as the data lifespan, while it consumes 890.26 KB. As future work, other kinds of dependencies analysis will be analyzed to provide local alternatives to LS.

International audience; We introduce the concurrent binary tree (CBT), a novel concurrent representation to build and update arbitrary binary trees in parallel. Fundamentally, our representation consists of a binary heap, i.e., a 1D array, that explicitly stores the sum-reduction tree of a bitfield. In this bitfield, each one-valued bit represents a leaf node of the binary tree encoded by the CBT, which we locate algorithmically using a binary-search over the sum-reduction. We show that this construction allows to dispatch down to one thread per leaf node and that, in turn, these threads can safely split and/or remove nodes concurrently via simple bitwise operations over the bitfield. The practical benefit of CBTs lies in their ability to accelerate binary-tree-based algorithm with parallel processors. To support this claim, we leverage our representation to accelerate a longest-edge-bisection-based algorithm that computes and renders adaptive geometry for large-scale terrains entirely on the GPU. For this specific algorithm, the CBT accelerates processing speed linearly with the number of processors.

This paper presents a deep reinforcement learning (DRL)-based task scheduling algorithm that is applied to an FPGA-based real-time digital simulation (FRTDS) system to generate arrangements to minimize the makespan of a task sequence with limited resources. The algorithm has two parts, which are synthetic cost construction and DRL processing to make arrangements. The synthetic cost represents the cost of different selections of arrangements in both resource usage and blockage arranging probability. This study uses the cost to measure the state-action value function to process the deep Q network (DQN) procedure to generate an optimized scheduling strategy. We establish the reinforcement learning strategy generation process by instantiating the computing components in the hardware as agents, and RAM resources and communication I/O ports as environment. A hardware-design-based decision rule is constructed to ensure that the computing variables are distributed as evenly as possible in storage, while making full use of the pipeline characteristics of FPGA. A compiler is written to generate an FRTDS binary stream to drive FRTDS. Accuracy and performance of the proposed method are verified and evaluated. We present simulation results of the modeling method, as well as from a classic method. Comparing these results, the makespan obtained by the proposed method is significantly shorter. It corresponds to the possibility of having higher computing power and dealing with larger-scale real-time simulation.

We propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of low-resolution (LR) image patches where the texture type is classified based on LBP features. On inference operation, a high-resolution (HR) image patch is reconstructed by multiplying an LR image patch with a linear mapping kernel, which is inferred by the LBP feature class of the corresponding LR patch. Since, the LBP is a highly efficient feature extraction operator for local texture classification, our method is extremely fast and power-efficient while showing competitive reconstruction quality to the latest machine learning-based SR techniques. We also present a fully pipe-lined hardware architecture and its implementation for real-time operations of the proposed SR method. The proposed SR algorithm has been implemented on a field-programmable-gate-array (FPGA) platform including Xilinx KCU105 that can process 63 frames-per-second (fps) while converting full-high-definition (FHD) images to 4K ultra-high-definition (UHD) images. Extensive experimental results show that the proposed proposed algorithm and its hardware implementation can achieve high reconstruction performance compared to the latest machine-learning-based SR methods while utilizing minimum hardware resources, thereby having remarkably less computational complexity. Sometimes, the latest deep-learning-based SR approaches offer slightly higher reconstruction quality, but they require significantly larger amount of hardware resources than the proposed method.

In near real-time photogrammetry, the first step in processing each new added image is determining the most relevant image in pre-sequence unordered images quickly and exactly, which is pivotal for accurate image matching and 3D reconstruction. This paper presents a hierarchical image retrieval algorithm based on multiple features and details the choice for representation of multiple features which is critical to the improvement of accuracy of this algorithm. First, we represent global features using AlexNet-FC7(fully connected layers) or ResNet101-Pool5(pooling layers) and local features using SIFT (scale-invariant feature transform) in two parallel threads with support of GPU (Graphics Processing Unit). Next, we obtain candidates based on cosine similarities between global features of each pre-sequence image and new added image. Finally, we determine the most relevant image from those candidates according to feature matching results for each candidate and new added image. The experimental results confirm that the second step is rather fast and the third step is necessary to tackle the problem that global features cannot distinguish objects from the same class. The total time our algorithm takes is about 83.6ms for determining the most relevant image in 5063 pre-sequence unordered images of size $1024\times 768$ , which outperforms exhaustive pairwise matching, Bag of Words and multi-vocabulary trees. Accuracy of our algorithm also perform better than the state-of-the-art methods on three benchmark datasets. SIFT matching results obtained in the third step after eliminating mismatches with RANSAC (Random Sample Consensus) can also be used for high-precision incremental SFM (Structure from Motion) reconstruction.

Transistors in three-phase voltage-source inverter often suffer from open-circuit failures due to the lifting of bonding wires caused by thermic cycling, resulting in performance degradation with ripple torque and current harmonics. Current-spectral-analysis based methods are widely applied to failure diagnosis; however, high calculation consumption and complex implementation limit their application in some real-time occasion. In this paper, a simplified Fourier series method is proposed by the product between reconstructed phase currents and reference signals. Meanwhile, a novel normalized method for DC and fundamental components of simplified Fourier series are proposed to locate twenty-one transistor open-circuit faults. Numerical results show that the proposed Fourier series method coincides with that of Fast Fourier Transform. Experimental results and the comparison with previous methods show high efficiency and merits of its application to transistor open-circuit fault location in the voltage-source inverter. Published version

To enhance surgeons’ efficiency and safety of patients, minimally invasive surgery (MIS) is widely used in a variety of clinical surgeries. Real-time surgical tool detection plays an important role in MIS. However, most methods of surgical tool detection may not achieve a good trade-off between detection speed and accuracy. We propose a real-time attention-guided convolutional neural network (CNN) for frame-by-frame detection of surgical tools in MIS videos, which comprises a coarse (CDM) and a refined (RDM) detection modules. The CDM is used to coarsely regress the parameters of locations to get the refined anchors and perform binary classification, which determines whether the anchor is a tool or background. The RDM subtly incorporates the attention mechanism to generate accurate detection results utilizing the refined anchors from CDM. Finally, a light-head module for more efficient surgical tool detection is proposed. The proposed method is compared to eight state-of-the-art detection algorithms using two public (EndoVis Challenge and ATLAS Dione) datasets and a new dataset we introduced (Cholec80-locations), which extends the Cholec80 dataset with spatial annotations of surgical tools. Our approach runs in real-time at 55.5 FPS and achieves 100, 94.05, and 91.65% mAP for the above three datasets, respectively. Our method achieves accurate, fast, and robust detection results by end-to-end training in MIS videos. The results demonstrate the effectiveness and superiority of our method over the eight state-of-the-art methods.

The developments in wireless technology and applications in recent years have increased the interest in downlink scheduling and resource allocations among researchers. Moreover, fair scheduling and balanced Quality of Service (QoS) delivery for various forms of traffic are needed for Long-Term Evolution (LTE) wireless systems. This paper proposes hybrid QoS-aware downlink scheduling approaches that aim to address different traffic classes and balance the QoS delivery with improvements to the overall system performance under channel and bandwidth constraints. Moreover, this research introduces a taxonomy that classifies the scheduling algorithms into four main classes: delay aware, queue aware, target bit-rate aware and hybrid aware. The latter class is the scheduling class that is proposed in this paper; it considers channel, queue and delay parameters in its scheduling metric. Using simulations, we compare and analyze different downlink scheduling rules for their network-centric performance metrics, e.g., average packet loss ratio, average throughput, average packet delay, system fairness, and system spectral efficiency. The simulation results show that the queue-aware and delay-aware scheduling rules deliver the best QoS performance for video traffic classes, whereas our proposed hybrid scheduling rules deliver balanced QoS for various types of traffic classes. Employing QoS balancing scheduling rules in an LTE downlink is suggested to provide high QoS delivery for different traffic classes.

Alert signals like sirens and home alarms are important as they warn people of precarious situations. This work presents the detection and separation of these acoustically important alert signals, not to be attenuated as noise, to assist the hearing impaired listeners. The proposed method is based on convolutional neural network (CNN) and convolutional-recurrent neural network (CRNN). The developed method consists of two blocks, the detector block, and the separator block. The entire setup is integrated with speech enhancement (SE) algorithms, and before the compression stage, used in a hearing aid device (HAD) signal processing pipeline. The detector recognizes the presence of alert signal in various noisy environments. The separator block separates the alert signal from the mixture of noisy signals before passing it through SE to ensure minimal or no attenuation of the alert signal. It is implemented on a smartphone as an application that seamlessly works with HADs in real-time. This smartphone assistive setup allows the hearing aid users to know the presence of the alert sounds even when these are out of sight. The algorithm is computationally efficient with a low processing delay. The key contribution of this paper includes the development and integration of alert signal separator block with SE and the realization of the entire setup on a smartphone in real-time. The proposed method is compared with several state-of-the-art techniques through objective measures in various noisy conditions. The experimental analysis demonstrates the effectiveness and practical usefulness of the developed setup in real-world noisy scenarios.

This paper reports on a service-based approach to enable real-time stream-processing of high-resolution videos. It presents a concept for integrating black-box image and video processing operations into a streaming framework. It further describes approaches to optimize data flow between the processing implementation and the framework to increase throughput and decrease latency. This enables the composition of streaming services to allow scaling for further throughput increase. We demonstrate the effectiveness of our approach by means of two real-time streaming-processing application examples.

Semantic segmentation is performed to understand an image at the pixel level; it is widely used in the field of autonomous driving. In recent years, deep neural networks achieve good accuracy performance; however, there exist few models that have a good trade-off between high accuracy and low inference time. In this paper, we propose a fast spatial feature network (FSFNet), an optimized lightweight semantic segmentation model using an accelerator, offering high performance as well as faster inference speed than current methods. FSFNet employs the FSF and MRA modules. The FSF module has three different types of subset modules to extract spatial features efficiently. They are designed in consideration of the size of the spatial domain. The multi-resolution aggregation module combines features that are extracted at different resolutions to reconstruct the segmentation image accurately. Our approach is able to run at over 203 FPS at full resolution ( $1024\times 2048$ ) in a single NVIDIA 1080Ti GPU, and obtains a result of 69.13% mIoU on the Cityscapes test dataset. Compared with existing models in real-time semantic segmentation, our proposed model retains remarkable accuracy while having high FPS that is over 30% faster than the state-of-the-art model. The experimental results proved that our model is an ideal approach for the Cityscapes dataset. The code is publicly available at: https://github.com/computervision8/FSFNet .

This paper addresses the problem of worst-case timing analysis of heterogeneous wormhole NoCs, i.e., routers with different buffer sizes and transmission speeds, when consecutive packet queuing (CPQ) occurs. The latter means that there are several consecutive packets of one flow queuing in the network. This scenario happens in the case of bursty traffic but also for non-schedulable traffic. Conducting such an analysis is known to be a challenging issue due to the sophisticated congestion patterns when enabling backpressure mechanisms. We tackle this problem through extending the applicability domain of our previous work for computing maximum delay bounds using Network Calculus, called Buffer-aware worst-case Timing Analysis (BATA). We propose a new Graph-based approach to improve the analysis of indirect blocking due to backpressure, while capturing the CPQ effect and keeping the information about dependencies between flows. Furthermore, the introduced approach improves the computation of indirect-blocking delay bounds in terms of complexity and ensures the safety of these bounds even for non-schedulable traffic. We provide further insights into the tightness and complexity issues of worst-case delay bounds yielded by the extended BATA with the Graph-based approach, denoted G-BATA. Our assessments show that the complexity has decreased by up to 100 times while offering an average tightness ratio of 71%, with reference to the basic BATA. Finally, we evaluate the yielded improvements with G-BATA for a realistic use case against a recent state-of-the-art approach. This evaluation shows the applicability of G-BATA under more general assumptions and the impact of such a feature on the tightness and computation time., 21 pages, 22 figures, 5 tables

Despite CNN-based deblur models have shown their superiority when solving motion blurs, restoring a photorealistic image from severe motion blur remains an ill-posed problem due to the loss of temporal information and textures. Event cameras such as Dynamic and Active-pixel Vision Sensor (DAVIS) can simultaneously produce gray-scale Active Pixel Sensor (APS) frames and events, which can capture fast motions as events of very high temporal resolution, i. e., $1~\mu s$ , can provide extra information for blurry APS frames. Due to the natural noise and sparsity of events, we employ a recurrent encoder-decoder architecture to generate dense recurrent event representations, which encode the overall historical information. We concatenate the original blurry image with the event representation as our hybrid input, from which the network learns to restore the sharp output. We conduct extensive experiments on GoPro dataset and a real event blurry dataset captured by DAVIS240C. Our experimental results on both synthetic and real images demonstrate state-of-the-art performance for $1280\times 720 $ images at 30 fps.

Deep Neural Networks (DNN) are widely applied to many mobile applications demanding real-time implementation and large memory space. Therefore, it presents a new challenge for low-power and efficient implementation of a diversity of applications, such as speech recognition and image classification, for embedded edge devices. This work presents a hardware-based DNN compression approach to address the limited memory resources in edge devices. We propose a new entropy-based compression algorithm for encoding DNN weights, as well as a real-time decoding method and efficient dedicated hardware implementation. The proposed approach enables a significant reduction of the required DNN weights memory (approximately 70% and 63% for AlexNet and VGG19, respectively), while allowing the decoding of one weight per clock cycle. Results show a high compression ratio compared to well-known lossless compression algorithms. The proposed hardware decoder enables an efficient implementation of large DNN networks in low-power edge devices with limited memory resources.

At present, human body moving target detection and recognition algorithms based on deep learning have made breakthrough progress. However, in some applications with high real-time requirements, the existing deep learning real-time detection and recognition network is difficult to achieve high detection accuracy. Therefore, how to achieve accurate positioning and recognition of human moving targets while ensuring real-time detection is still an urgent problem in this field. Based on the single shot multi-box detector (SSD) real-time detection network, this paper proposes a real-time detection positioning and recognition network based on multi-scale feature fusion (IMFF-SSD), which improves the positioning accuracy and identification accuracy. First, this article analyzes the multi-scale features extracted from the SSD network. It combines the position-sensitive information provided by low-level detail features with the context information provided by high-level semantic features through feature fusion, which effectively improves positioning accuracy of the target prediction layer in the SSD network. Secondly, a feature embedded prediction structure is designed to strengthen the semantics of target features without changing the spatial resolution of the SSD prediction layer, and embed low-scale detailed features in high-semantic features for collaborative prediction of targets. This improves the accuracy of the SSD network's recognition of human moving targets at all scales. The experimental results show that by combining the above two improvements, the real-time monitoring and recognition network based on multi-scale feature fusion proposed in this paper has achieved a greater degree of improvement in positioning accuracy and motion recognition accuracy than the original SSD, which is better than some current the human body moving object detection and recognition algorithm has great advantages.

Real-time state-of-health (SoH) estimation is often difficult to obtain due to the unavailability of capacity measurements in real-time monitoring. The equivalent internal resistance (EIR), which is easily obtained and closely related to battery deterioration, is studied as a possible solution for achieving real-time and reliable SoH estimation for lithium-ion batteries. A novel real-time SoH estimation method based on the EIR is introduced for lithium-ion batteries. First, an experimental study of the relationship between the EIR and battery degradation is implemented, and this study is used to develop an empirical description of battery degradation using the EIR vector. Second, a fast extraction method for identifying the EIR in real time is proposed by leveraging the relationship between the EIR vector and state of charge (SoC). Third, a support vector regression (SVR)-based method for real-time SoH estimation is introduced by characterizing the hidden relationship between the EIR vector and battery SoH. The proposed method is demonstrated using laboratory test data. The results show that the proposed method can predict the battery SoH in real time with good accuracy and robustness.

It is difficult for many semantic segmentation methods to perform useful inferences under extremely resource-constrained devices; therefore, an efficient fast semantic segmentation network (EFSNet) that employs a continuous shuffle dilated convolution (CSDC) and a up-sampling module is proposed in this paper. First, the number of parameters is reduced by group convolutions and the receptive field is enlarged by dilated convolution. Second, a up-sampling module is proposed for reducing noise and increasing inference speed. Finally, with the above CSDC module and up-sampling module, the EFSNet based on a fast semantic segmentation network is obtained. Our experiments on the CamVid and Cityscapes datasets show that the mean intersection over union (mIoU) values obtained by the proposed EFSNet with only 173 k parameters are 61.1% and 61.9%, respectively. The method is able to run in real-time at a speed of 332 frames per second (FPS) and 107 FPS on a single NVIDIA Titan Xp GPU. Compared with several existing methods, our network achieves efficient segmentation with low resource consumption.

The dark channel prior (DCP)-based single image removal algorithm achieved excellent performance. However, due to the high complexity of the algorithm, it is difficult to satisfy the demands of real-time processing. In this article, we present a Graphics Processing Unit (GPU) accelerated parallel computing method for the real-time processing of high-definition video haze removal. First, based on the memory access pattern, we propose a simple but effective filter method called transposed filter combined with the fast local minimum filter algorithm and integral image algorithm. The proposed method successfully accelerates the parallel minimum filter algorithm and the parallel mean filter algorithm. Meanwhile, we adopt the inter-frame atmospheric light constraint to suppress the flicker noise in the video haze removal and simplify the estimation of atmospheric light. Experimental results show that our implementation can process the 1080p video sequence with 167 frames per second. Compared with single thread Central Processing Units (CPU) implementation, the speedup is up to 226&times, with asynchronous stream processing and qualified for the real-time high definition video haze removal.

International audience; We tackle the problem of explaining faults in real-time systems. Intuitively, an explanation of the violation of a safety property by an execution is a concise excerpt of the faulty execution that retains only the elements that were relevant for entailing the violation, thus exhibiting how causes accumulate over time and propagate to entail the effect. Fault explanation therefore goes beyond the well-known concepts of fault diagnosis and localization.We provide a formal definition of causal explanations on dense-time models, based on the well-studied formalisms of timed automata and zone-based abstractions. Our approach is able to account for limited observability of the faulty execution. We propose a symbolic formalization to effectively construct such explanations, which we have implemented in a prototype tool. We illustrate our approach on several examples.

Real-time collision-avoidance navigation of autonomous ships is required by many application scenarios, such as carriage of goods by sea, search, and rescue. The collision avoidance algorithm is the core of autonomous navigation for Maritime autonomous surface ships (MASS). In order to realize real-time and free-collision under the condition of multi-ship encounter in an uncertain environment, a real-time collision avoidance framework is proposed using B-spline and optimal decoupling control. This framework takes advantage to handle the uncertain environment with limited sensing MASS which plans dynamically feasible, highly reliable, and safe feasible collision avoidance. First, owing to the collision risk assessment, a B-spline-based collision avoidance trajectory search (BCATS) algorithm is proposed to generate free-collision trajectories effectively. Second, a waypoint-based collision avoidance trajectory optimization is proposed with the path-speed decoupling control. Two benefits, a reduction of control cost and an improvement in the smoothness of the collision avoidance trajectory, are delivered. Finally, we conducted an experiment using the Electronic Chart System (ECS). The results reveal the robustness and real-time collision avoidance trajectory planned by the proposed collision avoidance system.

In this paper we are proposing an adaptive and real-time approach to resolve real-time financial data integration latency problems and semantic heterogeneity. Due to constraints that we have faced in some projects that requires real-time massive financial data integration and analysis, we decided to follow a new approach by combining a hybrid financial ontology, resilient distributed datasets and real-time discretized stream. We create a real-time data integration pipeline to avoid all problems of classic Extract-Transform-Load tools, which are data processing latency, functional miscomprehensions and metadata heterogeneity. This approach is considered as contribution to enhance reporting quality and availability in short time frames, the reason of the use of Apache Spark. We studied Extract-Transform-Load (ETL) concepts, data warehousing fundamentals, big data processing technics and oriented containers clustering architecture, in order to replace the classic data integration and analysis process by our new concept resilient distributed DataStream for online analytical process (RDD4OLAP) cubes which are consumed by using Spark SQL or Spark Core basics.

International audience; This work addresses the recurring challenge of real-time monophonic and polyphonic audio source classification. The whole normalized power spectrum (NPS) is directly involved in the proposed process, avoiding complex and hazardous traditional feature extraction. It is also a natural candidate for polyphonic events thanks to its additive property in such cases. The classification task is performed through a nonparametric kernel-based generative modeling of the power spectrum. Advantage of this model is twofold: it is almost hypothesis free and it allows to straightforwardly obtain the maximum a posteriori classification rule of online signals. Moreover it makes use of the monophonic dataset to build the polyphonic one. Then, to reach the real-time target, the complexity of the method can be tuned by using a standard hierarchical clustering preprocessing of the prototypes, revealing a particularly efficient computation time and classification accuracy trade-off. The proposed method, called RARE (for Real-time Audio Recognition Engine) reveals encouraging results both in monophonic and polyphonic classification tasks on benchmark and owned datasets, including also the targeted real-time situation. In particular, this method benefits from several advantages compared to the state-of-the-art methods including a reduced training time, no feature extraction, the ability to control the computation - accuracy trade-off and no training on already mixed sounds for polyphonic classification.

Accurate detection of air bubbles boundaries is of crucial importance in determining the performance and in the study of various gas/liquid two-phase flow systems. The main goal of this work is edge extraction of air bubbles rising in two-phase flow in real-time. To accomplish this, a fast algorithm based on local variance is improved and accelerated on the GPU to detect bubble contour. The proposed method is robust against changes of intensity contrast of edges and capable of giving high detection responses on low contrast edges. This algorithm is performed in two steps: in the first step, the local variance of each pixel is computed based on integral image, and then the resulting contours are thinned to generate the final edge map. We have implemented our algorithm on an NVIDIA GTX 780 GPU. The parallel implementation of our algorithm gives a speedup factor equal to 17x for high resolution images (1024×1024 pixels) compared to the serial implementation. Also, quantitative and qualitative assessments of our algorithm versus the most common edge detection algorithms from the literature were performed. A remarkable performance in terms of results accuracy and computation time is achieved with our algorithm.

Obtaining highly accurate depth from stereo images in real time has many applications across computer vision and robotics, but in some contexts, upper bounds on power consumption constrain the feasible hardware to embedded platforms such as FPGAs. Whilst various stereo algorithms have been deployed on these platforms, usually cut down to better match the embedded architecture, certain key parts of the more advanced algorithms, e.g. those that rely on unpredictable access to memory or are highly iterative in nature, are difficult to deploy efficiently on FPGAs, and thus the depth quality that can be achieved is limited. In this paper, we leverage a FPGA-CPU chip to propose a novel, sophisticated, stereo approach that combines the best features of SGM and ELAS-based methods to compute highly accurate dense depth in real time. Our approach achieves an 8.7% error rate on the challenging KITTI 2015 dataset at over 50 FPS, with a power consumption of only 5W., Comment: 6 pages, 7 figures, 2 tables, journal

This paper presents a technique and experimental validation for anonymous outdoor location tracking of all users residing on a mobile cellular network. The proposed technique does not require any intervention or cooperation on the mobile side but runs completely on the network side, which is useful to automatically monitor traffic, estimate population movements, or detect criminal activity. The proposed technique exploits the topology of a mobile cellular network, enriched open map data, mode of transportation, and advanced route filtering. Current tracking algorithms for cellular networks are validated in optimal or controlled environments on a small dataset or are merely validated by simulations. In this work, validation data consisting of millions of parallel location estimations from over a million users are collected and processed in real time, in cooperation with a major network operator in Belgium. Experiments are conducted in urban and rural environments near Ghent and Antwerp, with trajectories on foot, by bike, and by car, in the months May and September 2017. It is shown that the mode of transportation, smartphone usage, and environment impact the accuracy and that the proposed AMT location tracking algorithm is more robust and outperforms existing techniques with relative improvements up to 88%. Best performances were obtained in urban environments with median accuracies up to 112 m.

Modern onboard equipment complexes (OEC) utilize AFDX and FC-AE-ASM-RT switched networks implementing a virtual link-based approach to real-time data transfer. The main drawback of these networks is their limited or absent support for dynamic reconfiguration of virtual links, which makes impossible the dynamical recomposition of OEC operation modes, particularly in case of multiple equipment failures. To remove these drawbacks, in this paper an approach is proposed to use software-defined networks (SDN) as onboard real-time networks. The proposed approach is based on implementation of a virtual link-based technology (similar to those used in AFDX and FC-AE-ASMRT) in an SDN supporting OpenFlow 1.3 protocol. The approach was implemented as a functional prototype and experimentally evaluated in a virtual network environment based on Ofsoftswitch13 software SDN switches and RUNOS controller. The experiments indicated that the proposed data exchange scheme allows the transfer of messages within the given limits on delay and jitter, and does not allow violation of constraints on a virtual link bandwidth. The experiments also confirmed that dynamic reconfiguration of virtual links in SDN does not interrupt the data transfer through unchanged virtual links. An important direction for future work is development of algorithms for dynamic creation of virtual link routes in course of OEC reconfiguration. The final goal of the work is to create an SDN-based network technology supporting both real-time data transfer and automatic network reconfiguration in case of OEC mode change, including parrying multiple failures.

The Industry 4.0 era requires new quality management systems due to the ever increasing complexity of the global business environment and the advent of advanced digital technologies. This study presents new ideas for predictive quality management based on an extensive review of the literature on quality management and five real-world cases of predictive quality management based on new technologies. The results of the study indicate that advanced technology enabled predictive maintenance can be applied in various industries by leveraging big data analytics, smart sensors, artificial intelligence (AI), and platform construction. Such predictive quality management systems can become living ecosystems that can perform cause-effect analysis, big data monitoring and analytics, and effective decision-making in real time. This study proposes several practical implications for actual design and implementation of effective predictive quality management systems in the Industry 4.0 era. However, the living predictive quality management ecosystem should be the product of the organizational culture that nurtures collaborative efforts of all stakeholders, sharing of information, and co-creation of shared goals.

An accurate daylighting simulation can potentially improve the quality of pre-planing buildings and regulating daylighting to achieve the goal of green buildings. However, standard sky models can hardly reproduce real skies in real-time for a specific location within its micro-climate. This paper investigates an embedded photometric device based on high dynamic range (HDR) sky luminance monitoring with high resolution mapping in simulating real-time horizontal work-plane illuminance distribution. To increase time efficiency in the iterative process for the illuminance calculation, a matrix algebraic approach was employed and adapted for the device. The photometric device was validated experimentally in a daylighting test module with external Venetian blinds at different tilt angle of slats. The results indicate the embedded photometric device based on monitored sky can improve accuracy in simulating real-time daylighting provision by over 3 times, with 15%∼37% average error, compared with a common practice using the Perez all-weather model.