Search

This paper proposes an adaptive sliding mode control (ASMC) with a modified reduced-order proportional integral observer (MROPIO) to provide speed control for a permanent magnet synchronous motor against unknown disturbances, such as system parameter uncertainties and external load torque variation. The ASMC method, based on a new sliding mode reaching law, improves chattering in the control signal and reduces the time required for the system trajectory to reach the sliding mode surface. The MROPIO estimates unknown disturbances and provides feedforward compensation for the ASMC, thus overcoming incorrect estimations of unknown disturbances during speed jumps can cause overshoots in the speed response. All the proposed methods were implemented on an industrial motor drive. The experimental results confirm the validity of the proposed method for practical applications.

This paper proposes a robot calibration method using an extended Kalman filter (EKF) and an artificial neural network (ANN) based on a butterfly and flower pollination algorithm (ANN-BFPA) to improve the robot's absolute pose (position and orientation) accuracy. After establishing a geometric error model, the EKF, a robust optimization algorithm for a nonlinear system with Gaussian noise, was used to estimate geometric parameter errors and compensate for geometric errors. However, nongeometric errors caused by joint clearance, gear backlash, and link deflection could still affect the pose accuracy and interfere with the correctness of the model. Therefore, the ANN-BFPA was proposed to compensate for these errors. The ANN model was used to establish the complex relationship between joint lengths and pose error. In addition, BFPA was used to optimize weights and bias of the neural network. The efficiency of the proposed calibration method was evaluated using a Stewart platform. Experimental results demonstrated that the proposed method significantly improved the robot's pose accuracy and showed better performance than previous techniques.

Rectification is a standard process in every system that requires multiviews. Existing algorithms largely work on similar field of view (FoV) cases where the two views are mostly identical. Dissimilarities between different FoVs can generate unexpected errors during the optimization process, resulting in a large amount of rectification errors and unwanted geometric distortion. In this study, we present a full pipeline to rectify uncalibrated images captured by cameras that have dissimilar FoVs under the constraints of geometric distortion. The proposed method contains two main parts: Field of View Neutralization and Rectification with Adaptive Geometric Constraints. The Field of View Neutralization module estimates the transformation matrix to compensate for the imbalance between different views. In addition, this module improves the overall quality of correspondences by removing misleading feature matching pairs. Finally, by applying an adaptive optimization process with the geometric constraints involved, our method addresses the overdistortion issue while maintaining small rectification errors. Extensive experiments are conducted to demonstrate the robust performance of the proposed method. Besides the existing datasets, we provide our dissimilar FoVs dataset with multiple baselines to examine the performance. Our method outperforms the existing algorithms in terms of both rectification errors and geometric distortion rates.

As the integration of semiconductors increases, additional diagnosis sensor systems have been developed for the detection of defects. The sensor system for equipment diagnosis, which is added after mass production, performs diagnosis in a standalone format that is operated independently of the equipment. The sensor system mounted in this way cannot deliver the diagnostic data to the system in real-time, but delivers it to the user after a certain period through a separate server. In this paper, we analyzed the data and communication methods of sensors and components used in semiconductors and proposed a standard communication protocol that can be transmitted to the system in real-time. In addition, it is possible to collect equipment data in the protocol so that the sensing trigger signal and dynamic diagnosis algorithm can be utilized through other data in the equipment for intelligent diagnosis. And also, the proposed protocol was added to the data acquisition system and installed and evaluated in mass production equipment.

Stereo vision systems in autonomous vehicles can come with camera sensors with different models and lenses to detect objects at long and short ranges. The intrinsic/extrinsic parameters of the camera sensors, such as the focal length and distortion parameters used for calibration, are not provided. In this article, we present a novel rectification method for uncalibrated stereo cameras based on their corresponding feature points. To obtain reliable key/feature points and descriptors from stereo images, we propose a deep neural network, called a multilevel feature pooling network. The model processes images individually to learn both local and semiglobal features, from which the location and descriptor of each pixel in the image are estimated. We also introduce a method to balance the zoom effect caused by the different focal lengths of the camera lenses and rectify the images by estimating homography matrices for stereo images based on their refined corresponding feature points.

This article proposes a method to improve the accuracy of a vernier absolute magnetic encoder. The encoder consists of a master and a nonius multipolar magnetic track. Sinusoidal signals from the master and nonius tracks are used to infer the absolute information. Unfortunately, these signals are contaminated by nonideal factors such as different amplitudes, dc-offsets, phase shifts, and random noise. Moreover, harmonics existing in the encoder signals distort the vernier principle and significantly affect the accuracy of the encoder. To address these problems, the present article proposes an efficient method with three main parts. The first is an observer phase-locked loop (OPLL), which is used to estimate the phase and eliminate the nonideal factors. The second is nonlinear phase compensation, which is used to correct the vernier principle that deviated due to the existing harmonics. Finally, a pole pitch compensation method is introduced to modulate the master phase angle from the OPLL to eliminate the harmonic distortion. The proposed method can eliminate the nonideal factors, harmonic distortion and improve the accuracy of the encoder. All the proposed methods were implemented on an ARM STM32F407ZG. The experimental results confirm the validity of the proposed method for practical applications.

This article presents a portable smartphone-based laser measurement system (SLM) for measuring gaps and flushes on a car body. The system is designed to replace conventional gauges that are used by human operators. The developed device consists of a smartphone and an off-the-shelf laser-line projector held together by a three-dimensional–printed structure. It captures images of a laser line scattered on target surfaces, identifies the extreme points on the laser profiles, and obtains gap and flush measurements with minimal uncertainty. Since the smartphone is used as the operational unit, the measurement data can be stored and allows the capability for data transfer with other storage locations. Experiments were performed on the specimens and real cars to validate the application of the developed system. The measurement uncertainty on a real car was reported as $\pm$ 0.201 mm for gap and $\pm$ 0.154 mm for flush.

This paper proposes a linear compensator algorithm to improve the performance of an absolute multipolar magnetic encoder (AMPME). An AMPME is an absolute magnetic rotary encoder that uses a multipolar magnet (MPM) to increase the resolution. The resolution can be dramatically increased in proportion to the number of poles in the MPM. However, various hardware problems that occur during the encoder manufacturing process degrade the AMPME performance. Also, harmonic components occur in the raw data due to various problems, such as the resistance error of the analog circuit, magnetic field overlap between the magnets, position error between the sensor and magnet, and pole-pitch difference. In particular, during the magnetization process of the MPM, the pole-pitch difference becomes a problem when the sizes of each pole are not uniform. This problem causes harmonic components that reduce the absolute position accuracy. To solve these problems, this paper proposes a linear compensation method. The proposed linear compensator consists of two parts. The first part is the enhanced ratiometric linearization for phase calculation and calibration. The second is the phase compensator for removing the phase difference via the pole-pitch difference of the MPM. The linear compensator improves various parameters by precomputing the offset, amplitude, and phase corrections. After compensating for sinusoidal signals, the linear compensator applies appropriate parameters at the appropriate times. This method is faster, easier to set up, and more accurate than the conventional method. Furthermore, this method is experimentally verified against the existing harmonic rejection method. Experimental results are provided to verify the effectiveness of the proposed method.

As multiple functions have been added to single-core-based engine electronic control units (ECUs) in vehicles, automotive researchers and manufacturers have actively studied multi-core architecture for engine ECUs. Multi-core architecture can provide load balancing and parallelism that can meet the requirements of international organization standard (ISO) 26262. However, since real-world engine ECUs have the most complex automotive open system architecture (AUTOSAR)-based control logic and datasets among automotive ECUs, developing multi-core-based engine ECUs is a substantial amount of work. Thus, automotive researchers and manufacturers will need new methodologies for multi-core-based engine ECUs. In this paper, we focus on designing a multi-core migration methodology and applying it to a real-world AUTOSAR-based engine ECU from HYUNDAI. We verify its practicability and enhanced performance. In conclusion, through connection with other automotive domain ECUs, it is demonstrated that a multi-core engine ECU using our migration technology can be applied in real-world automotive vehicles, leading to a significant improvement in performance.

Rectification is a core programming process that involves multiple views. In this study, we focus on uncalibrated cases that neglect intrinsic and extrinsic camera information. Existing uncalibrated rectification methods use feature-matching techniques to form relationships between different views and use those features to estimate optimized homography matrices. However, outliers are inevitable in feature matching. Using these outliers in a rectification process produces vertical disparity errors and unwanted geometric distortion. To tackle the problem, we propose a novel method that can learn from the rectification results, re-select the matching pairs, and find superior solutions. The proposed method introduces a novel workflow for uncalibrated rectification that incorporates three cores: field of view (FoV) neutralization, rectification, and feature matching re-selection (FMR). While the FoV neutralization module handles FoV differences among views, a combination of the rectification module and FMR module results in the optimal homography matrices. The rectification module takes neutral correspondences and estimates the optimized rectified matrices. Applying the results from the rectification module, the FMR module optimizes the correspondences and return them to the rectification module. For the rectification module, we apply adaptive geometric constraints and our updated optimization strategy to secure satisfied vertical disparity errors while maintaining low distortion levels. Multidisciplinary experiments are performed to analyze the capabilities of the proposed method. We combine existing datasets with extra samples, including various outdoor environments, to gauge the performance better. Compared with existing methods, the proposed method produces fewer rectification errors and keeps the rectified images under low distortion rates.

This paper combines the extended Kalman filter (EKF), dual quantum-behaved particle swarm optimization (DQPSO), and adaptive neuro-fuzzy inference system (ANFIS) to propose a novel robot calibration method. Robot precision is influenced by kinematic and non-kinematic error sources. The EKF algorithm is robust for the nonlinear system with Gaussian noise to identify kinematic parameter errors for kinematic calibration. However, if inappropriate covariance matrices are selected, the EKF algorithm may converge to an incorrect solution. To increase the effectiveness of the EKF algorithm, we propose a DQPSO algorithm that consists of the QPSO-1 and QPSO-2 algorithms. The QPSO-1 algorithm adapts the covariance matrices of measurement noise and process noise, while the QPSO-2 algorithm optimizes the kinematic parameter errors estimated by the EKF algorithm. In addition, the used ANFIS predicts and compensates the non-kinematic error for non-kinematic calibration. Experiments have been performed on a five-bar parallel robot to confirm the effectiveness of the proposed method. The experimental results demonstrate that the proposed method significantly improves the positional accuracy, and is better than the previous methods.

This article presents a self-referencing lookup table (LUT) algorithm to enhance the multiturn accuracy of a battery-free multiturn absolute magnetic encoder. The encoder consists of a single-turn angle for the measurement of the degree of rotation within $2\pi $ using a multipolar magnet to achieve high resolution and a multiturn counter to monitor the total number of revolutions made by the encoder’s shaft through the different angles of the four bipolar magnets (BPMs) based on four-shaft gears. However, the signals from the BPMs are affected by factors, such as amplitude mismatch, dc offsets, random noises, and harmonic components that are as a result of electrical and mechanical distortions. To solve this problem, this article suggested a method derived from the single-turn angle and the raw signals from the BPMs to generate the LUTs for multiturn angles. These angles eliminate the effects of the disturbances. Additionally, a multiturn compensation algorithm is suggested to compensate for the number of revolutions made by the encoder. The proposed methods are simulated in MATLAB software and implemented in ARM STM32F407ZG. The effectiveness of the proposed method is demonstrated by the obtained results.

To improve the accuracy of permanent magnet (PM) rotor position estimation for a hybrid stepper motor (HSM), the authors propose a sensorless control design based on the back electromagnetic force (back-EMF) caused by magnetic induction and harmonic rejection via an orthogonal third-order phase-locked loop (PLL 3 rd) and an integral harmonic filter (IHF). The accuracy of estimation is analysed considering PM rotor position estimation errors. The harmonics of the back-EMF signal are eliminated by the IHF before entering the PLL 3 rd, which synchronises and decreases position estimation error. This technique is a simple, reliable, and effective method for implementing sensorless control of an HSM. An industrial test-bench was used to implement the proposed scheme. The experimental results are presented to validate the effectiveness of the proposed estimation method.

This paper proposes a stable adaptive gradient descent for harmonic-disturbance rejection as a tool for grid-power signal processing, magnetic rotary encoders, and permanent-magnet synchronous motors. The method can be widely applied to increase the accuracy of phase or position estimation in various systems in which harmonic disturbances exist. The proposed technique is based on learning the harmonic amplitudes by means of gradient descent on the feedback phase error. To compensate for the disadvantages of existing gradient-descent methods, the derivative expansion is fully developed with system transfer function integration for stable weighting update. In addition, an adaptive learning rate is also proposed based on discrete Lyapunov standard to achieve stability, and theoretical analysis is discussed. The performance of the method is evaluated by numerical simulation in MATLAB with various scenarios, and by the experiment with the rotary magnetic encoders. The results show that the proposed method achieves stability and high performance in harmonic rejection.

A network-based control system includes many connected devices, and time synchronization plays an important role in the unified operation of those systems. A clock servo needs a guarantee time offset and a clock rate to minimize the time difference between devices. This paper proposes an adaptive clock servo to improve time synchronization. This proposed algorithm uses a fuzzy-based proportional-integral clock servo (fuzzy-PI) based on IEEE 1588 to reduce the frequency of clock compensation. This method adapts the bandwidth and enhances noise reduction, improving both the time offset and rate difference between the slave and master. The node time can synchronize with the master time after just one cycle of synchronization. Experiments validated the effectiveness of the algorithm and demonstrated that the slave can track the master with the mean and standard deviation of the time offset are 0.432ns and 4.402ns. A cycle time of one second is used to ensure a low-bandwidth network. With these results, the number of nodes over a real-time network can be increased.

Traditional deep learning-based vehicle detection methods are often designed using a pyramid of filters with multiple scales and sizes; therefore, the processing time is slow due to the large number of scales used and because the classifier runs at all scales. Recently, a deep learning-based region proposal network was introduced to detect vehicles that only employ the network one time regardless of the size of the input image. In object detection, deep learning-based region proposal networks have achieved state-of-the-art performance in terms of accuracy. These systems achieve a very high accuracy under normal driving conditions; however, their performance decreases under difficult driving conditions such as in snow, rain, or fog. In addition, the current state-of-the-art system-based region proposal networks still fail to satisfy the real-time requirements of the driving assistant systems. More recently, the identification of local patterns has been shown to improve the performance of the traditional deep-learning systems; hence, this paper investigates local patterns in region proposal networks to improve their accuracy. Depth information is also investigated to improve the processing time of current region proposal networks. Our experimental results show that the proposed system obtains better performance than the state-of-the-art object region detection systems in terms of both accuracy and processing time.

Binocular stereo matching is a challenging problem in computer vision. Recently, convolutional neural networks (CNNs) have emerged as a promising approach. However, matching ambiguities on ill-posed regions remain an intractable challenge for current methods. In this paper, we propose a novel network model consisting of two main parts: a wide context learning network and stacked encoder–decoder 2D CNNs with a spatial diffusion module. The first part leverages the power of a dilated convolutional layer and spatial pyramid pooling to extract global context information and constitute a matching cost volume. The second part performs contextual aggregation over this matching cost volume to optimize and smooth the matching cost. Finally, we estimate a disparity map by computing the probability of each disparity from the predicted matching cost. The proposed network model allows us to train end-to-end without any further post-processing or refinement. In the experiments, we evaluate our method using the synthesis Scene Flow and real-world KITTI datasets. Our proposed method achieves performance results that are competitive against state-of-the-art methods while maintaining a fast run time.

In this paper, we propose an advanced approach to improve the accuracy of stepper motors based on an effective phase compensator. The proposed approach includes an adaptive current controller and an adaptive position controller. The proposed adaptive feed-forward proportional-resonant (AFPR) current control technique with a neural network (NN) is an improvement over the conventional proportional-integral algorithm. In which, the NN algorithm is applied to obtain the optimal parameters for the AFPR controller. This results in an enhancement of the stepper motor current tracking. In addition, the advanced position controller is a combination of a damping and a phase-compensated (D&PC) method to reduce steady-state error and variations during low-speed operation. The position tracking performance is further appreciated by applying an advanced proportional integral derivative (PID) controller during high-speed operation. A bandpass filter is designed to switch the motor between low- and high-speed operational regimes. The proposed approach does not require any changes in the structure of the hardware. Therefore, it can be widely applied to a diverse set of industrial applications. The performance of the proposed approach was validated by conducting experiments under practical conditions.

This paper proposes a method to improve the accuracy of an absolute magnetic encoder by using harmonic rejection (HR) and a dual-phase-locked loop (DPLL). The encoder consists of two permanent magnets: an edge-located multipolar magnet (MPM) and a center-located bipolar magnet, in which the signal-processing accuracy of the MPM is crucial for achieving high accuracy of the entire encoder. However, the MPM signals are disturbed by nonidealities such as dc offsets, amplitude mismatch, low- and high-order harmonics, and random noises. In this paper, the HR approach investigates the characteristics of nonidealities of the phase detector and rejects them by using gradient descent. In addition, the DPLL remains robust by maintaining a zero steady-state error during a phase jump, a constant frequency, and a ramp frequency. The proposed method is simulated with MATLAB software and implemented in ARM STM32F407ZG. The obtained results demonstrate efficient performance. This method can be applied to any use of quadrature sinusoidal signals, such as in power grids and in permanent-magnet synchronous motor phase detection.

Absolute magnetic encoders (AMEs) use two magnets: a ring multipolar magnet (MPM) generating high-resolution and improving the accuracy for the encoder, and a bipolar magnet in the center calculating the number cycle of MPM signals. The phase outputs of these AMEs are tracked from the sinusoidal signals of the MPM. However, these sine/cosine signals are disturbed by amplitude differences, offsets, phase-shift, harmonic components, and random noise. In order to solve this problem, this paper presents an adaptive linear neuron based on a third-order phase-locked loop (ALN-PLL) to improve the accuracy of AMEs. The proposed approach consists of two main parts: The first part is an ALN algorithm that uses the phase feedback of the third-order PLL in order to build the mathematical model of input signals, and then reject the disturbances. The second part is a third-order PLL that is designed based on a dominant pole approximation algorithm. The proposed PLL can reduce noise and eliminate dc-error during the phase step, frequency step, and frequency ramp. The simulation and experimental results demonstrate the effectiveness of the proposed approach.

Recently, advanced driver assistance systems (ADAS) have gained prominence in the vehicle industry, with respect to the importance of the stability and reliability of vehicular control technology or communication technology. In such situations, when data is transmitted, communication delays between different networks occur inevitably, and if the electronic control units (ECU) of each sensor are not time-synchronized, they result in control process issues of the ADAS. In this paper, we propose a time synchronization method of each node of the CAN-FD network by applying the end-to-end (E2E) mechanism of applying the precision time protocol (PTP).

This article aims at implementing a Deep Neural Network (DNN) using Field Programmable Gate Arrays (FPGAs) for real time deep learning inference in embedded systems. In now days DNNs are widely used where high accuracy is required. However, due to the structural complexity, deep learning models are highly computationally intensive. To improve the system performance, optimization techniques such as weight quantization and pruning are commonly adopted. Another approach to improve the system performance is by applying heterogeneous architectures. Processor with Graphics Processing Unit (GPU) architectures are commonly used for deep learning training and inference acceleration. However, GPUs are expensive and consume much power that not a perfect solution for embedded systems. In this paper, we implemented a deep neural network on a Zynq SoC which is a heterogenous system integrated of ARM processor and FPGA. We trained the model with MNIST database, quantized the model’s 32-bit floating point weights and bias into integer and implemented model to inference in FPGA. As a result, we deployed a network on an embedded system while maintaining inference accuracy and accelerated the system performance with using less resources.

With advancements in automotive software, it is necessary to improve the processing performance of the electronic control unit (ECU) that handles the software. Existing single-core ECUs are incapable of handling the heavy, complex, and constantly evolving vehicle software. Hence, automotive software based on multicore ECUs are being developed. New methods to schedule tasks and distribute load appropriately to each core are needed for multicore ECU-based automotive software. In addition, functional safety of multicore ECUs needs to be assessed. Herein, we propose a task load distribution (balancing) and scheduling method for multicore ECUs as well as a method to achieve freedom from interference (FFI) in accordance with ISO 26262 by avoiding resource interference that may occur owing to simultaneous operation of multiple cores.

In this paper, FPGA (Field-Programmable Gate Array) implementation of SIFT (Scale Invariant Feature Transform) algorithms is presented. SIFT algorithm extracts key points from the image. For implementation, basic study for SIFT algorithm and hardware design method is shown including pipeline line buffer and simplified gradient calculation. The SIFT key point extraction hardware was designed with Verilog HDL and implemented to Xilinx Vertex-7.

Feature point detection is a popular algorithm for techniques such as object detection and tracking. To detect objects using features in a vision system, feature points and descriptors are required. In this paper, we implemented a real-time vision system that outputs specific points and descriptors from input images in FPGA based on ORB algorithm. In the proposed hardware architecture, the feature detector module uses a total of 314,908 clocks to process the entire 640x480 image. The entire system including the descriptor generation module takes 525,230 Clock. In this paper, we describe the equations of the algorithm used in the proposed hardware structure, and then Describes the implemented hardware structure. Finally, it concludes by comparing it with SW. The proposed hardware structure can be used in various systems such as object detection and tracking in the future.

In intelligent traffic systems, vehicle detection and counting have become an important task. The counting information is essential for reducing traffic congestion and improving traffic signal capability. Traditional methods have been focusing on counting vehicles in a single frame or consecutive frames. However, they have not yet considered the movement of interest (MOI) of the vehicles moving in different lanes and directions. This paper proposes a region-and-trajectory movement matching method that aims to detect and count vehicles for each movement on the road. First, the YOLOv5 detection model is used to detect candidate vehicles in the region of interest (ROI). Second, the SORT tracking method associates vehicles of the same instance in consecutive images to create tracked trajectories. Then, the counting method using the combination of MOI regions and predefined movement tracks. Each tracked trajectory is assigned to the corresponding movement id and is outputted to the result file. The efficiency and effectiveness of the proposed method have been evaluated and ranked 3rd on AI City Challenge 2021 Track 1 leaderboard. Further experiments showed that the method could achieve around 120 fps on an NVIDIA Quadro RTX 8000 and 20 fps on an NVIDIA Jetson Xavier AGX.

In this study, we investigate the effect of smartphone camera exposure on laser extraction performance in a digital image. The laser extraction is the most crucial operation in laser triangulation systems that obtain object measurements from the projected laser light captured in the image. The exposure settings consist of sensor sensitivity (ISO), aperture, and shutter speed. This study provides insight into the smartphone settings and the exposure adjustment for further complicated laser extraction. A smartphone camera (Apple iPhone 11) and a laser stripe projector operating at 405 nm were employed. Analysis of different ISO values, the effect of aperture and shutter speed on the laser extraction quality was performed.

The dimensional inspection of the gap and flush (GF) created by two different surfaces has been a longstanding and challenging field of research. In this paper, a novel mobile vision-based measurement system has been developed with the goal to detect and measure the gap accurately. ArUco markers have been attached to two planar surfaces. A sequence of images is captured, and the 3D information of markers are estimated. The flush is the difference between the depth information obtained from the markers, while the width can be converted from the image using the pixel resolution computed from the marker's physical size. A working prototype of the proposed method has been implemented and extensively evaluated on synthetic gaps. The experimental results validate the robustness and applicability of the proposed method in a real-world environment.

Controller area network (CAN) was primarily employed as In-vehicle networks (IVNs). But this protocol has not been employed in industrial fieldbuses owing to its inadequate transmission speed and data rate. However, with the development of CANs with flexible data rates (CAN-FD) and CANopen, there has been increased interest regarding the application of CAN-based systems in industrial sites. Nevertheless, the CAN based protocol has the drawback of being vulnerable to unexpected wire breaking. Therefore, this paper proposes a redundancy method requiring 393ns delay time to address unexpected wire breakage in CAN-based systems.

The number of electronic control units employed in automobiles increases as the functions of automobiles diversify in response to demands for improved convenience and stability. Furthermore, the complexity and size of software loaded on controllers are increasing. As the number and types of software to be reprogrammed increase, the time to reprogram a vehicle also increases. In this paper, we propose an efficient method to store software in flash memory using the multicore function when reprogramming the electronic control unit. This method can efficiently transfer data from RAM to flash memory using two cores alternately. As part of this study, an experiment was performed using an embedded system, and the data transfer time was compared with that required when storing software in flash memory using only one core. The results of this comparison demonstrate that the proposed method can optimize memory usage during reprogramming.

Infrared cameras require constant nonuniformity correction because image nonuniformity occurs with environmental changes. In this paper, we propose a nonuniformity correction algorithm using feature pattern matching that can correct nonuniformities in real time. The proposed algorithm consists of motion estimation and nonuniformity correction steps. The motion estimation algorithm consists of feature extraction, feature point simplification, and feature point pattern generation steps and is proposed to calculate the amount of motion between frames in real time using a field programmable gate array. The experimental results confirm that the proposed method is robust against ghost phenomenon, compared to a statistics-based nonuniformity correction, and improves the real-time performance while providing the same performance as the existing interframe registration-based nonuniformity correction algorithm.

In this research, a multitask convolutional neural network that can do end-to-end road scene classification and semantic segmentation, which are the two crucial tasks for advanced driver assistance systems (ADAS), is proposed. We name the network TSS which means time-based semantic segmentation. The network contains three main modules: an image encoder, a scene classifier, and two time-based segmentation decoders. For each road scene image, the encoder extracts image features which will be used for classifier and decoders. Next, the image features are fed to the classifier to predict the scene type (in this case a day or a night scene). Then, based on the predicted scene type, the same extracted features are fed to a corresponding segmentation decoder to produce the final semantic segmentation result. By using this classification-driven decoder approach, we can improve the accuracy of the segmentation model, even when the model has been trained excessively earlier. Through the experiment, the validity of our proposed method has been proven. Our approach can be considered as stacking multiple segmentation modules on top of the classification module with all of them share the same image encoder. With this approach, we can utilize the result from classification to gain more accuracy in segmentation in one feed forward only.

Face detection is currently one of the most common technologies used in a variety of applications. This paper presents the design and implementation of face detection with some local patterns for achieving real-time performance. It is designed to process the image received in real time from an industrial camera using local patterns to detect the face on the FPGA. Each development result is separately analyzed specifically in terms of hardware resources.

This paper is a research finding on the inspection of the camera module assembled in the camera module automation line and automation of the adjustment process. Parts such as camera modules, where foreign particles of several micrometers might cause defects in sensor pixel should be produced with facilities with clean areas to prevent contamination. When managing workers, production by automated robots is essential to prevent defects caused by foreign objects and human mistakes due to monotonous and repetitive tasks. In order to cope with the production process where products are produced through several processes and changes, flexibility with the product production, an automatic sorting system that is available for line reconfiguration and inline production is required. Recently, as the need to produce parts in a clean area increases, a sorting system that can produce small products as well as camera modules and can be used continuously or alone is needed.

Vision Meets Drone: Multiple Object Tracking (VisDrone-MOT2021) challenge - the forth annual activity organized by the VisDrone team - focuses on benchmarking UAV MOT algorithms in realistic challenging environments. It is held in conjunction with ICCV 2021. VisDrone-MOT2021 contains 96 video sequences in total, including 56 sequences (~24K frames) for training, 7 sequences (~3K frames) for validation and 33 sequences (~13K frames) for testing. Bounding-box annotations for novel object categories are provided every frame and temporally consistent instance IDs are also given. Additionally, occlusion ratio and truncation ratio are provided as extra useful annotations. The results of eight state-of-the-art MOT algorithms are reported and discussed. We hope that our VisDrone-MOT2021 challenge will facilitate future research and applications in the field of UAV vision. The website of our challenge can be found at http://www.aiskyeye.com/.

Traffic light detection has a significant role in every driver assistance system (DAS). Using the information of recognized traffic signals, a DAS provides safe navigating suggestions to drivers. In this paper, we focus on horizontal traffic light formats, which are popular in South Korea, North American and most European countries. Despite their different shapes and sizes, we group important traffic light signals into six categories: red, yellow, green, green-turn, red-turn and no signal. This paper proposes a method to localize all traffic light positions in the current scene; and identify the main ones from them. To detect and classify traffic signals, we use YOLOv4, which is the most recent deep learning framework. Next, based on the relation of detected traffic lights, our system removes irrelevant signals. Finally, with the optimized set of detected traffic lights, we identify the main signal for the whole scene. Experiments were conducted on two large datasets captured in Seoul Korea containing both highways and urban areas. The results achieved 95% accuracy at 30FPS.

This paper proposes a gateway based on EtherCAT and DeviceNet for a smart factory. EtherCAT is chosen as the primary protocol and DeviceNet is used as secondary protocol. Due to its outstanding features, the gateway can support many devices and increase the quality of service (QoS) of the network. With proposal gateway, an embedded programmable logic controller (ePLC) can replace the hardware programmable logic controller. The gateway was implemented with an ARM-Cortex M3 and TwinCAT3 was used as the ePLC. Experiments show that this gateway provides a high QoS and also satisfies the real-time requirements of a factory system.

Real-time is a challenge facing the industrial communications technology. For this purpose, a large number of protocols have been promoted to enable the goal of real-time control. In the first time, many network types are developed based on CAN. In this time, Ethernet is a trend in modern industrial network. A chosen the best one plays an importance role in engineering. In this paper, the authors present an evaluation on the performance of two networking protocols: Mechatrolink III and SSCNET III. In each network type, the commonly theories are presented. We also consider how each protocol be used on the networked control system. Moreover, each protocol is developed on STM32F207 to evaluate the time communication individually. The cycle time is measured to demonstrate the performance of each network type.

Up to date, researchers have proposed various car detection models for driver safety. For each model, trade-off between precision and detection speed was inevitable. Most methods proposed for detecting cars in a scene do not quite satisfy these conditions simultaneously. The objective of this study is to propose a model using Recurrent Rolling Convolution (RRC) by adopting ResNet as the base net and utilizing pruning process within training. The pruned model has detection speed similar to that of the original network while having increased mean average precision (mAP).

Most autonomous driving algorithms are designed for normal-light images. Hence, insufficient lighting during image capture significantly degrades the visibility of images and hurts the performance of many computer vision systems. Retinex theory is an effective tool for enhancing the illumination and detail of images. In this paper, we collected a Low-Light Drive (LOL-Drive) dataset and applied a deep retinex neural network, named DriveRetinex, which was taught using this dataset. The deep Retinex-Net consists of two subnetworks: Decom-Net (decomposes a color image into a reflectance map and an illumination map) and Enhance-Net (enhances the light level in the illumination map). The whole architecture can be trained in an end-to-end fashion. Extensive experiments demonstrate that the proposed method not only achieves visually appealing low-light enhancement, but it also increases the accuracy of object detection in autonomous driving systems.

Many successful deep learning networks for object detection have been proposed in recent years, but direct comparison is difficult due to different resolutions, training validation data, and frameworks used to train these networks. In particular, we use DRIVE PX2 as a reference computing platform, which is manufactured by NVIDIA Corporation for development of autonomous vehicles and evaluate the performance of methods of autonomous driving on ARM-based embedded processing cores and Tegra-based embedded graphics processing units (GPUs).

SW currently under development is becoming complicated in structure and larger in size. Therefore, model-based programming is drawing considerable attention, and several people are participating in the development of a single SW. However, even with these tools, human errors such as typos and line mismatching still occur. Furthermore, the unity of the SW is degraded by reflecting individual development tendencies. In this paper, we propose a DB-based SW auto-generation method targeting ASCET. We developed an auto-generation tool that defines the structures of similar types in advance and completes this structure based on a DB. Our tool generates ASCET modules automatically by receiving data from a DB. Moreover, C code is generated from ASCET for use in the embedded systems. The generated SW is verified using a simulation equipment.

Image processing is a computationally intensive application in which FPGAs are used to accelerate system performance. New design methodologies using High-Level Synthesis have enabled rapid development. Still, there are some doubts as to whether HLS tools can produce as quickly and efficiently as traditional RTL-based designs. We presented that HLS-based design can accelerate equivalently to an RTL-based design and reduce the system development time.

As the functions of vehicles continue to diversify, the performances of the corresponding electronic control units (ECUs) are being significantly improved. Moreover, the communication speed and data size are also increasing. This can lead to a burden on the gateway, thereby causing problems to the gateway. Therefore, as the number of ECUs and communication performance increase, the performance of the gateway must also increase. In this paper, we propose a multi-core gateway, an operational algorithm, and a new scheduling algorithm for efficient data transfer. In addition, we confirmed that the proposed algorithm could address the problems in existing algorithms (FIFO, SP, and EDF) through comparison.

Result of image processing in computer vision varies with algorithms. In this study, we compare FPGA implemented Sobel edge detector and Canny edge detector. We design edge detectors with Verilog HDL and program Xilinx Vertex-7 FPGA. The goal of this study is to compare output result and resource usage of both edge detection algorithm.

When CAN was developed, its performance was sufficient for its application in vehicles. However, as the amount of data communicated in vehicles continued to increase, data processing using CAN became increasingly difficult. Therefore, the automotive industry required communication systems with higher-performance. To address this, an automaker was used to develop the new Ethernet communication 10base-T1, which reduced the bit-rate to 10Mbps and used a novel bus access method called PLCA.This paper compares CAN, CAN-FD and FlexRay to 10base-T1. The experiment implements bus conditions when using 10base-T1 to compare the data transfer times with other communication methods. The results proved that 10base-T1 was the superior means of communication for application in vehicles.

With the development of deep convolutional networks, significant advances in object detection task have been achieved. However, for applications in autonomous vehicles, it is necessary to have an efficient object detector that can process rapidly while maintaining high accuracy. This study presents our implementation and performance evaluation of two object detectors EfficientDet-Lite and Yolov3-tiny on Nvidia Jetson TX2 mobile embedded platform. Our experimental results on the KITTI dataset demonstrate that it is possible to achieve real-time and highly accurate object detection on edge devices with constrained resources.

As vehicles added many functions, the number and complexity of the controllers are increasing. Because of the complexity of software increasing, the CPU load on a single core based controller reaches its limits. To overcome the limitations of the single-core-based controller, a multicore based controller has been introduced. One of the problems encountered when converting software from a single-core-based controller to a multicore-based controller's software is scheduling. This paper proposes an algorithm for generating offset considering the period, execution time, and priority of controller software's internal tasks. This algorithm's offset allows you to reduce the maximum load on the controller among the measured loads every specific period. This period equals the period of task with the highest CPU load. We simulated the HMC engine electronic control unit (ECU) software with the Timing Architects Tool Suite (TA) tool. And using Hardware In the Loop Simulation (HILS), the offset was applied to the ECU of the HMC. Also, we verify the reduction of peak load among the loads measured every specific period that is the same as the task's period with the highest CPU load by offsets generated.

Image processing is a highly computationally intensive process in which heterogeneous system architectures are required for high-resolution images or real-time applications. Zynq SoC is integrated in a processor and FPGA so that it can be an answer for such problems. The HLS tools reduced the FPGA implantation time while the PYNQ framework enhanced productivity. We implemented the Sobel edge detection application as an example of image processing in a Zynq MPSoC device. We presented how FPGAs can accelerate image processing and how HLS tools and the PYNQ framework enable rapid implementation.