Search

The collaborative use of camera near-field sensors for monitoring the number and status of tourists is a crucial aspect of smart scenic spot management. This paper proposes a near-field perception technical system that achieves dynamic and accurate detection of tourist targets in mountainous scenic areas, addressing the challenges of real-time passive perception and safety management of tourists. The technical framework involves the following steps: Firstly, real-time video stream signals are collected from multiple cameras to create a distributed perception network. Then, the YOLOX network model is enhanced with the CBAM module and ASFF method to improve the dynamic recognition of preliminary tourist targets in complex scenes. Additionally, the BYTE target dynamic tracking algorithm is employed to address the issue of target occlusion in mountainous scenic areas, thereby enhancing the accuracy of model detection. Finally, the video target monocular spatial positioning algorithm is utilized to determine the actual geographic location of tourists based on the image coordinates. The algorithm was deployed in the Tianmeng Scenic Area of Yimeng Mountain in Shandong Province, and the results demonstrate that this technical system effectively assists in accurately perceiving and spatially positioning tourists in mountainous scenic spots. The system demonstrates an overall accuracy in tourist perception of over 90%, with spatial positioning errors less than 1.0 m and a root mean square error (RMSE) of less than 1.14. This provides auxiliary technical support and effective data support for passive real-time dynamic precise perception and safety management of regional tourist targets in mountainous scenic areas with no/weak satellite navigation signals. [ABSTRACT FROM AUTHOR]

During automated manufacturing processes, multiple sensors are attached to facilities to collect and analyze analog data for detecting operational anomalies. However, owing to facility devices being interlinked by a control system, simultaneous examination of the control system and analog data enhances the accuracy of anomaly detection and diagnosis of root causes. We proposed a system detecting anomalies by integrating an internal control system with external analog data and representing it in a graph structure. The system generates and combines the adjacency and feature matrices for training a convolutional autoencoder model to identify operational anomalies. Performance tests revealed distinct operational patterns in the cycle data flagged by the model as anomalies. The system diagnosed the root cause of anomalies, such as control operation sequencing, timing variances, and shifts in analog or video signals. This approach may enhance the productivity and quality of the manufacturing processes by facilitating anomaly detection and cause diagnosis. [ABSTRACT FROM AUTHOR]

Optical wireless video transmission technology combines the advantages of high data rates, enhanced security, large bandwidth capacity, and strong anti-interference capabilities inherent in optical communication, establishing it as a pivotal technology in contemporary data transmission networks. However, video data comprises a large volume of image information, resulting in substantial data flow with significant redundant bits. To address this, we propose an adaptive block sampling compressive sensing algorithm that overcomes the limitations of sampling inflexibility in traditional compressive sensing, which often leads to either redundant or insufficient local sampling. This method significantly reduces the presence of redundant bits in video images. First, the sampling mechanism of the block-based compressive sensing algorithm was optimized. Subsequently, a wireless optical video transmission experimental system was developed using a Field-Programmable Gate Array chip. Finally, experiments were conducted to evaluate the transmission of video optical signals. The results demonstrate that the proposed algorithm improves the peak signal-to-noise ratio by over 3 dB compared to other algorithms, with an enhancement exceeding 1.5 dB even in field tests, thereby significantly optimizing video transmission quality. This research contributes essential technical insights for the enhancement of wireless optical video transmission performance. [ABSTRACT FROM AUTHOR]

Effective video resources are assumed to promote learner engagement, but the extent to which this occurs is unclear. This study examines learners' engagement with instructor‐generated video. It contributes an analytical synthesis of qualitative and quantitative data that provides the basis for investigating the extent to which, and how, learners engage with video resources provided in their courses. Specifically, three dimensions of learner engagement with video are studied: behavioural, cognitive and affective. The study contributes to educational technology research by identifying distinctive patterns of learner control over the use of video which diverges from instructors' assumptions. It shows the complex and nuanced features of cognitive and affective engagement. Videos can have positive effects, but inappropriate use of technical features results in learner disengagement. This study contributes a novel use of signalling theory, suggesting the importance of instructor signals concerning relevance, focus and utility as a feature of video generation, as a prerequisite of cognitive engagement. A research and theory development agenda is developed, locating video‐based learning in student contexts as a basis for explaining both engagement and disengagement with video technology affordances. Practitioner notesWhat is already known about this topic Higher education tutors are increasingly expected to generate video content as a feature of their course delivery.Although effective video resources are assumed to promote learner engagement, the extent to which this occurs is unclear.Video technology affordances rarely feature in academic professional development programmes leaving tutors ill‐equipped to understand how and in what ways students engage with video resources as a part of their courses.What this paper adds A novel use of signalling theory to explain different patterns of learner engagement and disengagement.Learner agency and control results in video technology affordances being fitted and 'flexed' around factors that instructors rarely consider in the video generation process.A new model to describe factors that affect student engagement with instructor‐generated video.Implications for practice and/or policy Raise awareness for instructors about patterns of student engagement with video technology affordances that affect learning experience and outcomes.Implement professional development programmes to equip instructors to design and generate video content that takes account of patterns of behavioural, cognitive and affective engagement and signals relevance, focus and utility.Develop clear policies regarding technical features of video generation to avoid learner disengagement. [ABSTRACT FROM AUTHOR]

Mixed emotions have attracted increasing interest recently, but existing datasets rarely focus on mixed emotion recognition from multimodal signals, hindering the affective computing of mixed emotions. On this basis, we present a multimodal dataset with four kinds of signals recorded while watching mixed and non-mixed emotion videos. To ensure effective emotion induction, we first implemented a rule-based video filtering step to select the videos that could elicit stronger positive, negative, and mixed emotions. Then, an experiment with 80 participants was conducted, in which the data of EEG, GSR, PPG, and frontal face videos were recorded while they watched the selected video clips. We also recorded the subjective emotional rating on PANAS, VAD, and amusement-disgust dimensions. In total, the dataset consists of multimodal signal data and self-assessment data from 73 participants. We also present technical validations for emotion induction and mixed emotion classification from physiological signals and face videos. The average accuracy of the 3-class classification (i.e., positive, negative, and mixed) can reach 80.96% when using SVM and features from all modalities, which indicates the possibility of identifying mixed emotional states. [ABSTRACT FROM AUTHOR]

Automatic depression classification from multimodal input data is a challenging task. Modern methods use paralinguistic information such as audio and video signals. Using linguistic information such as speech signals and text data for depression classification is a complicated task in deep learning models. Best audio and video features are built to produce a dependable depression classification system. Textual signals related to depression classification are analyzed using text-based content data. Moreover, to increase the achievements of the depression classification system, audio, visual, and text descriptors are used. So, a deep learning-based depression classification model is developed to detect the person with depression from multimodal data. The EEG signal, Speech signal, video, and text are gathered from standard databases. Four stages of feature extraction take place. In the first stage, the features from the decomposed EEG signals are attained by the empirical mode decomposition (EMD) method, and features are extracted by means of linear and nonlinear feature extraction. In the second stage, the spectral features of the speech signals from the Mel-frequency cepstral coefficients (MFCC) are extracted. In the third stage, the facial texture features from the input video are extracted. In the fourth stage of feature extraction, the input text data are pre-processed, and from the pre-processed data, the textual features are extracted by using the Transformer Net. All four sets of features are optimally selected and combined with the optimal weights to get the weighted fused features using the enhanced mountaineering team-based optimization algorithm (EMTOA). The optimal weighted fused features are finally given to the hybrid attention-based dilated network (HADN). The HDAN is developed by combining temporal convolutional network (TCN) with bidirectional long short-term memory (Bi-LSTM). The parameters in the HDAN are optimized with the assistance of the developed EMTOA algorithm. At last, the classified output of depression is obtained from the HDAN. The efficiency of the developed deep learning HDAN is validated by comparing it with various traditional classification models. [ABSTRACT FROM AUTHOR]

The use of optical fibers is increasing in modern aircraft because this helps solve challenges of size, weight, communication, and reliability in new generation aircraft. This study describes a video and power transmission system using optical fibers (PoF) for in-flight entertainment (IFE) system application. We present the benefits and the limitations of this application, and we perform two practical experiments to demonstrate their performance. We used off-the-shelf devices in the experiments, such as one 15-Watt semiconductor laser operating at 808 nm, GaAs photovoltaic converters, optical transmitters and receivers, and video monitors. The power and video signals were transmitted using two 50-m length multimode fibers. In addition, we proposed and tested two types of energy transformation units (ETUs), which are responsible for supplying electrical energy to the IFE video monitor and the optical fiber receiver. [ABSTRACT FROM AUTHOR]

The monolithically integrated self-driven photoelectric detector (PD) with the light-emitting diode (LED) epitaxial structure completely relies on the built-in electric field in the multi-quantum wells region to separate the photogenerated carriers. Here, we propose a novel superlattices–electron barrier layer structure to expand the potential field region and enhance the detection capability of the integrated PD. The PD exhibits a record-breaking photo-to-dark current ratio of 5.14 × 107, responsivity of 110.3 A/W, and specific detectivity of 2.2 × 1013 Jones at 0 V bias, respectively. A clear open-eyed diagram of the monolithically integrated chip, including the PD, LED, and waveguide, is realized under a high-speed communication rate of 150 Mbps. The obtained transient response (rise/decay) time of 2.16/2.28 ns also illustrates the outstanding transient response capability of the integrated chip. The on-chip optical communication system is built to achieve the practical video signals transmission application, which is a formidable contender for the core module of future large-scale photonic integrated circuits. [ABSTRACT FROM AUTHOR]

Compressive sensing provides a promising sampling paradigm for video acquisition for resource‐limited sensor applications. However, the reconstruction of original video signals from sub‐sampled measurements is still a great challenge. To exploit the temporal redundancies within videos during the recovery, previous works tend to perform alignment on initial reconstructions, which are too coarse to provide accurate motion estimations. To solve this problem, the authors propose a novel reconstruction network, named TSRN, for compressive video sensing. Specifically, the authors utilise a number of stacked temporal shift reconstruction blocks (TSRBs) to enhance the initial reconstruction progressively. Each TSRB could learn the temporal structures by exchanging information with last and next time step, and no additional computations is imposed on the network compared to regular 2D convolutions due to the high efficiency of temporal shift operations. After the enhancement, a bidirectional alignment module to build accurate temporal dependencies directly with the help of optical flows is employed. Different from previous methods that only extract supplementary information from the key frames, the proposed alignment module can receive temporal information from the whole video sequence via bidirectional propagations, thus yielding better performance. Experimental results verify the superiority of the proposed method over other state‐of‐the‐art approaches quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Manual acupuncture manipulation (MAM) is essential in traditional Chinese medicine treatment. MAM action recognition is important for junior acupuncturists' training and education; however, there are obvious personalized differences in hand gestures among expert acupuncturists for the same type of MAM. In addition, during the MAM operations, the magnitude and frequency of the expert acupuncturists' hand shape and relative needle‐holding finger position changes are tiny and fast, resulting in difficulties in observing MAM action details. Thus, we propose a Spatial Multiscale Interactive Fusion MAM Recognition Network to solve the difficulties in MAM recognition. First, this paper presents an optical flow‐based hand motion contour global feature extraction method for acupuncture hand shape. Second, to explore the motion rule between the needle‐holding fingers during the MAM operations, we design a quantitative description method of the relative motion of the needle‐holding fingers: an "interactive attention module," which achieves feature fusion and mines the correlation between different scales of MAM action features. Finally, the proposed MAM recognition method was validated by 20 acupuncturists from the Beijing University of Traditional Chinese Medicine and 10 from the Beijing Zhongguancun Hospital who participated in the MAM video signal collection. The proposed recognition method achieves the highest average validation accuracy of 95.3% and the highest test accuracy of 96.0% for four typical MAMs, proving its feasibility and effectiveness. [ABSTRACT FROM AUTHOR]

Eye movements offer valuable insights for clinical interventions, diagnostics, and understanding visual perception. The process usually involves recording a participant's eye movements and analyzing them in terms of various gaze events. Manual identification of these events is extremely time-consuming. Although the field has seen the development of automatic event detection and classification methods, these methods have primarily focused on distinguishing events when participants remain stationary. With increasing interest in studying gaze behavior in freely moving participants, such as during daily activities like walking, new methods are required to automatically classify events in data collected under unrestricted conditions. Existing methods often rely on additional information from depth cameras or inertial measurement units (IMUs), which are not typically integrated into mobile eye trackers. To address this challenge, we present a framework for classifying gaze events based solely on eye-movement signals and scene video footage. Our approach, the Automatic Classification of gaze Events in Dynamic and Natural Viewing (ACE-DNV), analyzes eye movements in terms of velocity and direction and leverages visual odometry to capture head and body motion. Additionally, ACE-DNV assesses changes in image content surrounding the point of gaze. We evaluate the performance of ACE-DNV using a publicly available dataset and showcased its ability to discriminate between gaze fixation, gaze pursuit, gaze following, and gaze shifting (saccade) events. ACE-DNV exhibited comparable performance to previous methods, while eliminating the necessity for additional devices such as IMUs and depth cameras. In summary, ACE-DNV simplifies the automatic classification of gaze events in natural and dynamic environments. The source code is accessible at https://github.com/arnejad/ACE-DNV. [ABSTRACT FROM AUTHOR]

Looming perception, the ability to sense approaching objects, is crucial for the survival of humans and animals. After hundreds of millions of years of evolutionary development, biological entities have evolved efficient and robust looming perception visual systems. However, current artificial vision systems fall short of such capabilities. In this study, we propose a novel spiking neural network for looming perception that mimics biological vision to communicate motion information through action potentials or spikes, providing a more realistic approach than previous artificial neural networks based on sum-then-activate operations. The proposed spiking looming perception network (SLoN) comprises three core components. Neural encoding, known as phase coding, transforms video signals into spike trains, introducing the concept of phase delay to depict the spatial-temporal competition between phasic excitatory and inhibitory signals shaping looming selectivity. To align with biological substrates where visual signals are bifurcated into parallel ON/OFF channels encoding brightness increments and decrements separately to achieve specific selectivity to ON/OFF contrast stimuli, we implement eccentric down-sampling at the entrance of ON/OFF channels, mimicking the foveal region of the mammalian receptive field with higher acuity to motion, computationally modeled with a leaky integrate-and-fire (LIF) neuronal network. The SLoN model is deliberately tested under various visual collision scenarios, ranging from synthetic to real-world stimuli. A notable achievement is that the SLoN selectively spikes for looming features concealed in visual streams against other categories of movements, including translating, receding, grating, and near misses, demonstrating robust selectivity in line with biological principles. Additionally, the efficacy of the ON/OFF channels, the phase coding with delay, and the eccentric visual processing are further investigated to demonstrate their effectiveness in looming perception. The cornerstone of this study rests upon showcasing a new paradigm for looming perception that is more biologically plausible in light of biological motion perception. [ABSTRACT FROM AUTHOR]

This study conducted modernization of the VK-350 instrumented drop weight impact testing machine (DWITM) developed at the Pisarenko Institute of Problems of Strength of the National Academy of Sciences of Ukraine, equipped with a high-speed strain and force recording system that allows recording a load diagram with a sampling rate of up to 20 MHz, and heating and cooling systems for testing specimens in a wide temperature range. Modernization was carried out to expand the functionality of the DWITM, namely, a new dynamometer and specimen fixing units for impact shear, compression and dynamic pushing tests were installed. The new dynamometer was calibrated for two ranges of amplification and test specimens of various types were tested. The DWITM was also equipped with a video recording system, which includes a high-speed PHOTRON FASTCAM NOVA S9 camera that allows recording the deformation and fracture process at a speed of up to 200,000 frames per second. The camera is equipped with a 12x zoom lens and stroboscopic high-frequency lamps for illumination with the GSVITEC MultiLED G8 control system. Based on the results of the impact bending and shear tests, video signals of the deformation and fracture of the specimens were obtained, which made it possible to determine the time of onset of plastic deformation and formation of adiabatic shear zones in the impact shear specimens, as well as the moment of crack appearance on the side surface of the Charpy specimen and to estimate the rate of its propagation. [ABSTRACT FROM AUTHOR]

There is an increasing need for distress emotion acknowledgement and channeling the same is very important. There is an increasing need for machines to understand human and their complex emotions deeply. This research describes a unique framework for emotion detection that helps brain-computer interface/machine (BCI) to understand human emotions and brain complexity and working using multi-channel electroencephalograms (EEG). We have employed two datasets in our work, Database for Emotion Recognition Through EEG and ECG Signals from Wireless Low-Cost Off-the-Shelf Devices (DREAMER) and dataset for emotion analysis using EEG, physiological and video signals (DEAP) for LSTM model implementation and validation respectively. A linear model is considered for EEG signal mixing and an emotion timing model comprise the framework. Leveraging the contextual correlations within EEG feature sequences, our proposed methodology effectively recovers EEG source signals from the obtained EEG signals, thereby improving the accuracy rate of classification. Also, stress bins were set up for individual users to assess their degree of stress and calmness following exposure to external stimuli. DEAP dataset using LSTM framework was enforced for emotion recognition, and mean recognition accuracy using area under curve as evaluation matrix for valance and arousal was 82.02% and 76.52% respectively, validating competence of framework. Novelty of our work is it improved competency in feature extraction, use of context correlations increasing accuracy and use of spatio-temporal features in the proposed model framework. [ABSTRACT FROM AUTHOR]

Studies in human–human interaction have introduced the concept of F-formation to describe the spatial arrangement of participants during social interactions. This paper has two objectives. It aims at detecting F-formations in video sequences and at predicting the next speaker in a group conversation. The proposed approach exploits time information and multimodal signals of humans in video sequences. In particular, we rely on measuring the engagement level of people as a feature of group belonging. Our approach makes use of a recursive neural network, the Long Short Term Memory (LSTM), to predict who will take the speaker's turn in a conversation group. Experiments on the MatchNMingle dataset led to 85% true positives in group detection and 98% accuracy in predicting the next speaker. • New method for analyzing videos of groups of persons in interaction. • New method for detecting groups (F-formations) using temporal information and multimodal signals. • Consider engagement level between persons as a feature of group belonging. • New method for predicting the next speaker based on LSTM. • Experiments on the MatchNMingle dataset: 85% true positives in group detection, 98% accuracy in predicting next speaker. [ABSTRACT FROM AUTHOR]

Introduction: This study evaluated the accuracy of motion signals extracted from video monitoring data to differentiate epileptic motor seizures in patients with drug-resistant epilepsy. 3D near-infrared video was recorded by the Nelli® seizure monitoring system (Tampere, Finland). Methods: 10 patients with 130 seizures were included in the training dataset, and 17 different patients with 98 seizures formed the testing dataset. Only seizures with unequivocal hyperkinetic, tonic, and tonic-clonic semiology were included. Motion features from the catch22 feature collection extracted from video were explored to transformthe patients' videos into numerical time series for clustering and visualization. Results: Changes in feature generation provided incremental discrimination power to differentiate between hyperkinetic, tonic, and tonic-clonic seizures. Temporal motion features showed the best results in the unsupervised clustering analysis. Using these features, the system differentiated hyperkinetic, tonic and tonic-clonic seizures with 91, 88, and 45% accuracy after 100 cross-validation runs, respectively. F1-scores were 93, 90, and 37%, respectively. Overall accuracy and f1-score were 74%. Conclusion: The selected features of motion distinguished semiological differences within epileptic seizure types, enabling seizure classification to distinct motor seizure types. Further studies are needed with a larger dataset and additional seizure types. These results indicate the potential of video-based hybrid seizure monitoring systems to facilitate seizure classification improving the algorithmic processing and thus streamlining the clinical workflow for human annotators in hybrid (algorithmic-human) seizure monitoring systems. [ABSTRACT FROM AUTHOR]

Recognizing and regulating human emotion or a wave of riding emotions are a vital life skill as it can play an important role in how a person thinks, behaves and acts. Accurate real-time emotion detection can revolutionize the human–computer interaction industry and has the potential to provide a proactive approach to mental health care. Several untapped sources of data, including social media data (psycholinguistic markers), multimodal data (audio and video signals) combined with the sensor-based psychophysiological and brain signals, help to comprehend the affective states and emotional experiences. In this work, we propose a model that utilizes three modalities, i.e., visual (facial expression and body gestures), audio (speech) and text (spoken content), to classify emotion into discrete categories based on Ekman's model with an additional category for 'neutral' state. Transfer learning has been used with multistage fine-tuning for each modality instead of training on a single dataset to make the model generalizable. The use of multiple modalities allows integration of heterogeneous data from different sources effectively. The results of the three modalities are combined at the decision-level using weighted fusion technique. The proposed EmoHD model compares favorably to the state-of-the-art technique on two benchmark datasets MELD and IEMOCAP. [ABSTRACT FROM AUTHOR]

During a basketball game, the ball moves are dynamic, and it is very hard for athletes and trainers to track every move of the ball. An accurate image tracking of a basketball flight path provides the basis for basketball training and other applications. The flight trajectory tracking method based on video signal filtering is studied in this paper. Specifically, the adaptive median filtering algorithm is used to filter the basketball flight video signal. After applying median filtering, the image difference is selected to enhance the basketball trajectory flight images, followed by the Harris corner detection algorithm enhancing the images. Moreover, the SURF algorithm is used to extract features of basketball targets according to the detection results of corner points in the images. Finally, the Particle Swarm Optimization algorithm optimizes the basketball flight trajectory tracking results obtained through the Kalman filter algorithm. The experimental results show that the proposed method can accurately track the flight path of basketball, the real rate is 97%, and the maximum difference between the number of frames and the actual result is 1 frame. The position error and the end position error of the tracking result are both less than 5 cm, which is suitable for basketball training and other practical applications. [ABSTRACT FROM AUTHOR]

Sensitive information in images is can be leaked during attacks, resulting in the malicious acquisition of personal information. To improve the robustness of attacking defence for video images, a digital watermarking algorithm based on locality-sensitive hashing (LSH) is designed in this paper. Video signals are decomposed using a one-dimensional wavelet transform. According to the Yeung Mintzer (Y-M) algorithm, a marker watermark W1 is embedded in the low-frequency subband to identify any image tampering. A data string of hash function values and the exclusive OR (XOR) result of identification watermark W2 are embedded into the HH high-frequency subband, which was used to identify and counter pseudo-authentication attacks such as collage and Vector Quantization (VQ). The singular value decomposition (SVD) algorithm is used to decompose the hash-mapped watermark and adaptively adjust embedding strength of the watermark. The position-sensitive hash algorithm which is proposed in this work has good invisibility for embedding digital watermarks into images, with an average accuracy of approximately 97% for feature matching of digital images. The PSNR value of the image embedded with the watermark is approximately 49 dB. At the 50th minute of the experiment, the regulatory factor value of the research method was 0.3. Under different attack modes, the correlation coefficient between the watermark extracted by this method and the original watermark image is greater than 0.85. Due to the low compression quality of JPGE, the correlation coefficient between the watermark and the initial watermark is greater than 0.6, and its error rate is less than 0.10 bits, showing the effectiveness of the proposed methodologies. [ABSTRACT FROM AUTHOR]

Summary: Deep neural networks (DNN) are increasingly used as components of larger software systems that need to process complex data, such as images, written texts, audio/video signals. DNN predictions cannot be assumed to be always correct for several reasons, amongst which the huge input space that is dealt with, the ambiguity of some inputs data, as well as the intrinsic properties of learning algorithms, which can provide only statistical warranties. Hence, developers have to cope with some residual error probability. An architectural pattern commonly adopted to manage failure prone components is the supervisor, an additional component that can estimate the reliability of the predictions made by untrusted (e.g., DNN) components and can activate an automated healing procedure when these are likely to fail, ensuring that the deep learning‐based system (DLS) does not cause damages, despite its main functionality being suspended. In this paper, we consider DLS that implement a supervisor by means of uncertainty estimation. After overviewing the main approaches to uncertainty estimation and discussing their pros and cons, we motivate the need for a specific empirical assessment method that can deal with the experimental setting in which supervisors are used, where accuracy of the DNN matters only as long as the supervisor lets the DLS continue to operate. Then we present a large empirical study conducted to compare the alternative approaches to uncertainty estimation. We distilled a set of guidelines for developers that are useful to incorporate a supervisor based on uncertainty monitoring into a DLS. [ABSTRACT FROM AUTHOR]

The understanding of the video encoding/decoding processes is a key skill for students of computer science, engineering, or telecommunications, especially those specialized in multimedia. The special characteristics of the video signals, and how such features are leveraged to compress the video frames using coding theory, are hard to understand for engineering students. In this paper, we describe MPEG‐2 analyzer, an educational tool designed to provide a comprehensive and interactive way to understand MPEG‐2 video compression and encoding techniques. We have used our tool to evaluate its usefulness in reinforcing the knowledge about the MPEG‐2 video (de)coding process to Telecommunication Engineering students. Results show that the evaluation of the tool is very positive, and it has helped them to understand many of the complex concepts covered in the course. [ABSTRACT FROM AUTHOR]

On 6 January 1973, Chilean media artist Juan Downey exhibited Plato Now at the Everson Museum of Art in Syracuse, New York. A hybrid multichannel video installation and performance, this was Downey's restaging of Plato's Parable of the Cave—the "Now" registering what the myth might look like from the vantage of his own historical moment. And whereas Plato's original operated through an inflexible division between the space of the mind and a derivative sensual reality, Plato Now explicitly sought to blur those philosophical lines by assembling a relay of invisible energies, brain waves, video signals, and telepathic communications, such that the space of mind and sensual reality became speculatively entangled. This article clarifies just how Plato Now did this, and situates its philosophical vision as a significant, if relatively unremarked, aesthetic prefiguration of the new materialist tendencies towards relationality, hybrid assemblages, and vibrant conceptions of energy and matter. [ABSTRACT FROM AUTHOR]

Hybrid digital–analog transmission of video signals enables flexibility in dividing video information into two parts to utilize the available bandwidth better. This study proposes a compression scheme to reduce the utilized bandwidth. The scheme uses different subsampling in three-dimensional (3D) blocks, where subsampling factors are selected to minimize reconstruction distortion. The study evaluates various methods for subsampling and reconstruction to find the best combination in terms of reconstruction quality and complexity. Results show that medium-quality reconstructions can be obtained for compression ratios of about 0.125–0.3 samples per pixel. [ABSTRACT FROM AUTHOR]

This paper proposed a novel, efficient method to estimate challenging small structural motions from noisy video. To eliminate the phase limitation, ill‐posed problem, and high computational burden, the structural motion function is resampled and recovered. Because video signals have tremendous redundancies in spatial, block, and time domains, the objective is achieved by enforcing three levels of sparsity constraints. In the first step, the ill‐posed optimization is solved using the multi‐hypothesis prediction embedded with the Tikhonov regularization, to resample and recover the video signal in the spatial domain. The second step is to characterize the measurement uncertainty in the block domain, in which two weight matrices are proposed into the regularization terms of the sparse coding. Finally, the temporal correlation of video frames is exploited by the reweighted residual sparsity. The superiority of the sparsity‐enforcement method over existing methods was demonstrated through several case studies. Among the comparisons, the sparsity‐enforcement method yielded video magnifications, structural motions, and modal information more accurately. Meanwhile, it has the lowest computational burden. [ABSTRACT FROM AUTHOR]

Due to the strong correlation between symmetric frames, video signals have a high degree of temporal redundancy. Motion estimation techniques are computationally expensive and time-consuming processes used in symmetric video compression to reduce temporal redundancy. The block-matching technique is, on the other hand, the most popular and efficient of the different motion estimation and compensation techniques. Motion compensation based on the block-matching technique generally uses the minimization of either the mean square error (MSE) or mean absolute difference (MAD) in order to find the appropriate motion vector. This paper proposes to remove the highly temporally redundant information contained in each block of the video signal using the removing temporal redundancy (RTR) technique in order to improve the data rate and efficiency of the video signal. A comparison between the PSNR values of this technique and those of the JPEG video compression standard is made. As a result of its moderate memory and computation requirements, the algorithm was found to be suitable for mobile networks and embedded devices. Based on a detailed set of testing scenarios and the obtained results, it is evident that the RTR compression technique allowed a compression ratio of 22.71 and 95% loss in bit rate reduction while maintaining sufficient intact signal quality with minimized information loss. [ABSTRACT FROM AUTHOR]

Emotion recognition commonly relies on single-modal recognition methods, such as voice and video signals, which demonstrate a good practicability and universality in some scenarios. Nevertheless, as emotion-recognition application scenarios continue to expand and the data volume surges, single-modal emotion recognition proves insufficient to meet people's needs for accuracy and comprehensiveness when the amount of data reaches a certain scale. Thus, this paper proposes the application of multimodal thought to enhance emotion-recognition accuracy and conducts corresponding data preprocessing on the selected dataset. Appropriate models are constructed for both audio and video modalities: for the audio-modality emotion-recognition task, this paper adopts the "time-distributed CNNs + LSTMs" model construction scheme; for the video-modality emotion-recognition task, the "DeepID V3 + Xception architecture" model construction scheme is selected. Furthermore, each model construction scheme undergoes experimental verification and comparison with existing emotion-recognition algorithms. Finally, this paper attempts late fusion and proposes and implements a late-fusion method based on the idea of weight adaptation. The experimental results demonstrate the superiority of the multimodal fusion algorithm proposed in this paper. When compared to the single-modal emotion-recognition algorithm, the accuracy of recognition is increased by almost 4%, reaching 84.33%. [ABSTRACT FROM AUTHOR]

Emotion recognition is a crucial aspect of human communication, with applications in fields such as psychology, education, and healthcare. Identifying emotions accurately is challenging, as people use a variety of signals to express and perceive emotions. In this study, we address the problem of multimodal emotion recognition using both audio and video signals, to develop a robust and reliable system that can recognize emotions even when one modality is absent. To achieve this goal, we propose a novel architecture based on well-designed feature extractors for each modality and use model-level fusion based on a TFusion block to combine the information from both sources. To be more efficient in real-world scenarios, we trained our model on a compound dataset consisting of RAVDESS, RML, and eNTERFACE'05. It is then evaluated and compared to the state-of-the-art models. We find that our approach performs close to the modern solutions and can recognize emotions accurately when one of the modalities is missing. Additionally, we have developed a real-time emotion recognition application as a part of this work. [ABSTRACT FROM AUTHOR]

Anxiety is characterized by anxious anticipation and heightened vigilance to uncertain threat. However, if threat is not reliably indicated by a specific cue, the context in which threat was previously experienced becomes its best predictor, leading to anxiety. A suitable means to induce anxiety experimentally is context conditioning: In one context (CTX+), an unpredictable aversive stimulus (US) is repeatedly presented, in contrast to a second context (CTX−), in which no US is ever presented. In this EEG study, we investigated attentional mechanisms during acquisition and extinction learning in 38 participants, who underwent a context conditioning protocol. Flickering video stimuli (32 s clips depicting virtual offices representing CTX+/−) were used to evoke steady‐state visual evoked potentials (ssVEPs) as an index of visuocortical engagement with the contexts. Analyses of the electrocortical responses suggest a successful induction of the ssVEP signal by video presentation in flicker mode. Furthermore, we found clear indices of context conditioning and extinction learning on a subjective level, while cortical processing of the CTX+ was unexpectedly reduced during video presentation. The differences between CTX+ and CTX− diminished during extinction learning. Together, these results indicate that the dynamic sensory input of the video presentation leads to disruptions in the ssVEP signal, which is greater for motivationally significant, threatening contexts. The current results contribute to our understanding of the visuocortical processing of more ecologically valid context stimuli and its modulation during situations of potential threat. Besides clear indices of aversive learning, we provide evidence for a disruption of the visuocortical signal by the dynamic sensory input which is enhanced for motivationally significant stimuli. [ABSTRACT FROM AUTHOR]

Copyright of Journal of South China University of Technology (Natural Science Edition) is the property of South China University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Owing to the recent trends in remote health monitoring, real-time applications for measuring Heartbeat Rate and Respiration Rate (HARR) from video signals are growing rapidly. Photo Plethysmo Graphy (PPG) is a method that is operated by estimating the infinitesimal change in color of the human face, rigid motion of facial skin and head parts, etc. Ballisto Cardiography (BCG) is a nonsurgical tool for obtaining a graphical depiction of the human body's heartbeat by inducing repetitive movements found in the heart pulses. The resilience against motion artifacts induced by luminance fluctuation and the patient's mobility variation is the major difficulty faced while processing the real-time video signals. In this research, a video-based HARR measuring framework is proposed based on combined PPG and BCG. Here, the noise from the input video signals is removed by using an Adaptive Kalman filter (AKF). Three different algorithms are used for estimating the HARR from the noise-free input signals. Initially, the noise-free signals are subjected to Modified Adaptive Fourier Decomposition (MAFD) and then to Enhanced Hilbert vibration Decomposition (EHVD) and finally to Improved Variation mode Decomposition (IVMD) for attaining three various results of HARR. The obtained values are compared with each other and found that the EHVD is showing better results when compared with all the other methods. [ABSTRACT FROM AUTHOR]

Featured Application: The results of the research may be helpful in the setup of video quality assessment procedures in order to achieve results as close as possible to the quality experienced by the end users of the video streaming services. The paper presents the results of subjective and objective quality assessments of H.264-, H.265-, and VP9-encoded video. Most of the literature is devoted to subjective quality assessment in well-defined laboratory circumstances. However, the end users usually watch the films in their home environments, which may be different from the conditions recommended for laboratory measurements. This may cause significant differences in the quality assessment scores. Thus, the aim of the research is to show the impact of environmental conditions on the video quality perceived by the user. The subjective assessment was made in two different environments: in the laboratory and in users' homes, where people often watch movies on their laptops. The video signal was assessed by young viewers who were not experts in the field of quality assessment. The tests were performed taking into account different image resolutions and different bit rates. The research showed strong correlations between the obtained results and the coding bit rates used, and revealed a significant difference between the quality scores obtained in the laboratory and at home. As a conclusion, it must be underlined that the laboratory tests are necessary for comparative purposes, while the assessment of the video quality experienced by end users should be performed under circumstances that are as close as possible to the user's home environment. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We demonstrate the transmission of uncompressed 4K videos over the photonics‐based terahertz (THz) wireless link using a directly modulated distributed feedback laser diode (DFB‐LD). For optical heterodyne mixing and data modulation, a DFB‐LD was employed and directly modulated with a 5.94‐Gb/s non‐return‐to‐zero signal, which is related to a 6G‐serial digital interface standard to support ultra‐high‐definition video resolution. We derived the optimal frequency of the THz carrier by varying the wavelength difference between DFB‐LD output and Tunable LD output in the THz signal transmitter to obtain the best transmission performances of the uncompressed 4K video signals. Furthermore, we exploited the negative laser‐to‐filter detuning for the adiabatic chirp management of the DFB‐LD by the intentional discrepancy between the center wavelength of the optical band‐pass filter and the output wavelength of the DFB‐LD. With the help of the abovementioned methods, we successfully transmitted uncompressed 4K video signals over the 2.3‐m wireless transmission distance without black frames induced by time synchronization error. [ABSTRACT FROM AUTHOR]

The identification and classification of the herbs using the naked eye is difficult in forest or mountain areas like Marunthuvazh Malai of Kanyakumari district. The difficulties arise because of the variations in the crops identified are inaccurate. Mostly the manual prediction is taken place in those areas which require high expertise and more human resources. In this work both plant identification and tracking system based on fuzzy empowered Hybrid artificial neural networks (FHANN) are proposed. Here the input is taken from the video signals taken by the drone camera. The input video signals are converted into images. The fuzzy logic along with the HANN is used for the classification of the specific herbs from the set of plants. Some of the herbs included in the analysis are Parsley, Dill, Oregano, Chervil, Stevia, Basil, Catnip, Fennel and Lemon Grass. This approach used artificial neural networks (ANN) in combination with the K-Nearest neighbor (KNN) as the hybrid model for the herb prediction and classification in association with the fuzzy logic. The Linear Discriminant Analysis (LDA) and Convolutional Autoencoder are used as a hybrid model for the extraction of the feature from the obtained images. This approach considers various shapes, color features, and textures specifically representing the specific herbs. The experimental results show that the proposed model provides better results in the identification and classification of the various medicinal herbs. [ABSTRACT FROM AUTHOR]

Multimodal fusion is the idea of combining information in a joint representation of multiple modalities. The goal of multimodal fusion is to improve the accuracy of results from classification or regression tasks. This paper proposes a Bimodal Variational Autoencoder (BiVAE) model for audiovisual features fusion. Reliance on audiovisual signals in a speech recognition task increases the recognition accuracy, especially when an audio signal is corrupted. The BiVAE model is trained and validated on the CUAVE dataset. Three classifiers have evaluated the fused audiovisual features: Long-short Term Memory, Deep Neural Network, and Support Vector Machine. The experiment involves the evaluation of the fused features in the case of whether two modalities are available or there is only one modality available (i.e., cross-modality). The experimental results display the superiority of the proposed model (BiVAE) of audiovisual features fusion over the state-of-the-art models by an average accuracy difference ≃ 3.28% and 13.28% for clean and noisy, respectively. Additionally, BiVAE outperforms the state-of-the-art models in the case of cross-modality by an accuracy difference ≃ 2.79% when the only audio signal is available and 1.88% when the only video signal is available. Furthermore, SVM satisfies the best recognition accuracy compared with other classifiers. [ABSTRACT FROM AUTHOR]

Emotions play a crucial role in human-human communications with complex socio-psychological nature. In order to enhance emotion communication in human-computer interaction, this paper studies emotion recognition from audio and visual signals in video clips, utilizing facial expressions and vocal utterances. Thereby, the study aims to exploit temporal information of audio-visual cues and detect their informative time segments. Attention mechanisms are used to exploit the importance of each modality over time. We propose a novel framework that consists of bi-modal time windows spanning short video clips labeled with discrete emotions. The framework employs two networks, with each one being dedicated to one modality. As input to a modality-specific network, we consider a time-dependent signal deriving from the embeddings of the video and audio modalities. We employ the encoder part of the Transformer on the visual embeddings and another one on the audio embeddings. The research in this paper introduces detailed studies and meta-analysis findings, linking the outputs of our proposition to research from psychology. Specifically, it presents a framework to understand underlying principles of emotion recognition as functions of three separate setups in terms of modalities: audio only, video only, and the fusion of audio and video. Experimental results on two datasets show that the proposed framework achieves improved accuracy in emotion recognition, compared to state-of-the-art techniques and baseline methods not using attention mechanisms. The proposed method improves the results over baseline methods by at least 5.4%. Our experiments show that attention mechanisms reduce the gap between the entropies of unimodal predictions, which increases the bimodal predictions' certainty and, therefore, improves the bimodal recognition rates. Furthermore, evaluations with noisy data in different scenarios are presented during the training and testing processes to check the framework's consistency and the attention mechanism's behavior. The results demonstrate that attention mechanisms increase the framework's robustness when exposed to similar conditions during the training and the testing phases. Finally, we present comprehensive evaluations of emotion recognition as a function of time. The study shows that the middle time segments of a video clip are essential in the case of using audio modality. However, in the case of video modality, the importance of time windows is distributed equally. [ABSTRACT FROM AUTHOR]

In the present paper, we consider the issue of improving the accuracy of measurements and the peculiar features of the measurements of the geometric parameters of objects by optoelectronic systems, based on a television multiscan in the analogue mode in scanistor enabling. It is shown that the convolution of the input signal of the rectangular profile of the light zone with the impulse response of the multiscan explains the smearing of video signal edges. The value of smearing is completely determined by impulse response and does not depend on the zone width. It is established that signals of the action from narrow light zones are smeared out substantially relative to their width, and in this case, the conventional method for determining the zone width by the video signal derivatives stops working. A method is proposed for a full reconstruction of the rectangular profile of a light zone (including a narrow one) in a developed noise-proof optoelectronic measuring device based on a multiscan with the use of spatio-structural parameters, where the width of a light zone corresponds to an extent, illumination of the zone—to brightness, and its coordinate—to centroid. It is shown that the subtraction of the dissipation of the impulse response from the video signal dissipation allows for accounting of the value of the video signal smearing out with high accuracy at the estimation of the width of a light zone, in accordance with the property of the additive accumulation of dissipation. It is established that the use of the spatio-structural video signal model permits the extension of the application of multiscan-based optoelectronic devices for the high-accuracy control and measurement of the geometric parameters of small objects. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) is a promising contactless technology that uses videos of faces to extract health parameters, such as heart rate. Several methods for transforming red, green, and blue (RGB) video signals into rPPG signals have been introduced in the existing literature. The RGB signals represent variations in the reflected luminance from the skin surface of an individual over a given period of time. These methods attempt to find the best combination of color channels to reconstruct an rPPG signal. Usually, rPPG methods use a combination of prepossessed color channels to convert the three RGB signals to one rPPG signal that is most influenced by blood volume changes. This study examined simple yet effective methods to convert the RGB to rPPG, relying only on RGB signals without applying complex mathematical models or machine learning algorithms. A new method, GRGB rPPG, was proposed that outperformed most machine-learning-based rPPG methods and was robust to indoor lighting and participant motion. Moreover, the proposed method estimated the heart rate better than well-established rPPG methods. This paper also discusses the results and provides recommendations for further research. [ABSTRACT FROM AUTHOR]

This study describes important techniques for production of a series of video signals for use in fine focusing operations and near-perfect astigmatism correction in the general-purpose scanning electron microscopy (SEM) field. These techniques can enhance the stability of the signal greatly when used for focusing. As two particularly important fundamental techniques, SEM image acquisition with priority given to the signal-to-noise ratio and signal reinforcement based on the active image processing concept were utilized fully. The performance improvement was evaluated using the case of a previously reported support system for fine focusing and astigmatism correction based on image covariance. The method is almost completely robust against noise within practical limits and allows for focusing and astigmatism correction for even extremely noisy SEM images. The results of this study may be useful not only in the SEM field but also in many fields that use weak signals. [ABSTRACT FROM AUTHOR]

Cases of heavy traffic congestion (commonly referred to as traffic jams) have negative influences on the economy, public health, and social life. This makes it important to assist drivers in these situations with Internet of Things (IoT) systems. The aim of this article is to present a low-cost device that supports driving in traffic jams. The system, which is based on data from a laser sensor with a range of up to 40 m, controls the distances to vehicles in front of and behind a moving car. Through audio and video signals, it warns the driver whether the vehicle in front of or behind the equipped vehicle is too close. This allows the driver to control the distance and avoid minor collisions. [ABSTRACT FROM AUTHOR]

As a major daily task for the popularization of artificial intelligence technology, more and more attention has been paid to the scientific research of mental state electroencephalogram (EEG) in recent years. To retain the spatial information of EEG signals and fully mine the EEG timing-related information, this paper proposes a novel EEG emotion recognition method. First, to obtain the frequency, spatial, and temporal information of multichannel EEG signals more comprehensively, we choose the multidimensional feature structure as the input of the artificial neural network. Then, a neural network model based on depthwise separable convolution is proposed, extracting the input structure's frequency and spatial features. The network can effectively reduce the computational parameters. Finally, we modeled using the ordered neuronal long short-term memory (ON-LSTM) network, which can automatically learn hierarchical information to extract deep emotional features hidden in EEG time series. The experimental results show that the proposed model can reasonably learn the correlation and temporal dimension information content between EEG multi-channel and improve emotion classification performance. We performed the experimental validation of this paper in two publicly available EEG emotional datasets. In the experiments on the DEAP dataset (a dataset for emotion analysis using EEG, physiological, and video signals), the mean accuracy of emotion recognition for arousal and valence is 95.02% and 94.61%, respectively. In the experiments on the SEED dataset (a dataset collection for various purposes using EEG signals), the average accuracy of emotion recognition is 95.49%. [ABSTRACT FROM AUTHOR]

Automatic human posture recognition in surveillance videos has real world applications in monitoring old-homes, restoration centers, hospitals, disability, and child-care centers. It also has applications in other areas such as security and surveillance, sports, and abnormal activity recognition. Human posture recognition is a challenging problem due to occlusion, background clutter, illumination variations, camouflage, and noise in the captured video signal. In the current study, which is an extension of our previous work (Ali et al. Sensors, 18(6):1918, 2018), we propose a novel combination of a number of spatio-temporal features computed over human blobs in a temporal window. These features include aspect ratios, shape descriptors, geometric centroids, ellipse axes ratio, silhouette angles, and silhouette speed. In addition to these features, we also exploit the radon transform to get better shape based analysis. In order to obtain improved posture classification accuracy, we used J48 classifier under a boosting framework by employing the AdaBoost algorithm.The proposed algorithm is compared with eighteen existing state-of-the-art approaches on four publicly available datasets including MCF, UR Fall detection, KARD, and NUCLA. Our results demonstrate the excellent performance of the proposed algorithm compared to these existing methods. [ABSTRACT FROM AUTHOR]