Search

The development of automatic video understanding technology is highly needed due to the rise of mass video data, like surveillance videos, personal video data. Several methods have been presented previously for automatic video captioning. But, the existing methods have some problems, like more time consume during processing a huge number of frames, and also it contains over fitting problem. This is a difficult task to automate the process of video caption. So, it affects final result (Caption) accuracy. To overcome these issues, Automatic Video Captioning using Tree Hierarchical Deep Convolutional Neural Network and attention segmental recurrent neural network-bi-directional Long Short-Term Memory (ASRNN-bi-directional LSTM) is proposed in this paper. The captioning part contains two phases: Feature Encoder and Decoder. In feature encoder phase, the tree hierarchical Deep Convolutional Neural Network (Tree CNN) encodes the vector representation of video and extract three kinds of features. In decoder phase, the attention segmental recurrent neural network (ASRNN) decode vector into textual description. ASRNN-base methods struck with long-term dependency issue. To deal this issue, focuses on all generated words from the bi-directional LSTM and caption generator for extracting global context information presented by concealed state of caption generator is local and unfinished. Hence, Golden Eagle Optimization is exploited to enhance ASRNN weight parameters. The proposed method is executed in Python. The proposed technique achieves 34.89%, 29.06% and 20.78% higher accuracy, 23.65%, 22.10% and 29.68% lesser Mean Squared Error compared to the existing methods. [ABSTRACT FROM AUTHOR]

Special load customers such as electric vehicles are emerging in modern power systems. They lead to a higher penetration of special load patterns, raising difficulty for short-term load forecasting (STLF). We propose a hierarchical STLF framework to improve load forecasting accuracy. An improved adaptive K-means clustering algorithm is designed for load pattern recognition and avoiding local sub-optimal clustering centroids. We also design bi-directional long-short-term memory neural networks with an attention mechanism to filter important load information and perform load forecasting for each recognized load pattern. The numerical results on the public load dataset show that our proposed method effectively forecasts the residential load with a high accuracy. [ABSTRACT FROM AUTHOR]

Twitter has emerged as a powerful Microblogging site in recent years. It has become one of the immediately reported platforms about any crisis in today's world. During a crisis twitter is flooded with both informational and non-informational tweets. Informational tweets are the tweets which can help humanitarian response organizations or the victims to get useful information for better decision making This paper proposes a Semantic based Ontology Infused deep learning model for Informative tweet classification during natural disasters. The proposed model makes use of both text and image features present in a tweet during a classification, a Hierarchical ensemble of Vanilla and Bi-LSTM's is used along with ontologies and external knowledge from OntoCollab for text classification, a dilated convolution-based residual neural network is used for image classification, and a meaningful statistical fusion is performed with the results obtained from text and image for tweet classification. The usage of both image and text features and contextual understanding made the model distinctive and achieved exceptional results. The proposed method has proven, after experiments, to achieve evaluation metrics better than the up-to-date existing methods; evaluated on seven different datasets and compared with eight other methods with an increase in Accuracy and decrease in FNR for seven distinguished datasets which resulted an highest F1 score of 98.416% on Sri Lanka Floods dataset. [ABSTRACT FROM AUTHOR]

As extreme weather becomes more prevalent, it has serious implications for the insurance industry, real estate owners, and historic buildings. First of all, for the extreme weather, we summarize extreme weather into three parts: extreme temperature, extreme precipitation, and extreme humidity, and we process the data by combining the actual situation and applying the improved Critic method, and we demonstrate the data processing by taking the three data of Lhasa and Luxor as examples. anomalies. Subsequently, a future extreme climate prediction model for a region based on the Bi-directional LSTM method is developed. A time-weather model was fitted based on the collected data, with the highest and lowest points as the anomalies. At the same time, we simplify the formula to derive the insurance company's revenue, thus modeling the relationship between the degree of extreme weather hazard and insurance revenue, and derive that the insurance company will not be able to underwrite when the combined hazard is greater than the ratio of the insurance claim cost to the insurance payment. We applied the resulting model to Lhasa and Luxor, and plotted the combined hazard level of the actual data and the predicted data to help the insurance company visualize the combined hazard level of the region each month, and then choose whether to underwrite the policy in that region or not. [ABSTRACT FROM AUTHOR]

Satellite time series data, widely used for land cover classification, often contain missing values due to cloud contamination, which can negatively affect classification. Numerous strategies have been developed to reconstruct the missing values to produce regular time series for machine learning classifiers, among which the compositing followed by the linear interpolation is most widely used. However, the classification improvement of linear interpolation for land cover classification has not been examined. Recently developed deep learning models such as long short term memory (LSTM) and Transformer allow such examination as they can classify time series with missing values. In this study, we compared the time series composites with missing values (without linear interpolation) and the linearly interpolated time series composites (without missing values) for land cover classification. About 18 thousand Harmonized Landsat Sentinel-2 (HLS) images acquired over Amur River Basin of China (890,308 km2) in 2021 were composited to 14 16-day periods. Two time series composites were classified, i.e., (i) the 16-day composites without interpolation that have on average 15.35% 16-day periods with missing values and (ii) the linearly interpolated 16-day composites with no missing values. The classifications showed that (1) between classifications with and without linear interpolation there was < 0.2% overall accuracy differences for the bidirectional LSTM (Bi-LSTM) and < 0.5% for the Transformer both of which were smaller than model training randomness; and (2) the computation time can be saved using composites without linear interpolation. The findings suggested that it is unnecessary to use the time-consuming linear interpolation in Bi-LSTM and Transformer-based land cover classifications. The findings were confirmed by experiments for sensitivity to the number of cloud-free composites and to different classification legends using crop type classifications. It implied the linear interpolation algorithm cannot reconstruct reliable time series for land cover classifications and historical use of such method is more about mitigating the inability of traditional classifiers to handle missing values rather than improving classifications. Linear interpolation is not necessary for LSTM and Transformer with capability to handle missing values. The training datasets and developed codes in this study are made publicly available. [ABSTRACT FROM AUTHOR]

Lithium-ion batteries play an irreplaceable role in various areas of production and life as an efficient energy storage element. The state of health (SOH) of lithium-ion batteries is critical to the safe operation of the energy storage system. In this study, measurable data from the battery management system, such as voltage, current and temperature profiles that change due to aging, were used. Based on these data, the relationship between capacity and charging profiles is learned by neural networks. The experimental results achieved in this study are based on the NASA lithium-ion battery data set. It has been observed that compared to the LSTM, GRU and BiLSTM methods, which are deep learning methods, the proposed GRU method with attention mechanism is more successful in estimating the health of the battery in terms of average absolute percentage error by up to 35%, 27% and 20%, respectively. The simulation studies were carried out using the deep learning toolbox in the MATLAB environment. In recent years, attention mechanisms have emerged as a powerful tool to improve the performance of time series forecasting models. In this study, LSTM, BiLSTM and GRU, which are used to solve time series problems, were tested on the same NASA data sets, and GRU was preferred because it is faster and simpler than these three methods. The mechanism proposed in this study is the SoH prediction mechanism created by combining GRU and Attention Mechanism. [ABSTRACT FROM AUTHOR]

The growing significance of online reviews in e-commerce platforms has a profound impact on consumers' purchase decisions. Unfortunately, dishonest sellers exploit this system by engaging in deceptive practices and manipulating reviews. These fake reviews are very hard to be detected by humans. Hence, the issue of detecting fake online reviews has gained significant attention due to their prevalence. Typically, the datasets available have much higher number of real reviews as compared to fake reviews, which leads to data imbalance problem. Machine learning algorithms struggle to learn from these datasets and give biased results. However, existing studies on this topic have not adequately addressed the challenges posed by imbalanced data. They have mostly used under-sampling or over-sampling to handle imbalance in datasets, which leads to potential information loss or overfitting due to replication. Hence, to handle this class imbalance problem without using sampling techniques, this paper proposes a novel SBiLM model using Siamese Neural Network with Bidirectional LSTM and Manhattan Distance. SBiLM takes pair of reviews as inputs and computes similarity between them using distance metric. The outcomes indicate that the SBiLM model exhibits better performance in terms of both accuracy and loss, when compared with traditional deep learning models on a highly imbalanced restaurant review dataset. This presents a promising strategy for tackling the issues related to data imbalance in the domain of fake review detection. [ABSTRACT FROM AUTHOR]

In the recent scenario, it is important to extract the text from various formats, including handwritten and documents. The ability to accurately detect and recognize text, is a crucial task in many fields, such as OCR systems, document analysis, and image processing. Convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM) are used in a unique method for text detection. Additionally, CNN consist of six sequential layers by using Adam optimizer and SoftMax. The proposed approach is designed effectively to handle the variability and complexity of text in both handwritten and document formats. The proposed approach achieved a significant improvement in text detection and recognition accuracy, with 88.5 percentage while comparing with the existing techniques of CNN and LSTM which has an accuracy of 81.8 percentage. [ABSTRACT FROM AUTHOR]

In short-term load forecasting, deep learning models with recurrent units are widely used. For Long Short-term Memory (LSTM) models, in terms of the input features at one particular time-step, there is a weight series for each input feature. Hence, an input feature with a larger value of weight can be regarded as more important to the final forecasting result. Since the weights of the features are time-invariant, the model considers the input features as equally important at each time step, thus lack of the capability of reflecting the importance fluctuation through the time span. To tackle this issue, this paper proposes a forecasting scheme, which integrates mutual information (MI) into bi-directional long short-term memory (BILSTM) network. The MI method is used to extract the importance value of input features at different time-steps, and constitute the fluctuation matrix of the importance values for input features, which is used as coefficients to correct the original input feature. The corrected features are substituted into the BILSTM network, which further improves the forecasting accuracy. Numerical experiments have been carried out based on real-world load data. The results demonstrate the superior performance of the proposed scheme compared with a series of state-of-the-art methods and thus indicate the importance and effectiveness of capturing the fluctuation of the importance value.

Automatic seizure detection using electroen-cephalogram (EEG) can significantly expedite the diagnosis of epilepsy, thereby facilitating prompt treatment and reducing the risk of future seizures and associated complications. While most existing EEG-based epilepsy detection studies employ deep learning models, they often ignore the chronological relationships between different EEG channels. To tackle this limitation, a novel automatic epilepsy detection method is proposed, which leverages path signature and Bidirectional Long Short-Term Memory (Bi-LSTM) neural network with an attention mechanism. The path signature algorithm is used to extract discriminative features for capturing the dynamic dependencies between different channels of EEG, while Bi-LSTM with attention further analyzes the inherent temporal dependencies hidden in EEG signal features. Our method is evaluated on two public EEG databases with different sizes (CHB-MIT and TUEP) and a private database from a local hospital. Two experimental settings are used, i.e., five-fold cross-validation and leave-one-out cross-validation. Experimental results show that our method achieves 99.09%, 95.60%, and 99.87% average accuracies on CHB-MIT, TUEP with 250Hz, and TUEP with 256Hz, respectively. On the private dataset, our method also achieves 99.40% average accuracy, which outperforms other methods. Furthermore, our method exhibits robustness in patients, as demonstrated by the evaluation results of cross-patient experiments.

Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model's recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%. [ABSTRACT FROM AUTHOR]

Cloud computing is a new technology that provides services to customers anywhere, anytime, under varying conditions and managed by a third-party cloud provider. Even though cloud computing has progressed a lot, some attacks still happen. The recent anomalous and signature attacks use clever strategies such as low-rate attacks and attacking as an authenticated user. In this paper, a novel Attack Detection and Prevention (ADAPT) method is proposed to overcome this issue. The proposed system consists of three stages. An Intrusion Detection System is initially used to check whether there is an attack or not by comparing the IP address in the Blacklist IP Database. If an attack occurs, the IP address will be added to the Blacklist IP database and blocked. The second stage uses Bi-directional LSTM and Bi-directional GRU to check the anomalous and signature attack. In the third stage, classified output is sent to reinforcement learning, if any attack occurs the IP address is added to the blacklist IP database otherwise the packets are forwarded to the user. The proposed ADAPT technique achieves a higher accuracy range than existing techniques. [ABSTRACT FROM AUTHOR]

The solution of the distribution system state estimation (DSSE) relies on the presence of physical measurements in real time. Sometimes, these measurements may not reach the control center due to the defects in meter functionality, the large communication time delays, and denial-of-service (DoS) attacks on communication channels. Addressing this issue, a novel deep learning (DL) approach is proposed by using a transformer model with the integration of bi-directional long short-term memory (Bi-LSTM) layer. The proposed model can be leveraged to forecast the unavailable measurement data, which maintains the observability of the system to obtain an accurate solution from DSSE. The superiority of the proposed model is probed by comparing it with other forecasting-based DL models at various percentages of missing measurement data. Further, the effectiveness of the proposed model is evaluated for forecasting the power injections of residential, commercial, and industrial loads as well as renewable energy sources. Finally, the solution of the DSSE is tested with forecasted data and compared with the results of the DSSE for the original measurement data. The simulations are performed on modified IEEE 13-node and IEEE 37-node test systems, and their corresponding results highlighted the superiority of the proposed model in the forecasting of the incomplete data. [ABSTRACT FROM AUTHOR]

Special load customers such as electric vehicles are emerging in modern power systems. They lead to a higher penetration of special load patterns, raising difficulty for short-term load forecasting (STLF). We propose a hierarchical STLF framework to improve load forecasting accuracy. An improved adaptive K-means clustering algorithm is designed for load pattern recognition and avoiding local sub-optimal clustering centroids. We also design bi-directional long-short-term memory neural networks with an attention mechanism to filter important load information and perform load forecasting for each recognized load pattern. The numerical results on the public load dataset show that our proposed method effectively forecasts the residential load with a high accuracy.

Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model’s recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%.

One of the high-risk behaviors leading to severe traffic injuries is not wearing a seat belt. It is therefore very important to be able to automatically detect seat belts from surveillance images, encourage drivers to wear seat belts, and enhance passenger safety. In this paper, a novel deep neural network, Gated Bi-directional Long Short-Term Memory network with part-to-whole attention (GBL-PA), is proposed for seat belt detection from surveillance images. The innovation of our model lies in its unique diagonal sampling strategy, which meticulously captures the seat belt's fine details, typically oriented from top right to bottom left across the torso of vehicle occupants. Our framework's novelty is further encapsulated by the part-to-whole attention mechanism, which intelligently harmonizes the detailed local information from seat belt-specific patches with the broader contextual insights from the regional proposals. The pioneering design of a Gated Bi-directional LSTM network facilitates the dynamic integration of interactions across patches to deliver an optimized final prediction. The superiority of GBL-PA is established through rigorous comparison with the state-of-the-art methods on a new, large benchmark dataset comprising 14,936 images from traffic surveillance footage. Our framework demonstrates a notable improvement, achieving a mean Average Precision (mAP) of 72.3%, which surpasses the second best, YOLOX, by 0.9% mAP. This significant and consistent outperformance across various metrics underscores the transformative potential of GBL-PA in the realm of traffic safety enforcement. The source code of our framework is available at ANONYMISED. • Novel seat belt detection framework exploiting inherent properties of seat belts. • Patch Diagonal Sampling obtains patches to capture fine-grained seat belt details. • Part-to-whole Attention exploits attentive patch information under global context. • Gated Bi-LSTM captures interactions among patches to aggregate all information. • The framework greatly outperforms SoTA methods on the large, constructed dataset. [ABSTRACT FROM AUTHOR]

Retrieving relevant source code from large repositories is a significant and ongoing challenge in the field of software engineering, primarily due to the vast and ever-expanding amount of available code. Existing deep learning methods, although effective to some extent, exhibit limitations in capturing the intricate and complex structural information embedded within source code, which hinders their ability to provide highly accurate retrieval results. This study endeavors to tackle this prominent issue by introducing a novel and innovative approach known as the Joint Bi-directional LSTM and Graph Neural Networks (JBLG) model for source code retrieval. The central aim is to harness the combined strengths and capabilities of Bi-directional Long Short-Term Memory (LSTM) networks and Graph Neural Networks (GNNs) to significantly enhance the model's capacity to capture and interpret the complex structural characteristics intrinsic to source code. The proposed JBLG model employs a unique fusion of Bi-directional LSTM, which excels in capturing sequential and temporal dependencies within code, and GNN, which is adept at modeling the intricate graph structure of the code. By leveraging this hybrid architecture, the model aims to provide a comprehensive and highly effective solution for source code retrieval tasks. To assess the efficacy of the JBLG model, extensive experiments are conducted, and the model's performance is evaluated against well-established benchmarks, including LSTM, GNN, and ChatGPT, using two diverse datasets: CodeSearchNet and CosBench datasets. These evaluations span multiple programming languages, ensuring a comprehensive and robust assessment of the model's capabilities. The experimental results indicate that the JBLG model consistently outperforms its counterparts, including Bi-LSTM, GNN, ChatGPT, and DGMS, across various evaluation metrics. the JBLG model showcases an exceptional ability to handle and extract the intricate structural information inherent in source code, resulting in significantly enhanced retrieval accuracy. The JBLG model emerges as a highly promising solution for real-world source code retrieval applications, with the potential to revolutionize the field. The success of this model underscores the importance of combining deep learning techniques like Bi-directional LSTM and GNNs for tackling complex software engineering challenges. Furthermore, future research directions could involve exploring advanced techniques such as attention mechanisms and extending the model's applicability to other software engineering tasks like code summarization and code completion. The findings of this study are expected to have a lasting impact on the advancement of source code retrieval methodologies. [ABSTRACT FROM AUTHOR]