Search

Atrial septal defect (ASD) is common congenital heart disease. The detection rate of congenital heart disease has increased year by year, and ASD accounted for the largest proportion of it, reaching 37.31%. The ASD patient will suffer from shortness of breath, palpitation, weakness, etc., with symptoms worsening with advanced age. The ASD patient will not suffer from congenital heart disease if their condition is diagnosed early. Echocardiography is a powerful and cost-effective means of detecting ASD. However, the disadvantages of echocardiography, such as noise and poor imaging quality, cause misdiagnosis of ASD. Hence, research into echocardiography-based efficient and effective detection of ASD with a deep neural network is of great significance. For echocardiography is noisy and fuzzy, and the learning and feature expression ability of the traditional convolutional neural network architecture is limited, a feature view classification based atrial septal defect intelligent auxiliary diagnostic model architecture was proposed. The different views of echocardiography possess different features, demanding more precise model extraction and combined features from echocardiography. The proposed model architecture integrates the semantic characteristics of several views, significantly improving the accuracy of diagnosis. In addition, with the aim of denoising and preserving edges, a bilateral filtering algorithm was performed. Furthermore, an ASD intelligent auxiliary diagnostic system was built based on the proposed model. The results show that the accuracy of the ASD auxiliary diagnostic system reaches 97.8%, and the false-negative rate is greatly reduced compared with the traditional convolutional neural network architecture.

BackgroundContrast and non-contrast echocardiography are crucial for cardiovascular diagnoses and treatments. Correct view classification is a foundational step for the analysis of cardiac structure and function. View classification from all sequences of a patient is laborious and depends heavily on the sonographer’s experience. In addition, the intra-view variability and the inter-view similarity increase the difficulty in identifying critical views in contrast and non-contrast echocardiography. This study aims to develop a deep residual convolutional neural network (CNN) to automatically identify multiple views of contrast and non-contrast echocardiography, including parasternal left ventricular short axis, apical two, three, and four-chamber views.MethodsThe study retrospectively analyzed a cohort of 855 patients who had undergone left ventricular opacification at the Department of Ultrasound Medicine, Wuhan Union Medical College Hospital from 2013 to 2021, including 70.3% men and 29.7% women aged from 41 to 62 (median age, 53). All datasets were preprocessed to remove sensitive information and 10 frames with equivalent intervals were sampled from each of the original videos. The number of frames in the training, validation, and test datasets were, respectively, 19,370, 2,370, and 2,620 from 9 views, corresponding to 688, 84, and 83 patients. We presented the CNN model to classify echocardiographic views with an initial learning rate of 0.001, and a batch size of 4 for 30 epochs. The learning rate was decayed by a factor of 0.9 per epoch.ResultsOn the test dataset, the overall classification accuracy is 99.1 and 99.5% for contrast and non-contrast echocardiographic views. The average precision, recall, specificity, and F1 score are 96.9, 96.9, 100, and 96.9% for the 9 echocardiographic views.ConclusionsThis study highlights the potential of CNN in the view classification of echocardiograms with and without contrast. It shows promise in improving the workflow of clinical analysis of echocardiography.

Purpose: Interpreting echocardiographic exams requires substantial manual interaction as videos lack scan-plane information and have inconsistent image quality, ranging from clinically relevant to unrecognizable. Thus, a manual prerequisite step for analysis is to select the appropriate views that showcase both the target anatomy and optimal image quality. To automate this selection process, we present a method for automatic classification of routine views, recognition of unknown views, and quality assessment of detected views., Approach: We train a neural network for view classification and employ the logit activations from the neural network for unknown view recognition. Subsequently, we train a linear regression algorithm that uses feature embeddings from the neural network to predict view quality scores. We evaluate the method on a clinical test set of 2466 echocardiography videos with expert-annotated view labels and a subset of 438 videos with expert-rated view quality scores. A second observer annotated a subset of 894 videos, including all quality-rated videos., Results: The proposed method achieved an accuracy of 84.9 % ± 0.67 for the joint objective of routine view classification and unknown view recognition, whereas a second observer reached an accuracy of 87.6%. For view quality assessment, the method achieved a Spearman's rank correlation coefficient of 0.71, whereas a second observer reached a correlation coefficient of 0.62., Conclusion: The proposed method approaches expert-level performance, enabling fully automatic selection of the most appropriate views for manual or automatic downstream analysis., (© 2024 The Authors.)

In medical applications, retrieving similar images from repositories is most essential for supporting diagnostic imaging-based clinical analysis and decision support systems. However, this is a challenging task, due to the multi-modal and multi-dimensional nature of medical images. In practical scenarios, the availability of large and balanced datasets that can be used for developing intelligent systems for efficient medical image management is quite limited. Traditional models often fail to capture the latent characteristics of images and have achieved limited accuracy when applied to medical images. For addressing these issues, a deep neural network-based approach for view classification and content-based image retrieval is proposed and its application for efficient medical image retrieval is demonstrated. We also designed an approach for body part orientation view classification labels, intending to reduce the variance that occurs in different types of scans. The learned features are used first to predict class labels and later used to model the feature space for similarity computation for the retrieval task. The outcome of this approach is measured in terms of error score. When benchmarked against 12 state-of-the-art works, the model achieved the lowest error score of 132.45, with 9.62–63.14% improvement over other works, thus highlighting its suitability for real-world applications. [ABSTRACT FROM AUTHOR]

Echocardiography is a widely used and cost-effective medical imaging procedure that is used to diagnose cardiac irregularities. To capture the various chambers of the heart, echocardiography videos are captured from different angles called views to generate standard images/videos. Automatic classification of these views allows for faster diagnosis and analysis. In this work, we propose a representation for echo videos which encapsulates the motion profile of various chambers and valves that helps effective view classification. This variety of motion profiles is captured in a large Gaussian mixture model called universal motion profile model (UMPM). In order to extract only the relevant motion profiles for each view, a factor analysis based decomposition is applied to the means of the UMPM. This results in a low-dimensional representation called motion profile vector (MPV) which captures the distinctive motion signature for a particular view. To evaluate MPVs, a dataset called ECHO 1.0 is introduced which contains around 637 video clips of the four major views: a) parasternal long-axis view (PLAX), b) parasternal short-axis (PSAX), c) apical four-chamber view (A4C), and d) apical two-chamber view (A2C). We demonstrate the efficacy of motion profile-vectors over other spatio-temporal representations. Further, motion profile-vectors can classify even poorly captured videos with high accuracy which shows the robustness of the proposed representation. [ABSTRACT FROM AUTHOR]

Recently, automatic speech recognition (ASR) and visual speech recognition (VSR) have been widely researched owing to the development in deep learning. Most VSR research works focus only on frontal face images. However, assuming real scenes, it is obvious that a VSR system should correctly recognize spoken contents from not only frontal but also diagonal or profile faces. In this paper, we propose a novel VSR method that is applicable to faces taken at any angle. Firstly, view classification is carried out to estimate face angles. Based on the results, feature extraction is then conducted using the best combination of pre-trained feature extraction models. Next, lipreading is carried out using the features. We also developed audio-visual speech recognition (AVSR) using the VSR in addition to conventional ASR. Audio results were obtained from ASR, followed by incorporating audio and visual results in a decision fusion manner. We evaluated our methods using OuluVS2, a multi-angle audio-visual database. We then confirmed that our approach achieved the best performance among conventional VSR schemes in a phrase classification task. In addition, we found that our AVSR results are better than ASR and VSR results.

A proper echocardiographic study requires several video clips recorded from different acquisition angles for observation of the complex cardiac anatomy. However, these video clips are not necessarily labeled in a database. Identification of the acquired view becomes the first step of analyzing an echocardiogram. Currently, there is no consensus whether the mislabeled samples can be used to create a feasible clinical prediction model of ejection fraction (EF). The aim of this study was to test two types of input methods for the classification of images, and to test the accuracy of the prediction model for EF in a learning database containing mislabeled images that were not checked by observers. We enrolled 340 patients with five standard views (long axis, short axis, 3-chamber view, 4-chamber view and 2-chamber view) and 10 images in a cycle, used for training a convolutional neural network to classify views (total 17,000 labeled images). All DICOM images were rigidly registered and rescaled into a reference image to fit the size of echocardiographic images. We employed 5-fold cross validation to examine model performance. We tested models trained by two types of data, averaged images and 10 selected images. Our best model (from 10 selected images) classified video views with 98.1% overall test accuracy in the independent cohort. In our view classification model, 1.9% of the images were mislabeled. To determine if this 98.1% accuracy was acceptable for creating the clinical prediction model using echocardiographic data, we tested the prediction model for EF using learning data with a 1.9% error rate. The accuracy of the prediction model for EF was warranted, even with training data containing 1.9% mislabeled images. The CNN algorithm can classify images into five standard views in a clinical setting. Our results suggest that this approach may provide a clinically feasible accuracy level of view classification for the analysis of echocardiographic data.

Aims: Augmenting echocardiography with artificial intelligence would allow for automated assessment of routine parameters and identification of disease patterns not easily recognized otherwise. View classification is an essential first step before deep learning can be applied to the echocardiogram., Methods and Results: We trained two- and three-dimensional convolutional neural networks (CNNs) using transthoracic echocardiographic (TTE) studies obtained from 909 patients to classify nine view categories (10 269 videos). Transthoracic echocardiographic studies from 229 patients were used in internal validation (2582 videos). Convolutional neural networks were tested on 100 patients with comprehensive TTE studies (where the two examples chosen by CNNs as most likely to represent a view were evaluated) and 408 patients with five view categories obtained via point-of-care ultrasound (POCUS). The overall accuracy of the two-dimensional CNN was 96.8%, and the averaged area under the curve (AUC) was 0.997 on the comprehensive TTE testing set; these numbers were 98.4% and 0.998, respectively, on the POCUS set. For the three-dimensional CNN, the accuracy and AUC were 96.3% and 0.998 for full TTE studies and 95.0% and 0.996 on POCUS videos, respectively. The positive predictive value, which defined correctly identified predicted views, was higher with two-dimensional rather than three-dimensional networks, exceeding 93% in apical, short-axis aortic valve, and parasternal long-axis left ventricle views., Conclusion: An automated view classifier utilizing CNNs was able to classify cardiac views obtained using TTE and POCUS with high accuracy. The view classifier will facilitate the application of deep learning to echocardiography., Competing Interests: Conflict of interest: none declared., (© The Author(s) 2024. Published by Oxford University Press on behalf of the European Society of Cardiology.)

Background: View classification is a key step toward building a fully automated system for interpretation of echocardiograms. However, compared with adult echocardiograms, creating a view classification model for pediatric echocardiograms poses additional challenges, such as greater variation in anatomy, structure size, and views. The aim of this study was to develop a computer vision model to autonomously perform view classification on pediatric echocardiographic images., Methods: Using a training set of 12,067 echocardiographic images from patients aged 0 to 19 years, a convolutional neural network model was trained to identify 27 preselected standard pediatric echocardiographic views which included anatomic sweeps, color Doppler, and Doppler tracings. A validation set of 6,197 images was used for parameter tuning and model selection. A test set of 9,684 images from 100 different patients was then used to evaluate model accuracy. The model was also evaluated on a per study basis using a second test set consisting of 524 echocardiograms from children with leukemia to identify six preselected views pertinent to cardiac dysfunction surveillance., Results: The model identified the 27 preselected views with 90.3% accuracy. Accuracy was similar across age groups (89.3% for 0-4 years, 90.8% for 4-9 years, 90.0% for 9-14 years, and 91.2% for 14-19 years; P = .12). Examining the view subtypes, accuracy was 78.3% for the cine one location, 90.5% for sweeps with color Doppler, 82.2% for sweeps without color Doppler, and 91.1% for Doppler tracings. Among the leukemia cohort, the model identified the six preselected views on a per study basis with a positive predictive value of 98.7% to 99.2% and sensitivity of 76.9% to 94.8%., Conclusions: A convolutional neural network model was constructed for view classification of pediatric echocardiograms that was accurate across the spectrum of ages and view types. This work lays the foundation for automated quantitative analysis and diagnostic support to promote efficient, accurate, and scalable analysis of pediatric echocardiograms., (Published by Elsevier Inc.)

Abstract: Thousands of medical images are saved in databases every day and the need for algorithms able to handle such data in an unsupervised manner is steadily increasing. The classification of ultrasound images is an outstandingly difficult task, due to the high noise level of these images. We present a detailed description of an algorithm based on multiscale elastic registration capable of unsupervised, landmark-free classification of cardiac ultrasound images into their respective views (apical four chamber, two chamber, parasternal long axis and short axis views). We validated the algorithm with 90 unselected, consecutive echocardiographic images recorded during daily clinical work. When the two visually very similar apical views (four chamber and two chamber) are combined into one class, we obtained a 93.0% correct classification (χ2 = 123.8, p < 0.0001, cross-validation 93.0%; χ2 = 131.1, p < 0.0001). Classification into the 4 classes reached a 90.0% correct classification (χ2 = 205.4, p < 0.0001, cross-validation 82.2%; χ2 = 165.9, p < 0.0001). (E-mail: hunzikerp@uhbs.ch) [Copyright &y& Elsevier]

Recently, automatic speech recognition (ASR) and visual speech recognition (VSR) have been widely researched owing to the development in deep learning. Most VSR research works focus only on frontal face images. However, assuming real scenes, it is obvious that a VSR system should correctly recognize spoken contents from not only frontal but also diagonal or profile faces. In this paper, we propose a novel VSR method that is applicable to faces taken at any angle. Firstly, view classification is carried out to estimate face angles. Based on the results, feature extraction is then conducted using the best combination of pre-trained feature extraction models. Next, lipreading is carried out using the features. We also developed audio-visual speech recognition (AVSR) using the VSR in addition to conventional ASR. Audio results were obtained from ASR, followed by incorporating audio and visual results in a decision fusion manner. We evaluated our methods using OuluVS2, a multi-angle audio-visual database. We then confirmed that our approach achieved the best performance among conventional VSR schemes in a phrase classification task. In addition, we found that our AVSR results are better than ASR and VSR results. [ABSTRACT FROM AUTHOR]

This paper presents a novel computer vision based system to support automated PACP (Pipeline Assessment Certification Program) coding for cracks. The proposed system comprises five major steps: 1) identifying forward facing view (FFV) with pipeline viewpoint detector, 2) obtaining stable edge information using structure edge detector, 3) acquiring crack segments and inner circle area of the pipeline with image binarization, 4) generating a 2D angular-diameter histogram for each frame, and 5) training a crack category classifier with support vector machine (SVM). The experimental results demonstrated that the proposed system can not only detect the cracks and categorize the crack type per PACP standard effectively but can also run about 10 frames per second (fps) in real world CCTV videos with 320 × 240 resolutions. In terms of accuracy in detecting cracks, the proposed method reaches about 91%, 88% and 90% recall for longitudinal cracks (CL), circumferential cracks (CC) and multiple cracks (CM), respectively. This paper contributes to the overall body of knowledge by providing an innovative framework that supports real-time crack identification and coding per PACP standards, which will lay a strong foundation for the development of a fully automated PACP sewer pipeline inspection system. • Algorithms to classify subclass level cracks in sewer inspections were proposed. • The recall rate of the proposed crack classification algorithm can reach over 90%. • The algorithm showed sound performance while video running at a speed of 10 pfs. [ABSTRACT FROM AUTHOR]

Purpose: Echocardiography is the most commonly used modality for assessing the heart in clinical practice. In an echocardiographic exam, an ultrasound probe samples the heart from different orientations and positions, thereby creating different viewpoints for assessing the cardiac function. The determination of the probe viewpoint forms an essential step in automatic echocardiographic image analysis. Approach: In this study, convolutional neural networks are used for the automated identification of 14 different anatomical echocardiographic views (larger than any previous study) in a dataset of 8732 videos acquired from 374 patients. Differentiable architecture search approach was utilized to design small neural network architectures for rapid inference while maintaining high accuracy. The impact of the image quality and resolution, size of the training dataset, and number of echocardiographic view classes on the efficacy of the models were also investigated. Results: In contrast to the deeper classification architectures, the proposed models had significantly lower number of trainable parameters (up to 99.9% reduction), achieved comparable classification performance (accuracy 88.4% to 96%, precision 87.8% to 95.2%, recall 87.1% to 95.1%) and real-time performance with inference time per image of 3.6 to 12.6 ms. Conclusion: Compared with the standard classification neural network architectures, the proposed models are faster and achieve comparable classification performance. They also require less training data. Such models can be used for real-time detection of the standard views., (© 2021 Society of Photo-Optical Instrumentation Engineers (SPIE).)