Search

In this paper, we investigate the spontaneity issue in facial expression sequence generation. Current leading methods in the field are commonly reliant on manually adjusted conditional variables to direct the model to generate a specific class of expression. We propose a neural network-based method which uses Gaussian noise to model spontaneity in the generation process, removing the need for manual control of conditional generation variables. Our model takes two sequential images as input, with additive noise, and produces the next image in the sequence. We trained two types of models: single-expression, and mixed-expression. With single-expression, unique facial movements of certain emotion class can be generated; with mixed expressions, fully spontaneous expression sequence generation can be achieved. We compared our method to current leading generation methods on a variety of publicly available datasets. Initial qualitative results show our method produces visually more realistic expressions and facial action unit (AU) trajectories; initial quantitative results using image quality metrics (SSIM and NIQE) show the quality of our generated images is higher. Our approach and results are novel in the field of facial expression generation, with potential wider applications to other sequence generation tasks.

Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. Current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than tracking frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression duration. As baseline for MEGC2021 using leave-one-subject-out evaluation method, we show that our solution performed better in a high frame-rate (200 fps) SAMM long videos dataset (SAMM-LV) than a low frame-rate (30 fps) (CAS(ME)2) dataset. We introduce a new unseen dataset for MEGC2022 challenge (MEGC2022-testSet) and achieves F1-Score of 0.1531 as baseline result.

Objective: To test automated in vivo estimation of active and passive skeletal muscle states using ultrasonic imaging. Background: Current technology (electromyography, dynamometry, shear wave imaging) provides no general, non-invasive method for online estimation of skeletal muscle states. Ultrasound (US) allows non-invasive imaging of muscle, yet current computational approaches have never achieved simultaneous extraction nor generalisation of independently varying, active and passive states. We use deep learning to investigate the generalizable content of 2D US muscle images. Method: US data synchronized with electromyography of the calf muscles, with measures of joint moment/angle were recorded from 32 healthy participants (7 female, ages: 27.5, 19-65). We extracted a region of interest of medial gastrocnemius and soleus using our prior developed accurate segmentation algorithm. From the segmented images, a deep convolutional neural network was trained to predict three absolute, driftfree, components of the neurobiomechanical state (activity, joint angle, joint moment) during experimentally designed, simultaneous, independent variation of passive (joint angle) and active (electromyography) inputs. Results: For all 32 held-out participants (16-fold cross-validation) the ankle joint angle, electromyography, and joint moment were estimated to accuracy 55±8%, 57±11%, and 46±9% respectively. Significance: With 2D US imaging, deep neural networks can encode in generalizable form, the activitylength-tension state relationship of these muscles. Observation only, low power, 2D US imaging can provide a new category of technology for non-invasive estimation of neural output, length and tension in skeletal muscle. This proof of principle has value for personalised muscle assessment in pain, injury, neurological conditions, neuropathies, myopathies and ageing.

Objective: To provide objective visualization and pattern analysis of neck muscle boundaries to inform and monitor treatment of cervical dystonia. Methods: We recorded transverse cervical ultrasound (US) images and whole-body motion analysis of sixty-one standing participants (35 cervical dystonia, 26 age matched controls). We manually annotated 3,272 US images sampling posture and the functional range of pitch, yaw, and roll head movements. Using previously validated methods, we used 60-fold cross validation to train, validate and test a deep neural network (U-net) to classify pixels to 13 categories (five paired neck muscles, skin, ligamentum nuchae, vertebra). For all participants for their normal standing posture, we segmented US images and classified condition (Dystonia/Control), sex and age (higher/lower) from segment boundaries. We performed an explanatory, visualization analysis of dystonia muscle-boundaries. Results: For all segments, agreement with manual labels was Dice Coefficient (64±21%) and Hausdorff Distance (5.7±4 mm). For deep muscle layers, boundaries predicted central injection sites with average precision 94±3%. Using leave-one-out cross-validation, a support-vector-machine classified condition, sex, and age from predicted muscle boundaries at accuracy 70.5%, 67.2%, 52.4% respectively, exceeding classification by manual labels. From muscle boundaries, Dystonia clustered optimally into three sub-groups. These sub-groups are visualized and explained by three eigen-patterns which correlate significantly with truncal and head posture. Conclusion: Using US, neck muscle shape alone discriminates dystonia from healthy controls. Significance: Using deep learning, US imaging allows online, automated visualization, and diagnostic analysis of cervical dystonia and segmentation of individual muscles for targeted injection. The dataset is available (DOI: 10.23634/MMUDR.00624643).

This dataset supports the publication "Objective analysis of neck muscle boundaries for cervical dystonia using ultrasound imaging and deep learning, DOI: 10.1109/JBHI.2020.2964098". The data includes transverse US images of the neck (2172), corresponding manually annotated labels segmenting 13 muscle/ligament/vertebra segments from 61 participants (35 cervical dystonia, and 26 age matched controls). The dataset includes the accompanying clinical (Dystonia, Control), sex (M, F), age, and anonymised participant ID’s for the 61 participants. The instructions are contained within the file ‘Instructions for DataSet.docx’ and the data, which can be opened in MATLAB once downloaded, is contained in ‘DataAccompanyingPaper.mat.

The aim of this study was to provide automated identification of postural point-features required to estimate the location and orientation of the head, multi-segmented trunk and arms from videos of the clinical test ‘Segmental Assessment of Trunk Control’ (SATCo). Three expert operators manually annotated 13 point-features in every fourth image of 177 short (5–10 s) videos (25 Hz) of 12 children with cerebral palsy (aged: 4.52 ± 2.4 years), participating in SATCo testing. Linear interpolation for the remaining images resulted in 30 825 annotated images. Convolutional neural networks were trained with cross-validation, giving held-out test results for all children. The point-features were estimated with error 4.4 ± 3.8 pixels at approximately 100 images per second. Truncal segment angles (head, neck and six thoraco-lumbar–pelvic segments) were estimated with error 6.4 ± 2.8°, allowing accurate classification (F1 > 80%) of deviation from a reference posture at thresholds up to 3°, 3° and 2°, respectively. Contact between arm point-features (elbow and wrist) and supporting surface was classified at F1 = 80.5%. This study demonstrates, for the first time, technical feasibility to automate the identification of (i) a sitting segmental posture including individual trunk segments, (ii) changes away from that posture, and (iii) support from the upper limb, required for the clinical SATCo.

The level of intracellular diadenosine 5, 5-P1,P4-tetraphosphate (Ap4A) increases several fold in mammalian cells treated with non-cytotoxic doses of interstrand DNA-crosslinking agents such as mitomycinC. It is also increased in cells lacking DNA repair proteins including XRCC1, PARP1, APTX and FANCG,while >50-fold increases (up to around 25 M) are achieved in repair mutants exposed to mitomycin C.Part of this induced Ap4A is converted into novel derivatives, identified as mono- and di-ADP-ribosylatedAp4A. Gene knockout experiments suggest that DNA ligase III is primarily responsible for the synthesisof damage-induced Ap4A and that PARP1 and PARP2 can both catalyze its ADP-ribosylation. Degrada-tive proteins such as aprataxin may also contribute to the increase. Using a cell-free replication system,Ap4A was found to cause a marked inhibition of the initiation of DNA replicons, while elongation wasunaffected. Maximum inhibition of 70–80% was achieved with 20 M Ap4A. Ap3A, Ap5A, Gp4G and ADP-ribosylated Ap4A were without effect. It is proposed that Ap4A acts as an important inducible ligand inthe DNA damage response to prevent the replication of damaged DNA.