Search

Real-world systems often encounter new data over time, which leads to experiencing target domain shifts. Existing Test-Time Adaptation (TTA) methods tend to apply computationally heavy and memory-intensive backpropagation-based approaches to handle this. Here, we propose a novel method that uses a backpropagation-free approach for TTA for the specific case of 3D data. Our model uses a two-stream architecture to maintain knowledge about the source domain as well as complementary target-domain-specific information. The backpropagation-free property of our model helps address the well-known forgetting problem and mitigates the error accumulation issue. The proposed method also eliminates the need for the usually noisy process of pseudo-labeling and reliance on costly self-supervised training. Moreover, our method leverages subspace learning, effectively reducing the distribution variance between the two domains. Furthermore, the source-domain-specific and the target-domain-specific streams are aligned using a novel entropy-based adaptive fusion strategy. Extensive experiments on popular benchmarks demonstrate the effectiveness of our method. The code will be available at \url{https://github.com/abie-e/BFTT3D}., Comment: CVPR 2024

Federated Class-Incremental Learning (FCIL) is an underexplored yet pivotal issue, involving the dynamic addition of new classes in the context of federated learning. In this field, Data-Free Knowledge Transfer (DFKT) plays a crucial role in addressing catastrophic forgetting and data privacy problems. However, prior approaches lack the crucial synergy between DFKT and the model training phases, causing DFKT to encounter difficulties in generating high-quality data from a non-anchored latent space of the old task model. In this paper, we introduce LANDER (Label Text Centered Data-Free Knowledge Transfer) to address this issue by utilizing label text embeddings (LTE) produced by pretrained language models. Specifically, during the model training phase, our approach treats LTE as anchor points and constrains the feature embeddings of corresponding training samples around them, enriching the surrounding area with more meaningful information. In the DFKT phase, by using these LTE anchors, LANDER can synthesize more meaningful samples, thereby effectively addressing the forgetting problem. Additionally, instead of tightly constraining embeddings toward the anchor, the Bounding Loss is introduced to encourage sample embeddings to remain flexible within a defined radius. This approach preserves the natural differences in sample embeddings and mitigates the embedding overlap caused by heterogeneous federated settings. Extensive experiments conducted on CIFAR100, Tiny-ImageNet, and ImageNet demonstrate that LANDER significantly outperforms previous methods and achieves state-of-the-art performance in FCIL. The code is available at https://github.com/tmtuan1307/lander., Comment: Accepted at CVPR 2024

3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity. However, the substantial Gaussians and their associated attributes necessitate effective compression techniques. Nevertheless, the sparse and unorganized nature of the point cloud of Gaussians (or anchors in our paper) presents challenges for compression. To address this, we make use of the relations between the unorganized anchors and the structured hash grid, leveraging their mutual information for context modeling, and propose a Hash-grid Assisted Context (HAC) framework for highly compact 3DGS representation. Our approach introduces a binary hash grid to establish continuous spatial consistencies, allowing us to unveil the inherent spatial relations of anchors through a carefully designed context model. To facilitate entropy coding, we utilize Gaussian distributions to accurately estimate the probability of each quantized attribute, where an adaptive quantization module is proposed to enable high-precision quantization of these attributes for improved fidelity restoration. Additionally, we incorporate an adaptive masking strategy to eliminate invalid Gaussians and anchors. Importantly, our work is the pioneer to explore context-based compression for 3DGS representation, resulting in a remarkable size reduction of over $75\times$ compared to vanilla 3DGS, while simultaneously improving fidelity, and achieving over $11\times$ size reduction over SOTA 3DGS compression approach Scaffold-GS. Our code is available here: https://github.com/YihangChen-ee/HAC, Comment: Project Page: https://yihangchen-ee.github.io/project_hac/ Code: https://github.com/YihangChen-ee/HAC

Machine unlearning has become a pivotal task to erase the influence of data from a trained model. It adheres to recent data regulation standards and enhances the privacy and security of machine learning applications. In this work, we present a new machine unlearning approach Scissorhands. Initially, Scissorhands identifies the most pertinent parameters in the given model relative to the forgetting data via connection sensitivity. By reinitializing the most influential top-k percent of these parameters, a trimmed model for erasing the influence of the forgetting data is obtained. Subsequently, Scissorhands fine-tunes the trimmed model with a gradient projection-based approach, seeking parameters that preserve information on the remaining data while discarding information related to the forgetting data. Our experimental results, conducted across image classification and image generation tasks, demonstrate that Scissorhands, showcases competitive performance when compared to existing methods., Comment: Machine Unlearning, Deep Learning

In this work, we introduce an unlearning algorithm for diffusion models. Our algorithm equips a diffusion model with a mechanism to mitigate the concerns related to data memorization. To achieve this, we formulate the unlearning problem as a constraint optimization problem, aiming to preserve the utility of the diffusion model on the remaining data and scrub the information associated with forgetting data by deviating the learnable generative process from the ground-truth denoising procedure. To solve the resulting problem, we adopt a first-order method, having superior practical performance while being vigilant about the diffusion process. Empirically, we demonstrate that our algorithm can preserve the model utility, effectiveness, and efficiency while removing across the widely-used diffusion models and in both conditional and unconditional image generation scenarios., Comment: Diffusion Model, Machine Unlearning

In recent years, Neural Radiance Field (NeRF) has demonstrated remarkable capabilities in representing 3D scenes. To expedite the rendering process, learnable explicit representations have been introduced for combination with implicit NeRF representation, which however results in a large storage space requirement. In this paper, we introduce the Context-based NeRF Compression (CNC) framework, which leverages highly efficient context models to provide a storage-friendly NeRF representation. Specifically, we excavate both level-wise and dimension-wise context dependencies to enable probability prediction for information entropy reduction. Additionally, we exploit hash collision and occupancy grids as strong prior knowledge for better context modeling. To the best of our knowledge, we are the first to construct and exploit context models for NeRF compression. We achieve a size reduction of 100$\times$ and 70$\times$ with improved fidelity against the baseline Instant-NGP on Synthesic-NeRF and Tanks and Temples datasets, respectively. Additionally, we attain 86.7\% and 82.3\% storage size reduction against the SOTA NeRF compression method BiRF. Our code is available here: https://github.com/YihangChen-ee/CNC., Comment: Project Page: https://yihangchen-ee.github.io/project_cnc/ Code: https://github.com/yihangchen-ee/cnc/. We further propose a 3DGS compression method HAC, which is based on CNC: https://yihangchen-ee.github.io/project_hac

BackgroundThere has been substantial progress in developing approaches to measure mistreatment of women during childbirth. However, less is known about the differences in measurement approaches. In this study, we compare measures of mistreatment obtained from the same women using labour observations and community-based surveys in Ghana, Guinea and Nigeria.MethodsExperiences of mistreatment during childbirth are person-centred quality measures. As such, we assessed individual-level and population-level accuracy of labour observation relative to women's self-report for different types of mistreatment. We calculated sensitivity, specificity, percent agreement and population-level inflation factor (IF), assessing prevalence of mistreatment in labour observation divided by 'true' prevalence in women's self-report. We report the IF degree of bias as: low (0.75Results1536 women across Ghana (n=779), Guinea (n=425) and Nigeria (n=332) were included. Most mistreatment items demonstrated better specificity than sensitivity: observation of any physical abuse (44% sensitive, 89% specific), any verbal abuse (61% sensitive, 73% specific) and presence of a labour companion (19% sensitive, 93% specific). Items for stigma (IF 0.16), pain relief requested (IF 0.38), companion present (IF 0.32) and lack of easy access to fluids (IF 0.46) showed high risk of bias, meaning labour observations would substantially underestimate true prevalence. Other items showed low or moderate bias.ConclusionUsing self-report as the reference standard, labour observations demonstrated moderate-to-high specificity (accurately identifying lack of mistreatment) but low-to-moderate sensitivity (accurately identifying presence of mistreatment) among women. For overall prevalence, either women's self-report or observations can be used with low-moderate bias for most mistreatment items. However, given the dynamicity, complexity, and limitations in 'objectivity', some experiences of mistreatment (stigma, pain r

BACKGROUND: There has been substantial progress in developing approaches to measure mistreatment of women during childbirth. However, less is known about the differences in measurement approaches. In this study, we compare measures of mistreatment obtained from the same women using labour observations and community-based surveys in Ghana, Guinea and Nigeria. METHODS: Experiences of mistreatment during childbirth are person-centred quality measures. As such, we assessed individual-level and population-level accuracy of labour observation relative to women's self-report for different types of mistreatment. We calculated sensitivity, specificity, percent agreement and population-level inflation factor (IF), assessing prevalence of mistreatment in labour observation divided by 'true' prevalence in women's self-report. We report the IF degree of bias as: low (0.75

Despite global attention, physical and verbal abuse remains prevalent in maternity and newborn healthcare. We aimed to establish theoretical principles for interventions to reduce such abuse. We undertook a mixed methods systematic review of health and social care literature (MEDLINE, SocINDEX, Global Index Medicus, CINAHL, Cochrane Library, Sept 29th 2020 and March 22nd 2022: no date or language restrictions). Papers that included theory were analysed narratively. Those with suitable outcome measures were meta-analysed. We used convergence results synthesis to integrate findings. In September 2020, 193 papers were retained (17,628 hits). 154 provided theoretical explanations; 38 were controlled studies. The update generated 39 studies (2695 hits), plus five from reference lists (12 controlled studies). A wide range of explicit and implicit theories were proposed. Eleven non-maternity controlled studies could be meta-analysed, but only for physical restraint, showing little intervention effect. Most interventions were multi-component. Synthesis suggests that a combination of systems level and behavioural change models might be effective. The maternity intervention studies could all be mapped to this approach. Two particular adverse contexts emerged; social normalisation of violence across the socio-ecological system, especially for ‘othered’ groups; and the belief that mistreatment is necessary to minimise clinical harm. The ethos and therefore the expression of mistreatment at each level of the system is moderated by the individuals who enact the system, through what they feel they can control, what is socially normal, and what benefits them in that context. Interventions to reduce verbal and physical abuse in maternity care should be locally tailored, and informed by theories encompassing all socio-ecological levels, and the psychological and emotional responses of individuals working within them. Attention should be paid to social normalisation of violence agains

Despite global attention, physical and verbal abuse remains prevalent in maternity and newborn healthcare. We aimed to establish theoretical principles for interventions to reduce such abuse. We undertook a mixed methods systematic review of health and social care literature (MEDLINE, SocINDEX, Global Index Medicus, CINAHL, Cochrane Library, Sept 29th 2020 and March 22nd 2022: no date or language restrictions). Papers that included theory were analysed narratively. Those with suitable outcome measures were meta-analysed. We used convergence results synthesis to integrate findings. In September 2020, 193 papers were retained (17,628 hits). 154 provided theoretical explanations; 38 were controlled studies. The update generated 39 studies (2695 hits), plus five from reference lists (12 controlled studies). A wide range of explicit and implicit theories were proposed. Eleven non-maternity controlled studies could be meta-analysed, but only for physical restraint, showing little intervention effect. Most interventions were multi-component. Synthesis suggests that a combination of systems level and behavioural change models might be effective. The maternity intervention studies could all be mapped to this approach. Two particular adverse contexts emerged; social normalisation of violence across the socio-ecological system, especially for ‘othered’ groups; and the belief that mistreatment is necessary to minimise clinical harm. The ethos and therefore the expression of mistreatment at each level of the system is moderated by the individuals who enact the system, through what they feel they can control, what is socially normal, and what benefits them in that context. Interventions to reduce verbal and physical abuse in maternity care should be locally tailored, and informed by theories encompassing all socio-ecological levels, and the psychological and emotional responses of individuals working within them. Attention should be paid to social normalisation of violence agains

Despite global attention, physical and verbal abuse remains prevalent in maternity and newborn healthcare. We aimed to establish theoretical principles for interventions to reduce such abuse. We undertook a mixed methods systematic review of health and social care literature (MEDLINE, SocINDEX, Global Index Medicus, CINAHL, Cochrane Library, Sept 29th 2020 and March 22nd 2022: no date or language restrictions). Papers that included theory were analysed narratively. Those with suitable outcome measures were meta-analysed. We used convergence results synthesis to integrate findings. In September 2020, 193 papers were retained (17,628 hits). 154 provided theoretical explanations; 38 were controlled studies. The update generated 39 studies (2695 hits), plus five from reference lists (12 controlled studies). A wide range of explicit and implicit theories were proposed. Eleven non-maternity controlled studies could be meta-analysed, but only for physical restraint, showing little intervention effect. Most interventions were multi-component. Synthesis suggests that a combination of systems level and behavioural change models might be effective. The maternity intervention studies could all be mapped to this approach. Two particular adverse contexts emerged; social normalisation of violence across the socio-ecological system, especially for ‘othered’ groups; and the belief that mistreatment is necessary to minimise clinical harm. The ethos and therefore the expression of mistreatment at each level of the system is moderated by the individuals who enact the system, through what they feel they can control, what is socially normal, and what benefits them in that context. Interventions to reduce verbal and physical abuse in maternity care should be locally tailored, and informed by theories encompassing all socio-ecological levels, and the psychological and emotional responses of individuals working within them. Attention should be paid to social normalisation of violence agains

Learning deep discrete latent presentations offers a promise of better symbolic and summarized abstractions that are more useful to subsequent downstream tasks. Inspired by the seminal Vector Quantized Variational Auto-Encoder (VQ-VAE), most of work in learning deep discrete representations has mainly focused on improving the original VQ-VAE form and none of them has studied learning deep discrete representations from the generative viewpoint. In this work, we study learning deep discrete representations from the generative viewpoint. Specifically, we endow discrete distributions over sequences of codewords and learn a deterministic decoder that transports the distribution over the sequences of codewords to the data distribution via minimizing a WS distance between them. We develop further theories to connect it with the clustering viewpoint of WS distance, allowing us to have a better and more controllable clustering solution. Finally, we empirically evaluate our method on several well-known benchmarks, where it achieves better qualitative and quantitative performances than the other VQ-VAE variants in terms of the codebook utilization and image reconstruction/generation.

In 2004, the Province of British Columbia (BC) announced a multi-year $1.5 billion seismic retrofit program for the province's 750 at-risk public schools. The purpose of this program was to quantify the seismic risk of the province's public-school buildings and to expedite the seismic upgrading of the most at-risk schools. In order to provide a safe and cost-effective implementation of this program, the Engineers and Geoscientists BC, in collaboration with the University of British Columbia, has developed a performance-based probabilistic method and guidelines for the seismic risk assessment and retrofit of low-rise buildings. As part of this initiative, a number of laboratory experiments have been conducted to provide data that would support the recommendations provided in the guidelines. The laboratory experiments included several full-scale shake table tests of wood frame systems. The specimens were subjected to sequences of earthquake motions to simulate their performance under main shock-aftershocks. This paper presents details of seven of the experiment and wood-frame buildings considered. A detailed discussion of the results of the analyses of the shake table tests data, and performance assessment using drift- and energy-based damage indices is presented. This study highlights the importance of considering the effects of subduction earthquake and mainshock-aftershock sequences for design and retrofit of wood frame structures. © 2023 John Wiley & Sons Ltd.

In Magnetic Resonance Imaging (MRI), image acquisitions are often undersampled in the measurement domain to accelerate the scanning process, at the expense of image quality. However, image quality is a crucial factor that influences the accuracy of clinical diagnosis; hence, high-quality image reconstruction from undersampled measurements has been a key area of research. Recently, deep learning (DL) methods have emerged as the state-of-the-art for MRI reconstruction, typically involving deep neural networks to transform undersampled MRI images into high-quality MRI images through data-driven processes. Nevertheless, there is clear and significant room for improvement in undersampled DL MRI reconstruction to meet the high standards required for clinical diagnosis, in terms of eliminating aliasing artifacts and reducing image noise. In this paper, we introduce a self-supervised pretraining procedure using contrastive learning to improve the accuracy of undersampled DL MRI reconstruction. We use contrastive learning to transform the MRI image representations into a latent space that maximizes mutual information among different undersampled representations and optimizes the information content at the input of the downstream DL reconstruction models. Our experiments demonstrate improved reconstruction accuracy across a range of acceleration factors and datasets, both quantitatively and qualitatively. Furthermore, our extended experiments validate the proposed framework's robustness under adversarial conditions, such as measurement noise, different k-space sampling patterns, and pathological abnormalities, and also prove the transfer learning capabilities on MRI datasets with completely different anatomy. Additionally, we conducted experiments to visualize and analyze the properties of the proposed MRI contrastive learning latent space.

Hyperbolic geometry, a Riemannian manifold endowed with constant sectional negative curvature, has been considered an alternative embedding space in many learning scenarios, \eg, natural language processing, graph learning, \etc, as a result of its intriguing property of encoding the data's hierarchical structure (like irregular graph or tree-likeness data). Recent studies prove that such data hierarchy also exists in the visual dataset, and investigate the successful practice of hyperbolic geometry in the computer vision (CV) regime, ranging from the classical image classification to advanced model adaptation learning. This paper presents the first and most up-to-date literature review of hyperbolic spaces for CV applications. To this end, we first introduce the background of hyperbolic geometry, followed by a comprehensive investigation of algorithms, with geometric prior of hyperbolic space, in the context of visual applications. We also conclude this manuscript and identify possible future directions., Comment: First survey paper for the hyperbolic geometry in CV applications

Continual learning aims to efficiently learn from a non-stationary stream of data while avoiding forgetting the knowledge of old data. In many practical applications, data complies with non-Euclidean geometry. As such, the commonly used Euclidean space cannot gracefully capture non-Euclidean geometric structures of data, leading to inferior results. In this paper, we study continual learning from a novel perspective by exploring data geometry for the non-stationary stream of data. Our method dynamically expands the geometry of the underlying space to match growing geometric structures induced by new data, and prevents forgetting by keeping geometric structures of old data into account. In doing so, making use of the mixed curvature space, we propose an incremental search scheme, through which the growing geometric structures are encoded. Then, we introduce an angular-regularization loss and a neighbor-robustness loss to train the model, capable of penalizing the change of global geometric structures and local geometric structures. Experiments show that our method achieves better performance than baseline methods designed in Euclidean space., Comment: CVPR 2023

We introduce a language-grounded visual prompting method to adapt the visual encoder of vision-language models for downstream tasks. By capitalizing on language integration, we devise a parameter-efficient strategy to adjust the input of the visual encoder, eliminating the need to modify or add to the model's parameters. Due to this design choice, our algorithm can operate even in black-box scenarios, showcasing adaptability in situations where access to the model's parameters is constrained. We will empirically demonstrate that, compared to prior art, grounding visual prompts with language enhances both the accuracy and speed of adaptation. Moreover, our algorithm excels in base-to-novel class generalization, overcoming limitations of visual prompting and exhibiting the capacity to generalize beyond seen classes. We thoroughly assess and evaluate our method across a variety of image recognition datasets, such as EuroSAT, UCF101, DTD, and CLEVR, spanning different learning situations, including few-shot learning, base-to-novel class generalization, and transfer learning., Comment: The 38th Annual AAAI Conference on Artificial Intelligence

Auscultation for neonates is a simple and non-invasive method of providing diagnosis for cardiovascular and respiratory disease. Such diagnosis often requires high-quality heart and lung sounds to be captured during auscultation. However, in most cases, obtaining such high-quality sounds is non-trivial due to the chest sounds containing a mixture of heart, lung, and noise sounds. As such, additional preprocessing is needed to separate the chest sounds into heart and lung sounds. This paper proposes a novel deep-learning approach to separate such chest sounds into heart and lung sounds. Inspired by the Conv-TasNet model, the proposed model has an encoder, decoder, and mask generator. The encoder consists of a 1D convolution model and the decoder consists of a transposed 1D convolution. The mask generator is constructed using stacked 1D convolutions and transformers. The proposed model outperforms previous methods in terms of objective distortion measures by 2.01 dB to 5.06 dB in the artificial dataset, as well as computation time, with at least a 17-time improvement. Therefore, our proposed model could be a suitable preprocessing step for any phonocardiogram-based health monitoring system.

The objective of Continual Test-time Domain Adaptation (CTDA) is to gradually adapt a pre-trained model to a sequence of target domains without accessing the source data. This paper proposes a Dynamic Sample Selection (DSS) method for CTDA. DSS consists of dynamic thresholding, positive learning, and negative learning processes. Traditionally, models learn from unlabeled unknown environment data and equally rely on all samples' pseudo-labels to update their parameters through self-training. However, noisy predictions exist in these pseudo-labels, so all samples are not equally trustworthy. Therefore, in our method, a dynamic thresholding module is first designed to select suspected low-quality from high-quality samples. The selected low-quality samples are more likely to be wrongly predicted. Therefore, we apply joint positive and negative learning on both high- and low-quality samples to reduce the risk of using wrong information. We conduct extensive experiments that demonstrate the effectiveness of our proposed method for CTDA in the image domain, outperforming the state-of-the-art results. Furthermore, our approach is also evaluated in the 3D point cloud domain, showcasing its versatility and potential for broader applicability., Comment: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision

Data-Free Knowledge Distillation (DFKD) has made significant recent strides by transferring knowledge from a teacher neural network to a student neural network without accessing the original data. Nonetheless, existing approaches encounter a significant challenge when attempting to generate samples from random noise inputs, which inherently lack meaningful information. Consequently, these models struggle to effectively map this noise to the ground-truth sample distribution, resulting in prolonging training times and low-quality outputs. In this paper, we propose a novel Noisy Layer Generation method (NAYER) which relocates the random source from the input to a noisy layer and utilizes the meaningful constant label-text embedding (LTE) as the input. LTE is generated by using the language model once, and then it is stored in memory for all subsequent training processes. The significance of LTE lies in its ability to contain substantial meaningful inter-class information, enabling the generation of high-quality samples with only a few training steps. Simultaneously, the noisy layer plays a key role in addressing the issue of diversity in sample generation by preventing the model from overemphasizing the constrained label information. By reinitializing the noisy layer in each iteration, we aim to facilitate the generation of diverse samples while still retaining the method's efficiency, thanks to the ease of learning provided by LTE. Experiments carried out on multiple datasets demonstrate that our NAYER not only outperforms the state-of-the-art methods but also achieves speeds 5 to 15 times faster than previous approaches. The code is available at https://github.com/tmtuan1307/nayer., Comment: Accepted at CVPR 2024

Nowadays, understanding the geometry of the loss landscape shows promise in enhancing a model's generalization ability. In this work, we draw upon prior works that apply geometric principles to optimization and present a novel approach to improve robustness and generalization ability for constrained optimization problems. Indeed, this paper aims to generalize the Sharpness-Aware Minimization (SAM) optimizer to Riemannian manifolds. In doing so, we first extend the concept of sharpness and introduce a novel notion of sharpness on manifolds. To support this notion of sharpness, we present a theoretical analysis characterizing generalization capabilities with respect to manifold sharpness, which demonstrates a tighter bound on the generalization gap, a result not known before. Motivated by this analysis, we introduce our algorithm, Riemannian Sharpness-Aware Minimization (RSAM). To demonstrate RSAM's ability to enhance generalization ability, we evaluate and contrast our algorithm on a broad set of problems, such as image classification and contrastive learning across different datasets, including CIFAR100, CIFAR10, and FGVCAircraft. Our code is publicly available at \url{https://t.ly/RiemannianSAM}.

Audio-visual zero-shot learning aims to classify samples consisting of a pair of corresponding audio and video sequences from classes that are not present during training. An analysis of the audio-visual data reveals a large degree of hyperbolicity, indicating the potential benefit of using a hyperbolic transformation to achieve curvature-aware geometric learning, with the aim of exploring more complex hierarchical data structures for this task. The proposed approach employs a novel loss function that incorporates cross-modality alignment between video and audio features in the hyperbolic space. Additionally, we explore the use of multiple adaptive curvatures for hyperbolic projections. The experimental results on this very challenging task demonstrate that our proposed hyperbolic approach for zero-shot learning outperforms the SOTA method on three datasets: VGGSound-GZSL, UCF-GZSL, and ActivityNet-GZSL achieving a harmonic mean (HM) improvement of around 3.0%, 7.0%, and 5.3%, respectively., Comment: ICCV 2023

The performance of a lifelong learning (L3) model degrades when it is trained on a series of tasks, as the geometrical formation of the embedding space changes while learning novel concepts sequentially. The majority of existing L3 approaches operate on a fixed-curvature (e.g., zero-curvature Euclidean) space that is not necessarily suitable for modeling the complex geometric structure of data. Furthermore, the distillation strategies apply constraints directly on low-dimensional embeddings, discouraging the L3 model from learning new concepts by making the model highly stable. To address the problem, we propose a distillation strategy named L3DMC that operates on mixed-curvature spaces to preserve the already-learned knowledge by modeling and maintaining complex geometrical structures. We propose to embed the projected low dimensional embedding of fixed-curvature spaces (Euclidean and hyperbolic) to higher-dimensional Reproducing Kernel Hilbert Space (RKHS) using a positive-definite kernel function to attain rich representation. Afterward, we optimize the L3 model by minimizing the discrepancies between the new sample representation and the subspace constructed using the old representation in RKHS. L3DMC is capable of adapting new knowledge better without forgetting old knowledge as it combines the representation power of multiple fixed-curvature spaces and is performed on higher-dimensional RKHS. Thorough experiments on three benchmarks demonstrate the effectiveness of our proposed distillation strategy for medical image classification in L3 settings. Our code implementation is publicly available at https://github.com/csiro-robotics/L3DMC., Comment: MICCAI 2023 (Early Accept)

An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL., Comment: Neural Networks (submitted May 2022, accepted July 2023)

Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate representations of 3D scenes. To address this issue, we propose a novel neural-field-based method, called EndoSurf, which effectively learns to represent a deforming surface from an RGBD sequence. In EndoSurf, we model surface dynamics, shape, and texture with three neural fields. First, 3D points are transformed from the observed space to the canonical space using the deformation field. The signed distance function (SDF) field and radiance field then predict their SDFs and colors, respectively, with which RGBD images can be synthesized via differentiable volume rendering. We constrain the learned shape by tailoring multiple regularization strategies and disentangling geometry and appearance. Experiments on public endoscope datasets demonstrate that EndoSurf significantly outperforms existing solutions, particularly in reconstructing high-fidelity shapes. Code is available at https://github.com/Ruyi-Zha/endosurf.git., Comment: MICCAI 2023(Oral, Student Travel Award, Top 3%); Ruyi Zha and Xuelian Cheng made equal contributions. Corresponding author: Ruyi Zha (ruyi.zha@gmail.com)

INTRODUCTION: Pregnancy and childbearing among adolescents-especially younger adolescents-is associated with health complications and lost opportunities for education and personal development. In addition to established challenges adolescents and young women face in sexual and reproductive healthcare, evidence suggests that they also face mistreatment during childbirth. METHODS: This is a secondary analysis of the WHO study 'How women are treated during facility-based childbirth' cross-sectional community survey in Ghana, Guinea, Myanmar and Nigeria. We used descriptive analysis to assess experiences of mistreatment among adolescents (15-19 years) and young women (20-24 years) and multivariable logistic regression models to assess the association between experiences of mistreatment and satisfaction with care during childbirth. RESULTS: 862 participants are included (15-19 years: 287, 33.3%; 20-24 years: 575, 66.7%). The most common mistreatment was verbal abuse (15-19 years: 104/287, 36.2%; 20-24 years: 181/575, 31.5%). There were high levels of poor communication (15-19 years: 92/287, 32.1%; 20-24 years: 171/575, 29.7%), lack of supportive care (15-19 years: 22/287, 42.5%; 20-24 years: 195/575, 33.9%) and lack of privacy (15-19 years: 180/287, 62.7%; 20-24 years: 395/575, 68.7%). Women who were verbally abused were less likely to report satisfaction with care (adjusted OR (AOR): 0.19, 95% CI: 0.12 to 0.31) and less likely to recommend the facility (AOR: 0.24, 95% CI: 0.15 to 0.38). There were similar reports among those who were physically abused, had long waiting time, did not mobilise and did not give consent for vaginal examinations. CONCLUSION: Our study shows that adolescents and young women mistreatment during childbirth, contributing to low satisfaction with care. It is critical to recognise adolescents and young women's unique needs in maternal healthcare and how their needs may intersect with social stigma around sex and pregnancy.

Some individuals with autism spectrum disorder (ASD) carry functional mutations rarely observed in the general population. We explored the genes disrupted by these variants from joint analysis of protein-truncating variants (PTVs), missense variants and copy number variants (CNVs) in a cohort of 63,237 individuals. We discovered 72 genes associated with ASD at false discovery rate (FDR) ≤ 0.001 (185 at FDR ≤ 0.05). De novo PTVs, damaging missense variants and CNVs represented 57.5%, 21.1% and 8.44% of association evidence, while CNVs conferred greatest relative risk. Meta-analysis with cohorts ascertained for developmental delay (DD) (n = 91,605) yielded 373 genes associated with ASD/DD at FDR ≤ 0.001 (664 at FDR ≤ 0.05), some of which differed in relative frequency of mutation between ASD and DD cohorts. The DD-associated genes were enriched in transcriptomes of progenitor and immature neuronal cells, whereas genes showing stronger evidence in ASD were more enriched in maturing neurons and overlapped with schizophrenia-associated genes, emphasizing that these neuropsychiatric disorders may share common pathways to risk.

In structural health monitoring (SHM) placement of the sensors is often a problem and the more we can do to simplify this the better. In this paper, we will consider the case where displacement, angle and curvature is measured in the transition piece of an offshore wind turbine instead of the traditional solution where sensors are distributed over different heights of the tower. The two cases are investigated with respect to estimation of the modal parameters in an SHM application.

A new concept of length of a continuous mode shape has been recently defined by the authors, which depends on the mode shape and how the volume is distributed in the structure. This concept was then extended to discrete systems by introducing the concept of a volume matrix. However, finite element programs do not provide the lengths of the mode shapes according to this new definition. Moreover, the volume matrices cannot be exported from the finite element programs. In this paper, an approximate approach is proposed to calculate the length of numerical mode shapes from the nodal components. It has been demonstrated that the length can be estimated with a reasonable accuracy if small finite elements are used. These new techniques are illustrated by several numerical models assembled in ABAQUS. The length of numerical mode shapes can be used to estimate the length of experimental mode shapes using the structural dynamic modification, which is valuable information to validate the modal masses estimated with the existing techniques to determine the modal masses in operational modal analysis.

Modal identification procedures are used to evaluate the vibration properties of structures and can be divided into two branches, that is Input-Output and Output-Only analyses. Several methods are developed to assess the accuracy of the modal parameter estimates in the Input-Output framework, whereas deeper investigation is still needed to assess the confidence boundaries of Output-Only methods. Such accuracy of the estimates is generally given by comparing them with the expected set of modal parameters provided by a numerical model. In this paper, the accuracy of the modal parameters estimated using Output-Only techniques is investigated aided by the use of both time and frequencybased estimating methods coupled to a sensitivity analysis when varying data acquisition settings. Furthermore, insights into the robustness of the estimating process to the measuring chain is obtained from several independent tests and changes to the boundary conditions. Indeed, the experimental boundary conditions are altered by using materials with different elastic properties and different connection configurations. The uncertainties arising from the estimating process and the experimental setup mounting are evaluated for the natural frequencies, damping ratios, mode shapes through several modal assurance criterion formulations. The tests show that the accuracy of modal parameter estimates can be assessed and increased by weighting the similarity between multiple estimates of the same vibration mode. The accuracy depends on mounting configurations due to changes of structural properties. An evaluation of the overall accuracy of the modal parameter estimates finds that the accuracy is increased when information from multiple OMA algorithms is used.

Structural Health Monitoring (SHM) using OMA techniques utilizes the dynamic characteristics, as indicators of the health state of the monitored structure. In fact, changes in these dynamic properties can reflect either a potential degradation of the structural integrity or a change in the environmental conditions (e.g., temperature, humidity, etc.). Therefore, to apply a successful SHM approach to the monitored structures, the damage detection technique must identify and clearly separate the different sources of changes in the monitored dynamic properties. In this context, the present study aims to detect structural changes in a structure exposed to ambient vibrations and varying environmental conditions. The investigated structure consists of a wooden mast and a steel frame topside that is clamped to a concrete block at the bottom. In the monitoring campaign of the wooden mast, the vibration responses are collected by means of a measurement system whose sensors are placed at strategic locations on the topside, whereas the environmental conditions are recorded by a nearby weather station. The collected data is utilized to (i) estimate the environmental conditions; (ii) remove their influence from the estimated modal parameters, and finally (iii) detect structural changes induced by damages added with different severities to the mast.

In frequency domain experimental modal identification, the main goal is to extract the modal properties from the measured transfer function or power spectral densities. The modal parameter estimation can be carried out in a Linear Least Squares (LLS) sense by means of parametric identification methods. When the accuracy of such estimates is not accurate enough, non-linear identification methods based on Maximum Likelihood (ML) can by employed to improve precision of the LLS estimates in an iterative manner. One the main advantages of the ML-based optimization techniques is that, apart from providing more accurate estimates, they are also capable of estimating the uncertainties on the optimized estimates if the noise information is also taken in account in the optimization process. In this paper, the performance of a new frequency-domain ML-based technique formulated in Z-domain modal is investigated by means of a simulated example.

The magnitude and the units of the modal mass of a mode shape is not unique but it depends on the normalization method used to define the mode shape. Moreover, the magnitude can also depend depends on the number of degrees of freedom (DOFs) used to discretize the model. Recently, a new definition of the length of a mode shape, which depends on the mode shape and how the volume is distributed in the structure, has been proposed by the authors. This definition allows a better definition of the modal mass, which is physically meaningful and does not depend on the number of DOFs of a discrete model. With this new definition, the modal mass in constant mass-density systems is equal to the product between the total mass of the structure and the length squared. This property can be used advantageously to validate the modal masses estimated with the techniques proposed by different authors to determine the modal masses in operational modal analysis. In this paper, these new concepts are explained by analytical, numerical, and experimental examples. The model masses of an experimental steel beam structure were estimated by experimental modal analysis and validated with the equations proposed in this paper. Moreover, the modal masses and lengths of a rigid beam supported on two springs, were calculated using different sets of DOF’s and different types of normalization, demonstrating that the same mass normalized mode shapes are obtained.

Robust elimination of the environmental and operational variability from monitored vibration properties is essential for accurate vibration-based damage detection in civil engineering structures. This study uses the two-story wooden frame model of a shear building placed in real ambient conditions to extract natural frequencies of the structure by means of automated operational modal analysis. Further, inputoutput and output-only environmental models based on multiple linear regression and principal component analysis are developed to remove environmental influences from the identified frequency time series. This study aims to assess the robustness and performance of those methods in detecting damage and structural changes in civil structures exposed to highly variable environmental and operational conditions. The initial results show the superiority of the principal component analysis for modelling environmental effects and damage detection under challenging environments

The mobile communication enabled by cellular networks is the one of the main foundations of our modern society. Optimizing the performance of cellular networks and providing massive connectivity with improved coverage and user experience has a considerable social and economic impact on our daily life. This performance relies heavily on the configuration of the network parameters. However, with the massive increase in both the size and complexity of cellular networks, network management, especially parameter configuration, is becoming complicated. The current practice, which relies largely on experts' prior knowledge, is not adequate and will require lots of domain experts and high maintenance costs. In this work, we propose a learning-based framework for handover parameter configuration. The key challenge, in this case, is to tackle the complicated dependencies between neighboring cells and jointly optimize the whole network. Our framework addresses this challenge in two ways. First, we introduce a novel approach to imitate how the network responds to different network states and parameter values, called auto-grouping graph convolutional network (AG-GCN). During the parameter configuration stage, instead of solving the global optimization problem, we design a local multi-objective optimization strategy where each cell considers several local performance metrics to balance its own performance and its neighbors. We evaluate our proposed algorithm via a simulator constructed using real network data. We demonstrate that the handover parameters our model can find, achieve better average network throughput compared to those recommended by experts as well as alternative baselines, which can bring better network quality and stability. It has the potential to massively reduce costs arising from human expert intervention and maintenance.

Recently, grant-free transmission paradigm has been introduced for massive Internet of Things (IoT) networks to save both time and bandwidth and transmit the message with low latency. In order to accurately decode the message of each device at the base station (BS), first, the active devices at each transmission frame must be identified. In this work, first we investigate the problem of activity detection as a threshold comparing problem. We show the convexity of the activity detection method through analyzing its probability of error which makes it possible to find the optimal threshold for minimizing the activity detection error. Consequently, to achieve an optimum solution, we propose a deep learning (DL)-based method called convolutional neural network (CNN)-activity detection (AD). In order to make it more practical, we consider unknown and time-varying activity rate for the IoT devices. Our simulations verify that our proposed CNN-AD method can achieve higher performance compared to the existing non-Bayesian greedy-based methods. This is while existing methods need to know the activity rate of IoT devices, while our method works for unknown and even time-varying activity rates, Comment: Accepted in International Conference on Computing, Networking and Communications (ICNC 2023)

Point cloud learning is receiving increasing attention, however, most existing point cloud models lack the practical ability to deal with the unavoidable presence of unknown objects. This paper mainly discusses point cloud learning under open-set settings, where we train the model without data from unknown classes and identify them in the inference stage. Basically, we propose to solve open-set point cloud learning using a novel Point Cut-and-Mix mechanism consisting of Unknown-Point Simulator and Unknown-Point Estimator modules. Specifically, we use the Unknown-Point Simulator to simulate out-of-distribution data in the training stage by manipulating the geometric context of partial known data. Based on this, the Unknown-Point Estimator module learns to exploit the point cloud's feature context for discriminating the known and unknown data. Extensive experiments show the plausibility of open-set point cloud learning and the effectiveness of our proposed solutions. Our code is available at \url{https://github.com/ShiQiu0419/pointcam}.

Content-based medical image retrieval is an important diagnostic tool that improves the explainability of computer-aided diagnosis systems and provides decision making support to healthcare professionals. Medical imaging data, such as radiology images, are often multimorbidity; a single sample may have more than one pathology present. As such, image retrieval systems for the medical domain must be designed for the multi-label scenario. In this paper, we propose a novel multi-label metric learning method that can be used for both classification and content-based image retrieval. In this way, our model is able to support diagnosis by predicting the presence of diseases and provide evidence for these predictions by returning samples with similar pathological content to the user. In practice, the retrieved images may also be accompanied by pathology reports, further assisting in the diagnostic process. Our method leverages proxy feature vectors, enabling the efficient learning of a robust feature space in which the distance between feature vectors can be used as a measure of the similarity of those samples. Unlike existing proxy-based methods, training samples are able to assign to multiple proxies that span multiple class labels. This multi-label proxy assignment results in a feature space that encodes the complex relationships between diseases present in medical imaging data. Our method outperforms state-of-the-art image retrieval systems and a set of baseline approaches. We demonstrate the efficacy of our approach to both classification and content-based image retrieval on two multimorbidity radiology datasets.

One of the challenges in studying the interactions in large graphs is to learn their diverse pattern and various interaction types. Hence, considering only one distribution and model to study all nodes and ignoring their diversity and local features in their neighborhoods, might severely affect the overall performance. Based on the structural information of the nodes in the graph and the interactions between them, the main graph can be divided into multiple sub-graphs. This graph partitioning can tremendously affect the learning process, however the overall performance is highly dependent on the clustering method to avoid misleading the model. In this work, we present a new framework called KD-SGL to effectively learn the sub-graphs, where we define one global model to learn the overall structure of the graph and multiple local models for each sub-graph. We assess the performance of the proposed framework and evaluate it on public datasets. Based on the achieved results, it can improve the performance of the state-of-the-arts spatiotemporal models with comparable results compared to ensemble of models with less complexity.

We present LAVA, a simple yet effective method for multi-domain visual transfer learning with limited data. LAVA builds on a few recent innovations to enable adapting to partially labelled datasets with class and domain shifts. First, LAVA learns self-supervised visual representations on the source dataset and ground them using class label semantics to overcome transfer collapse problems associated with supervised pretraining. Secondly, LAVA maximises the gains from unlabelled target data via a novel method which uses multi-crop augmentations to obtain highly robust pseudo-labels. By combining these ingredients, LAVA achieves a new state-of-the-art on ImageNet semi-supervised protocol, as well as on 7 out of 10 datasets in multi-domain few-shot learning on the Meta-dataset. Code and models are made available., Comment: Accepted in WACV2023

Naively trained AI models can be heavily biased. This can be particularly problematic when the biases involve legally or morally protected attributes such as ethnic background, age or gender. Existing solutions to this problem come at the cost of extra computation, unstable adversarial optimisation or have losses on the feature space structure that are disconnected from fairness measures and only loosely generalise to fairness. In this work we propose a differentiable approximation of the variance of demographics, a metric that can be used to measure the bias, or unfairness, in an AI model. Our approximation can be optimised alongside the regular training objective which eliminates the need for any extra models during training and directly improves the fairness of the regularised models. We demonstrate that our approach improves the fairness of AI models in varied task and dataset scenarios, whilst still maintaining a high level of classification accuracy. Code is available at https://bitbucket.org/nelliottrosa/base_fairness.

As intensities of MRI volumes are inconsistent across institutes, it is essential to extract universal features of multi-modal MRIs to precisely segment brain tumors. In this concept, we propose a volumetric vision transformer that follows two windowing strategies in attention for extracting fine features and local distributional smoothness (LDS) during model training inspired by virtual adversarial training (VAT) to make the model robust. We trained and evaluated network architecture on the FeTS Challenge 2022 dataset. Our performance on the online validation dataset is as follows: Dice Similarity Score of 81.71%, 91.38% and 85.40%; Hausdorff Distance (95%) of 14.81 mm, 3.93 mm, 11.18 mm for the enhancing tumor, whole tumor, and tumor core, respectively. Overall, the experimental results verify our method's effectiveness by yielding better performance in segmentation accuracy for each tumor sub-region. Our code implementation is publicly available : https://github.com/himashi92/vizviva_fets_2022

Deep Metric Learning algorithms aim to learn an efficient embedding space to preserve the similarity relationships among the input data. Whilst these algorithms have achieved significant performance gains across a wide plethora of tasks, they have also failed to consider and increase comprehensive similarity constraints; thus learning a sub-optimal metric in the embedding space. Moreover, up until now; there have been few studies with respect to their performance in the presence of noisy labels. Here, we address the concern of learning a discriminative deep embedding space by designing a novel, yet effective Deep Class-wise Discrepancy Loss (DCDL) function that segregates the underlying similarity distributions (thus introducing class-wise discrepancy) of the embedding points between each and every class. Our empirical results across three standard image classification datasets and two fine-grained image recognition datasets in the presence and absence of noise clearly demonstrate the need for incorporating such class-wise similarity relationships along with traditional algorithms while learning a discriminative embedding space.

Federated Learning (FL) is a distributed learning paradigm that enhances users privacy by eliminating the need for clients to share raw, private data with the server. Despite the success, recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users private data via eavesdropping on the shared gradient information. We hypothesize that a key factor in the success of such attacks is the low entanglement among gradients per data within the batch during stochastic optimization. This creates a vulnerability that an adversary can exploit to reconstruct the sensitive data. Building upon this insight, we present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples. To achieve this, we propose synthesizing concealed samples to mimic the sensitive data at the gradient level while ensuring their visual dissimilarity from the actual sensitive data. Compared to the previous art, our empirical evaluations suggest that the proposed technique provides the strongest protection while simultaneously maintaining the FL performance., Comment: Defence against model inversion attack in federated learning

Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature. However, the success of the geometry constraints, posed in the embedding space, indicates that curved spaces might encode more structural information, leading to better discriminative power and hence richer representations. In this work, we investigate benefits of the curved space for analyzing anomalies or out-of-distribution objects in data. This is achieved by considering embeddings via three geometry constraints, namely, spherical geometry (with positive curvature), hyperbolic geometry (with negative curvature) or mixed geometry (with both positive and negative curvatures). Three geometric constraints can be chosen interchangeably in a unified design given the task at hand. Tailored for the embeddings in the curved space, we also formulate functions to compute the anomaly score. Two types of geometric modules (i.e., Geometric-in-One and Geometric-in-Two models) are proposed to plug in the original Euclidean classifier, and anomaly scores are computed from the curved embeddings. We evaluate the resulting designs under a diverse set of visual recognition scenarios, including image detection (multi-class OOD detection and one-class anomaly detection) and segmentation (multi-class anomaly segmentation and one-class anomaly segmentation). The empirical results show the effectiveness of our proposal through the consistent improvement over various scenarios.

The self-attention mechanism, successfully employed with the transformer structure is shown promise in many computer vision tasks including image recognition, and object detection. Despite the surge, the use of the transformer for the problem of stereo matching remains relatively unexplored. In this paper, we comprehensively investigate the use of the transformer for the problem of stereo matching, especially for laparoscopic videos, and propose a new hybrid deep stereo matching framework (HybridStereoNet) that combines the best of the CNN and the transformer in a unified design. To be specific, we investigate several ways to introduce transformers to volumetric stereo matching pipelines by analyzing the loss landscape of the designs and in-domain/cross-domain accuracy. Our analysis suggests that employing transformers for feature representation learning, while using CNNs for cost aggregation will lead to faster convergence, higher accuracy and better generalization than other options. Our extensive experiments on Sceneflow, SCARED2019 and dVPN datasets demonstrate the superior performance of our HybridStereoNet., Comment: Accepted to MICCAI 2022; Xuelian Cheng and Yiran Zhong made equal contributions. Code:https://github.com/XuelianCheng/HybridStereoNet-main.git

Global correlations are widely seen in human anatomical structures due to similarity across tissues and bones. These correlations are reflected in magnetic resonance imaging (MRI) scans as a result of close-range proton density and T1/T2 parameters. Furthermore, to achieve accelerated MRI, k-space data are undersampled which causes global aliasing artifacts. Convolutional neural network (CNN) models are widely utilized for accelerated MRI reconstruction, but those models are limited in capturing global correlations due to the intrinsic locality of the convolution operation. The self-attention-based transformer models are capable of capturing global correlations among image features, however, the current contributions of transformer models for MRI reconstruction are minute. The existing contributions mostly provide CNN-transformer hybrid solutions and rarely leverage the physics of MRI. In this paper, we propose a physics-based stand-alone (convolution free) transformer model titled, the Multi-head Cascaded Swin Transformers (McSTRA) for accelerated MRI reconstruction. McSTRA combines several interconnected MRI physics-related concepts with the transformer networks: it exploits global MR features via the shifted window self-attention mechanism; it extracts MR features belonging to different spectral components separately using a multi-head setup; it iterates between intermediate de-aliasing and k-space correction via a cascaded network with data consistency in k-space and intermediate loss computations; furthermore, we propose a novel positional embedding generation mechanism to guide self-attention utilizing the point spread function corresponding to the undersampling mask. Our model significantly outperforms state-of-the-art MRI reconstruction methods both visually and quantitatively while depicting improved resolution and removal of aliasing artifacts.

In this paper, we present and study a new image segmentation task, called Generalized Open-set Semantic Segmentation (GOSS). Previously, with the well-known open-set semantic segmentation (OSS), the intelligent agent only detects the unknown regions without further processing, limiting their perception of the environment. It stands to reason that a further analysis of the detected unknown pixels would be beneficial. Therefore, we propose GOSS, which unifies the abilities of two well-defined segmentation tasks, OSS and generic segmentation (GS), in a holistic way. Specifically, GOSS classifies pixels as belonging to known classes, and clusters (or groups) of pixels of unknown class are labelled as such. To evaluate this new expanded task, we further propose a metric which balances the pixel classification and clustering aspects. Moreover, we build benchmark tests on top of existing datasets and propose a simple neural architecture as a baseline, which jointly predicts pixel classification and clustering under open-set settings. Our experiments on multiple benchmarks demonstrate the effectiveness of our baseline. We believe our new GOSS task can produce an expressive image understanding for future research. Code will be made available.

We propose a new video camouflaged object detection (VCOD) framework that can exploit both short-term dynamics and long-term temporal consistency to detect camouflaged objects from video frames. An essential property of camouflaged objects is that they usually exhibit patterns similar to the background and thus make them hard to identify from still images. Therefore, effectively handling temporal dynamics in videos becomes the key for the VCOD task as the camouflaged objects will be noticeable when they move. However, current VCOD methods often leverage homography or optical flows to represent motions, where the detection error may accumulate from both the motion estimation error and the segmentation error. On the other hand, our method unifies motion estimation and object segmentation within a single optimization framework. Specifically, we build a dense correlation volume to implicitly capture motions between neighbouring frames and utilize the final segmentation supervision to optimize the implicit motion estimation and segmentation jointly. Furthermore, to enforce temporal consistency within a video sequence, we jointly utilize a spatio-temporal transformer to refine the short-term predictions. Extensive experiments on VCOD benchmarks demonstrate the architectural effectiveness of our approach. We also provide a large-scale VCOD dataset named MoCA-Mask with pixel-level handcrafted ground-truth masks and construct a comprehensive VCOD benchmark with previous methods to facilitate research in this direction. Dataset Link: https://xueliancheng.github.io/SLT-Net-project., Comment: Accepted to CVPR 2022; Xuelian Cheng and Huan Xiong made equal contributions; Corresponding author: Deng-Ping Fan (dengpfan@gmail.com). Dataset: https://xueliancheng.github.io/SLT-Net-project