Search

Semi-supervised segmentation presents a promising approach for large-scale medical image analysis, effectively reducing annotation burdens while achieving comparable performance. This methodology holds substantial potential for streamlining the segmentation process and enhancing its feasibility within clinical settings for translational investigations. While cross-supervised training, based on distinct co-training sub-networks, has become a prevalent paradigm for this task, addressing critical issues such as predication disagreement and label-noise suppression requires further attention and progress in cross-supervised training. In this paper, we introduce an Evidential Tri-Branch Consistency learning framework (ETC-Net) for semi-supervised medical image segmentation. ETC-Net employs three branches: an evidential conservative branch, an evidential progressive branch, and an evidential fusion branch. The first two branches exhibit complementary characteristics, allowing them to address prediction diversity and enhance training stability. We also integrate uncertainty estimation from the evidential learning into cross-supervised training, mitigating the negative impact of erroneous supervision signals. Additionally, the evidential fusion branch capitalizes on the complementary attributes of the first two branches and leverages an evidence-based Dempster-Shafer fusion strategy, supervised by more reliable and accurate pseudo-labels of unlabeled data. Extensive experiments conducted on LA, Pancreas-CT, and ACDC datasets demonstrate that ETC-Net surpasses other state-of-the-art methods for semi-supervised segmentation. The code will be made available in the near future at https://github.com/Medsemiseg.

The growth of Graph Convolution Network (GCN) model sizes has revolutionized numerous applications, surpassing human performance in areas such as personal healthcare and financial systems. The deployment of GCNs in the cloud raises privacy concerns due to potential adversarial attacks on client data. To address security concerns, Privacy-Preserving Machine Learning (PPML) using Homomorphic Encryption (HE) secures sensitive client data. However, it introduces substantial computational overhead in practical applications. To tackle those challenges, we present LinGCN, a framework designed to reduce multiplication depth and optimize the performance of HE based GCN inference. LinGCN is structured around three key elements: (1) A differentiable structural linearization algorithm, complemented by a parameterized discrete indicator function, co-trained with model weights to meet the optimization goal. This strategy promotes fine-grained node-level non-linear location selection, resulting in a model with minimized multiplication depth. (2) A compact node-wise polynomial replacement policy with a second-order trainable activation function, steered towards superior convergence by a two-level distillation approach from an all-ReLU based teacher model. (3) an enhanced HE solution that enables finer-grained operator fusion for node-wise activation functions, further reducing multiplication level consumption in HE-based inference. Our experiments on the NTU-XVIEW skeleton joint dataset reveal that LinGCN excels in latency, accuracy, and scalability for homomorphically encrypted inference, outperforming solutions such as CryptoGCN. Remarkably, LinGCN achieves a 14.2x latency speedup relative to CryptoGCN, while preserving an inference accuracy of 75% and notably reducing multiplication depth., Comment: NeurIPS 2023 accepted publication

Multispectral and Hyperspectral Image Fusion (MHIF) is a practical task that aims to fuse a high-resolution multispectral image (HR-MSI) and a low-resolution hyperspectral image (LR-HSI) of the same scene to obtain a high-resolution hyperspectral image (HR-HSI). Benefiting from powerful inductive bias capability, CNN-based methods have achieved great success in the MHIF task. However, they lack certain interpretability and require convolution structures be stacked to enhance performance. Recently, Implicit Neural Representation (INR) has achieved good performance and interpretability in 2D tasks due to its ability to locally interpolate samples and utilize multimodal content such as pixels and coordinates. Although INR-based approaches show promise, they require extra construction of high-frequency information (\emph{e.g.,} positional encoding). In this paper, inspired by previous work of MHIF task, we realize that HR-MSI could serve as a high-frequency detail auxiliary input, leading us to propose a novel INR-based hyperspectral fusion function named Implicit Neural Feature Fusion Function (INF). As an elaborate structure, it solves the MHIF task and addresses deficiencies in the INR-based approaches. Specifically, our INF designs a Dual High-Frequency Fusion (DHFF) structure that obtains high-frequency information twice from HR-MSI and LR-HSI, then subtly fuses them with coordinate information. Moreover, the proposed INF incorporates a parameter-free method named INR with cosine similarity (INR-CS) that uses cosine similarity to generate local weights through feature vectors. Based on INF, we construct an Implicit Neural Fusion Network (INFN) that achieves state-of-the-art performance for MHIF tasks of two public datasets, \emph{i.e.,} CAVE and Harvard. The code will soon be made available on GitHub.

Two-party computation (2PC) is promising to enable privacy-preserving deep learning (DL). However, the 2PC-based privacy-preserving DL implementation comes with high comparison protocol overhead from the non-linear operators. This work presents PASNet, a novel systematic framework that enables low latency, high energy efficiency & accuracy, and security-guaranteed 2PC-DL by integrating the hardware latency of the cryptographic building block into the neural architecture search loss function. We develop a cryptographic hardware scheduler and the corresponding performance model for Field Programmable Gate Arrays (FPGA) as a case study. The experimental results demonstrate that our light-weighted model PASNet-A and heavily-weighted model PASNet-B achieve 63 ms and 228 ms latency on private inference on ImageNet, which are 147 and 40 times faster than the SOTA CryptGPU system, and achieve 70.54% & 78.79% accuracy and more than 1000 times higher energy efficiency., Comment: DAC 2023 accepeted publication, short version was published on AAAI 2023 workshop on DL-Hardware Co-Design for AI Acceleration: RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference

Semi-supervised medical image segmentation offers a promising solution for large-scale medical image analysis by significantly reducing the annotation burden while achieving comparable performance. Employing this method exhibits a high degree of potential for optimizing the segmentation process and increasing its feasibility in clinical settings during translational investigations. Recently, cross-supervised training based on different co-training sub-networks has become a standard paradigm for this task. Still, the critical issues of sub-network disagreement and label-noise suppression require further attention and progress in cross-supervised training. This paper proposes a cross-supervised learning framework based on dual classifiers (DC-Net), including an evidential classifier and a vanilla classifier. The two classifiers exhibit complementary characteristics, enabling them to handle disagreement effectively and generate more robust and accurate pseudo-labels for unlabeled data. We also incorporate the uncertainty estimation from the evidential classifier into cross-supervised training to alleviate the negative effect of the error supervision signal. The extensive experiments on LA and Pancreas-CT dataset illustrate that DC-Net outperforms other state-of-the-art methods for semi-supervised segmentation. The code will be released soon., Comment: 13 pages, 4 figures, 5 tables. Code will come soon

Consistency learning plays a crucial role in semi-supervised medical image segmentation as it enables the effective utilization of limited annotated data while leveraging the abundance of unannotated data. The effectiveness and efficiency of consistency learning are challenged by prediction diversity and training stability, which are often overlooked by existing studies. Meanwhile, the limited quantity of labeled data for training often proves inadequate for formulating intra-class compactness and inter-class discrepancy of pseudo labels. To address these issues, we propose a self-aware and cross-sample prototypical learning method (SCP-Net) to enhance the diversity of prediction in consistency learning by utilizing a broader range of semantic information derived from multiple inputs. Furthermore, we introduce a self-aware consistency learning method that exploits unlabeled data to improve the compactness of pseudo labels within each class. Moreover, a dual loss re-weighting method is integrated into the cross-sample prototypical consistency learning method to improve the reliability and stability of our model. Extensive experiments on ACDC dataset and PROMISE12 dataset validate that SCP-Net outperforms other state-of-the-art semi-supervised segmentation methods and achieves significant performance gains compared to the limited supervised training. Our code will come soon., Comment: 14 pages, Early accepted in MICCAI 2023, code will be released soon

Denosing diffusion model, as a generative model, has received a lot of attention in the field of image generation recently, thanks to its powerful generation capability. However, diffusion models have not yet received sufficient research in the field of image fusion. In this article, we introduce diffusion model to the image fusion field, treating the image fusion task as image-to-image translation and designing two different conditional injection modulation modules (i.e., style transfer modulation and wavelet modulation) to inject coarse-grained style information and fine-grained high-frequency and low-frequency information into the diffusion UNet, thereby generating fused images. In addition, we also discussed the residual learning and the selection of training objectives of the diffusion model in the image fusion task. Extensive experimental results based on quantitative and qualitative assessments compared with benchmarks demonstrates state-of-the-art results and good generalization performance in image fusion tasks. Finally, it is hoped that our method can inspire other works and gain insight into this field to better apply the diffusion model to image fusion tasks. Code shall be released for better reproducibility.

The proliferation of deep learning (DL) has led to the emergence of privacy and security concerns. To address these issues, secure Two-party computation (2PC) has been proposed as a means of enabling privacy-preserving DL computation. However, in practice, 2PC methods often incur high computation and communication overhead, which can impede their use in large-scale systems. To address this challenge, we introduce RRNet, a systematic framework that aims to jointly reduce the overhead of MPC comparison protocols and accelerate computation through hardware acceleration. Our approach integrates the hardware latency of cryptographic building blocks into the DNN loss function, resulting in improved energy efficiency, accuracy, and security guarantees. Furthermore, we propose a cryptographic hardware scheduler and corresponding performance model for Field Programmable Gate Arrays (FPGAs) to further enhance the efficiency of our framework. Experiments show RRNet achieved a much higher ReLU reduction performance than all SOTA works on CIFAR-10 dataset., Comment: This is work is a updated version of arXiv:2209.09424, the original version has been withdrawn

Recently cloud-based graph convolutional network (GCN) has demonstrated great success and potential in many privacy-sensitive applications such as personal healthcare and financial systems. Despite its high inference accuracy and performance on cloud, maintaining data privacy in GCN inference, which is of paramount importance to these practical applications, remains largely unexplored. In this paper, we take an initial attempt towards this and develop $\textit{CryptoGCN}$--a homomorphic encryption (HE) based GCN inference framework. A key to the success of our approach is to reduce the tremendous computational overhead for HE operations, which can be orders of magnitude higher than its counterparts in the plaintext space. To this end, we develop an approach that can effectively take advantage of the sparsity of matrix operations in GCN inference to significantly reduce the computational overhead. Specifically, we propose a novel AMA data formatting method and associated spatial convolution methods, which can exploit the complex graph structure and perform efficient matrix-matrix multiplication in HE computation and thus greatly reduce the HE operations. We also develop a co-optimization framework that can explore the trade offs among the accuracy, security level, and computational overhead by judicious pruning and polynomial approximation of activation module in GCNs. Based on the NTU-XVIEW skeleton joint dataset, i.e., the largest dataset evaluated homomorphically by far as we are aware of, our experimental results demonstrate that $\textit{CryptoGCN}$ outperforms state-of-the-art solutions in terms of the latency and number of homomorphic operations, i.e., achieving as much as a 3.10$\times$ speedup on latency and reduces the total Homomorphic Operation Count by 77.4\% with a small accuracy loss of 1-1.5$\%$., Comment: Accepted in Conference on Neural Information Processing Systems (NeurIPS 2022)

The rapid growth and deployment of deep learning (DL) has witnessed emerging privacy and security concerns. To mitigate these issues, secure multi-party computation (MPC) has been discussed, to enable the privacy-preserving DL computation. In practice, they often come at very high computation and communication overhead, and potentially prohibit their popularity in large scale systems. Two orthogonal research trends have attracted enormous interests in addressing the energy efficiency in secure deep learning, i.e., overhead reduction of MPC comparison protocol, and hardware acceleration. However, they either achieve a low reduction ratio and suffer from high latency due to limited computation and communication saving, or are power-hungry as existing works mainly focus on general computing platforms such as CPUs and GPUs. In this work, as the first attempt, we develop a systematic framework, PolyMPCNet, of joint overhead reduction of MPC comparison protocol and hardware acceleration, by integrating hardware latency of the cryptographic building block into the DNN loss function to achieve high energy efficiency, accuracy, and security guarantee. Instead of heuristically checking the model sensitivity after a DNN is well-trained (through deleting or dropping some non-polynomial operators), our key design principle is to em enforce exactly what is assumed in the DNN design -- training a DNN that is both hardware efficient and secure, while escaping the local minima and saddle points and maintaining high accuracy. More specifically, we propose a straight through polynomial activation initialization method for cryptographic hardware friendly trainable polynomial activation function to replace the expensive 2P-ReLU operator. We develop a cryptographic hardware scheduler and the corresponding performance model for Field Programmable Gate Arrays (FPGA) platform., Comment: Uploaded a new version of the paper in another new submission: RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference [arXiv:2302.02292]

IoT devices are increasingly being implemented with neural network models to enable smart applications. Energy harvesting (EH) technology that harvests energy from ambient environment is a promising alternative to batteries for powering those devices due to the low maintenance cost and wide availability of the energy sources. However, the power provided by the energy harvester is low and has an intrinsic drawback of instability since it varies with the ambient environment. This paper proposes EVE, an automated machine learning (autoML) co-exploration framework to search for desired multi-models with shared weights for energy harvesting IoT devices. Those shared models incur significantly reduced memory footprint with different levels of model sparsity, latency, and accuracy to adapt to the environmental changes. An efficient on-device implementation architecture is further developed to efficiently execute each model on device. A run-time model extraction algorithm is proposed that retrieves individual model with negligible overhead when a specific model mode is triggered.Experimental results show that the neural networks models generated by EVE is on average 2.5X times faster than the baseline models without pruning and shared weights.

Membership inference attacks (MIAs) against machine learning models can lead to serious privacy risks for the training dataset used in the model training. In this paper, we propose a novel and effective Neuron-Guided Defense method named NeuGuard against membership inference attacks (MIAs). We identify a key weakness in existing defense mechanisms against MIAs wherein they cannot simultaneously defend against two commonly used neural network based MIAs, indicating that these two attacks should be separately evaluated to assure the defense effectiveness. We propose NeuGuard, a new defense approach that jointly controls the output and inner neurons' activation with the object to guide the model output of training set and testing set to have close distributions. NeuGuard consists of class-wise variance minimization targeting restricting the final output neurons and layer-wise balanced output control aiming to constrain the inner neurons in each layer. We evaluate NeuGuard and compare it with state-of-the-art defenses against two neural network based MIAs, five strongest metric based MIAs including the newly proposed label-only MIA on three benchmark datasets. Results show that NeuGuard outperforms the state-of-the-art defenses by offering much improved utility-privacy trade-off, generality, and overhead.

In a recent letter (PRL,121,246101,2018), Sun et al. reported that combined MD simulation and sum frequency generation vibrational spectroscopy (SFG-VS) measurements led to conclusions of a broad and exponentially decaying orientational distribution, and the presence of the free O-H group pointing down to the bulk at the air/water interface. In this comment, we show that their main conclusions are based on questionable interpretation of the SFG-VS data presented in the letter [1], and are also contrary to the established data analysis and interpretations in the literature [2-5]., Comment: 2 pages, 0 figures

The existence of sufficiently many finite order meromorphic solutions of a differential equation, or difference equation, or differential-difference equation, appears to be a good indicator of integrability. In this paper, we investigate the nonlinear differential-difference equations of form \begin{equation*} f(z)^{n}+L(z,f)=q(z)e^{p(z)},\eqno(*) \end{equation*} where $n\geq 2, L(z,f)(\not\equiv 0)$ is a linear differential-difference polynomial in $f(z)$, with small functions as its coefficients, $p(z)$ and $q(z)$ are non-vanishing polynomials. We first obtain that $n=2$ and $f(z)$ satisfies $\overline{\lambda}(f)=\sigma(f)=\deg p(z)$ under the assumption that the equation (*) possesses a transcendental entire solution of hyper order $\sigma_{2}(f)<1$. Furthermore, we give the exact form of the solutions of equation (*) when $p(z)=a, q(z)=b$, $\eta$ are constants and $L(z,f)=g(z)f(z+\eta)+h(z)f^{'}(z)+u(z)f(z)+v(z)$ is a linear differential-difference polynomial in $f(z)$ with polynomial coefficients $g(z), h(z), u(z)$ and $v(z)$ such that $L(z,f)\not\equiv 0$ and $a b \eta\neq 0$., Comment: 12 pages

In this paper, we investigate the delay differential equations of Malmquist type of form \begin{equation*} w(z+1)-w(z-1)+a(z)\frac{w'(z)}{w(z)}=R(z, w(z)),~~~~~~~~~~~~~~(*) \end{equation*} where $R(z, w(z))$ is an irreducible rational function in $w(z)$ with rational coefficients and $a(z)$ is a rational function. We characterize all reduced forms when the equation $(*)$ admits a transcendental entire solutions with hyper-order less than one. When we compare with the results obtained by Halburd and Korhonen[Proc.Amer.Math.Soc., forcoming],we obtain the reduced forms without the assumptions that the denominator of rational function $R(z,w(z))$ has roots that are nonzero rational functions in $z$. The growth order and value distribution of transcendental entire solutions for the reduced forms are also investigated.

Sum frequency generation vibrational spectra of the water molecules at the NaF and KF aqueous solution surfaces showed significantly different spectral features and different concentration dependence. This result is the first direct observation of the cation effects of the simple alkali cations, which have been believed to be depleted from the aqueous surface, on the hydrogen bonding structure of the water molecules at the electrolyte solution surfaces. These observations may provide important clue to understand the fundamental phenomenon of ions at the air/water interface., Comment: 15 pages, 2 figures

Here we report the polarization dependent non-resonant second harmonic generation (SHG) measurement of the interfacial water molecules at the aqueous solution of the following salts: NaF, NaCl, NaBr, KF, KCl, and KBr. Through quantitative polarization analysis of the SHG data,the orientational parameter D value and the relative surface density of the interfacial water molecules at these aqueous solution surfaces were determined. From these results we found that addition of each of the six salts caused increase of the thickness of the interfacial water layer at the surfaces to a certain extent. Noticeably, both the cations and the anions contributed to the changes, and the abilities to increase the thickness of the interfacial water layer were in the following order: KBr > NaBr > KCl > NaCl ~ NaF > KF. Since these changes can not be factorized into individual anion and cation contributions, there are possible ion pairing or association effects, especially for the NaF case. We also found that the orientational parameter D values of the interfacial water molecules changed to opposite directions for the aqueous solutions of the three sodium salts versus the aqueous solutions of the three potassium salts. These findings clearly indicated unexpected specific Na+ and K+ cation effects at the aqueous solution surface. These effects were not anticipated from the recent molecular dynamics (MD) simulation results, which concluded that the Na+ and K+ cations can be treated as small non-polarizable hard ions and they are repelled from the aqueous interfaces. These results suggest that the electrolyte aqueous solution surfaces are more complex than the currently prevalent theoretical and experimental understandings., Comment: 33 pages, 5 figures

Here we report a detailed study on spectroscopy, structure and dynamics of water molecules at air/water interface, investigated with Sum Frequency Generation Vibrational Spectroscopy (SFG-VS). Quantitative polarization and experimental configuration analysis of the SFG data in different polarizations with four sets of experimental configurations can shed new lights on our present understanding of the air/water interface. Firstly, we concluded that the motion of the interfacial water molecules can only be in a limited angular range, instead rapidly varying over a broad angular range in the vibrational relaxation time suggested previously. Secondly, because different vibrational modes of different molecular species at the interface has different symmetry properties, polarization and symmetry analysis of the SFG-VS spectral features can help assignment of the SFG-VS spectra peaks to different interfacial species. These analysis concluded that the narrow 3693cm-1 and broad 3550cm-1 peaks belong to C_ v symmetry, while the broad 3250cm-1 and 3450cm-1 peaks belong to the symmetric stretching modes with C_2v symmetry. Thus, the 3693cm-1 peak is assigned to the free OH, the 3550cm-1 peak is assigned to the single hydrogen bonded OH stretching mode, and the 3250cm-1 and 3450cm-1 peaks are assigned to interfacial water molecules as two hydrogen donors for hydrogen bonding (with C_2v symmetry), respectively. Thirdly, analysis of the SFG-VS spectra concluded that the singly hydrogen bonded water molecules at the air/water interface have their dipole vector direct almost parallel to the interface, and is with a very narrow orientational distribution. The doubly hydrogen bond donor water molecules have their dipole vector point away from the liquid phase.