Search

Integrating textual data with imaging in liver tumor segmentation is essential for enhancing diagnostic accuracy. However, current multi-modal medical datasets offer only general text annotations, lacking lesion-specific details critical for extracting nuanced features, especially for fine-grained segmentation of tumor boundaries and small lesions. To address these limitations, we developed datasets with lesion-specific text annotations for liver tumors and introduced the TexLiverNet model. TexLiverNet employs an agent-based cross-attention module that integrates text features efficiently with visual features, significantly reducing computational costs. Additionally, enhanced spatial and adaptive frequency domain perception is proposed to precisely delineate lesion boundaries, reduce background interference, and recover fine details in small lesions. Comprehensive evaluations on public and private datasets demonstrate that TexLiverNet achieves superior performance compared to current state-of-the-art methods.

Background: Liver cancer ranks as the fifth most common malignant tumor and the second most fatal in our country. Early diagnosis is crucial, necessitating that physicians identify liver cancer in patients at the earliest possible stage. However, the diagnostic process is complex and demanding. Physicians must analyze a broad spectrum of patient data, encompassing physical condition, symptoms, medical history, and results from various examinations and tests, recorded in both structured and unstructured medical formats. This results in a significant workload for healthcare professionals. In response, integrating knowledge graph technology to develop a liver cancer knowledge graph-assisted diagnosis and treatment system aligns with national efforts toward smart healthcare. Such a system promises to mitigate the challenges faced by physicians in diagnosing and treating liver cancer. Methods: This paper addresses the major challenges in building a knowledge graph for hepatocellular carcinoma diagnosis, such as the discrepancy between public data sources and real electronic medical records, the effective integration of which remains a key issue. The knowledge graph construction process consists of six steps: conceptual layer design, data preprocessing, entity identification, entity normalization, knowledge fusion, and graph visualization. A novel Dynamic Entity Replacement and Masking Strategy (DERM) for named entity recognition is proposed. Results: A knowledge graph for liver cancer was established, including 7 entity types such as disease, symptom, and constitution, containing 1495 entities. The recognition accuracy of the model was 93.23%, the recall was 94.69%, and the F1 score was 93.96%.

Recent advances have demonstrated the powerful capability of transformer architecture in image restoration. However, our analysis indicates that existing transformerbased methods can not establish both exact global and local dependencies simultaneously, which are much critical to restore the details and missing content of degraded images. To this end, we present an efficient image processing transformer architecture with hierarchical attentions, called IPTV2, adopting a focal context self-attention (FCSA) and a global grid self-attention (GGSA) to obtain adequate token interactions in local and global receptive fields. Specifically, FCSA applies the shifted window mechanism into the channel self-attention, helps capture the local context and mutual interaction across channels. And GGSA constructs long-range dependencies in the cross-window grid, aggregates global information in spatial dimension. Moreover, we introduce structural re-parameterization technique to feed-forward network to further improve the model capability. Extensive experiments demonstrate that our proposed IPT-V2 achieves state-of-the-art results on various image processing tasks, covering denoising, deblurring, deraining and obtains much better trade-off for performance and computational complexity than previous methods. Besides, we extend our method to image generation as latent diffusion backbone, and significantly outperforms DiTs.

Editing real images authentically while also achieving cross-domain editing remains a challenge. Recent studies have focused on converting real images into latent codes and accomplishing image editing by manipulating these codes. However, merely manipulating the latent codes would constrain the edited images to the generator's image domain, hindering the attainment of diverse editing goals. In response, we propose an innovative image editing method called HyperEditor, which utilizes weight factors generated by hypernetworks to reassign the weights of the pre-trained StyleGAN2's generator. Guided by CLIP's cross-modal image-text semantic alignment, this innovative approach enables us to simultaneously accomplish authentic attribute editing and cross-domain style transfer, a capability not realized in previous methods. Additionally, we ascertain that modifying only the weights of specific layers in the generator can yield an equivalent editing result. Therefore, we introduce an adaptive layer selector, enabling our hypernetworks to autonomously identify the layers requiring output weight factors, which can further improve our hypernetworks' efficiency. Extensive experiments on abundant challenging datasets demonstrate the effectiveness of our method., Comment: Accepted by AAAI2024

Multi-frame high dynamic range (HDR) imaging aims to reconstruct ghost-free images with photo-realistic details from content-complementary but spatially misaligned low dynamic range (LDR) images. Existing HDR algorithms are prone to producing ghosting artifacts as their methods fail to capture long-range dependencies between LDR frames with large motion in dynamic scenes. To address this issue, we propose a novel image fusion transformer, referred to as IFT, which presents a fast global patch searching (FGPS) module followed by a self-cross fusion module (SCF) for ghost-free HDR imaging. The FGPS searches the patches from supporting frames that have the closest dependency to each patch of the reference frame for long-range dependency modeling, while the SCF conducts intra-frame and inter-frame feature fusion on the patches obtained by the FGPS with linear complexity to input resolution. By matching similar patches between frames, objects with large motion ranges in dynamic scenes can be aligned, which can effectively alleviate the generation of artifacts. In addition, the proposed FGPS and SCF can be integrated into various deep HDR methods as efficient plug-in modules. Extensive experiments on multiple benchmarks show that our method achieves state-of-the-art performance both quantitatively and qualitatively.

Neuromorphic photonic computing has emerged as a competitive computing paradigm to overcome the bottlenecks of the von-Neumann architecture. Linear weighting and nonlinear spiking activation are two fundamental functions of a photonic spiking neural network (PSNN). However, they are separately implemented with different photonic materials and devices, hindering the large-scale integration of PSNN. Here, we propose, fabricate and experimentally demonstrate a photonic neuro-synaptic chip enabling the simultaneous implementation of linear weighting and nonlinear spiking activation based on a distributed feedback (DFB) laser with a saturable absorber (DFB-SA). A prototypical system is experimentally constructed to demonstrate the parallel weighted function and nonlinear spike activation. Furthermore, a four-channel DFB-SA array is fabricated for realizing matrix convolution of a spiking convolutional neural network, achieving a recognition accuracy of 87% for the MNIST dataset. The fabricated neuro-synaptic chip offers a fundamental building block to construct the large-scale integrated PSNN chip.

Photonic neuromorphic computing has emerged as a promising avenue toward building a low-latency and energy-efficient non-von-Neuman computing system. Photonic spiking neural network (PSNN) exploits brain-like spatiotemporal processing to realize high-performance neuromorphic computing. However, the nonlinear computation of PSNN remains a significant challenging. Here, we proposed and fabricated a photonic spiking neuron chip based on an integrated Fabry-P\'erot laser with a saturable absorber (FP-SA) for the first time. The nonlinear neuron-like dynamics including temporal integration, threshold and spike generation, refractory period, and cascadability were experimentally demonstrated, which offers an indispensable fundamental building block to construct the PSNN hardware. Furthermore, we proposed time-multiplexed spike encoding to realize functional PSNN far beyond the hardware integration scale limit. PSNNs with single/cascaded photonic spiking neurons were experimentally demonstrated to realize hardware-algorithm collaborative computing, showing capability in performing classification tasks with supervised learning algorithm, which paves the way for multi-layer PSNN for solving complex tasks., Comment: 10 pages, 8 figures

In this paper, we consider a special class of nonlinear optimal control problems, where the control variables are box-constrained and the objective functional is strongly convex corresponding to control variables and separable with respect to the state variables and control variables. We convert solving the original nonlinear problem into solving a sequence of constrained linear-quadratic optimal control problems by quasilinearization method. In order to solve each linear-quadratic problem efficiently we turn to study its dual problem. We formulate dual problem by the scheme of Fenchel duality, the strong duality property and the saddle point property corresponding to primal and dual problem are also proved, which together ensure that solving dual problem is effective. Thus solving the sequence of control constrained linear-quadratic optimal control problems obtained by quasilinearization technique is substituted by solving the sequence of their dual problem. We solve the sequence of dual problem and obtain the solution to primal control constrained linear-quadratic problem by the saddle point property. Furthermore, the fact that solution to each subproblem finally converges to the solution to the optimality conditions of original nonlinear problem is also proved. After that we carry out numerical experiments using present approach, for each subproblem we formulate the discretized primal and dual problem by Euler discretization scheme in our experiments. Efficiency of the present method is validated by numerical results., Comment: 25 pages, 5 figures

This paper is concerned with the optimal control problem governed by a linear parabolic equation and subjected to box constraints on control variables. This type of problem has important applications in heating and cooling systems. By applying the scheme of Fenchel duality, we derive the dual problem explicitly where the control constraints in primal problem are embedded in the dual problem's objective functional. The existence and uniqueness of the solution to the dual problem are proved and the first-order optimality conditions are also derived. In addition, we discuss the saddle point property between solution of the primal problem and the dual problem. The solution of primal problem can be readily obtained by the solution of the dual problem. To solve the dual problem numerically, we design two implementable method: conjugate gradient method and semismooth Newton method. Three example problems are solved, numerical results show that the proposed method is efficient and accurate., Comment: 26 pages, 7 figures

Recently, deep hashing methods have been widely used in image retrieval task. Most existing deep hashing approaches adopt one-to-one quantization to reduce information loss. However, such class-unrelated quantization cannot give discriminative feedback for network training. In addition, these methods only utilize single label to integrate supervision information of data for hashing function learning, which may result in inferior network generalization performance and relatively low-quality hash codes since the inter-class information of data is totally ignored. In this paper, we propose a dual semantic asymmetric hashing (DSAH) method, which generates discriminative hash codes under three-fold constraints. Firstly, DSAH utilizes class prior to conduct class structure quantization so as to transmit class information during the quantization process. Secondly, a simple yet effective label mechanism is designed to characterize both the intra-class compactness and inter-class separability of data, thereby achieving semantic-sensitive binary code learning. Finally, a meaningful pairwise similarity preserving loss is devised to minimize the distances between class-related network outputs based on an affinity graph. With these three main components, high-quality hash codes can be generated through network. Extensive experiments conducted on various datasets demonstrate the superiority of DSAH in comparison with state-of-the-art deep hashing methods.

Effective features can improve the performance of a model, which can thus help us understand the characteristics and underlying structure of complex data. Previous feature selection methods usually cannot keep more local structure information. To address the defects previously mentioned, we propose a novel supervised orthogonal least square regression model with feature weighting for feature selection. The optimization problem of the objection function can be solved by employing generalized power iteration (GPI) and augmented Lagrangian multiplier (ALM) methods. Experimental results show that the proposed method can more effectively reduce the feature dimensionality and obtain better classification results than traditional feature selection methods. The convergence of our iterative method is proved as well. Consequently, the effectiveness and superiority of the proposed method are verified both theoretically and experimentally.