Search

Fine-tuning Large Language Models (LLMs) for clinical Natural Language Processing (NLP) poses significant challenges due to the domain gap and limited data availability. This study investigates the effectiveness of various adapter techniques, equivalent to Low-Rank Adaptation (LoRA), for fine-tuning LLMs in a resource-constrained hospital environment. We experimented with four structures-Adapter, Lightweight, TinyAttention, and Gated Residual Network (GRN)-as final layers for clinical notes classification. We fine-tuned biomedical pre-trained models, including CamemBERT-bio, AliBERT, and DrBERT, alongside two Transformer-based models. Our extensive experimental results indicate that i) employing adapter structures does not yield significant improvements in fine-tuning biomedical pre-trained LLMs, and ii) simpler Transformer-based models, trained from scratch, perform better under resource constraints. Among the adapter structures, GRN demonstrated superior performance with accuracy, precision, recall, and an F1 score of 0.88. Moreover, the total training time for LLMs exceeded 1000 hours, compared to under 6 hours for simpler transformer-based models, highlighting that LLMs are more suitable for environments with extensive computational resources and larger datasets. Consequently, this study demonstrates that simpler Transformer-based models can be effectively trained from scratch, providing a viable solution for clinical NLP tasks in low-resource environments with limited data availability. By identifying the GRN as the most effective adapter structure, we offer a practical approach to enhance clinical note classification without requiring extensive computational resources., Comment: Under revisions

Semi-grant-free non-orthogonal multiple access (semi-GF NOMA) has emerged as a promising technology for the fifth-generation new radio (5G-NR) networks supporting the coexistence of a large number of random connections with various quality of service requirements. However, implementing a semi-GF NOMA mechanism in 5G-NR networks with heterogeneous services has raised several resource management problems relating to unpredictable interference caused by the GF access strategy. To cope with this challenge, the paper develops a novel hybrid optimization and multi-agent deep (HOMAD) reinforcement learning-based resource allocation design to maximize the energy efficiency (EE) of semi-GF NOMA 5G-NR systems. In this design, a multi-agent deep Q network (MADQN) approach is employed to conduct the subchannel assignment (SA) among users. While optimization-based methods are utilized to optimize the transmission power for every SA setting. In addition, a full MADQN scheme conducting both SA and power allocation is also considered for comparison purposes. Simulation results show that the HOMAD approach outperforms other benchmarks significantly in terms of the convergence time and average EE.

This paper aims to jointly determine linear precoding (LP) vectors, beam hopping (BH), and discrete DVB-S2X transmission rates for the GEO satellite communication systems to minimize the payload power consumption and satisfy ground users' demands within a time window. Regarding constraint on the maximum number of illuminated beams per time slot, the technical requirement is formulated as a sparse optimization problem in which the hardware-related beam illumination energy is modeled in a sparsity form of the LP vectors. To cope with this problem, the compressed sensing method is employed to transform the sparsity parts into the quadratic form of precoders. Then, an iterative window-based algorithm is developed to update the LP vectors sequentially to an efficient solution. Additionally, two other two-phase frameworks are also proposed for comparison purposes. In the first phase, these methods aim to determine the MODCOD transmission schemes for users to meet their demands by using a heuristic approach or DNN tool. In the second phase, the LP vectors of each time slot will be optimized separately based on the determined MODCOD schemes., Comment: 6 pages, 7 figures, accepted - Globecom 2022

5G wireless systems will extend mobile communication services beyond mobile telephony, mobile broadband, and massive machine-type communication into new application domains, namely the so-called vertical domains including the smart factory, smart vehicles, smart grid, smart city, etc. Supporting these vertical domains comes with demanding requirements: high-availability, high-reliability, low-latency, and in some cases, high-accuracy positioning. In this survey, we first identify the potential key performance requirements of 5G communication in support of automation in the vertical domains and highlight the 5G enabling technologies conceived for meeting these requirements. We then discuss the key challenges faced both by industry and academia which have to be addressed in order to support automation in the vertical domains. We also provide a survey of the related research dedicated to automation in the vertical domains. Finally, our vision of 6G wireless systems is discussed briefly.

Data compression has the potential to significantly improve the computation offloading performance in hierarchical fog-cloud systems. However, it remains unknown how to optimally determine the compression ratio jointly with the computation offloading decisions and the resource allocation. This joint optimization problem is studied in the current paper where we aim to minimize the maximum weighted energy and service delay cost (WEDC) of all users. First, we consider a scenario where data compression is performed only at the mobile users. We prove that the optimal offloading decisions have a threshold structure. Moreover, a novel three-step approach employing convexification techniques is developed to optimize the compression ratios and the resource allocation. Then, we address the more general design where data compression is performed at both the mobile users and the fog server. We propose three efficient algorithms to overcome the strong coupling between the offloading decisions and resource allocation. We show that the proposed optimal algorithm for data compression at only the mobile users can reduce the WEDC by a few hundred percent compared to computation offloading strategies that do not leverage data compression or use sub-optimal optimization approaches. Besides, the proposed algorithms for additional data compression at the fog server can further reduce the WEDC.

In this paper, we study the joint computation offloading and resource allocation problem in the two-tier wireless heterogeneous network (HetNet). Our design aims to optimize the computation offloading to the cloud jointly with the subchannel allocation to minimize the maximum (min-max) weighted energy consumption subject to practical constraints on bandwidth, computing resource and allowable latency for the multi-user multitask computation system. To tackle this non-convex mixed integer non-linear problem (MINLP), we employ the bisection search method to solve it where we propose a novel approach to transform and verify the feasibility of the underlying problem in each iteration. In addition, we propose a low-complexity algorithm, which can decrease the number of binary optimization variables and enable more scalable computation offloading optimization in the practical wireless HetNets. Numerical studies confirm that the proposed design achieves the energy saving gains about 55% in comparison with the local computation scheme under the strict required latency of 0.1s., Comment: 6 pages, IEEE Global Communications Conference 2017 (GLOBECOM 2017)