Search

Despite the remarkable success of Large Language Models (LLMs), they still exhibit a limited capability to align their outputs to the user instructions. In this work, we introduce a simple and effective method, which we name GUIDE, that mechanistically increases attention scores in instruction tokens. To support this operation, we present Influence, a novel metric that highlights how the user's instructions propagate through the transformer layers and impact the LLM output. Our results show that GUIDE improves the accuracy of following instructions 29.4 % to 60.4%, outperforming natural prompting alternatives and Supervised Fine-Tuning up to 1M tokens.

The emergence of large language models (LLMs) has significantly impacted various fields, from natural language processing to sectors like medicine and finance. However, despite their rapid proliferation, the applications of LLMs in telecommunications remain limited, often relying on general-purpose models that lack domain-specific specialization. This lack of specialization results in underperformance, particularly when dealing with telecommunications-specific technical terminology and their associated mathematical representations. This paper addresses this gap by first creating and disseminating Tele-Data, a comprehensive dataset of telecommunications material curated from relevant sources, and Tele-Eval, a large-scale question-and-answer dataset tailored to the domain. Through extensive experiments, we explore the most effective training techniques for adapting LLMs to the telecommunications domain, ranging from examining the division of expertise across various telecommunications aspects to employing parameter-efficient techniques. We also investigate how models of different sizes behave during adaptation and analyze the impact of their training data on this behavior. Leveraging these findings, we develop and open-source Tele-LLMs, the first series of language models ranging from 1B to 8B parameters, specifically tailored for telecommunications. Our evaluations demonstrate that these models outperform their general-purpose counterparts on Tele-Eval while retaining their previously acquired capabilities, thus avoiding the catastrophic forgetting phenomenon.

With the advent of large language models (LLMs), managing scientific literature via LLMs has become a promising direction of research. However, existing approaches often overlook the rich structural and semantic relevance among scientific literature, limiting their ability to discern the relationships between pieces of scientific knowledge, and suffer from various types of hallucinations. These methods also focus narrowly on individual downstream tasks, limiting their applicability across use cases. Here we propose LitFM, the first literature foundation model designed for a wide variety of practical downstream tasks on domain-specific literature, with a focus on citation information. At its core, LitFM contains a novel graph retriever to integrate graph structure by navigating citation graphs and extracting relevant literature, thereby enhancing model reliability. LitFM also leverages a knowledge-infused LLM, fine-tuned through a well-developed instruction paradigm. It enables LitFM to extract domain-specific knowledge from literature and reason relationships among them. By integrating citation graphs during both training and inference, LitFM can generalize to unseen papers and accurately assess their relevance within existing literature. Additionally, we introduce new large-scale literature citation benchmark datasets on three academic fields, featuring sentence-level citation information and local context. Extensive experiments validate the superiority of LitFM, achieving 28.1% improvement on retrieval task in precision, and an average improvement of 7.52% over state-of-the-art across six downstream literature-related tasks, Comment: 18 pages, 12 figures

The application of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems in the telecommunication domain presents unique challenges, primarily due to the complex nature of telecom standard documents and the rapid evolution of the field. The paper introduces Telco-RAG, an open-source RAG framework designed to handle the specific needs of telecommunications standards, particularly 3rd Generation Partnership Project (3GPP) documents. Telco-RAG addresses the critical challenges of implementing a RAG pipeline on highly technical content, paving the way for applying LLMs in telecommunications and offering guidelines for RAG implementation in other technical domains., Comment: 6 pages, 5 Figure, 4 Tables, accepted to IEEE Globecom 2024 (see https://github.com/netop-team/telco-rag)

In this paper, we examine a multi-sensor system where each sensor may monitor more than one time-varying information process and send status updates to a remote monitor over a common channel. We consider that each sensor's status update may contain information about more than one information process in the system subject to the system's constraints. To investigate the impact of this correlation on the overall system's performance, we conduct an analysis of both the average Age of Information (AoI) and source state estimation error at the monitor. Building upon this analysis, we subsequently explore the impact of the packet arrivals, correlation probabilities, and rate of processes' state change on the system's performance. Next, we consider the case where sensors have limited sensing abilities and distribute a portion of their sensing abilities across the different processes. We optimize this distribution to minimize the total AoI of the system. Interestingly, we show that monitoring multiple processes from a single source may not always be beneficial. Our results also reveal that the optimal sensing distribution for diverse arrival rates may exhibit a rapid regime switch, rather than smooth transitions, after crossing critical system values. This highlights the importance of identifying these critical thresholds to ensure effective system performance.

Next-generation cellular networks will evolve into more complex and virtualized systems, employing machine learning for enhanced optimization and leveraging higher frequency bands and denser deployments to meet varied service demands. This evolution, while bringing numerous advantages, will also pose challenges, especially in mobility management, as it will increase the overall number of handovers due to smaller coverage areas and the higher signal attenuation. To address these challenges, we propose a deep learning based algorithm for predicting the future serving cell utilizing sequential user equipment measurements to minimize the handover failures and interruption time. Our algorithm enables network operators to dynamically adjust handover triggering events or incorporate UAV base stations for enhanced coverage and capacity, optimizing network objectives like load balancing and energy efficiency through transfer learning techniques. Our framework complies with the O-RAN specifications and can be deployed in a Near-Real-Time RAN Intelligent Controller as an xApp leveraging the E2SM-KPM service model. The evaluation results demonstrate that our algorithm achieves a 92% accuracy in predicting future serving cells with high probability. Finally, by utilizing transfer learning, our algorithm significantly reduces the retraining time by 91% and 77% when new handover trigger decisions or UAV base stations are introduced to the network dynamically.

As deep learning models become increasingly large, they pose significant challenges in heterogeneous devices environments. The size of deep learning models makes it difficult to deploy them on low-power or resource-constrained devices, leading to long inference times and high energy consumption. To address these challenges, we propose FlexTrain, a framework that accommodates the diverse storage and computational resources available on different devices during the training phase. FlexTrain enables efficient deployment of deep learning models, while respecting device constraints, minimizing communication costs, and ensuring seamless integration with diverse devices. We demonstrate the effectiveness of FlexTrain on the CIFAR-100 dataset, where a single global model trained with FlexTrain can be easily deployed on heterogeneous devices, saving training time and energy consumption. We also extend FlexTrain to the federated learning setting, showing that our approach outperforms standard federated learning benchmarks on both CIFAR-10 and CIFAR-100 datasets., Comment: Workshop on Advancing Neural Network Training (WANT) at NeurIPS 2023

We introduce TeleQnA, the first benchmark dataset designed to evaluate the knowledge of Large Language Models (LLMs) in telecommunications. Comprising 10,000 questions and answers, this dataset draws from diverse sources, including standards and research articles. This paper outlines the automated question generation framework responsible for creating this dataset, along with how human input was integrated at various stages to ensure the quality of the questions. Afterwards, using the provided dataset, an evaluation is conducted to assess the capabilities of LLMs, including GPT-3.5 and GPT-4. The results highlight that these models struggle with complex standards related questions but exhibit proficiency in addressing general telecom-related inquiries. Additionally, our results showcase how incorporating telecom knowledge context significantly enhances their performance, thus shedding light on the need for a specialized telecom foundation model. Finally, the dataset is shared with active telecom professionals, whose performance is subsequently benchmarked against that of the LLMs. The findings illustrate that LLMs can rival the performance of active professionals in telecom knowledge, thanks to their capacity to process vast amounts of information, underscoring the potential of LLMs within this domain. The dataset has been made publicly accessible on GitHub.

Large Language Models (LLMs), AI-driven models that can achieve general-purpose language understanding and generation, have emerged as a transformative force, revolutionizing fields well beyond Natural Language Processing (NLP) and garnering unprecedented attention. As LLM technology continues to progress, the telecom industry is facing the prospect of its impact on its landscape. To elucidate these implications, we delve into the inner workings of LLMs, providing insights into their current capabilities and limitations. We also examine the use cases that can be readily implemented in the telecom industry, streamlining tasks, such as anomalies resolutions and technical specifications comprehension, which currently hinder operational efficiency and demand significant manpower and expertise. Furthermore, we uncover essential research directions that deal with the distinctive challenges of utilizing the LLMs within the telecom domain. Addressing them represents a significant stride towards fully harnessing the potential of LLMs and unlocking their capabilities to the fullest extent within the telecom domain.

In this paper, we introduce a new metric, named Penalty upon Decision (PuD), for measuring the impact of communication delays and state changes at the source on a remote decision maker. Specifically, the metric quantifies the performance degradation at the decision maker's side due to delayed, erroneous, and (possibly) missed decisions. We clarify the rationale for the metric and derive closed-form expressions for its average in M/GI/1 and M/GI/1/1 with blocking settings. Numerical results are then presented to support our expressions and to compare the infinite and zero buffer regimes. Interestingly, comparing these two settings sheds light on a buffer length design challenge that is essential to minimize the average PuD.

This paper considers an anomaly detection problem in which a detection algorithm assigns anomaly scores to multi-dimensional data points, such as cellular networks' Key Performance Indicators (KPIs). We propose an optimization framework to refine these anomaly scores by leveraging side information in the form of a causality graph between the various features of the data points. The refinement block builds on causality theory and a proposed notion of confidence scores. After motivating our framework, smoothness properties are proved for the ensuing mathematical expressions. Next, equipped with these results, a gradient descent algorithm is proposed, and a proof of its convergence to a stationary point is provided. Our results hold (i) for any causal anomaly detection algorithm and (ii) for any side information in the form of a directed acyclic graph. Numerical results are provided to illustrate the advantage of our proposed framework in dealing with False Positives (FPs) and False Negatives (FNs). Additionally, the effect of the graph's structure on the expected performance advantage and the various trade-offs that take place are analyzed.

In this paper, we study network reliability in relation to a periodic time-dependent utility function that reflects the system's functional performance. When an anomaly occurs, the system incurs a loss of utility that depends on the anomaly's timing and duration. We analyze the long-term average utility loss by considering exponential anomalies' inter-arrival times and general distributions of maintenance duration. We show that the expected utility loss converges in probability to a simple form. We then extend our convergence results to more general distributions of anomalies' inter-arrival times and to particular families of non-periodic utility functions. To validate our results, we use data gathered from a cellular network consisting of 660 base stations and serving over 20k users. We demonstrate the quasi-periodic nature of users' traffic and the exponential distribution of the anomalies' inter-arrival times, allowing us to apply our results and provide reliability scores for the network. We also discuss the convergence speed of the long-term average utility loss, the interplay between the different network's parameters, and the impact of non-stationarity on our convergence results.

In this paper, we analyze status update systems modeled through the Stochastic Hybrid Systems (SHSs) tool. Contrary to previous works, we allow the system's transition dynamics to be polynomial functions of the Age of Information (AoI). This dependence allows us to encapsulate many applications and opens the door for more sophisticated systems to be studied. However, this same dependence on the AoI engenders technical and analytical difficulties that we address in this paper. Specifically, we first showcase several characteristics of the age processes modeled through the SHSs tool. Then, we provide a framework to establish the Lagrange stability and positive recurrence of these processes. Building on this, we provide an approach to compute the m-th moment of the age processes. Interestingly, this technique allows us to approximate the average age by solving a simple set of linear equations. Equipped with this approach, we also provide a sequential convex approximation method to optimize the average age by calibrating the parameters of the system. Finally, we consider an age-dependent CSMA environment where the backoff duration depends on the instantaneous age. By leveraging our analysis, we contrast its performance to the age-blind CSMA and showcase the age performance gain provided by the former.

This paper examines the average age minimization problem where only a fraction of the network users can transmit simultaneously over unreliable channels. Finding the optimal scheduling scheme, in this case, is known to be challenging. Accordingly, the Whittle's index policy was proposed in the literature as a low-complexity heuristic to the problem. Although simple to implement, characterizing this policy's performance is recognized to be a notoriously tricky task. In the sequel, we provide a new mathematical approach to establish its optimality in the many-users regime for specific network settings. Our novel approach is based on intricate techniques, and unlike previous works in the literature, it is free of any mathematical assumptions. These findings showcase that the Whittle's index policy has analytically provable asymptotic optimality for the AoI minimization problem. Finally, we lay out numerical results that corroborate our theoretical findings and demonstrate the policy's notable performance in the many-users regime.

In this paper, we introduce the Age of Incorrect Information (AoII) as an enabler for semantics-empowered communication, a newly advocated communication paradigm centered around data's role and its usefulness to the communication's goal. First, we shed light on how the traditional communication paradigm, with its role-blind approach to data, is vulnerable to performance bottlenecks. Next, we highlight the shortcomings of several proposed performance measures destined to deal with the traditional communication paradigm's limitations, namely the Age of Information (AoI) and the error-based metrics. We also show how the AoII addresses these shortcomings and captures more meaningfully the purpose of data. Afterward, we consider the problem of minimizing the average AoII in a transmitter-receiver pair scenario. We prove that the optimal transmission strategy is a randomized threshold policy, and we propose an algorithm that finds the optimal parameters. Furthermore, we provide a theoretical comparison between the AoII framework and the standard error-based metrics counterpart. Interestingly, we show that the AoII-optimal policy is also error-optimal for the adopted information source model. Concurrently, the converse is not necessarily true. Finally, we implement our policy in various applications, and we showcase its performance advantages compared to both the error-optimal and the AoI-optimal policies.

In this paper, we consider a transmission scheduling problem, in which several streams of status update packets with diverse priority levels are sent through a shared channel to their destinations. We introduce a notion of Lexicographic age optimality, or simply lex-age-optimality, to evaluate the performance of multi-class status update policies. In particular, a lex-age-optimal scheduling policy first minimizes the Age of Information (AoI) metrics for high-priority streams, and then, within the set of optimal policies for high-priority streams, achieves the minimum AoI metrics for low-priority streams. We propose a new scheduling policy named Preemptive Priority, Maximum Age First, Last-Generated, First-Served (PP-MAF-LGFS), and prove that the PP-MAF-LGFS scheduling policy is lex-age-optimal. This result holds (i) for minimizing any time-dependent, symmetric, and non-decreasing age penalty function; (ii) for minimizing any non-decreasing functional of the stochastic process formed by the age penalty function; and (iii) for the cases where different priority classes have distinct arrival traffic patterns, age penalty functions, and age penalty functionals. For example, the PP-MAF-LGFS scheduling policy is lex-age-optimal for minimizing the mean peak age of a high-priority stream and the time-average age of a low-priority stream. Numerical results are provided to illustrate our theoretical findings.

In this paper, we consider the average age minimization problem where a central entity schedules M users among the N available users for transmission over unreliable channels. It is well-known that obtaining the optimal policy, in this case, is out of reach. Accordingly, the Whittle's index policy has been suggested in earlier works as a heuristic for this problem. However, the analysis of its performance remained elusive. In the sequel, we overcome these difficulties and provide rigorous results on its asymptotic optimality in the many-users regime. Specifically, we first establish its optimality in the neighborhood of a specific system's state. Next, we extend our proof to the global case under a recurrence assumption, which we verify numerically. These findings showcase that the Whittle's index policy has analytically provable optimality in the many-users regime for the AoI minimization problem. Finally, numerical results that showcase its performance and corroborate our theoretical findings are presented.

In this paper, we introduce a new performance metric in the framework of status updates that we will refer to as the Age of Incorrect Information (AoII). This new metric deals with the shortcomings of both the Age of Information (AoI) and the conventional error penalty functions as it neatly extends the notion of fresh updates to that of fresh "informative" updates. The word informative in this context refers to updates that bring new and correct information to the monitor side. After properly motivating the new metric, and with the aim of minimizing its average, we formulate a Markov Decision Process (MDP) in a transmitter-receiver pair scenario where packets are sent over an unreliable channel. We show that a simple "always update" policy minimizes the aforementioned average penalty along with the average age and prediction error. We then tackle the general, and more realistic case, where the transmitter cannot surpass a specific power budget. The problem is formulated as a Constrained Markov Decision Process (CMDP) for which we provide a Lagrangian approach to solve. After characterizing the optimal transmission policy of the Lagrangian problem, we provide a rigorous mathematical proof to showcase that a mixture of two Lagrange policies is optimal for the CMDP in question. Equipped with this, we provide a low complexity algorithm that finds the AoII-optimal operating point of the system in the constrained scenario. Lastly, simulation results are laid out to showcase the performance of the proposed policy and highlight the differences with the AoI framework.

In this paper, we consider N information streams sharing a common service facility. The streams are supposed to have different priorities based on their sensitivity. A higher priority stream will always preempt the service of a lower priority packet. By leveraging the notion of Stochastic Hybrid Systems (SHS), we investigate the Age of Information (AoI) in the case where each stream has its own waiting room; when preempted by a higher priority stream, the packet is stored in the waiting room for future resume. Interestingly, it will be shown that a "no waiting room" scenario, and consequently discarding preempted packets, is better in terms of average AoI in some cases. The exact cases where this happen are discussed and numerical results that corroborate the theoretical findings and highlight this trade-off are provided., Comment: arXiv admin note: text overlap with arXiv:1901.00481

In this paper, we examine the potentials of Non- Orthogonal Multiple Access (NOMA), currently rivaling Orthogonal Multiple Access (OMA) in 3rd Generation Partnership Project (3GPP) standardization for future 5G networks Machine Type Communications (MTC), in the framework of minimizing the average Age of Information (AoI). By leveraging the notion of Stochastic Hybrid Systems (SHS), we find the total average AoI of the network in simple NOMA and conventional OMA environments. Armed with this, we provide a comparison between the two schemes in terms of average AoI. Interestingly, it will be shown that even when NOMA achieves better spectral efficiency in comparison to OMA, this does not necessarily translates into a lower average AoI in the network.

In this paper, we investigate a network of N interfering links contending for the channel to send their data by employing the well-known Carrier Sense Multiple Access (CSMA) scheme. By leveraging the notion of stochastic hybrid systems, we find a closed form of the total average age of the network in this setting. Armed with this expression, we formulate the optimization problem of minimizing the total average age of the network by calibrating the back-off time of each link. By analyzing its structure, the optimization problem is then converted to an equivalent convex problem that can be solved efficiently to find the optimal back-off time of each link. Insights on the interaction between the links is provided and numerical implementations of our optimized CSMA scheme in an IEEE 802.11 environment is presented to highlight its performance. We also show that, although optimized, the standard CSMA scheme still lacks behind other distributed schemes in terms of average age in some special cases. These results suggest the necessity to find new distributed schemes to further minimize the average age of any general network.

In this paper, we examine a system where status updates are generated by a source and are forwarded in a First-Come-First-Served (FCFS) manner to the monitor. We consider the case where the server has other tasks to fulfill, a simple example being relaying the packets of another stream. Due to the server's necessity to go on vacations, the age process of the stream of interest becomes complicated to evaluate. By leveraging specific queuing theory tools, we provide a closed form of the average age for both streams which enables us to optimize the generation rate of packets belonging to each stream to achieve the minimum possible average age. The tools used can be further adopted to provide insights on more general multi-hop scenarios. Numerical results are provided to corroborate the theoretical findings and highlight the interaction between the two streams., Comment: 7 figures

Massive MIMO is widely considered as a key enabler of the next generation 5G networks. With a large number of antennas at the Base Station, both spectral and energy efficiencies can be enhanced. Unfortunately, the downlink channel estimation overhead scales linearly with the number of antennas. This burden is easily mitigated in TDD systems by the use of the channel reciprocity property. However, this is unfeasible for FDD systems and the method of two-stage beamforming was therefore developed to reduce the amount of channel state information feedback. The performance of this scheme being highly dependent on the users grouping and scheduling mechanims, we introduce in this paper a new similarity measure coupled with a novel clustering procedure to achieve the appropriate users grouping. We also proceed to formulate the optimal users scheduling policy in JSDM and prove that it is NP-hard. This result is of paramount importance since it suggests that, unless P=NP, there are no polynomial time algorithms that solve the general scheduling problem to global optimality and the use of sub-optimal scheduling strategies is more realistic in practice. We therefore use graph theory to develop a sub-optimal users scheduling scheme that runs in polynomial time and outperforms the scheduling schemes previously introduced in the literature for JSDM in both sum-rate and throughput fairness., Comment: arXiv admin note: text overlap with arXiv:1712.03022

Massive MIMO is considered as one of the key enablers of the next generation 5G networks.With a high number of antennas at the BS, both spectral and energy efficiencies can be improved. Unfortunately, the downlink channel estimation overhead scales linearly with the number of antenna. This does not create complications in Time Division Duplex (TDD) systems since the channel estimate of the uplink direction can be directly utilized for link adaptation in the downlink direction. However, this channel reciprocity is unfeasible for the Frequency Division Duplex (FDD) systems where different physical transmission channels are existent for the uplink and downlink. In the aim of reducing the amount of Channel State Information (CSI) feedback for FDD systems, the promising method of two stage beamforming transmission was introduced. The performance of this transmission scheme is however highly influenced by the users grouping and selection mechanisms. In this paper, we first introduce a new similarity measure coupled with a novel clustering technique to achieve the appropriate users partitioning. We also use graph theory to develop a low complexity groups scheduling scheme that outperforms currently existing methods in both sum-rate and throughput fairness. This performance gain is demonstrated through computer simulations., Comment: 7 pages, 6 figures

Carrier Sense Multiple Access (CSMA) is widely used as a Medium Access Control (MAC) in wireless networks due to its simplicity and distributed nature. This motivated researchers to find CSMA schemes that achieve throughput optimality. In 2008, it has been shown that a simple CSMA-type algorithm is able to achieve optimality in terms of throughput and has been given the name "adaptive" CSMA. Lately, new technologies emerged where prolonged battery life is crucial such as environment and industrial monitoring. This inspired the foundation of new CSMA based MAC schemes where links are allowed to transition into sleep mode to reduce the power consumption. However, throughput optimality of these schemes was not established. This paper therefore aims to find a new CSMA scheme that combines both throughput optimality and energy efficiency by adapting to the throughput and power consumption needs of each link. This is done by controlling operational parameters such as back-off and sleeping timers with the aim of optimizing a certain objective function. The resulting CSMA scheme is characterized by being asynchronous, completely distributed and being able to adapt to different power consumption profiles required by each link while still ensuring throughput optimality. The performance gain in terms of energy efficiency compared to the conventional adaptive CSMA scheme is demonstrated through computer simulations.

Non-Orthogonal Multiple Access (NOMA) is recognized today as a most promising technology for future 5G cellular networks and a large number of papers have been published on the subject over the past few years. Interestingly, none of these authors seems to be aware that the foundation of NOMA actually dates back to the year 2000, when a series of papers introduced and investigated multiple access schemes using two sets of orthogonal signal waveforms and iterative interference cancellation at the receiver. The purpose of this paper is to shed light on that early literature and to describe a practical scheme based on that concept, which is particularly attractive for Machine-Type Communications (MTC) in future 5G cellular networks. Using this approach, NOMA appears as a convenient extension of orthogonal multiple access rather than a strictly competing technology, and most important of all, the power imbalance between the transmitted user signals that is required to make the receiver work in other NOMA schemes is not required here., Comment: 6 pages, 8 figures, submitted to IEEE WCNC 2018