Search

In this paper, we focus on a wireless-powered sensor network coordinated by a multi-antenna access point (AP). Each node can generate sensing information and report the latest information to the AP using the energy harvested from the AP's signal beamforming. We aim to minimize the average age-of-information (AoI) by adapting the nodes' transmission scheduling and the transmission control strategies jointly. To reduce the transmission delay, an intelligent reflecting surface (IRS) is used to enhance the channel conditions by controlling the AP's beamforming vector and the IRS's phase shifting matrix. Considering dynamic data arrivals at different sensing nodes, we propose a hierarchical deep reinforcement learning (DRL) framework to for AoI minimization in two steps. The users' transmission scheduling is firstly determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and then the inner-loop optimization is used to adapt either the uplink information transmission or downlink energy transfer to all nodes. A simple and efficient approximation is also proposed to reduce the inner-loop rum time overhead. Numerical results verify that the hierarchical learning framework outperforms typical baselines in terms of the average AoI and proportional fairness among different nodes., Comment: 31 pages, 6 figures, 2 tables, 3 algorithms

In this paper, we employ multiple UAVs coordinated by a base station (BS) to help the ground users (GUs) to offload their sensing data. Different UAVs can adapt their trajectories and network formation to expedite data transmissions via multi-hop relaying. The trajectory planning aims to collect all GUs' data, while the UAVs' network formation optimizes the multi-hop UAV network topology to minimize the energy consumption and transmission delay. The joint network formation and trajectory optimization is solved by a two-step iterative approach. Firstly, we devise the adaptive network formation scheme by using a heuristic algorithm to balance the UAVs' energy consumption and data queue size. Then, with the fixed network formation, the UAVs' trajectories are further optimized by using multi-agent deep reinforcement learning without knowing the GUs' traffic demands and spatial distribution. To improve the learning efficiency, we further employ Bayesian optimization to estimate the UAVs' flying decisions based on historical trajectory points. This helps avoid inefficient action explorations and improves the convergence rate in the model training. The simulation results reveal close spatial-temporal couplings between the UAVs' trajectory planning and network formation. Compared with several baselines, our solution can better exploit the UAVs' cooperation in data offloading, thus improving energy efficiency and delay performance., Comment: 15 pages, 10 figures, 2 algorithms

A smart grid is an important application in Industry 4.0 with a lot of new technologies and equipment working together. Hence, sensitive data stored in the smart grid is vulnerable to malicious modification and theft. This paper proposes a framework to build a smart grid based on a highly effective private Ethereum network. Our framework provides a real smart grid that includes modern hardware and a smart contract to secure data in the blockchain network. To obtain high throughput but a low uncle rate, the difficulty calculation method used in the mining process of the Ethereum consensus mechanism is modified to adapt to the practical smart grid setup. The performance in terms of throughput and latency are evaluated by simulation and verified by the real smart grid setup. The enhanced private Ethereum-based smart grid has significantly better performance than the public one. Moreover, this framework can be applied to any system used to store data in the Ethereum network., Comment: 6 pages, conference

Autonomous Vehicles (AVs) are required to operate safely and efficiently in dynamic environments. For this, the AVs equipped with Joint Radar-Communications (JRC) functions can enhance the driving safety by utilizing both radar detection and data communication functions. However, optimizing the performance of the AV system with two different functions under uncertainty and dynamic of surrounding environments is very challenging. In this work, we first propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions in selecting JRC operation functions under the dynamic and uncertainty of the surrounding environment. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV without requiring any prior information about surrounding environment. Furthermore, to make our proposed framework more scalable, we develop a Transfer Learning (TL) mechanism that enables the AV to leverage valuable experiences for accelerating the training process when it moves to a new environment. Extensive simulations show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches. With the deep reinforcement learning and transfer learning approaches, our proposed solution can find its applications in a wide range of autonomous driving scenarios from driver assistance to full automation transportation. Agency for Science, Technology and Research (A*STAR) Ministry of Education (MOE) National Research Foundation (NRF) This research was supported in part by the Australian Research Council under the DECRA project DE210100651. This research is supported in part by the programme DesCartes - the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme and under its Emerging Areas Research Projects (EARP) Funding Initiative, Alibaba Group through Alibaba Innovative Research (AIR) Program and Alibaba-NTU Singapore Joint Research Institute (JRI), the National Research Foundation, Singapore under the AI Singapore Programme (AISG) (AISG2- RP-2020-019), and Singapore Ministry of Education (MOE) Tier 1 (RG16/20). This research was supported in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant 2021R1A2C2007638 and the MSIT under the ICT Creative Consilience program (IITP-2020-0-01821) supervised by the IITP. This research is supported by A*STAR under its RIE2020 Advanced Manufacturing and Engineering (AME) Industry Alignment Fund – Pre Positioning (IAF-PP) (Grant No. A19D6a0053).

This article proposes a cooperative friendly jamming framework for swarm unmanned aerial vehicle (UAV)-assisted amplify-and-forward (AF) relaying networks with wireless energy harvesting. Due to the limited energy of the UAVs, we develop a collaborative time-switching relaying protocol which allows the UAVs to collaborate to harvest wireless energy, relay information, and jam the eavesdropper. To evaluate the secrecy rate, we derive the secrecy outage probability (SOP) for two popular detection techniques at the eavesdropper, i.e., selection combining and maximum-ratio combining. Monte Carlo simulations are then used to validate the theoretical SOP derivation and to show the effectiveness of the proposed framework in terms of SOP as compared with the conventional amplify-and-forward relaying system. Using the derived SOP, one can obtain engineering insights to optimize the energy harvesting time and the number of UAVs in the swarm to achieve a given secrecy protection level. The analytical SOP derived in this work can also be helpful in future UAV secure-communications optimizations (e.g., trajectory, locations of UAVs). As an example, we present a case study to find the optimal corridor to locate the swarm so as to minimize the system SOP., Comment: 30 pages, 7 figures, journal

Joint radar and communication (JRC) has recently attracted substantial attention. The first reason is that JRC allows individual radar and communication systems to share spectrum bands and thus improves the spectrum utilization. The second reason is that JRC enables a single hardware platform, e.g., an autonomous vehicle or a UAV, to simultaneously perform the communication function and the radar function. As a result, JRC is able to improve the efficiency of resources, i.e., spectrum and energy, reduce the system size, and minimize the system cost. However, there are several challenges to be solved for the JRC design. In particular, sharing the spectrum imposes the interference caused by the systems, and sharing the hardware platform and energy resource complicates the design of the JRC transmitter and compromises the performance of each function. To address the challenges, several resource management approaches have been recently proposed, and this paper presents a comprehensive literature review on resource management for JRC. First, we give fundamental concepts of JRC, important performance metrics used in JRC systems, and applications of the JRC systems. Then, we review and analyze resource management approaches, i.e., spectrum sharing, power allocation, and interference management, for JRC. In addition, we present security issues to JRC and provide a discussion of countermeasures to the security issues. Finally, we highlight important challenges in the JRC design and discuss future research directions related to JRC.

Due to the proliferation of smart devices and emerging applications, many next-generation technologies have been paid for the development of wireless networks. Even though commercial 5G has just been widely deployed in some countries, there have been initial efforts from academia and industrial communities for 6G systems. In such a network, a very large number of devices and applications are emerged, along with heterogeneity of technologies, architectures, mobile data, etc., and optimizing such a network is of utmost importance. Besides convex optimization and game theory, swarm intelligence (SI) has recently appeared as a promising optimization tool for wireless networks. As a new subdivision of artificial intelligence, SI is inspired by the collective behaviors of societies of biological species. In SI, simple agents with limited capabilities would achieve intelligent strategies for high-dimensional and challenging problems, so it has recently found many applications in next-generation wireless networks (NGN). However, researchers may not be completely aware of the full potential of SI techniques. In this work, our primary focus will be the integration of these two domains: NGN and SI. Firstly, we provide an overview of SI techniques from fundamental concepts to well-known optimizers. Secondly, we review the applications of SI to settle emerging issues in NGN, including spectrum management and resource allocation, wireless caching and edge computing, network security, and several other miscellaneous issues. Finally, we highlight open challenges and issues in the literature, and introduce some interesting directions for future research., Comment: Submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Internet of things (IoT) produces massive data from devices embedded with sensors. The IoT data allows creating profitable services using machine learning. However, previous research does not address the problem of optimal pricing and bundling of machine learning-based IoT services. In this paper, we define the data value and service quality from a machine learning perspective. We present an IoT market model which consists of data vendors selling data to service providers, and service providers offering IoT services to customers. Then, we introduce optimal pricing schemes for the standalone and bundled selling of IoT services. In standalone service sales, the service provider optimizes the size of bought data and service subscription fee to maximize its profit. For service bundles, the subscription fee and data sizes of the grouped IoT services are optimized to maximize the total profit of cooperative service providers. We show that bundling IoT services maximizes the profit of service providers compared to the standalone selling. For profit sharing of bundled services, we apply the concepts of core and Shapley solutions from cooperative game theory as efficient and fair allocations of payoffs among the cooperative service providers in the bundling coalition., Comment: 17 pages

This paper proposes to use Intelligent Reflecting Surface (IRS) as a green jammer to attack a legitimate communication without using any internal energy to generate jamming signals. In particular, the IRS is used to intelligently reflect the signals from the legitimate transmitter to the legitimate receiver (LR) to guarantee that the received signals from direct and reflecting links can be added destructively, which thus diminishes the Signal-to-Interference-plus-Noise Ratio (SINR) at the LR. To minimize the received signal power at the LR, we consider the joint optimization of magnitudes of reflection coefficients and discrete phase shifts at the IRS. Based on the block coordinate descent, semidefinite relaxation, and Gaussian randomization techniques, the solution can be obtained efficiently. Through simulation results, we show that by using the IRS-based jammer, we can reduce the signal power received at the LR by up to 99\%. Interestingly, the performance of the proposed IRS-based jammer is even better than that of the conventional active jamming attacks in some scenarios., Comment: 12 pages, 5 figures

Future IoT networks consist of heterogeneous types of IoT devices (with various communication types and energy constraints) which are assumed to belong to an IoT service provider (ISP). To power backscattering-based and wireless-powered devices, the ISP has to contract with an energy service provider (ESP). This article studies the strategic interactions between the ISP and its ESP and their implications on the joint optimal time scheduling and energy trading for heterogeneous devices. To that end, we propose an economic framework using the Stackelberg game to maximize the network throughput and energy efficiency of both the ISP and ESP. Specifically, the ISP leads the game by sending its optimal service time and energy price request (that maximizes its profit) to the ESP. The ESP then optimizes and supplies the transmission power which satisfies the ISP's request (while maximizing ESP's utility). To obtain the Stackelberg equilibrium (SE), we apply a backward induction technique which first derives a closed-form solution for the ESP. Then, to tackle the non-convex optimization problem for the ISP, we leverage the block coordinate descent and convex-concave procedure techniques to design two partitioning schemes (i.e., partial adjustment (PA) and joint adjustment (JA)) to find the optimal energy price and service time that constitute local SEs. Numerical results reveal that by jointly optimizing the energy trading and the time allocation for heterogeneous IoT devices, one can achieve significant improvements in terms of the ISP's profit compared with those of conventional transmission methods. Different tradeoffs between the ESP's and ISP's profits and complexities of the PA/JA schemes can also be numerically tuned. Simulations also show that the obtained local SEs approach the socially optimal welfare when the ISP's benefit per transmitted bit is higher than a given threshold., Comment: Submitted to TWC, 16 pages, 10 figures

In this article, we introduce a novel deception strategy which is inspired by the "Borrowing Arrows with Thatched Boats", one of the most famous military tactics in the history, in order to defeat reactive jamming attacks for low-power IoT networks. Our proposed strategy allows resource-constrained IoT devices to be able to defeat powerful reactive jammers by leveraging their own jamming signals. More specifically, by stimulating the jammer to attack the channel through transmitting fake transmissions, the IoT system can not only undermine the jammer's power, but also harvest energy or utilize jamming signals as a communication means to transmit data through using RF energy harvesting and ambient backscatter techniques, respectively. Furthermore, we develop a low-cost deep reinforcement learning framework that enables the hardware-constrained IoT device to quickly obtain an optimal defense policy without requiring any information about the jammer in advance. Simulation results reveal that our proposed framework can not only be very effective in defeating reactive jamming attacks, but also leverage jammer's power to enhance system performance for the IoT network., Comment: 16 pages, 5 figures, magazine

© 2002-2012 IEEE. In this paper, we study the transmission strategy adaptation problem in an RF-powered cognitive radio network, in which hybrid secondary users are able to switch between the harvest-then-transmit mode and the ambient backscatter mode for their communication with the secondary gateway. In the network, a monetary incentive is introduced for managing the interference caused by the secondary transmission with imperfect channel sensing. The sensing-pricing-transmitting process of the secondary gateway and the transmitters is modeled as a single-leader-multi-follower Stackelberg game. Furthermore, the follower sub-game among the secondary transmitters is modeled as a generalized Nash equilibrium problem with shared constraints. Based on our theoretical discoveries regarding the properties of equilibria in the follower sub-game and the Stackelberg game, we propose a distributed, iterative strategy searching scheme that guarantees the convergence to the Stackelberg equilibrium. The numerical simulations show that the proposed hybrid transmission scheme always outperforms the schemes with fixed transmission modes. Furthermore, the simulations reveal that the adopted hybrid scheme is able to achieve a higher throughput than the sum of the throughput obtained from the schemes with fixed transmission modes.

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are used to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs.