Search

This paper presents a computation-efficient stochastic dynamic programming algorithm for solving energy storage price arbitrage considering variable charge and discharge efficiencies. We formulate the price arbitrage problem using stochastic dynamic programming and model real-time prices as a Markov process. Then we propose an analytical solution algorithm using a piecewise linear approximation of the value-to-go function. Our solution algorithm achieves extreme computation performance and solves the proposed arbitrage problem for one operating day in less than one second on a personal computer. We demonstrate our approach using historical price data from four price zones in New York Independent System Operator, with case studies comparing the performance of different stochastic models and storage settings. Our results show that the proposed method captures 50% to 90% of arbitrage profit compared to perfect price forecasts. In particular, our method captures more than 80% of arbitrage profit in three out of the four price zones when considering batteries with more than two-hour duration and realistic degradation cost. [ABSTRACT FROM AUTHOR]

Renewable energy is an advisable choice to reduce fuel consumption and $\rm CO_{2}$ emission. Therein, wind energy and solar energy are the most promising contributors to reach this goal. Although the hybrid wind/solar system has been widely studied, the real-time current sharing based on their maximum capacities is rarely achieved in terms of seconds. Based on this, this paper proposes an accurate current sharing and voltage regulation approach in hybrid wind/solar systems, which is based on distributed adaptive dynamic programming approach. Firstly, the equivalent wind/solar model is built, which is an indispensable preprocessing to achieve the complementary between wind energy and solar energy. Therein, the wind energy and solar energy can output relative current according to their respective capacity ratio, which ensure the maximum utilization ratio of renewable energy source. Furthermore, current sharing and voltage regulation problem is switched into optimal control problem. Under this effect, each source agent aims to obtain the optimal control variable and achieve accurate current sharing/voltage regulation. Moreover, an adaptive dynamic programming approach based on Bellman principle is proposed. It can achieve accurate current sharing and voltage regulation. Finally, the simulation results are provided to illustrate the performance of the proposed adaptive dynamic programming approach. [ABSTRACT FROM AUTHOR]

As for fuel cell hybrid electric vehicle equipped with battery (BAT) and ultracapacitor (UC), its dynamic topology structure is complex and different characteristics of three power sources induce challenges in energy management for fuel economy, power sources lifespan, and dynamic performance of the vehicle. In this paper, an energy management strategy (EMS) based on a hierarchical power splitting structure and deep reinforcement learning (DRL) is proposed. In the higher layer strategy of the proposed EMS, the UC is employed to supply peak power and recover braking energy through the adaptive filter based on fuzzy control. Then, the integrated DRL and equivalent consumption minimization strategy framework is proposed to optimize the power allocation of fuel cell (FC) and BAT in the lower layer, to ensure the highly efficient operation of FC and reduce hydrogen consumption. And the action trimming based on heuristic technique is proposed to further restrain the adverse effect of sudden peak power on FC lifespan. The simulation results show the proposed EMS can make the output of FC smoother, improve its working efficiency to alleviate the stress of BAT, and increase by 14.8% compared with the Q-learning strategy in fuel economy under WLTP driving cycle. Meanwhile, the obtained results under UDDSHDV show fuel economy of the proposed EMS can reach dynamic programming (DP) benchmark level of 89.7 $\%$. [ABSTRACT FROM AUTHOR]

This paper proposes a dynamic valuation framework to determine the opportunity value of battery capacity degradation in grid applications based on the internal degradation mechanism and utilization scenarios. The proposed framework follows a dynamic programming approach and includes a piecewise linear value function approximation solution that solves the optimization problem over a long planning horizon. The paper provides two case studies on price arbitrage and frequency regulation using real market and system data to demonstrate the broad applicability of the proposed framework. Results show that the battery lifetime value is critically dependent on both the external market environment and its internal state of health. On the grid service side, results show that second-life batteries can provide more than 50% of the value compared to new batteries, and frequency regulation provides two times more revenue than price arbitrage throughout the battery lifetime. [ABSTRACT FROM AUTHOR]

This paper aims to explore the performance potential of a novel dual-mode power-split hybrid powertrain. First, the steady-state power split characteristics for the proposed hybrid powertrain in different modes are analyzed. The mode switching strategy is developed to maximize powertrain efficiency, which is verified by using dynamic programming based on direct transmit points (DTPs). Moreover, a novel multi-objective evolutionary algorithm based on the decomposition (MOEA/D) method used in a nested way with dynamic programming is proposed to solve the multi-objective optimization of the PS-HEVs for the first time, and the computational efficiency and superiority of the proposed algorithm is compared with the commonly used NSGA-II algorithm. The results show that the MOEA/D based multi-objective optimization framework has similar performance in the search for the Pareto frontier but significantly higher computational efficiency than that of the NSGA-II algorithm. The obtained Pareto frontier provides optimal design candidates for hybrid powertrain systems. [ABSTRACT FROM AUTHOR]

A multiperiodic structure always underlies the procurement problem in practice. Option contracts are widely used to make a firm capable of responding flexibly to high demand uncertainty. However, so far, the impact of different option contracts on the multiperiod decisions and performance of a firm has been not discussed. Motivated by this problem, in this paper we analyze the multiperiod procurement problem within the framework of wholesale price, call, put, and bidirectional option contracts. Based on the dynamic programming theory, we characterize the structure of the firm's optimal replenishment policy for each period in four cases. We then develop a heuristic to approximate the corresponding policy parameters. Through system comparisons of four cases, we examine the value of option contracts for the firm and discuss the best option contract type for the firm. The results show that the firm will gain higher profits after the option contracts are introduced. Moreover, if the unit purchase and exercise costs of the bidirectional option are equal to those of the unilateral option, then bidirectional option contracts are preferred over unilateral option contracts. If the unit purchase cost of the bidirectional option is higher than that of the unilateral option, unilateral option contracts may be more profitable than bidirectional option contracts. Furthermore, if the unit backlogging cost is high and the unit inventory holding cost is low, call option contracts are better than put option contracts; if the unit backlogging cost is low and the unit inventory holding cost is high, then put option contracts work better than call option contracts. [ABSTRACT FROM AUTHOR]

Remote entanglement distribution plays a crucial role in large-scale quantum networks, and the key enabler for entanglement distribution is quantum routers (or repeaters) that can extend the entanglement transmission distance. However, the performance of quantum routers is far from perfect yet. Amongst the causes, the limited quantum memories in quantum routers largely affect the rate and efficiency of entanglement distribution. To overcome this challenge, this paper presents a new modeling for the maximization of entanglement distribution rate (EDR) on a memory-limited path, which is then transformed into entanglement generation and swapping sub-problems. We propose a greedy algorithm for short-distance entanglement generation so that the quantum memories can be efficiently used. As for the entanglement swapping sub-problem, we model it using an Entanglement Graph (EG), whose solution is yet found to be at least NP-complete. In light of it, we propose a heuristic algorithm by dividing the original EG into several sub-problems, each of which can be solved using dynamic programming (DP) in polynomial time. By conducting simulations, the results show that our proposed scheme can achieve a high EDR, and the developed algorithm has a polynomial-time upper bound and reasonable average runtime complexity. [ABSTRACT FROM AUTHOR]

We observed that remarkable and impressive performance on image-based human pose estimation have been achieved by deep Convolutional Neural Networks (CNN). Nevertheless, directly applying these image-based models on videos is not only computionally intensive, but also may cause jitter and loss. The main reason is that the image-based models purely focus on the local features of individual frames and totally ignore the temporal information among adjacent frames. Some existing methods are proposed to address the temporal coherency issue. However, these methods need to be designed carefully and cannot be combined with existing image-based methods. In this paper, we propose a simple yet effective module to refine the estimated pose by exploiting the temporal coherency among the heatmaps of adjacent frames, which can be easily inserted into image-based networks as a plug-in. We show that the temporal coherency issue among the heatmap frames could be re-formulated as a graph path selection optimization problem. Moreover, to speed up the refinement process, we propose a hierarchical graph optimization to achieve the refinement from coarse to fine. Experimental results on two large-scale video pose estimation benchmarks show that our module can improve the performance with little speed loss when combined with image-based methods as an efficient plug-in. [ABSTRACT FROM AUTHOR]

The edge computing service mode has entered the edge federation service stage, where comprehensive resources are supported to collaborate between vertical cloud-edge and horizontal edge nodes. The massive data processing poses a severe challenge to the traditional storage mechanism. In this paper, we propose an efficient storage resource collaboration model that leverages the selection of storage collaborative edge nodes and the allocation of storage resources among collaborative nodes, which can provide efficient and balanced storage services solutions. Specifically, selecting storage collaborative edge nodes is a multi-objective integer linear programming problem. We transform it into a single objective 0-1 integer linear programming problem and use the dynamic programming method to obtain the optimal solution. In addition, we condense the storage resource allocation problem into a mixed-linear integer programming problem and use the greedy auction algorithm to obtain an approximate optimal solution, which significantly improves efficiency and performance. We evaluate the performance of the proposed two mechanisms by performing extensive experiments. The experimental results show that they are efficient solutions to solve the problem of the storage resource collaboration model among edge nodes in edge federation service. [ABSTRACT FROM AUTHOR]

Dynamic ridesharing refers to services that arrange one-time shared rides on short notice. It underpins various real-world intelligent transportation applications such as car-pooling, food delivery and last-mile logistics. A core operation in dynamic ridesharing is the “insertion operator”. Given a worker and a feasible route which contains a sequence of origin-destination pairs from previous requests, the insertion operator inserts a new origin-destination pair from a newly arrived request into the current route such that certain objective is optimized. Common optimization objectives include minimizing the maximum/sum flow time of all requests and minimizing the total travel time of the worker. Despite its frequent usage, the insertion operator has a time complexity of $O(n^3)$ O (n 3) , where $n$ n is the number of all requests assigned to the worker. The cubic running time of insertion fundamentally limits the efficiency of urban-scale dynamic ridesharing based applications. In this paper, we propose a novel partition framework and a dynamic programming based insertion with a time complexity of $O(n^2)$ O (n 2) . We further improve the time efficiency of the insertion operator to $O(n)$ O (n) harnessing efficient index structures, such as fenwick tree. Evaluations on two real-world large-scale datasets show that our methods can accelerate insertion by 1.5 to 998.1 times. [ABSTRACT FROM AUTHOR]

Millimeter wave (mmW) is being deployed in hotspot scenarios for providing high bandwidths. However, the coverage of mmW cell is limited due to the rapid attenuation during the propagation and sensitivity to blockage by obstacles. In this paper, we propose a novel dynamic programming approach for automated dimensioning and appropriate placement of mmW small cells (SCs) in dense urban under the constraints of coverage and budget. Being cognizant of the blockage by city buildings, a fast search algorithm based on vector is introduced to enable most line-of-sight (LOS) communication links between base stations (BSs) and mobile stations (MSs). The analysis of time complexity is also derived in closed forms. The evaluation of our approach was carried out in both hypothetic and realistic scenarios. Results show that the studied model is effective for a real large-scaled network and 4 or 5 times faster than the conventional ones based on grid-map. [ABSTRACT FROM AUTHOR]

Speed planning is one of the tasks that a self-driving vehicle carries out. A complete planner should consider and balance passengers comfort, trip time and energy consumption. This paper proposes a computationally efficient global speed planner for autonomous vehicles that explicitly includes comfort as one of the main objectives. In particular, our approach considers the trip time as a user-specified constraint and optimizes a cost function that accounts for both energy consumption and comfort. Since passenger comfort plays a critical role for self driving vehicle, we propose a comfort model that captures different aspects: planar and vertical accelerations and the contribution of different frequency components. We test the algorithm on a realistic case study and we quantify the trade-off between energy consumption and comfort. [ABSTRACT FROM AUTHOR]

The use of multiple low-power fuel cells (FCs), instead of a high-power one, in the powertrain of a FC-hybrid electric vehicle (FC-HEV) has recently received considerable attention. This is mainly due to the fact that this configuration can lead to higher efficiency, durability, and reliability. However, the added degrees of freedom require an advanced multi-agent energy management strategy (EMS) for an effective power distribution among power sources. This paper puts forward an EMS based on game theory (GT) for a multi-stack FC-HEV with three FCs and a battery pack. GT is a well-approved method for characterizing the interactions in multi-agent systems. Unlike the other strategies, the proposed EMS is equipped with an online identification system to constantly update the time-varying characteristics of the power sources. The performance of the suggested strategy is investigated through two case studies. Firstly, a comparative study with two other EMSs, dynamic programming (offline), and a competent rule-based strategy (online), is conducted to realize the capability of GT. Secondly, to justify the necessity of online system identification, the degradation effect of each power source on the EMS performance is examined. The carried-out studies show that the total cost (hydrogen consumption and degradation) of the proposed strategy is almost 6% better than the rule-based EMS while keeping a reasonable difference with dynamic programming. Moreover, health unawareness of power sources can increase the hydrogen consumption up to 7% in the studied system. [ABSTRACT FROM AUTHOR]

With theincreased penetration of renewable sources, power gridsare becoming stressed due to fluctuating generation. To alleviate stress from inconsistent sources, utilities employ energy storage systems alongside renewable sources and rely on dispatching synchronous generators. However, optimal dispatch of such devices is limited by traditional AC optimal power flow methods that do not account for time-dependent constraints, such as the state of charge of energy storage systems and generator ramping constraints. Multi-period optimal power flow is proposed as a large non-convex non-linear problem to optimally dispatch and control generators and energy storage elements across multiple time periods. In this paper, we introduce a scalable, robust framework to solve multi-period optimal power flow using a differential dynamic programming scheme that makes it capable of scaling to large systems containing energy storage devices. We demonstrate the efficacy of this solution by optimizing the SyntheticUSA testcase over a set of time periods. A robust homotopy method is applied to achieve fast simulation times that can be parallelized for further improvements. [ABSTRACT FROM AUTHOR]

In this paper, we propose a parallel dual dynamic programming (PDDP)-based decentralized algorithm for the multi-area optimal power flow (MAOPF), which can preserve the information privacy and operational independence of each area. The MAOPF problem is decomposed into a series of subproblems for individual areas by the dual dynamic programming (DDP) algorithm, and the Benders cut-based value functions are used to reflect the impacts of one area’s decisions to the subsequent areas. The optimal solution of MAOPF can be obtained in a decentralized fashion, requiring only a limited amount of information exchange among neighbor areas. Moreover, a parallel processing technique is designed to avoid the waiting process of the basic DDP algorithm, thus accelerating the computing speed of the proposed decentralized algorithm. Compared with the existing decentralized algorithms, the proposed algorithm has better performance in terms of convergence and computational efficiency. In addition, there is no need for parameter tuning. Case studies on several IEEE test systems and a real 2298-bus system demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

To find the optimal powertrain design for hybrid and electric vehicles in terms of energy efficiency and performance, automatised optimisation tools are often applied. These require a mathematical model of the powertrain configuration, which is usually done on the level of components, such as planetary gear sets (PGSs), clutches and brakes. This, however, leads to a huge design space with lots of functional redundancy. In fact, each functional powertrain mode can be realised by various component combinations. In this paper, a novel mode-based modelling technique for arbitrary hybrid and electric powertrain configurations with up to one internal combustion engine (ICE) and two electrical machines (EMs) is proposed. This allows to easily analyse multi-mode hybrid and electric powertrains on a purely functional basis. Additionally, it reduces mode describing parameters to the minimum amount required. First, an existing graphical representation of hybrid powertrains with one ICE and one EM is analysed and formalised. Based on that, virtual powertrain nodes are defined. These are used to compose all hybrid and electric powertrain modes. Five of them are presented in detail and plotted within a three-dimensional ratio plot. Using a first case study, it is shown how the ratio plot can be used to analyse arbitrary multi-mode powertrains. Within a second case study, it is demonstrated how optimal parameter sets for single-mode powertrain configurations can be identified using the derived model. [ABSTRACT FROM AUTHOR]

Distribution systems are operated with an increasing level of uncertainty. Energy storage is playing an important role in shaving the peak load and mitigating uncertainty. This paper proposes a multistage robust optimization model for distribution system operation with energy storage under uncertainty. Unlike the conventional robust optimization paradigm which minimizes the worst-case cost, the proposed formulation optimizes the cost in the nominal scenario. In analogy to dynamic programming, we define dynamic robust feasible regions in a recursive manner. In each period, the dynamic robust feasible region is shown to be polyhedral, and a linear programming based projection algorithm is developed to compute such regions offline. In the online stage, the method is executed following a rolling horizon manner: renewable output is observed at the beginning of each period, and the cost of remaining periods in the forecast scenario is to be minimized subject to operation constraints and dynamic robust feasible regions, giving rise to a linear program. In this way, the dispatch strategy ensures multistage operation security regardless of future realizations of renewable power. In numeric tests on a modified IEEE 33-bus distribution system, the dynamic robust feasible regions are visualized and analyzed, and the proposed method is compared with two prevailing robust optimization methods, verifying its advantages in terms of optimality and robustness. [ABSTRACT FROM AUTHOR]

Autonomous driving of unmanned ground vehicle (UGV) at road intersection is a challenging task due to the complicated traffic conditions. In this paper, an event-triggered model predictive adaptive dynamic programming (MPADP) algorithm is proposed for path planning of UGV at road intersection. Following the critic-actor scheme of adaptive dynamic programming (ADP), cost function approximation and control policy generation are combined to formulate MPADP. The infinite horizon cost function of ADP is stacked over predictive horizon of model predictive control (MPC), and then the infinite horizon cost function is converted to the finite horizon-stacked cost function in MPADP. By minimizing the approximation error within predictive horizon, the approximation accuracy is enhanced. Considering the limitation of energy consumption, the event-triggered mechanism is designed based on the mismatch of cost function approximation. Three triggering conditions are designed, and the corresponding boundedness of approximation error is proved. Simulation results illustrate the effectiveness, efficiency and feasibility in application of the event-triggered MPADP method for path planning at road intersection. [ABSTRACT FROM AUTHOR]

Recent improvements in vehicle-to-everything (V2X) communication and onboard computing power have enabled the development of control algorithms that jointly optimize the vehicle velocity and powertrain control in Connected and Automated Vehicles (CAVs), commonly referred to as the Eco-Driving problem. This paper presents a novel and computationally efficient algorithm to optimize the velocity planning and energy management in a CAV with a hybrid electric powertrain. The Eco-Driving problem is formulated as a dynamic, constrained optimization problem in the spatial domain, where information about the upcoming speed limits and road topography is assumed known. This problem is solved by embedding an Equivalent Consumption Minimization Strategy (ECMS) into a Dynamic Programming (DP) optimization to obtain a sub-optimal solution that provides results close to the global optimum at a fraction of the computational cost. Further, a multi-layer hierarchical control architecture is proposed as a path to a causal, real-time implementation. The DP-ECMS algorithm is converted into a Model Predictive Control (MPC) framework by using principles of Approximate Dynamic Programming (ADP). This causal implementation is finally benchmarked to a global optimal solution obtained with DP for different scenarios. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of multiple graph matching (MGM) by considering both offline batch mode and online setting. We explore the concept of cycle-consistency over pairwise matchings and formulate the problem as finding optimal composition path on the supergraph, whose vertices refer to graphs and edge weights denote score function regarding consistency and affinity. By our theoretical study we show that the offline and online MGM on supergraph can be converted to finding all pairwise shortest paths and single-source shortest paths respectively. We adopt the Floyd algorithm and shortest path faster algorithm (SPFA) , to effectively find the optimal path. Extensive experimental results show our methods surpass state-of-the-art MGM methods, including CAO , MISM , IMGM , and many other recent methods in offline and online settings. Source code will be made publicly available. [ABSTRACT FROM AUTHOR]

Advanced algorithms can promote the development of energy management strategies (EMSs) as a key technology in hybrid electric vehicles (HEVs). Reinforcement learning (RL) with distributed structure can significantly improve training efficiency in complex environments, and multi-threaded parallel computing provides a reliable algorithm basis for promoting adaptability. Dedicated to trying more efficient deep reinforcement learning (DRL) algorithms, this paper proposed a deep q-network (DQN)-based energy and emission management strategy (E&EMS) at first. Then, two distributed DRL algorithms, namely asynchronous advantage actor-critic (A3C) and distributed proximal policy optimization (DPPO), were adopted to propose EMSs, respectively. Finally, emission optimization was taken into account and then distributed DRL-based E&EMSs were proposed. Regarding dynamic programming (DP) as the optimal benchmark, simulation results show that three DRL-based control strategies can achieve near-optimal fuel economy and outstanding computational efficiency, and compared with DQN, two distributed DRL algorithms have improved the learning efficiency by four times. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) has achieved unprecedented performance in various computer vision tasks. However, most existing NAS methods are defected in search efficiency and model generalizability. In this paper, we propose a novel NAS framework, termed MIGO-NAS, with the aim to guarantee the efficiency and generalizability in arbitrary search spaces. On the one hand, we formulate the search space as a multivariate probabilistic distribution, which is then optimized by a novel multivariate information-geometric optimization (MIGO). By approximating the distribution with a sampling, training, and testing pipeline, MIGO guarantees the memory efficiency, training efficiency, and search flexibility. Besides, MIGO is the first time to decrease the estimation error of natural gradient in multivariate distribution. On the other hand, for a set of specific constraints, the neural architectures are generated by a novel dynamic programming network generation (DPNG), which significantly reduces the training cost under various hardware environments. Experiments validate the advantages of our approach over existing methods by establishing a superior accuracy and efficiency i.e., 2.39 test error on CIFAR-10 benchmark and 21.7 on ImageNet benchmark, with only 1.5 GPU hours and 96 GPU hours for searching, respectively. Besides, the searched architectures can be well generalize to computer vision tasks including object detection and semantic segmentation, i.e., 25 × FLOPs compression, with 6.4 mAP gain over Pascal VOC dataset, and 29. 9 × FLOPs compression, with only 1.41 percent performance drop over Cityscapes dataset. The code is publicly available. [ABSTRACT FROM AUTHOR]

Mobile edge computing (MEC) can use wireless access network (RAN) to provide the services required by user's information technology (IT) and cloud computing functions nearby, which can create a high-performance and low latency service environment. Performing task offloading and data caching at access points (APs) in a cooperative manner can reduce the heavy backhaul load and the retransmission of content downloading. However, in edge networks (ENs), how to maximize storage utilization while reducing service latency and energy consumption is still a key issue, because the heterogeneity of ENs and the uneven distribution of users make it difficult to determine which MEC server and what data should be cached. In this paper, we study a two-tier MEC system, which enables data caching and computing offloading policy to minimize the network cost at the user equipment (UE) side, while satisfying the constraints of task offloading deadline, the cache capacity at APs and the computing capability of MEC servers. The optimization problem is formulated as a mixed integer nonlinear program (MINLP) problem. In order to solve the problem, we transform it into an equivalent task offloading convex optimization problem by fixing an optimization variable. Furthermore, we solve a cache placement problem by dynamic programming (DP) algorithm. Then we propose a distributed collaborative data caching and computing offloading (CDCCO) iterative algorithm. Simulation results demonstrate that our proposed CDCCO algorithm can significantly reduce the network cost and achieve better performance than other existing schemes. [ABSTRACT FROM AUTHOR]

We consider the use of the well-known dual capacity bounding technique for deriving upper bounds on the capacity of indecomposable finite-state channels (FSCs) with finite input and output alphabets. In this technique, capacity upper bounds are obtained by choosing suitable test distributions on the sequence of channel outputs. We propose test distributions that arise from certain graphical structures called Q-graphs. As we show in this paper, the advantage of this choice of test distribution is that, for the important sub-classes of unifilar and input-driven FSCs, the resulting upper bounds can be formulated as a dynamic programming (DP) problem, which makes the bounds tractable. We illustrate this for several examples of FSCs, where we are able to solve the associated DP problems explicitly to obtain capacity upper bounds that either match or beat the best previously reported bounds. For instance, for the classical trapdoor channel, we improve the best known upper bound of 0.661 (due to Lutz (2014)) to 0.584, shrinking the gap to the best known lower bound of 0.572, all bounds being in units of bits per channel use. [ABSTRACT FROM AUTHOR]

With the recent technological innovation, unmanned aerial vehicles, known as drones, have found numerous applications including package and parcel delivery for shippers. Drone delivery offers benefits over conventional ground-based vehicle delivery in terms of faster speed, lower cost, more environment-friendly, and less manpower needed. However, most of existing studies on drone delivery planning and scheduling focus on a single shipper and ignore uncertainty factors. As such, in this paper, we consider a scenario that multiple shippers can cooperate to minimize their drone delivery cost. We propose the Bayesian Shipper Cooperation in Stochastic Drone Delivery (BCoSDD) framework. The framework is composed of three functions, i.e., package assignment, shipper cooperation formation and cost management. The uncertainties of drone breakdown and misbehavior of cooperative shippers are taken into account by using multistage stochastic programming optimization and dynamic Bayesian coalition formation game. We conduct extensive performance evaluation of the BCoSDD framework by using customer locations from Solomon benchmark suite and a real Singapore logistics industry. As a result, the framework can help the shippers plan and schedule their drone delivery effectively. [ABSTRACT FROM AUTHOR]

Thermal management strategies implemented on-board fuel cell electrified vehicles (FCEVs) are currently based on heuristic reactive approaches. In this framework, developing predictive thermal management approaches that anticipate the travel needs of FCEV users could lead to improved hydrogen savings. However, the theoretical hydrogen saving achievable by predictive thermal management needs assessment first to quantify the technical and economical viability of the technology proposal. This paper lays the foundations in this domain by analyzing the a priori optimal thermal management of a fuel cell system in an FCEV. Initially, an electrochemical and thermal modeling technique for the fuel cell system is described. A reactive rule-based approach is then selected as the baseline controller for the coolant rate and the instantaneous radiator fan state of the FCEV. Then, the optimal control problem formulation for the thermal management of fuel cell systems in FCEVs is discussed and solved using a dynamic programming (DP) based optimization approach that makes use of a priori information about the entire driving mission. The fuel cell system is evaluated while the FCEV performs one to ten repetitions of the Worldwide Harmonized Light Vehicle Test Cycle (WLTC) at different ambient temperatures ranging from −20 °C to 40 °C. When compared to the baseline reactive control technique, the offline optimal benchmark can save up to 10.2% hydrogen. Results presented in this paper demonstrate the potential of hydrogen saving achievable by improving the thermal management of automotive fuel cell systems. Moreover, they may be used to develop and benchmark real-time capable predictive thermal management strategies for fuel cell systems in FCEVs.

With the exponential growth of data traffic in wireless networks, edge caching has been regarded as a promising solution to offload data traffic and alleviate backhaul congestion, where the contents can be cached by an unmanned aerial vehicle (UAV) and user terminal (UT) with local data storage. In this article, a cooperative caching architecture of UAV and UTs with scalable video coding (SVC) is proposed, which provides the high transmission rate content delivery and personalized video viewing qualities in hotspot areas. In the proposed cache-enabling UAV-D2D networks, we formulate a joint optimization problem of UT caching placement, UAV trajectory, and UAV caching placement to maximize the cache utility. To solve this challenging mixed integer nonlinear programming problem, the optimization problem is decomposed into three sub-problems. Specifically, we obtain UT caching placement by a many-to-many swap matching algorithm, then obtain the UAV trajectory and UAV caching placement by approximate convex optimization and dynamic programming, respectively. Finally, we propose a low complexity iterative algorithm for the formulated optimization problem to improve the system capacity, fully utilize the cache space resource, and provide diverse delivery qualities for video traffic. Simulation results reveal that: i) the proposed cooperative caching architecture of UAV and UTs obtains larger cache utility than the cache-enabling UAV networks with same data storage capacity and radio resource; ii) compared with the benchmark algorithms, the proposed algorithm improves cache utility and reduces backhaul offloading ratio effectively. [ABSTRACT FROM AUTHOR]

This paper presents a control strategy to optimize both the vehicle speed and gearbox position on curved roads with the aim of maximizing fuel economy. Previous studies have focused only on the vehicle speed optimization during cornering. Combining vehicle speed and gearbox position optimization can significantly maximize the energy-saving for an Internal combustion engine (ICE) vehicle. The problem is formulated as two successive optimization problems. In the first one, based on the road map and traffic data, the vehicle’s optimal speed profile is calculated by minimizing the energy spent for the entire curve. In the second one, the gearbox position is optimized for fuel consumption minimization using realistic engine map. The presented control structure’s effectiveness was evaluated through co-simulation of Matlab/Simulink and CarSim in various driving scenarios. The results show that approximately 5.25% to 11.44 % of fuel savings can be achieved compared with a typical driver model.

The objective of this paper is to formulate and analyze the benefits of a predictive non-linear multi objective optimization method for a platoon of mild-hybrid line haul trucks. In this study a group of three trucks with hybrid electric powertrain are considered in a platoon formation where each truck has a predictive optimal control to save fuel with out any loss of trip time. While the controller on each truck uses the look ahead knowledge of the entire route in terms of road grade, the overall platoon controller used a multi agent method (Metropolis algorithm) to define coordination between the trucks. While the individual trucks, showed significant improvement in fuel economy when running on predictive mode, the true savings came from the entire platoon and showed promising results in terms of absolute fuel economy without trading off on total trip time. The proposed algorithm also proved to be significantly emission efficient. A platoon of 3 trucks achieved an average of 10% fuel savings while cutting back 13% on engine out NOx emissions for engine off coasting and 9.3% fuel saving with 8% emissions reduction for engine idle coast configuration when compared to non-predictive non-platoon configuration.

Energy consumption and environmental pollution have stimulated the development of electric vehicles (EVs), among which the in-wheel motors (IWMs) are convinced to be the ultimate driving technology for the simplification of the powertrain. In this paper, the advantages and disadvantages of different driving configurations of conventional IWMs are discussed. Subsequently, four dual-motor IWMs including dual-motor torque-coupling driving configuration (DMTDC), dual-motor speed-coupling driving configuration (DMSDC), dual-motor multi-modes driving configuration (DMMDC) and dual-motor power-split driving configuration (DMPDC) are proposed. Thereafter, a two-layer hierarchical multi-objective optimization regarding the parameters of different configurations is proposed. The upper layer contains four multi-objective optimization algorithms while the lower layer runs dynamic programming (DP) to obtain the optimal energy consumption and acceleration time. The results show that the performances of the dual-motor IWM driving configurations have improved significantly compared to single-motor IWM driving configurations.

In this paper, we propose parallel heuristic methods to accelerate the generation of $(n,m)$ best equivocation code (BEC), where $n$ and $m$ are code and message lengths, respectively. The proposed dynamic programming (DP) method and greedy method extend a previous heuristics method by reducing the time complexity of the code generation process. The DP method produces the same codes as the previous method but incurs an overhead for data reuse. In contrast, the greedy method avoids this overhead but generates slightly different codes due to its heuristic approach. We parallelize the proposed methods by exploiting coarse-grained and fine-grained parallelisms, which achieve further acceleration on multicore CPU and graphics processing unit (GPU) systems, respectively. Experimental results demonstrate that the proposed DP and greedy methods reduce the sequential generation time to a quarter, as indicated by theoretical complexity analysis. In addition, the parallel implementation achieves linear speedup on a multicore CPU system, and the GPU implementation realizes coalesced memory accesses, resulting in $17\times $ acceleration over the eight-core CPU implementation. We found that the greedy method produced different codes that differ from the previous and DP methods; however, the generated codes had higher equivocation rates than those generated by a naive random method. We believe that the proposed parallel methods can effectively accelerate BEC generation for large $m$ and $n$ values, especially with larger values of $n$ relative to $m$ .

The goal of this paper is to study model-free data-driven control evaluation and design strategies for discrete-time linear time-invariant systems, where the system model is unknown. In particular, our main contribution is twofold: 1) new state-input exploration and data collection schemes from experiences; 2) new data-driven linear matrix inequalities and dynamic programming methods for stabilization and optimal control problems. The proposed exploration and data collection schemes theoretically guarantee to acquire sufficient information from the system’s state-input trajectories that can solve the underlying control design problems. We prove that under mild assumptions, as more and more data is accumulated, the collected data can solve the problems with higher probability along with the proposed algorithms.

In the construction process of intelligent pipeline system, pipeline monitoring is an important content to improve the safety of pipeline operation. Small leak location, in particular, is the primary focus of pipeline monitoring due to the unclear pressure drop point. To solve this, in this article, a data driven-based method of small leak location is proposed. First, through the collected pipeline parameters, empirical flow variables, and historical pressure values in pipeline system, a pipeline model based on pressure along pipeline is presented by a three-layer neural network to close to the industrial scenarios. Then, on the basis of the analyzed propagation process of negative pressure wave, an action-dependent heuristic dynamic programming with pressure–distance physical constraints is proposed to obtain the small leak location result. The proposed method is suitable for the collection of only pressure data scenario, which expands the application range. Finally, different cases of small leak location results indicate that the proposed method can locate the leak point, and the field tests further show that the proposed method has satisfactory performances in pipeline leak analysis. [ABSTRACT FROM AUTHOR]

This article proposes an offline control algorithm, called recurrent model predictive control, to solve large-scale nonlinear finite-horizon optimal control problems. It can be regarded as an explicit solver of traditional model predictive control (MPC) algorithms, which can adaptively select appropriate model prediction horizon according to current computing resources, so as to improve the policy performance. Our algorithm employs a recurrent function to approximate the optimal policy, which maps the system states and reference values directly to the control inputs. The output of the learned policy network after $N$ recurrent cycles corresponds to the nearly optimal solution of $N$ -step MPC. A policy optimization objective is designed by decomposing the MPC cost function according to the Bellman’s principle of optimality. The optimal recurrent policy can be obtained by directly minimizing the designed objective function, which is applicable for general nonlinear and noninput-affine systems. Both simulation-based and real-robot path-tracking tasks are utilized to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this article, we develop a new waveform codesign approach for radar–communications spectral coexistence using a decision-theoretic framework called partially observable Markov decision process (POMDP). The POMDP framework’s natural look-ahead feature allows us to trade off short-term for long-term performance, which is necessary in waveform codesign problems with competing objectives and dynamic user needs. As POMDPs are computationally intractable, we extend two approximation methods called nominal belief-state optimization and random-sampling multipath hypothesis propagation to make the codesign approaches tractable. [ABSTRACT FROM AUTHOR]

We propose a new method to compute the feedback capacity of unifilar finite state channels (FSCs) with memory using reinforcement learning (RL). The feedback capacity was previously estimated using its formulation as a Markov decision process (MDP) with dynamic programming (DP) algorithms. However, their computational complexity grows exponentially with the channel alphabet size. Therefore, we use RL, and specifically, its ability to parameterize value functions and policies with neural networks, to evaluate numerically the feedback capacity of channels with a large alphabet size. The outcome of the RL algorithm is a numerical lower bound on the feedback capacity, which is used to reveal the structure of the optimal solution. The structure is modeled by a graph-based auxiliary random variable that is utilized to derive an analytic upper bound on the feedback capacity with the duality bound. The capacity computation is concluded by verifying the tightness of the upper bound by testing whether it is Bahl-Cocke-Jelinek-Raviv (BCJR) invariant. We demonstrate this method on the Ising channel with an arbitrary alphabet size. For an alphabet size smaller than or equal to 8, we derive the analytic solution of the capacity. Next, the structure of the numerical solution is used to deduce a simple coding scheme that achieves the feedback capacity and serves as a lower bound for larger alphabets. For an alphabet size greater than 8, we present an upper bound on the feedback capacity. For an asymptotically large alphabet size, we present an asymptotic optimal coding scheme. [ABSTRACT FROM AUTHOR]

The cluster analysis of real-life data often encounters the challenges of noisy data or may rely heavily on the uncertainty of the main clustering variable owing to its stochastic nature, which has a potential influence on its performance. In this paper, we propose a novel clustering technique that is efficient in dealing with noise and uncertainty in a dataset by adopting a stochastic approach that uses realistic values of data points by assuming a continuous probability distribution instead of exact values. By estimating the best-fit probability distribution of the clustering variable, the proposed method formulates the problem of determining the most homogeneous clusters by determining the optimum cluster partitions (OCP) as a mathematical programming problem (MPP). A computer-intensive dynamic programming technique was used to solve the MPP and determine the OCP, which minimized the sum of the weighted intracluster standard deviations. The proposed technique is then demonstrated in this study using univariate data that follows a normal distribution, which is a symmetric distribution, as well as the Weibull distribution, which is a skewed distribution. Numerical examples were also presented to illustrate the computational details of the proposed method. Finally, using both simulated and real datasets, a comparative analysis of the effectiveness of the proposed technique was performed against four advanced clustering methods: k-means, fuzzy c-means, expectation maximization, and Genie++ hierarchical clustering. The results reveal that the proposed method works well and produces more efficient clusters than other methods.

The constrained discrete optimization (CDO) problems pose an immense challenge to solve with provable accuracy and computational efficiency. Dynamic programming (DP) is an elegant technique that is used to solve a class of such problems with linear constraints that follow a particular structure, namely Bellman’s principle of optimality (BPO). Unfortunately, many of the CDO problems do not fall into this category. This work focuses on solving a class of CDO problems, which we call problem class $H$ , that do not satisfy BPO if the constraint functions are considered. There are no conditions placed on the constraint functions of $H$ . However, the objective function alone satisfies the BPO. Such problems are ubiquitous in wireless communication, signal processing, and machine learning. These problems are, in general, NP-Hard. This paper attempts to unify this class of problems to be solvable using the DP framework. Using the theory of multi-objective optimization and assisted by an information-theoretic measure, we establish provable near-optimality guarantees with reduced computational complexity. We describe two algorithms to solve $H$ . We support our claims by solving the power-constrained analog-to-digital converter bit allocation (BA) problem in massive Multiple-Input Multiple-Output (MaMIMO) receivers. The optimal BA thus obtained ensures the maximum energy efficiency of the MaMIMO receiver.

In this brief, a novel formulation of the value function with dynamic event-triggered strategy is introduced for the optimal tracking problem (TP) of nonlinear discrete-time systems to eliminate the tracking error by using adaptive dynamic programming (ADP). Firstly, different from the existing ADP methods, a creative formulation of the cost function introduces the control input into the tracking error, and ignores the quadratic form of the control input directly. Second, a novel static triggering mechanism is designed. The proposed approach samples the states and updates the control signal only when the triggered condition is satisfied, and critic-actor Neural network (NN) is designed to approach the performance index and control input. The stability analysis of the closed-loop system is provided based on the Lyapunov’s theorem. Finally, in order to obtain a larger event triggering time interval, a dynamic triggering mechanism is given. Theoretical results show that the system state and the network weight errors are uniformly ultimately bounded (UUB). A simulation result also verify the theoretical claims. [ABSTRACT FROM AUTHOR]

We propose an energy-efficient supervisory control method for the power management of parallel hybrid electric vehicles (HEVs) to improve the fuel economy and reduce exhaust gas emissions. Plug-in HEVs ((P)HEVs) have multiple power sources (e.g., an engine and motor) that should be cooperatively operated to meet the required instantaneous traction power for the desired vehicle speed while satisfying their physical limits. Because the efficiencies of the engine and motor vary with different operating speeds and torques, the main issue of energy-efficient power management is to allocate the power demand among the power sources by achieving maximum power conversion efficiencies and satisfy the operating limits. For an efficient power allocation, an optimal control problem is formulated, and a global solution is found through deterministic dynamic programming (DP). Owing to the curse of dimensionality and uncertainties in real driving, DP solutions are not directly applicable in real time. To resolve the limitations of DP, we employ a non-parametric Bayesian function approximation technique using a Gaussian process (GP). The offline DP solutions obtained from a set of real vehicle driving test data were used to learn a state-dependent probabilistic value function through Gaussian process regression. For online implementations, a receding horizon control scheme was applied for the feedback control of the power management. In comparison with the existing charge sustaining strategy and charge depleting and charge sustaining mixed controllers, we recorded fuel efficiency improvements of over 4.8% and 7.3%, respectively, in a mixed urban-suburban route. [ABSTRACT FROM AUTHOR]

In this article, we present a novel maintenance policy optimization method for systems with two balanced components. The components in the system are assumed to degrade over time according to a bivariate Wiener process. The maintenance actions aim at eliminating the differences of degradation levels of system components at the cost of aggravating the degradation. Utilizing the Markov decision process, the maintenance model is put forward under both the finite and the infinite planning horizons, from which we find the structural properties of the optimal policies. Backwards dynamic programming and value iteration algorithms are employed to optimize the maintenance decisions. Examples along with sensitivity analysis are presented to facilitate the illustration and insight attainment. We find that the maintenance policies are to a great extent regulated by the absolute degradation difference between the two components. [ABSTRACT FROM AUTHOR]

A novel event-triggered optimal tracking control (ETOTC) method is developed for discrete-time nonlinear systems in this study. For the time-invariant desired trajectory, we prove that the tracking error is asymptotically stable, and an upper bound of the real performance index can be predetermined by a design parameter. For the time-varying desired trajectory, the developed triggering condition reduces communication costs by relaxing the restriction of the asymptotic stability of the closed-loop system, and we prove that the tracking error is uniformly ultimately bounded (UUB). The developed ETOTC method entails obtaining the next state of the real system. Therefore, a parallel control approach is proposed to predict the next state by constructing a parallel system for the real system. Neural networks (NNs) and adaptive dynamic programming (ADP) techniques are utilized in the parallel control approach. Moreover, the stability analysis of the closed-loop system is shown, and the tracking error and NN weight estimation errors are proved to be UUB using the Lyapunov approach. Finally, we validate the developed ETOTC method through two simulations. [ABSTRACT FROM AUTHOR]

Energy storage can contribute to the resource-adequacy needs of power systems. However, the energy-limited nature of energy storage complicates estimating its resource-adequacy contribution. Energy storage that discharges to mitigate a loss-of-load event may have less energy available to mitigate a subsequent loss-of-load event. We present a stochastic-dynamic-optimization approach to capture such impacts endogenously. We demonstrate our approach using an example and two case studies, which show that energy storage's capacity value is sensitive to the load patterns of the system in which it is deployed. [ABSTRACT FROM AUTHOR]

Battery storage management that involves multiple revenue streams would affect customers’ monthly electricity costs. In this article, a three-level model of battery storage management is proposed for achieving various functionalities, including energy arbitrage, peak shaving, and frequency regulation. The original joint optimization problem with the horizon of one month is decomposed into three subproblems according to time scales, covering the daily battery power baseline decision, hourly regulation capacity submission, and real-time regulation response in seconds. A dynamic programming framework is adopted to integrate three submodels, featuring reduced solving complexity and the overall optimality of the proposed model. Also, the unexpected regulation capacity of customers and the uncertainty of exogenous information are considered to accurately depict the characteristics of practical scenarios. Extensive case studies for two types of customers, i.e., the pure consumer and the prosumer, are conducted to validate the effectiveness of the proposed model, which achieves at least 6% (consumer) and 19% (prosumer) benefit improvement as compared with other strategies. [ABSTRACT FROM AUTHOR]

Hydrogen energy plays a fundamental role during the decarbonization transition of electricity systems. However, there is still a lack of a systematic framework analyzing the carbon-reduction capability of integrating a hydrogen-energy storage system (HESS) into a power grid. This research combines the grid dispatch model and storage-control model to define the metric and formulate a framework for assessing the HESS’s value of reducing carbon emissions in a power system. A dynamic programming algorithm is developed to estimate the maximal carbon-reduction value of HESS, which is a nonconvex optimization problem. We apply our model to assess the carbon value of HESS in the ERCOT market and discover the tradeoff between using HESS to reduce carbon and fuel costs. Using the HESS can even cause an increase in $CO_2$ emissions in ERCOT if the charging strategy is designed only to reduce the economic cost. We further identify the factors determining the carbon value of the HESS. The wind energy penetration level decisively determines how much carbon emissions the HESS can reduce. The technical characteristics of HESS also affect its carbon value. [ABSTRACT FROM AUTHOR]

The structural control of floating wind turbines using active tuned mass damper is investigated in this article. To our knowledge, this is for the first time that reinforcement learning-based control approach is employed to this type of application. Specifically, an adaptive dynamic programming (ADP) algorithm is used to derive the optimal control law based on the nonlinear structural dynamics, and the large-scale machine learning platform Tensorflow is employed for the design and implementation of the neural network (NN) structure. Three fully connected NNs, i.e., a plant network, a critic network, and an action network, are included in the proposed NN structure. Their training requires the gradient information flowing through the whole network, which is tackled by automatic differentiation, a popular technique for deriving the gradients of complex networks automatically. While to our knowledge, the network structures in the existing literature are rather simple and the training of the hidden layer is usually ignored. This allows their gradients to be derived analytically, which is infeasible with complex network structures. Thus, automatic differentiation greatly improves the employed ADP algorithm’s ability in solving complex problems. The simulation results of structural control of floating wind turbines show that ADP controller performs very well in both normal and extreme conditions, with the standard deviation of the platform pitch displacement being reduced by around 40%. A clear advantage of ADP controllers over the $H_{\infty }$ controller is observed, especially in extreme conditions. Moreover, our design considers the tradeoff between the control performance and power consumption. [ABSTRACT FROM AUTHOR]

Inferring appropriate information from large datasets has become important. In particular, identifying relationships among variables in these datasets has far-reaching impacts. In this article, we introduce the uniform information coefficient (UIC), which measures the amount of dependence between two multidimensional variables and is able to detect both linear and non-linear associations. Our proposed UIC is inspired by the maximal information coefficient (MIC) [1].; however, the MIC was originally designed to measure dependence between two one-dimensional variables. Unlike the MIC calculation that depends on the type of association between two variables, we show that the UIC calculation is less computationally expensive and more robust to the type of association between two variables. The UIC achieves this by replacing the dynamic programming step in the MIC calculation with a simpler technique based on the uniform partitioning of the data grid. This computational efficiency comes at the cost of not maximizing the information coefficient as done by the MIC algorithm. We present theoretical guarantees for the performance of the UIC and a variety of experiments to demonstrate its quality in detecting associations. [ABSTRACT FROM AUTHOR]

Microgrids are recognized as a relevant tool to absorb decentralized renewable energies in the energy mix. However, the sequential handling of multiple stochastic productions and demands, and of storage, make their management a delicate issue. We add another layer of complexity by considering microgrids where different buildings stand at the nodes of a network and are connected by the arcs; some buildings host local production and storage capabilities, and can exchange with others their energy surplus. We formulate the problem as a multistage stochastic optimization problem, corresponding to the minimization of the expected temporal sum of operational costs, while satisfying the energy demand at each node, for all time. The resulting mathematical problem has a large-scale nature, exhibiting both spatial and temporal couplings. However, the problem displays a network structure that makes it amenable to a mix of spatial decomposition-coordination with temporal decomposition methods. We conduct numerical simulations on microgrids of different sizes and topologies, with up to 48 nodes and 64 state variables. Decomposition methods are faster and provide more efficient policies than a state-of-the-art Stochastic Dual Dynamic Programming algorithm. Moreover, they scale almost linearly with the state dimension, making them a promising tool to address more complex microgrid optimal management problems. [ABSTRACT FROM AUTHOR]

This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response. [ABSTRACT FROM AUTHOR]

This article considers the problem of event-triggered optimal control for discrete-time switched nonlinear systems with constrained control input. First, an event-triggered condition is given to make the closed-loop switched system asymptotically stable. Second, a novel method, event-triggered heuristic dynamic programming (ETHDP), is applied to derive the optimal control policy. Two neural networks (NNs) are utilized to approximate the value function and control law, respectively. When the event-triggered condition is violated, the weights of the two NNs are updated, which can decrease the networks calculation and transmission load notably. A proof of the convergence of the ETHDP is also carried out. Finally, the effectiveness of the proposed method is verified by an example. [ABSTRACT FROM AUTHOR]