Search

The work presented in this paper concerns a switching-based control formulation for multi-intersection and multiphase traffic light systems. A macroscopic traffic flow modeling approach is first presented, which is instrumental to the development of a model-based and switching-based optimization method for traffic signal operation, in the framework of adaptive dynamic programming (ADP). The main advantage of the switching-based formulation is its capability to determine both “when”’ to switch and “which” mode to switch on without the need to use the cycle-based average flow approximation typical of state-of-the-art formulations. In addition, the framework can handle different cycle times across intersections without the need for synchronization constraints and, moreover, minimum dwell-time constraints can be directly enforced to comply with minimum green/red times in each phase. The simulation experiments on a multi-intersection and multiphase traffic light systems are presented to show the effectiveness of the method. [ABSTRACT FROM AUTHOR]

This paper describes a Markov decision process approach to a robust spacecraft mission control policy that maximizes the expected value of science reward assuming a circular orbit. The control policy that governs mission steps can be computed off-board or onboard depending upon the availability of communication bandwidth and on-board computational resources. This paper considers a sample science mission, where the spacecraft collects data from celestial objects viewable only within a certain orbit true anomaly window. Science data collection requires the spacecraft to slew its instrument(s) toward each target, and continue pointing in the direction of the target while the spacecraft traverses its orbit. Robustness and stochastic optimization of scientific reward, is achieved at the cost of computational complexity. Approximate dynamic programming (ADP) is exploited to reduce the computational time and effort to manageable levels and to treat larger problem sizes. The proposed ADP algorithm partitions the state-space based on true anomaly regions, enabling grouping of adjacent science targets. Results of a simulation case study demonstrate that our proposed ADP approach performs quite well for reasonable ranges of key problem parameters. [ABSTRACT FROM AUTHOR]

This paper examines the problem of predictive gear scheduling for fuel consumption minimization in connected/automated heavy trucks. The literature highlights the fuel economy benefits of such predictive scheduling, but there is a need to optimize such scheduling online, in real time. To address this need, we begin by using dynamic programming (DP) to schedule gear shifting offline, in a manner that achieves a globally optimal Pareto tradeoff between the conflicting objectives of minimizing fuel consumption and shift frequency. The computational cost of DP is unfavorable for online implementation, but we present two algorithms addressing this challenge. Both algorithms rely on the fact that in the Pareto limit where fuel consumption minimization is the sole objective, DP furnishes a simple static shift map. Our first algorithm trains a recurrent neural network to prune the shift schedule generated by this map. The second algorithm performs this pruning in a direct manner tailored to reduce the schedule's rain flow count. We simulate these algorithms for different drive cycles. Both algorithms achieve a reasonable tradeoff between fuel consumption and gear shift frequency. However, the rain flow count algorithm is both more effective in approaching the DP-based Pareto front and more computationally efficient. [ABSTRACT FROM AUTHOR]

With the proliferation of portable devices, the exponential growth of the global mobile traffic brings great challenges to the traditional communication networks and the traditional wireless communication technologies. In this context, converged networks and cache-based data offloading have drawn more and more attention based on the strong correlation of services. This paper proposes a novel popular services pushing and caching scheme by using converged overlay networks. The most popular services are pushed by terrestrial broadcasting networks. And they are cached in ${n}$ router-nodes with limited cache sizes. Each router-node only interconnects with its neighbor nodes. Users are served through the router’s WiFi link. If the services requested are cached in the routers, the user can be immediately responded; otherwise, the requests can be responded through the link from cellular stations to the router. In the proposed scheme, the cache size of the router, the maximum number of requests each router can serve, and the whole-time delay are limited. Three node-selecting and dynamic programming algorithms are adopted to maximize the equivalent throughput. Analytical and numerical results demonstrate that the proposed scheme is very effective. [ABSTRACT FROM AUTHOR]

This paper validates two approximate dynamic programming approaches on a maritime interdiction problem involving the allocation of multiple heterogeneous assets over a large area of responsibility to interdict multiple drug smugglers using heterogeneous types of transportation on the sea with varying contraband weights. The asset allocation is based on a probability of activity surface, which represents spatio-temporal target activity obtained by integrating intelligence data on drug smuggler whereabouts/waypoints for contraband transportation, behavior models, and meteorological and oceanographic information. We validate the proposed architectural and algorithmic concepts via several realistic mission scenarios. We conduct sensitivity analyses to quantify the robustness and proactivity of our approach, as well as to measure the value of information used in the allocation process. The contributions of this paper have been transitioned to and are currently being tested by Joint Interagency Task Force—South, an organization tasked with providing the initial line of defense against drug trafficking in the East Pacific and Caribbean Oceans. [ABSTRACT FROM AUTHOR]

In this paper, the optimization of the finite-time consensus of a multiagent system is studied when the leader is disturbed by white noise. Different from the consensus problem itself, this paper is devoted to optimizing the performance of the consensus and the constructive method and martingale method are used to guarantee the effectiveness of the optimization. Another contribution is that the problem of the optimal control with the piecewise constant controller gains under a stochastic environment is solved, and the reformed dynamic principle and the reformed Hamilton–Jacobi–Bellman (HJB) partial differential equations (PDEs) are proposed by using the splitting method and Feynman–Kac formula. Furthermore, since the multiagent system often runs on a high-dimensional space, a heuristic method is proposed based on the derivation of the reformed HJB PDEs and the scalar of the computational complexity of the PDEs is equivalent to that of a system of ordinary differential equations with the same dimensions when the multiagent system can be described as a piecewise linear system. Finally, the heuristic method is compared with usual filtering methods for solving engineering problems by a numerical example and it is found that the heuristic method achieves higher precision in finite time. [ABSTRACT FROM AUTHOR]

This paper presents a computation-efficient stochastic dynamic programming algorithm for solving energy storage price arbitrage considering variable charge and discharge efficiencies. We formulate the price arbitrage problem using stochastic dynamic programming and model real-time prices as a Markov process. Then we propose an analytical solution algorithm using a piecewise linear approximation of the value-to-go function. Our solution algorithm achieves extreme computation performance and solves the proposed arbitrage problem for one operating day in less than one second on a personal computer. We demonstrate our approach using historical price data from four price zones in New York Independent System Operator, with case studies comparing the performance of different stochastic models and storage settings. Our results show that the proposed method captures 50% to 90% of arbitrage profit compared to perfect price forecasts. In particular, our method captures more than 80% of arbitrage profit in three out of the four price zones when considering batteries with more than two-hour duration and realistic degradation cost. [ABSTRACT FROM AUTHOR]

This paper presents a multi-period active distribution network planning (ADNP) with distributed generation (DG). The objective of the proposed ADNP is to minimize the total planning cost, subject to both investment and operation constraints. The paper proposes a multi-stage stochastic optimization model to address DG uncertainties over several periods, in which the decisions are made sequentially by only using the present-stage information. A nested decomposition method is proposed which applies the stochastic dual dynamic integer programming (SDDIP) method to address computational intractabilities of the proposed ADNP approach. The presented numerical results and discussions on a 33-bus distribution system and a large-scale 906-bus system verify the effectiveness of the proposed ADNP method and its solution method. [ABSTRACT FROM AUTHOR]

The ever-increasing integration of variable wind energy requests for a power system with high flexibility. In this paper, we formulate the real-time economic dispatch problem as a multi-stage robust program to leverage flexible resources, especially energy storage systems, in a broader timescale, and also to avoid the risk of high costs in critical scenarios. The multi-stage robust program is decomposed to the dynamic programming form and the fast robust dual dynamic programming (FRDDP) algorithm is developed to solve it efficiently. To tackle the nonconvexities caused by the inter-temporal constraints of ESSs, an estimation-correction method is developed, and measures of the approximation accuracy are presented. Comprehensive case studies comprising units up to 1371 are carried out to verify the effectiveness of the proposed model. Numerical results show that compared with the existing ones, the proposed multi-stage robust model achieves lower cost under the worst-case scenario and shows better scalability in large-scale systems. [ABSTRACT FROM AUTHOR]

Renewable energy is an advisable choice to reduce fuel consumption and $\rm CO_{2}$ emission. Therein, wind energy and solar energy are the most promising contributors to reach this goal. Although the hybrid wind/solar system has been widely studied, the real-time current sharing based on their maximum capacities is rarely achieved in terms of seconds. Based on this, this paper proposes an accurate current sharing and voltage regulation approach in hybrid wind/solar systems, which is based on distributed adaptive dynamic programming approach. Firstly, the equivalent wind/solar model is built, which is an indispensable preprocessing to achieve the complementary between wind energy and solar energy. Therein, the wind energy and solar energy can output relative current according to their respective capacity ratio, which ensure the maximum utilization ratio of renewable energy source. Furthermore, current sharing and voltage regulation problem is switched into optimal control problem. Under this effect, each source agent aims to obtain the optimal control variable and achieve accurate current sharing/voltage regulation. Moreover, an adaptive dynamic programming approach based on Bellman principle is proposed. It can achieve accurate current sharing and voltage regulation. Finally, the simulation results are provided to illustrate the performance of the proposed adaptive dynamic programming approach. [ABSTRACT FROM AUTHOR]

As for fuel cell hybrid electric vehicle equipped with battery (BAT) and ultracapacitor (UC), its dynamic topology structure is complex and different characteristics of three power sources induce challenges in energy management for fuel economy, power sources lifespan, and dynamic performance of the vehicle. In this paper, an energy management strategy (EMS) based on a hierarchical power splitting structure and deep reinforcement learning (DRL) is proposed. In the higher layer strategy of the proposed EMS, the UC is employed to supply peak power and recover braking energy through the adaptive filter based on fuzzy control. Then, the integrated DRL and equivalent consumption minimization strategy framework is proposed to optimize the power allocation of fuel cell (FC) and BAT in the lower layer, to ensure the highly efficient operation of FC and reduce hydrogen consumption. And the action trimming based on heuristic technique is proposed to further restrain the adverse effect of sudden peak power on FC lifespan. The simulation results show the proposed EMS can make the output of FC smoother, improve its working efficiency to alleviate the stress of BAT, and increase by 14.8% compared with the Q-learning strategy in fuel economy under WLTP driving cycle. Meanwhile, the obtained results under UDDSHDV show fuel economy of the proposed EMS can reach dynamic programming (DP) benchmark level of 89.7 $\%$. [ABSTRACT FROM AUTHOR]

In this paper, downlink transmission scheduling of popular files is optimized with the assistance of wireless cache nodes. Specifically, the requests of each file, which is further divided into a number of segments, are modeled as a Poisson point process within its finite lifetime. Two downlink transmission modes are considered: 1) the base station reactively multicasts the file segments to the requesting users and selected cache nodes and (2) the base station proactively multicasts some file segments to the selected cache nodes without requests. The cache nodes with decoded file segments can help to offload the traffic via other spectrum. Without the proactive multicast, we formulate the downlink transmission resource minimization as a dynamic programming problem with random stage number, which can be approximated via a finite-horizon Markov decision process (MDP) with fixed stage number. To address the prohibitively huge state space, we propose a low-complexity scheduling policy by linearly approximating the value functions of the MDP, where the bound on the approximation error is derived. Moreover, we propose a learning-based algorithm to evaluate the approximated value functions for unknown geographical distribution of requesting users. Finally, given the above reactive multicast policy, a proactive multicast policy is introduced to exploit the temporal diversity of shadowing effect. It is shown by simulation that the proposed low-complexity reactive multicast policy can significantly reduce the resource consumption at the base station, and the proactive multicast policy can further improve the performance. [ABSTRACT FROM AUTHOR]

On multi-core processors, applications are run sharing the cache. This paper presents optimization theory to co-locate applications to minimize cache interference and maximize performance. The theory precisely specifies MRC-based composition, optimization, and correctness conditions. The paper also presents a new technique called footprint symbiosis to obtain the best shared cache performance under fair CPU allocation as well as a new sampling technique which reduces the cost of locality analysis. When sampling and optimization are combined, the paper shows that it takes less than 0.1 second analysis per program to obtain a co-run that is within 1.5 percent of the best possible performance. In an exhaustive evaluation with 12,870 tests, the best prior work improves co-run performance by 56 percent on average. The new optimization improves it by another 29 percent. Without single co-run test, footprint symbiosis is able to choose co-run choices that are just 8 percent slower than the best co-run solutions found with exhaustive testing. [ABSTRACT FROM PUBLISHER]

This paper proposes a dynamic valuation framework to determine the opportunity value of battery capacity degradation in grid applications based on the internal degradation mechanism and utilization scenarios. The proposed framework follows a dynamic programming approach and includes a piecewise linear value function approximation solution that solves the optimization problem over a long planning horizon. The paper provides two case studies on price arbitrage and frequency regulation using real market and system data to demonstrate the broad applicability of the proposed framework. Results show that the battery lifetime value is critically dependent on both the external market environment and its internal state of health. On the grid service side, results show that second-life batteries can provide more than 50% of the value compared to new batteries, and frequency regulation provides two times more revenue than price arbitrage throughout the battery lifetime. [ABSTRACT FROM AUTHOR]

In this paper, a data-based control method based on adaptive dynamic programming (ADP) algorithm is proposed for a class of discrete-time (DT) systems in the case of multiple delays. Data-based ADP method is implemented by virtue of the measured input and output data. The condition of the existence of the corresponding equivalent multiple delays system is derived according to the characteristics of time-delay system. A novel data-based state equation is developed that only composed of input and output data, which is very meaningful in practical applications. By using the data-based ADP method, the output feedback control problem is solved only by measuring the input and output of the system with multiple delays. The convergence proofs of the designed policy iteration and value iteration algorithms are given, respectively. The simulation example is presented to demonstrate the validity of the proposed data-based ADP method. [ABSTRACT FROM AUTHOR]

To contribute to a smart grid, the unit commitment and dispatch (UCD) is a very important problem to be revisited in the future microgrid environment. This paper studies the UCD problem by taking account of both traditional thermal units and distributed generation units while considering demand response and carbon emissions trading. First, the UCD problem is reformulated as an optimal switching problem of a hybrid system. The reformulation allows for convenient handling of time-dependent start-up cost by dynamic programming. Then, we propose a closed-loop hierarchical operation (CLHO) algorithm, which produces the optimal schedule with a closed-loop feature. Theoretical analysis shows the equivalence of the reformulation and the effectiveness of the CLHO algorithm. Finally, the closed-loop feature and the effectiveness of the proposed algorithm are further verified by simulations. The simulations also show that low emission allowance and high emissions trading price can reduce the total carbon emissions. [ABSTRACT FROM AUTHOR]

Uncertain power sources are increasingly integrated into distribution networks and causes more challenges for the traditional load modeling. A variety of distributed load components present dynamic characteristics with time-varying parameters. Toward the end, this paper proposes a robust time-varying parameter identification (TVPI) method for synthesis load modeling in distribution networks, including time-varying ZIP, induction motor, and equivalent impedance models. The nonlinear optimization model is developed and solved by the nonlinear least square (NLS) to find the minimum error between estimated outputs and measurements. To cope with TVPI deteriorated by voltage disturbances, dynamic programming is first used to detect the disturbance. Then, a robust TVPI engine is designed to constrain the estimated time-varying parameters within a stable range. Furthermore, advanced tolerance thresholds are also required during iterations of NLS. Numerical simulations on the 9- and 129-bus distribution systems verify the effectiveness and robustness of the proposed TVPI method. Also, this method can be robust to the ambient noise of measurements. [ABSTRACT FROM AUTHOR]

This paper presents a novel method, advisory dynamic programming (AD-DP), for power management of a fuel cell hybrid vehicle (FCHEV). The presented method embraces a new understanding of vehicle states as driver-dependent and -independent states in time domain to define a suitable state space to be used for dynamic programming. Driver-dependent states are defined in terms of multiple characteristic parameters for vehicle speed and power demand, corporately. Driver-independent states are defined in terms of discrete values for supercapacitor's state of charge $SoC_{sc}$. Transition costs between all states in the state space are calculated offline and tabulated in look-up tables for online implementation. A state predictive model is developed based on the transition statistics of driver-dependent states for a suitable number of driving cycles. Backward calculation of the total transition cost for the predicted horizon in state space is used to define optimal power split strategy for the powertrain. The formulation of optimal control problem, in terms of situation-based solutions related to vehicle states, enables a significant reduction of computational steps and hence addresses the main challenge of real-time applications. The algorithm is adapted in terms of optimization horizon and number of discrete states for DIS to suit the real-time application. Experimental application of AD-DP, using an emulation test-rig, is conducted over different driving cycles. The obtained results reveal an improvement in energy efficiency up to 29% compared to the adaptive rule-based method. The contribution of this paper can be identified as: first, development of a corporate definition for vehicle states, that can be further implemented in optimization-based power management methods. Second, the formulation of an adaptive DP that requires lower computational steps and hence suits real-time applications in hybrid electric vehicles. [ABSTRACT FROM AUTHOR]

This paper presents a cache-based popular services pushing solution on high-speed train (HST) by using converged wireless broadcasting and cellular networks. Pushing and caching popular services on the HST to improve the capacity of the network is a very efficient way; and it can also bring a better user experience. The most popular services are transmitted and cached on the vehicle relay station of the train ahead the departure time in the proposed model. Then, the most popular services are broadcasted and cached on the User Equipment after all the passengers are on the train; the less popular services are delivered to the passengers by P2P mode through the relayed cellular network on the train. Specifically, we firstly use the dynamic programming algorithm to maximize the network capacity in limited pushing time, which can be converted to the 0–1 Knapsack problem. Furthermore, we propose three greedy algorithms to approximate the optimal solution on account of the high time complexity of dynamic programming when the input scale gets bigger. And simulation results show that the proposed popularity-based greedy algorithm performs well. Moreover, as the passengers may get on and off the HST when arriving at an intermediate station, a services rebroadcast algorithm is employed when more intermediate stations are considered. U-shaped distribution is adopted to indicate the number of passengers getting on and off the train. Simulations also show that the proposed rebroadcast algorithm can efficiently improve the capacity of the converged networks. [ABSTRACT FROM AUTHOR]

Our contemporary society survives on the services provided by several network infrastructures, such as communication, power, and transportation, so their reliability should be accurately evaluated from various aspects. While past studies defined network reliability in terms of connectivity, some of the network connections that have been established may suffer from insufficient resources, i.e., vertices may be connected on a network but flows might fail due to resource contention. As a first step to address resource protection, this paper introduces path disjointness to the field of network reliability analysis, i.e., we evaluate the probability that given terminals are connected via edge- or vertex-disjoint paths. In addition, we also deal with identifying critical links under path disjointness. Since network reliability analysis is a computationally tough problem, we propose efficient algorithms utilizing the data structure of binary decision diagrams. Numerical experiments show that our method scales up to a network with 189 links. We show that network reliability and the criticality of links are greatly dependent on path disjointness; this validates the importance of the proposed method. [ABSTRACT FROM AUTHOR]

Layout-dependent effect causes variation in device performance as well as mismatch in model-hardware correlation in sub-10 nm nodes. In order to effectively explore the power-performance envelope for IC design, cell libraries must provide cells with different diffusion heights, leading to neighbor diffusion effect (NDE) due to inter-cell diffusion height change (diffusion steps). Special filler cells can protect against steps to functional cells, but with nontrivial area overhead. In this paper, we develop dynamic programming (DP)-based single-row and multi-row (MR) detailed placement optimizations that optimally reduce inter-cell diffusion steps to mitigate the impacts of NDE. Compared to previous works, our algorithms are capable of exploring richer solution spaces as they support cell flipping, relocating, and reordering across cell rows; we also consider cell displacement, flipping, and wirelength costs. Notably, to our knowledge, our MR DP-based optimization algorithm is the first to optimally handle inter-row cell relocating and reordering. We also explore various metaheuristic configurations to further improve the solution quality. Last, we develop a timing-aware approach, which is capable of creating intentional steps that can potentially improve the drive strength of critical cells. 1 The optimality is in terms of maximum diffusion step reduction, for given displacement range, reordering range and cell variants. Additionally, the above range definitions, including the definition of the ordering of cells, depend on assumptions described in and. [ABSTRACT FROM AUTHOR]

The optimal design regarding energy efficiency of hybrid and electric drivetrains is a problem that includes topology generation, topology optimization, component sizing, and control. One of the challenges of this design problem is that the topology defines the plant sizing and control parameters. Furthermore, the topology also determines the transmission model that is needed for the evaluation of the sizing and control problems. To enable automated component sizing and control optimization, a novel method is presented in this paper for the automated dynamic modeling of arbitrary hybrid and electric drivetrain topologies. In this paper, topologies are modeled at the level of (planetary) gears and clutches, making the method suitable for complex and unconventional drivetrain topologies. A generic transmission model is defined, for which the model parameters are automatically determined. The parameter determination is based on the analytic evaluation of the kinematic and kinetic properties of the components of the topology. All transmission modes are identified and classified, and infeasible and redundant modes are automatically excluded. As the method is analytic, the computation time to determine the model parameters is short (less than a second). The model, in this case, provides the rotational speeds and torques of all power sources as a function of the rotational speed and torque at the wheels, and the control variables. Using a case study, it is shown that the method can be used to automatically solve the control and sizing problems for complex drivetrain topologies. [ABSTRACT FROM AUTHOR]

Interarea oscillation is one of the main challenges for the secure and stable operation of a large-scale interconnected power system. To address this problem, this paper presents an adaptive supplementary damping controller (SDC) for voltage source converter based high voltage direct current (VSC-HVDC) by using the goal representation heuristic dynamic programming (GrHDP). Compared with conventional heuristic dynamic programming (HDP), GrHDP adds a new goal representation network to generate an adaptive reward signal to achieve better mapping relationship between the system states and the control action. To compensate phase shift between the output and input signals of a large-scale interconnected AC/DC power system, this paper proposes a parallel phase shift channel for the GrHDP. Moreover, the proposed GrHDP-SDC is a model-free control scheme, which does not need the mathematic model of the power system and has quick online learning ability to adapt to the variation of system operating conditions. Case studies are carried out on the 10-machine 39-bus system and 16-machine 68-bus system, respectively. Compared with the conventional lead-lag SDC (LL-SDC) and HDP-SDC, simulation results show that the proposed GrHDP-SDC can always provide satisfactory damping performance over a wide range of system operating conditions and disturbances. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a discriminative framework for online simultaneous segmentation and classification of modeled visual actions that can be performed in the context of other unknown actions. To this end, we employ Hough transform to vote in a 3D space for the begin point, the end point, and the label of the segmented part of the input stream. A support vector machine is used to model each class and to suggest putative labeled segments on the timeline. To identify the most plausible segments among the putative ones, we apply a dynamic programming algorithm, which maximizes the likelihood for label assignment in linear time. The performance of our method is evaluated on synthetic as well as on real data (Weizmann, TUM Kitchen, UTKAD, and Berkeley Multimodal Human Action databases). Extensive quantitative results obtained on a number of standard data sets demonstrate that the proposed approach is of comparable accuracy with the state-of-the-art approaches for online stream segmentation and classification when all performed actions are known, and performs considerably better in the presence of unmodeled actions. [ABSTRACT FROM AUTHOR]

This paper aims to explore the performance potential of a novel dual-mode power-split hybrid powertrain. First, the steady-state power split characteristics for the proposed hybrid powertrain in different modes are analyzed. The mode switching strategy is developed to maximize powertrain efficiency, which is verified by using dynamic programming based on direct transmit points (DTPs). Moreover, a novel multi-objective evolutionary algorithm based on the decomposition (MOEA/D) method used in a nested way with dynamic programming is proposed to solve the multi-objective optimization of the PS-HEVs for the first time, and the computational efficiency and superiority of the proposed algorithm is compared with the commonly used NSGA-II algorithm. The results show that the MOEA/D based multi-objective optimization framework has similar performance in the search for the Pareto frontier but significantly higher computational efficiency than that of the NSGA-II algorithm. The obtained Pareto frontier provides optimal design candidates for hybrid powertrain systems. [ABSTRACT FROM AUTHOR]

The computationally demanding dynamic programming (DP) algorithm is frequently used in academic research to solve the energy management problem of a hybrid electric vehicle (HEV). This paper is exclusively focused on how the computational demand of such a computation can be reduced. The main idea is to use a local approximation of the gridded cost-to-go and derive an analytic solution for the optimal torque split decision at each point in the time and state grid. Thereby, it is not necessary to quantize the torque split and identify the optimal decision by interpolating in the cost-to-go. Two different approximations of the cost-to-go are considered in this paper: 1) a local linear approximation and 2) a quadratic spline approximation. The results indicate that computation time can be reduced by orders of magnitude with only a slight degradation in simulated fuel economy. Furthermore, with a spline approximated cost-to-go, it is also possible to significantly reduce the memory storage requirements. A parallel plug-in HEV is considered in this paper, but the method is also applicable to an HEV. [ABSTRACT FROM PUBLISHER]

This paper addresses an NP-hard problem, refered to as Network Topology Design with minimum Cost subject to a Reliability constraint (NTD-CR), to design a minimal-cost communication network topology that satisfies a pre-defined reliability constraint. The paper describes a dynamic programming (DP) scheme to solve the NTD-CR problem, and proposes a DP approach, called Dynamic Programming Algorithm to solve NTD-CR (DPCR-ST), to generate the topology using a selected sequence of spanning trees of the network, STXmin. The paper shows that our DPCR-ST approach always provides a feasible solution, and produces an optimal topology given an optimal order of spanning trees. The paper proves that the problem of optimally ordering the spanning trees is NP-complete, and proposes three greedy heuristics to generate and order only k spanning trees of the network. Each heuristic allows the DPCR-ST approach to generate STXmin using only k spanning trees, which improves the time complexity while producing a near optimal topology. Simulations based on fully connected networks that contain up to 2.3\times 10^9 spanning trees show the merits of using the ordering methods and the effectiveness of our algorithm vis-à-vis to four existing state-of-the-art techniques. Our DPCR-ST approach is able to generate 81.5% optimal results, while using only 0.77% of the spanning trees contained in networks. Further, for a typical 2\,\times\,100 grid network that contains up to 1.899^102 spanning trees, DPCR-ST approach requires only k=1214 spanning trees to generate a topology with a reliability no larger than 5.05% off from optimal. [ABSTRACT FROM PUBLISHER]

The Ising channel, which was introduced in 1990, is a channel with memory that models intersymbol interference. In this paper, we consider the Ising channel with feedback and find the capacity of the channel together with a capacity-achieving coding scheme. To calculate the channel capacity, an equivalent dynamic programming (DP) problem is formulated and solved. Using the DP solution, we establish that the feedback capacity is the expression \(C=({2H_{b}(a)}/{3+a})\approx 0.575522\) , where \(a\) is a particular root of a fourth-degree polynomial and \(H_{b}(x)\) denotes the binary entropy function. Simultaneously, \(a=\arg \max _{0\leq x \leq 1} ({2H_{b}(x)}/{3+x})\) . Finally, an error-free, capacity-achieving coding scheme is provided together with the outlining of a strong connection between the DP results and the coding scheme. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive block coding order for intra prediction is proposed. Modern video coding standards, including the most recent High Efficiency Video Coding (HEVC), utilize fixed scan orders in processing blocks during intra coding. However, the fixed scan orders typically result in residual blocks with noticeable edge patterns. That means the fixed scan orders cannot fully exploit the content-adaptive spatial correlations between adjacent blocks, thus the bitrate after compression tends to be large. To reduce the bitrate induced by inaccurate intra prediction, the proposed approach adaptively chooses both the block and subblock coding orders by minimizing the coding cost. Specifically, determining the block coding order is formulated as a traveling salesman problem that is solved using dynamic programming. Besides the block coding order, we also design the subblock coding order in each block with an adaptive manner. The experimental results demonstrate a Bjøntegaard-Delta-rate reduction of up to 4.4% compared with HEVC anchor. [ABSTRACT FROM AUTHOR]

A multiperiodic structure always underlies the procurement problem in practice. Option contracts are widely used to make a firm capable of responding flexibly to high demand uncertainty. However, so far, the impact of different option contracts on the multiperiod decisions and performance of a firm has been not discussed. Motivated by this problem, in this paper we analyze the multiperiod procurement problem within the framework of wholesale price, call, put, and bidirectional option contracts. Based on the dynamic programming theory, we characterize the structure of the firm's optimal replenishment policy for each period in four cases. We then develop a heuristic to approximate the corresponding policy parameters. Through system comparisons of four cases, we examine the value of option contracts for the firm and discuss the best option contract type for the firm. The results show that the firm will gain higher profits after the option contracts are introduced. Moreover, if the unit purchase and exercise costs of the bidirectional option are equal to those of the unilateral option, then bidirectional option contracts are preferred over unilateral option contracts. If the unit purchase cost of the bidirectional option is higher than that of the unilateral option, unilateral option contracts may be more profitable than bidirectional option contracts. Furthermore, if the unit backlogging cost is high and the unit inventory holding cost is low, call option contracts are better than put option contracts; if the unit backlogging cost is low and the unit inventory holding cost is high, then put option contracts work better than call option contracts. [ABSTRACT FROM AUTHOR]

This paper presents predictive energy management of hybrid electric vehicles (HEVs) via computationally efficient multi-layer control. First, we formulate an optimization problem by considering driveability and a penalty for using service brakes in the objective function to optimize gear, engine on/off, engine clutch state, and power-split decisions subject to constraints on the battery state of charge (SOC) and charge sustenance. Then, we split it into two control layers, including a supervisory control in a higher layer and a local power-split control in a lower layer. In the supervisory layer, a gear and powertrain mode manager (PM) is designed, and optimal gear, engine on/off and clutch states are obtained by using a combination of dynamic programming (DP) and Pontryagin's minimum principle (PMP). Moreover, a real-time iteration Secant method is proposed to calculate optimal battery costate such that the constraint on charge sustenance is satisfied. In the local controller layer, a linear quadratic tracking method (LQT) is used to optimally split power between the engine and the electric machine and keep battery SOC within its bounds. [ABSTRACT FROM AUTHOR]

Long queues of vehicles are often found at signalized intersections, which increases the energy consumption of all the vehicles involved. This paper proposes an enhanced eco-approach control (EEAC) strategy with consideration of the queue ahead for connected electric vehicles (EVs) at a signalized intersection. The discharge movement of the vehicle queue is predicted by an improved queue discharge prediction method (IQDP), which takes both vehicle and driver dynamics into account. Based on the prediction of the queue, the EEAC strategy is designed with a hierarchical framework: the upper-stage uses dynamic programming to find the general trend of the energy-efficient speed profile, which is followed by the lower-stage model predictive controller to computes the explicit solution for a short horizon with guaranteed safe inter-vehicular distance. Finally, numerical simulations are conducted to demonstrate the energy efficiency improvement of the EEAC strategy. Besides, the effects of the queue prediction accuracy on the performance of the EEAC strategy are also investigated. [ABSTRACT FROM AUTHOR]

This paper investigates the fundamental limits of communication over a noisy discrete memoryless channel that wears out, in the sense of signal-dependent catastrophic failure. In particular, we consider a channel that starts as a memoryless binary-input channel and when the number of transmitted ones causes a sufficient amount of damage, the channel ceases to convey signals. Constant composition codes are adopted to obtain an achievability bound, and the left-concave right-convex inequality is then refined to obtain a converse bound on the log-volume throughput for channels that wear out. Since infinite blocklength codes will always wear out the channel for any finite threshold of failure, and therefore cannot convey information at positive rates, we analyze the performance of finite blocklength codes to determine the maximum expected transmission volume at a given level of average error probability. We show that this maximization problem has a recursive form and can be solved by dynamic programming. Numerical results demonstrate that a sequence of block codes is preferred to a single block code for streaming sources. [ABSTRACT FROM AUTHOR]

Due to technology scaling, aging issue is becoming one of major concerns in the design of network-on-chip (NoC). The imbalanced workload distribution and routing algorithm cause aging hotspots, where a certain group of routers have higher aging effect than others. This can possibly lead to shorter lifetime of NoC. Most existing aging-aware routing algorithms are based on minimal routing, which suffers from less degree of adaptiveness compared to non-minimal routing. Thus, they are inefficient to mitigate the aging effect of routers. In this paper, we propose to use a non-minimal routing scheme to detour the traffic away from the aging hotspots, with the objective of mitigating the aging effect for NoCs. The problem is formulated as a bottleneck shortest path problem and solved using a dynamic programming approach. Finally, the experimental results show that compared to the state-of-the-art aging-aware routing algorithm, the non-minimal routing algorithm has up to 20% lifetime improvement for hotspot traffic patterns and realistic workload traces. [ABSTRACT FROM AUTHOR]

This paper addresses the design of two-stage successively refinable unrestricted polar quantizers for bivariate circularly symmetric sources in the entropy-constrained and fixed-rate cases. The proposed solutions are globally optimal when the thresholds of the magnitude quantizers are confined to finite discretizations of the interval $\left[0,\infty\right)$. The algorithm developed for the entropy-constrained case involves a series of stages, including solving the minimum-weight path problem for multiple node pairs in certain weighted directed acyclic graphs. The asymptotical time complexity is ${O}({K}_{1}{K}_{2}^{2}{P}_{max})$ , where ${K}_{1}$ and ${K}_{2}$ are the sizes of the sets of possible magnitude thresholds of the coarse and refined unrestricted polar quantizers (UPQs), respectively, while ${P}_{max}$ is an upper bound on the number of phase levels in any phase quantizer of the coarse UPQ. The solution algorithm for the fixed-rate case is based on solving a succession of dynamic programming problems for multiple coarse quantizer bins. The time complexity in the fixed-rate case amounts to ${O}({K}_{1}{K}_{2}{N}^{2}{N}_{1})$ , where ${N}_{1}$ is the number of cells of the coarse UPQ and $N$ is the ratio between the number of bins of the fine and coarse UPQs. The extensive experimental results on a bivariate circularly symmetric Gaussian source show the effectiveness of the proposed schemes. [ABSTRACT FROM AUTHOR]

In this paper, we propose an enhanced branch-and-bound (B&B) framework for a class of sequencing problems, which aim to find a permutation of all involved elements to minimize a given objective function. We require that the sequencing problems satisfy three conditions: 1) incrementally computable; 2) monotonic; and 3) overlapping subproblems. Our enhanced B&B framework is built on the classical B&B process by introducing two techniques, i.e., dominance rules and caching search states. Following the enhanced B&B framework, we conduct empirical studies on three typical and challenging sequencing problems, i.e., quadratic traveling salesman problem, traveling repairman problem, and talent scheduling problem. The computational results demonstrate the effectiveness of our enhanced B&B framework when compared to classical B&B and some exact approaches, such as dynamic programming and constraint programming. Additional experiments are carried out to analyze different configurations of the algorithm. [ABSTRACT FROM AUTHOR]

Remote entanglement distribution plays a crucial role in large-scale quantum networks, and the key enabler for entanglement distribution is quantum routers (or repeaters) that can extend the entanglement transmission distance. However, the performance of quantum routers is far from perfect yet. Amongst the causes, the limited quantum memories in quantum routers largely affect the rate and efficiency of entanglement distribution. To overcome this challenge, this paper presents a new modeling for the maximization of entanglement distribution rate (EDR) on a memory-limited path, which is then transformed into entanglement generation and swapping sub-problems. We propose a greedy algorithm for short-distance entanglement generation so that the quantum memories can be efficiently used. As for the entanglement swapping sub-problem, we model it using an Entanglement Graph (EG), whose solution is yet found to be at least NP-complete. In light of it, we propose a heuristic algorithm by dividing the original EG into several sub-problems, each of which can be solved using dynamic programming (DP) in polynomial time. By conducting simulations, the results show that our proposed scheme can achieve a high EDR, and the developed algorithm has a polynomial-time upper bound and reasonable average runtime complexity. [ABSTRACT FROM AUTHOR]

We observed that remarkable and impressive performance on image-based human pose estimation have been achieved by deep Convolutional Neural Networks (CNN). Nevertheless, directly applying these image-based models on videos is not only computionally intensive, but also may cause jitter and loss. The main reason is that the image-based models purely focus on the local features of individual frames and totally ignore the temporal information among adjacent frames. Some existing methods are proposed to address the temporal coherency issue. However, these methods need to be designed carefully and cannot be combined with existing image-based methods. In this paper, we propose a simple yet effective module to refine the estimated pose by exploiting the temporal coherency among the heatmaps of adjacent frames, which can be easily inserted into image-based networks as a plug-in. We show that the temporal coherency issue among the heatmap frames could be re-formulated as a graph path selection optimization problem. Moreover, to speed up the refinement process, we propose a hierarchical graph optimization to achieve the refinement from coarse to fine. Experimental results on two large-scale video pose estimation benchmarks show that our module can improve the performance with little speed loss when combined with image-based methods as an efficient plug-in. [ABSTRACT FROM AUTHOR]

The edge computing service mode has entered the edge federation service stage, where comprehensive resources are supported to collaborate between vertical cloud-edge and horizontal edge nodes. The massive data processing poses a severe challenge to the traditional storage mechanism. In this paper, we propose an efficient storage resource collaboration model that leverages the selection of storage collaborative edge nodes and the allocation of storage resources among collaborative nodes, which can provide efficient and balanced storage services solutions. Specifically, selecting storage collaborative edge nodes is a multi-objective integer linear programming problem. We transform it into a single objective 0-1 integer linear programming problem and use the dynamic programming method to obtain the optimal solution. In addition, we condense the storage resource allocation problem into a mixed-linear integer programming problem and use the greedy auction algorithm to obtain an approximate optimal solution, which significantly improves efficiency and performance. We evaluate the performance of the proposed two mechanisms by performing extensive experiments. The experimental results show that they are efficient solutions to solve the problem of the storage resource collaboration model among edge nodes in edge federation service. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the cycle-based sizing optimization for diesel-engine two-cell selective catalytic reduction (SCR) systems. In modeling the SCR system, it is generally assumed that the states inside the catalyst can are homogeneous to simplify the modeling work. However, the actual states are not homogeneous, and the modeling error becomes large if the catalyst length is long. To reduce the modeling error, the states are assumed piecewise homogeneous, i.e., the catalyst is sliced into multicells and the states for each cell are assumed homogeneous. In this paper, we focus on the two-cell SCR systems where the two cells are connected in series and where the chemical reaction rates are assumed identical for the two cells. Considering the main chemical reactions and assuming that each cell is a continuous stirred tank reactor, a nonlinear system model with six states is obtained. To simultaneously reduce the tailpipe \mboxNOx and ammonia slip, it is quite interesting to study the optimal control of the urea dosing such that the \mboxNOx emissions can be maximally converted and where the ammonia slip is constrained. A dynamic programming (DP) algorithm that has a tolerable computational load is proposed. Since the whole catalyst is sliced into two cells, the next motivation is how to divide the catalyst such that the \mboxNOx conversion efficiency can be maximized while meeting the tailpipe ammonia constraint. The best sizing ratio of the SCR system in our application for the US06 test cycle is found to be 60%–40%. [ABSTRACT FROM PUBLISHER]

In this paper, concurrent design of energy management (EM) and traction control algorithms for a vehicle equipped with a parallel hybrid powertrain is studied. This paper focuses on designing the two control algorithms together as one control design problem, which are traditionally considered separately. First, optimal control actions and operating points are obtained by applying dynamic programming (DP). Then, this information is used for developing a rule-based supervisory controller. Our objective is to minimize the fuel consumption and the wheel slip simultaneously. Two control problems are also solved separately and compared with the concurrent solution. Results show that promising benefits can be obtained by using the concurrent design approach rather than considering two control problems separately. Under the same conditions, the vehicle with the concurrent supervisory controller is 16% more efficient in fuel consumption and experiences 12% less wheel slip, assuming slippery road friction conditions. [ABSTRACT FROM PUBLISHER]

We consider spectrum detection in multiple homogeneous two-state channels. At each time, a secondary user (SU) can simultaneously detect multiple channels with imperfect spectrum detectors. The objective of the SU is to obtain the idle channels as quickly as possible under a false detection constraint. This detection problem is formulated as a restless multiarmed bandit (RMAB) problem that is proven to be PSPACE-hard. A feasible approach, namely myopic policy, is to detect the best channels until idle channels are caught. In this paper, we establish the structure of the myopic policy, prove that the myopic policy is optimal in the case of detecting N-\mbox1 of $N$ channels under certain mild assumptions, and further show that the myopic policy is not optimal generally by constructing a counterexample. [ABSTRACT FROM PUBLISHER]

This paper presents a novel function of energy-optimal adaptive cruise control (EACC) for electric vehicles based on model predictive control (MPC), which plans the host car's speed trajectory in real time for higher energy efficiency by taking look-ahead traffic information and road conditions into consideration. After linear MPC (LMPC) is formulated in time domain, a nonlinear MPC (NMPC) formulation in space domain is proposed in this paper to overcome the drawbacks of LMPC in time domain. To cope with computational complexity of NMPC in space domain, the nonlinear equality constraints are relaxed to inequality constraints to yield a convex optimization problem. Moreover, it is proven that the relaxed optimization problem can be recast as a second-order cone programming (SOCP) problem, for which the efficient numerical optimizers exist. The performance of NMPC is evaluated in simulation, which is compared with the time domain LMPC and a theoretically optimal receding horizon dynamic programming (RH-DP) solution. Results indicate that the proposed space domain NMPC outperforms LMPC in time domain and the optimal solution of NMPC is very close to the result of RH-DP. [ABSTRACT FROM AUTHOR]

The past decade has witnessed a surge in research activities related to adaptive dynamic programming (ADP) and reinforcement learning (RL), particularly for control applications. Several books [item 1)–5) in the Appendix] and survey papers [item 6)–10) in the Appendix] have been published on the subject. Both ADP and RL provide approximate solutions to dynamic programming problems. In a 1995 article by Barto et al. [item 11) in the Appendix], they introduced the so-called “adaptive real-time dynamic programming,” which was specifically to apply ADP for real-time control. Later, in 2002, Murray et al. [item 12) in the Appendix] developed an ADP algorithm for optimal control of continuous-time affine nonlinear systems. On the other hand, the most famous algorithms in RL are the temporal difference algorithm [item 13) in the Appendix] and the Q-learning algorithm [item 14) and 15) in the Appendix]. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive dynamic programming (ADP) algorithm is developed for the infinite-horizon ($H_{\infty}$) optimal control problems with unknown continuous-time (CT) nonlinear systems subject to external disturbances. To facilitate the implementation of the algorithm, generalized fuzzy hyperbolic models (GFHMs) are utilized to establish an identifier–critic architecture, where the identifier is designed to reconstruct the unknown system dynamics, and the GFHM-based critic network is employed to approximate the value functions. The CT $H_{\infty}$ optimal control issue is converted into a two-player zero-sum game and the corresponding Hamilton–Jacobi–Isaacs equation is derived. The learning procedure of the critic design is adaptively implemented with the help of the reconstructed model, thus the requirement of the complete system dynamics is relaxed. Furthermore, by the means of Lyapunov direct method, the uniform ultimate boundedness stability analysis of the closed-loop control system is explicitly provided. Finally, to compare the control performances and disturbance attenuation properties of the proposed method and the existing ADP algorithms, two numerical examples are given. [ABSTRACT FROM AUTHOR]

In this paper, we develop a decentralized event-triggered control (ETC) strategy for a class of nonlinear systems with uncertain interconnections. To begin with, we show that the decentralized ETC policy for the whole system can be represented by a group of optimal ETC laws of auxiliary subsystems. Then, under the framework of adaptive critic learning, we construct the critic networks to solve the event-triggered Hamilton–Jacobi–Bellman equations related to these optimal ETC laws. The weight vectors used in the critic networks are updated by using the gradient descent approach and the experience replay (ER) technique together. With the aid of the ER technique, we can conquer the difficulty arising in the persistence of excitation condition. Meanwhile, by using classic Lyapunov approaches, we prove that the estimated weight vectors used in the critic networks are uniformly ultimately bounded. Moreover, we demonstrate that the obtained decentralized ETC can force the overall system to be asymptotically stable. Finally, we present an interconnected nonlinear plant to validate the proposed decentralized ETC scheme. [ABSTRACT FROM AUTHOR]

In this paper, an approximately optimal control strategy is developed for the tracking control of a class of continuous-time nonlinear systems with unmatched uncertainties. By transforming the unmatched uncertain term, the auxiliary system associated with the uncertain nonlinear system is established. The auxiliary system is divided into steady and transient parts, and the related controllers are separately solved, meanwhile the transient tracking error system is also obtained by introducing the reference system. A neural network-based adaptive dynamic programming method is used to get the approximately optimal tracking control law of uncertain nonlinear systems with a predefined cost function. Furthermore, the ultimately uniform boundedness of neural network weights and the stability of tracking error systems are both proved through Lyapunov theory. Two cases of nonlinear systems with unmatched uncertainties are investigated to illustrate the effectiveness of the proposed robust tracking control strategy. [ABSTRACT FROM AUTHOR]

Conventionally, as the system’s dynamics is known, the optimal consensus control problem relies on solving the coupled Hamilton–Jacobi–Bellman (HJB) equations. In this paper, with the unknown system dynamics being considered, a local ${Q}$ -function-based adaptive dynamic programming method is put forward to deal with the optimal consensus control problem for unknown discrete-time nonlinear multiagent systems by approximating the solutions of the coupled HJB equations. First, a local ${Q}$ -function is defined, which considers the local consensus error and the actions of the agent and its neighbors. Using the ${Q}$ -function, it is convenient to get the derivatives with regard to the weights of the consensus control policies, even without the model of system dynamics. Then, with the defined local ${Q}$ -function, a distributed policy iteration technique is developed, which is theoretically proved to be convergent to the solutions of the coupled HJB equations. An actor–critic neural network framework for implementing the developed model-free optimal consensus control method is constructed to approximate the local ${Q}$ -functions and the control policies. Finally, the feasibility and effectiveness of the developed method are verified by a series of simulations. [ABSTRACT FROM AUTHOR]

In this paper, a novel integral reinforcement learning method is proposed based on value iteration (VI) to design the $H_{\infty }$ controller for continuous-time nonlinear systems subject to input constraints. To confront the control constraints, a nonquadratic function is introduced to reconstruct the ${L_{2}}$ -gain condition for the $H_{\infty }$ control problem. Then, the VI method is proposed to solve the corresponding Hamilton–Jacobi–Isaacs equation initialized with an arbitrary positive semi-definite value function. Compared with most existing works developed based on policy iteration, the initial admissible control policy is no longer required which results in a more free initial condition. The iterative process of the proposed VI method is analyzed and the convergence to the saddle point solution is proved in a general way. For the implementation of the proposed method, only one neural network is introduced to approximate the iterative value function, which results in a simpler architecture with less computational load compared with utilizing three neural networks. To verify the effectiveness of the VI-based method, two nonlinear cases are presented, respectively. [ABSTRACT FROM AUTHOR]

Power buffers are power electronic converters, with large capacitors, that decouple volatile loads and a low-inertia distribution network in a DC microgrid. In this work, a set of distributed optimal control policies enable power buffers to reciprocally assist each other during abrupt load changes. While the majority of existing control paradigms are localized, enabling communication among buffers extends their effective range of assistance and helps them minimize a shared objective in a cooperative fashion. The control law's weights surfaces are learned for a mesh of reference loads of each power buffer. Hamilton-Jacobi-Bellman equation is solved by a continuous-time adaptive dynamic programming (ADP) approach with off-policy learning to directly provide a feedback controller, instead of existing approaches that obtain open-loop policies via Pontryagin's minimum principle. This paper presents the first attempt in using ADP techniques for the control of power buffers that respects their original nonlinear dynamics, overcoming the limitations of previous approaches based on small-signal analysis. Compared to the current literature, the proposed approach provides trained controllers that are known a priori, avoiding player-by-player solutions or real-time optimization procedures that could degrade performances or become computationally intensive. Hardware-in-the-loop emulations of a low-voltage DC microgrid validates the proposed approach. [ABSTRACT FROM AUTHOR]