Search

In the reverse Stackelberg mechanism, by considering a decision function for the leader rather than a decision value in the conventional Stackelberg game, the leader can explore a wider decision space. This flexibility can result in realizing the globally optimal solution of the leader’s objective function, while controlling the reaction function of the followers, simultaneously. We consider an aggregator who purchases energy from the wholesale energy market. The aggregator acts as the leader for a group of plugged in electric vehicles (PEVs) and determines the price of energy versus consumption at each hour a day as its decision function. In the followers level, since the optimal charging strategies of the PEVs are coupled through the electricity price, the PEVs in a group are considered to cooperate in finding their Nash–Pareto-optimal charging strategy, by minimizing a social cost function. For a large number of PEVs, the cooperative cost minimization of PEVs can be modeled as a cooperative mean-field (MF) game. We propose a decentralized MF optimal control algorithm and prove that the algorithm converges to leader–follower MF $\varepsilon _{N}$ -Nash equilibrium point of the game. Furthermore, a decentralized reverse Stackelberg algorithm is implemented to achieve the optimal linear price function of the leader. Simulation results and comparison with benchmark methods are performed to demonstrate the advantages of the proposed method. Note to Practitioners—The effect of a large population of PEVs on the power grid such as overload and voltage drop is inevitable. Motivated by this, there are many research articles which propose different centralized and decentralized charging coordination solutions to address this problem. The core idea in the most of literature is: “How to IMPLEMENT a demand response (DR) program appropriately for charging coordination problem to avoid high peak load?” However, none of them propose a practical algorithm on “How to DESIGN a DR optimally by having limited information from the clients?” We propose a bilevel optimization algorithm to both design a DR program (i.e., price function) and also implement DR program to flatten the demand curve, accordingly. Another advantage of the proposed method is that the information structure of the problem is close to reality. There is no information exchange among the clients and also the utility company does not need to know the private information of the clients. The Utility Company only knows the charging profile of the PEVs in each day and broadcasts the price signal to the clients for the next day. The method is illustrated in an IEEE 5-bus system that supports our claims. [ABSTRACT FROM AUTHOR]

This study proposes a decentralised robust fuzzy control strategy for islanded operation of an AC microgrid with voltage source inverters. The objective is to design a robust controller for regulating the load voltage and sharing power among distributed generators (DGs) in the presence of uncertainties in the system and non-linear loads. The AC microgrid consists of parallel DGs connected to a main AC bus. A Takagi-Sugeno fuzzy approach is developed in this article to achieve stability and desired performance in dealing with non-linearities in the islanded microgrid and H∞ criterion is used to obtain a robust control strategy in presence of uncertainties raised by unmodelled and high frequency dynamics. Therefore, a non-convex condition in H∞ optimisation problem is converted to a convex linear matrix inequality (LMI) condition and is solved by MATLAB LMI toolbox. In order to develop a decentralized system, P-f and Q-V droop controllers are used to specify set-points for local inverter controllers in each DG. The effectiveness of the proposed control strategy in presence of constant power load and DG accidental outage is validated by the simulation of a microgrid test system in MATLAB SimPowerSystems toolbox. Comparison with cascaded proportional integral controller shows advantages of robust T-S controller. [ABSTRACT FROM AUTHOR]

The authors consider the problem of multi-player evolutionary game with two strategies and an arbitrary number of players. A randomised probabilistic access model is considered where the users randomly select the strategies. The strategies are a selection between two radio access technologies to receive service from. For a fair resource allocation between a number of users, the payoff function of each player is considered to be inversely proportional to the number of players choosing a strategy. It is valid for the radio access technology selection problem where resources are divided fairly between the users. The necessary and sufficient conditions for the existence and uniqueness of a mixed Nash equilibrium point of the game is provided. In addition, it is proved that the mixed Nash equilibrium point of the game is globally asymptotically stable and also evolutionarily stable strategy. [ABSTRACT FROM AUTHOR]

This study proposes a decentralised robust fuzzy control strategy for islanded operation of an AC microgrid with voltage source inverters. The objective is to design a robust controller for regulating the load voltage and sharing power among distributed generators (DGs) in the presence of uncertainties in the system and non‐linear loads. The AC microgrid consists of parallel DGs connected to a main AC bus. A Takagi‐Sugeno fuzzy approach is developed in this article to achieve stability and desired performance in dealing with non‐linearities in the islanded microgrid and H∞ criterion is used to obtain a robust control strategy in presence of uncertainties raised by unmodelled and high frequency dynamics. Therefore, a non‐convex condition in H∞ optimisation problem is converted to a convex linear matrix inequality (LMI) condition and is solved by MATLAB LMI toolbox. In order to develop a decentralized system, P–f and Q–V droop controllers are used to specify set‐points for local inverter controllers in each DG. The effectiveness of the proposed control strategy in presence of constant power load and DG accidental outage is validated by the simulation of a microgrid test system in MATLAB SimPowerSystems toolbox. Comparison with cascaded proportional integral controller shows advantages of robust T‐S controller. [ABSTRACT FROM AUTHOR]

The authors consider the problem of multi‐player evolutionary game with two strategies and an arbitrary number of players. A randomised probabilistic access model is considered where the users randomly select the strategies. The strategies are a selection between two radio access technologies to receive service from. For a fair resource allocation between a number of users, the payoff function of each player is considered to be inversely proportional to the number of players choosing a strategy. It is valid for the radio access technology selection problem where resources are divided fairly between the users. The necessary and sufficient conditions for the existence and uniqueness of a mixed Nash equilibrium point of the game is provided. In addition, it is proved that the mixed Nash equilibrium point of the game is globally asymptotically stable and also evolutionarily stable strategy. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of energy consumption management of a large population of plug-in hybrid electric vehicles (PHEVs). A PHEV can be derived either on its gasoline or the battery modes while traveling, and can be either charged from or discharged to the grid while parking at home, respectively. This feature is the main novel consideration in our system model, besides taking into account the constraints like traveling time and distance, and the bounds on the charging power and state of charge of the battery. On the basis of the proposed model of PHEV, the optimal battery-gasoline energy management strategy is obtained in order to minimize the cost of energy and battery degradation for each user. Such a strategy can also compensate the peak load. Since the strategy of each PHEV affects other users’ cost through the electricity price, we model such an interaction as a game problem and we prove that there exist a unique Nash equilibrium point of the game. Then, due to the fact that the effect of a single PHEV of a large population on the other users’ costs is negligible, the theory of mean filed (MF) game is utilized to develop a decentralized control strategy for PHEVs. The lightweight information exchange of the proposed MF algorithm ensures a low computational cost and communication overhead. The proof of convergence for the proposed algorithm is given and it is shown that as the number of PHEVs goes to infinity the MF optimal strategy of the players converges to the unique Nash equilibrium point of the game. In simulation results, the effectiveness of the proposed method in terms of demand side management, peak shaving, convergence rate, and robustness is studied. [ABSTRACT FROM AUTHOR]

virtually, all decisions in financial markets are made in the presence of uncertainty. Stochastic dominance is a well-known concept that is broadly implemented for decision making under uncertainty. Thanks to the advantages of this approach, a decision maker is able to exploit available information related to uncertainties with stochastic nature. In this paper, to cope with the uncertainties in the procedure of decision making for forward contracts in an electricity market, a modified stochastic dominance approach is proposed. In the suggested framework, instead of considering a single value for desirable goal in each scenario of benchmark, a profit interval has been allocated to reflect the economic targets of decision maker. In the next step, the allocated profit interval is utilized for decision making problem in the stochastic dominance framework. In order to find the optimal profit profile in such a setting, a two level optimization structure is suggested. To this end, at the lower level of optimization, three different methods namely the prospect stochastic dominance, conditional value at risk (CvaR) and hypothesis testing are applied to find the optimal profit profiles for a number of benchmarks, those are stochastically generated in an interval around a pre-specified profit benchmark. In the upper level, the optimal profit profile is computed by using the Mean-CvaR method. To show the practical aspects and generalizability of our proposed approach, the methods are applied to two different cases, including a retailer and an electricity producer’s decision making problems to determine their involvement in the futures market by maximizing their expected profit over a given planning horizon, while controlling the risk of profit volatility in the electricity market, raised from the uncertainties in the spot market price and the consumer demand. The performance of suggested framework is evaluated through simulation results and relevant conclusions are drawn. [ABSTRACT FROM AUTHOR]

This study addresses a robust counterpart of the deterministic mean field control in a multi‐agent system. A decentralised mean field algorithm is proposed to solve a min–max control problem for a large population of heterogeneous agents. In the proposed leader following scheme, the leader tracks a reference signal which is unknown to the followers and each follower tracks a convex combination of the population state average and the leader's state. The leader plays a robust min–max game against disturbance and the followers play a mean field εN ‐Nash game against each other and at the same time, each follower plays a robust min–max game against the disturbance. For all the players, finite horizon quadratic cost is considered. In the proposed decentralised algorithm, followers do not need the knowledge about each of leader's and other followers' states and they only use an estimate of the population state average. In this way, propose a policy iteration method which guarantees the convergence to the saddle point mean field εN ‐Nash solution. The proposed method is applied to a large population of agents and compared with centralised algorithm to demonstrate the results. [ABSTRACT FROM AUTHOR]

Device-to-device (D2D) communication has been recently proposed to reduce cellular traffic by directly connecting the local nodes. In this paper, we consider a cellular-based primary network coexisting with a cognitive D2D pair. The primary network consists of a base station and multiple cellular users, which can sell their spectrum to the D2D user. Additionally, under low-interference conditions, the spectrum can be simultaneously shared between cellular users and D2D-capable users, which is called the nonorthogonal sharing mode. We introduce a double-sided bandwidth-auction game for cognitive D2D communication. In the auction, the D2D transmitter has three types of service providers (SPs) or sharing modes. The SPs operate on different frequency spectra, and a D2D-capable pair intends to share these spectra. This situation is formulated as an auction game, where a D2D transmitter bids a demand price–bandwidth curve, and each SP offers a price–bandwidth supply curve. The Nash equilibrium point of the auction game is analytically obtained. Then, a learning method is proposed for the decision making of the players in an incomplete-information repeated game. Moreover, the convergence of the proposed learning method to the Nash equilibrium point is studied through the converse Lyapunov theorem. Finally, the proposed auction game is extended to the multiple-D2D case, and the optimal bandwidth allocation strategy to maximize the sum rate is derived. [ABSTRACT FROM PUBLISHER]

In an incomplete information game, a big challenge is to find the best way of exploiting available information for optimal decision making of the agents. In this paper, two decision making methods, namely model-based and learning-based bidding strategies, are proposed and compared, for repeated Cournot competition of the generators in a day-ahead electricity market. The sum of the rivals’ offered quantities (SROQ) is considered as the state of the agent and its value is estimated using an adaptive expectation method. In the model-based approach, the convergence of the agents’ strategies to the Nash equilibrium point is also studied in two different cases. In the learning-based approach, the optimal bidding strategy is learned through combination of state estimation and a reinforcement learning method. Using the estimated state (SROQ), the optimal decision is learned through a fuzzy Q-learning algorithm. Through a case study, which is performed on the three-bus benchmark Cournot model, the convergence of the generators’ bids to the Nash-Cournot equilibrium is examined. [ABSTRACT FROM AUTHOR]

In this paper, the competition among suppliers in an oligopolistic market is studied using supply function equilibrium model. Since the decision of each supplier affects other suppliers’ profit, we have a game problem rather than optimization. In the proposed supply function equilibrium game, both of the cooperative and non-cooperative behaviors of the players are studied. In non-cooperative case, the Nash equilibrium point of the game is obtained for n -player in constant and price sensitive demand cases. In cooperative game, the concept of Nash bargaining solution is utilized to study the collusive behavior of the suppliers. The convexity and closeness of the players’ payoff set in supply function equilibrium model is proved as the necessary and sufficient condition for existence and uniqueness of the solution and then, the mathematical formulation of Nash bargaining solution is derived. In addition, the cooperative and non-cooperative behavior of players is compared over a wide range of parameters in a case study. [ABSTRACT FROM AUTHOR]

The aim of this article is to propose that an intelligent agent can be able to decide properly in an incomplete information repeated Cournot game. The market model and the competitors’ decision models are not known to the players. The proposed agent employs a combination of the k-nearest neighbor (KNN) method and the Bayes classifier to predict the next action of its rivals, using the market decision history. The agent takes the predicted actions as an estimate of its next state and learns the expected payoff of its state-action pairs interactively using the reinforcement learning (RL) algorithm. The results of the proposed agent's competition with two benchmark competitors in different simulated Cournot games are presented. The simulation results show that the proposed agent can significantly earn more payoffs in comparison with the two benchmark agents. [ABSTRACT FROM PUBLISHER]

Abstract: Motivated by analyzing the dynamic adjustment process of players in an -player Cournot game, in this paper a discrete-time quadratic dynamical system is proposed and the stability of its equilibrium is analyzed. Several output adjustment mechanisms (e.g. best reply, adaptive adjustment and myopic) and expectations (e.g. naïve and adaptive) in a Cournot game with linear price function and quadratic costs, which form a quadratic dynamical system, are the special cases of the proposed model. The stability of the proposed quadratic dynamical system is analyzed with (1) time invariant parameters, (2) time invariant parameters in the presence of disturbance and (3) bounded time varying parameters in the presence of disturbance. In each case, the sufficient condition to find the region that belongs to the basin of attraction is derived using some discrete-time converse Lyapunov theorems. In addition, the proposed model and theorems are utilized to analyze the stability of the boundary and Nash equilibrium points of a Cournot game with three heterogeneous players. [Copyright &y& Elsevier]

In this paper, the problem of short-term load forecasting is redefined and solved with a new metric, which is the extension of the conventional sum of squared error (SSE) metric. The proposed metric is a nonsymmetric penalty function with different penalties for over-forecasting and under-forecasting. Therefore, a large family of approaches that utilize gradient-based methods such as artificial neural networks with back propagation learning and regressions method with least squares estimate are not useful in this case. To solve this problem, a modified radial basis function (RBF) network, which uses the genetic algorithm to estimate the weights of the network is presented. This network has the ability to handle the new penalty function. In addition, a fuzzy inference system is combined with the modified RBF network to incorporate the impact of temperature on load. As a real case study, we tried to forecast the electric power load of Mazandaran area in Iran. The comparison between the proposed method and the well-known RBF network demonstrates the efficiency of the proposed method with the new forecasting metric. [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, an agent-based system for bilateral contracts of energy is proposed. The generating companies submit their offers to the demand companies. The demand companies also submit their bids to the generators. Each load or generator’s agent wants to match with an opponent, which offers the most valuable proposal. However, the problem of simultaneous decision-making causes decision conflicts among the agents. To overcome this conflict, we assume loads as the leaders and generators as the followers. We use Stackelberg game to match the seller and buyer agents. The negotiation process between a buyer and its potential seller will determine the power price between them. This process is carried out through a proposed combined time-behavioral protocol (TBP). With negligible changes in around the agreed price, this protocol can reduce the negotiation time considerably. After successful negotiation, the seller and buyer agents could sign a bilateral contract of energy if the market conditions allow it. The applicability of the proposed method is illustrated through a case study. [Copyright &y& Elsevier]

In this paper, an agent-based simultaneous negotiation method for bilateral contracts in a multi agent market is proposed. Each agent has a representation of its desired attributes for a trading commodity using fuzzy linguistic terms. The negotiation process begins between every buyer and its potential sellers simultaneously through a new soft-bargaining protocol. This method offers flexibility for the agents to represent their satisfaction for the combinations of different attributes using a multi-dimensional membership functions (MDMFs), taking into account the interdependencies between the attributes. For each pair of buyer and seller at each step of negotiation, a point that has the maximum degree of agreement in the intersection of MDMFs is a candidate for agreement. During the negotiation, the agents adjust their satisfaction membership degrees on the attribute space as well as their acceptance thresholds to increase the chance of agreement. Bargaining is carried out under incomplete information, keeping agents’ attitudes private. Every buyer/seller simultaneously receives the maximum degrees of agreement with other opponents through a double-blind mediating software, and then it checks them against its acceptance threshold in order to choose the best matching opponent for signing the contract. Through a case study for an energy market, the capabilities of the proposed method are illustrated. [Copyright &y& Elsevier]

In this paper, an agent matching method for bilateral contracts in a multi-agent market is proposed. Each agent has a hierarchical representation of its trading commodity attributes by a tree structure of fuzzy attributes. Using this structure, the similarity between the trees of each pair of buyer and seller is computed using a new ordered fuzzy similarity algorithm. Then, using the concept of Stackelberg equilibrium in a leader–follower game, matchmaking is performed among the sellers and buyers. The fuzzy similarities of each agent with others in its personal viewpoint have been used as its payoffs in a bimatrix game. Through a case study for bilateral contracts of energy, the capabilities of the proposed agent-based system are illustrated. [ABSTRACT FROM AUTHOR]

In this paper, multi-agent dynamic optimization with a coupling constraint is studied. The aim is to minimize a strongly convex social cost function, by considering a linear stochastic dynamics for each agent and also coupling constraints among the agents. In order to handle the coupling constraint and also, to avoid high computational cost imposed by a centralized method for large scale systems, the dual decomposition method is used to decompose the problem into multiple individual sub-problems, while the dual variable is adjusted by a coordinator. Nevertheless, since each sub-problem is not a linear–quadratic (LQ) optimal control problem, and hence its closed-form solution does not exist, approximate dynamic programming (ADP) is utilized to solve the sub-problems. The main contribution of the paper is to propose an algorithm by considering the interrelated iterations of dual variable adjustment and ADP, and to prove the convergence of the algorithm to the global optimal solution of the social cost function. Additionally, the implementation of the proposed algorithm using a neural network is presented. Also, the computational advantage of the proposed algorithm in comparison with other bench-marking methods is discussed in simulation results. [ABSTRACT FROM AUTHOR]

Due to the shared nature of wireless channels, competition among the nodes to access media is inevitable. P-persistent carrier sense multiple access (CSMA) is a medium access control scheme widely used for resource allocation in wireless environments. The probability of transmission highly affects the throughput. We model the wireless nodes as agents in a network game. The strategy of an agent is defined as the probability of transmission. Agents don’t have a priori information about the network (e.g., number of nodes, other agents strategies) and learn their optimal strategy using the historical sensory information including the number of collisions or successful transmissions. In this paper, a multi-state reinforcement learning (RL) method is designed for this purpose. The main challenge in designing an RL agent is to define the states of the environment from agent’s perspective. For this purpose, in this paper, various state representations are proposed in a multi-state Q-learning model. This leads to different agents personalities ranging from cautious agents with risk aversion to aggressive risky agents. We also propose agents with combined personalities of cautiousness and aggressiveness. The performance of the proposed Q-learning agents with different state definitions in comparison with each other and also other benchmarking agents is examined via comprehensive simulation results. [ABSTRACT FROM AUTHOR]

In a competitive electricity market, suppliers seek the optimal bidding strategy, maximising their individual profit. Due to the uncertainties, unknown parameters and dynamics of electricity market, the optimal bidding strategies cannot be attained through straightforward optimisation methods. In this study, the supply function equilibrium model is considered together with uniform price market clearing mechanism. The interaction among suppliers is modelled as an incomplete information game problem and multi-agent reinforcement learning (MARL) is utilised to find the optimal bidding strategy of the suppliers in a non-cooperative game. The proposed framework is an extension, from single agent Markov decision process to Markov game process, which is suitable for studying multi-agent decision making problems in stochastic environments. In this study, a Markov game model of the electricity market is proposed and a new state-action and Markov model of electricity market is proposed to built up the MARL environment. In addition, a learning-based non-cooperative MARL method is utilised for learning the optimal bidding strategies of the suppliers in a day-ahead electricity market. The proposed method is successfully applied to IEEE-30-bus power system. To examine the statistical significance of the results, the T-test method is applied to compare the competitive behaviour of the players in both multi- and single-agent frameworks. [ABSTRACT FROM AUTHOR]

In a competitive electricity market, suppliers seek the optimal bidding strategy, maximising their individual profit. Due to the uncertainties, unknown parameters and dynamics of electricity market, the optimal bidding strategies cannot be attained through straightforward optimisation methods. In this study, the supply function equilibrium model is considered together with uniform price market clearing mechanism. The interaction among suppliers is modelled as an incomplete information game problem and multi‐agent reinforcement learning (MARL) is utilised to find the optimal bidding strategy of the suppliers in a non‐cooperative game. The proposed framework is an extension, from single agent Markov decision process to Markov game process, which is suitable for studying multi‐agent decision making problems in stochastic environments. In this study, a Markov game model of the electricity market is proposed and a new state‐action and Markov model of electricity market is proposed to built up the MARL environment. In addition, a learning‐based non‐cooperative MARL method is utilised for learning the optimal bidding strategies of the suppliers in a day‐ahead electricity market. The proposed method is successfully applied to IEEE‐30‐bus power system. To examine the statistical significance of the results, the T‐test method is applied to compare the competitive behaviour of the players in both multi‐ and single‐agent frameworks. [ABSTRACT FROM AUTHOR]