Search

Bayesian policy reuse (BPR) is a general policy transfer framework for selecting a source policy from an offline library by inferring the task belief based on some observation signals and a trained observation model. In this article, we propose an improved BPR method to achieve more efficient policy transfer in deep reinforcement learning (DRL). First, most BPR algorithms use the episodic return as the observation signal that contains limited information and cannot be obtained until the end of an episode. Instead, we employ the state transition sample, which is informative and instantaneous, as the observation signal for faster and more accurate task inference. Second, BPR algorithms usually require numerous samples to estimate the probability distribution of the tabular-based observation model, which may be expensive and even infeasible to learn and maintain, especially when using the state transition sample as the signal. Hence, we propose a scalable observation model based on fitting state transition functions of source tasks from only a small number of samples, which can generalize to any signals observed in the target task. Moreover, we extend the offline-mode BPR to the continual learning setting by expanding the scalable observation model in a plug-and-play fashion, which can avoid negative transfer when faced with new unknown tasks. Experimental results show that our method can consistently facilitate faster and more efficient policy transfer.

Identifying the Hamiltonian of an unknown quantum system is a critical task in the area of quantum information. In this article, we propose a systematic Hamiltonian identification approach via quantum ensemble multiclass classification (HI-QEMC). This approach is implemented by a three-step iterative refining process, i.e., parameter interval guess, verification, and judgment. In the parameter interval guess step, the parameter interval is divided into several sub-intervals and the true Hamiltonian parameter is guessed in one of them. In the parameter interval verification step, cross verification is applied to verify the accuracy of the guess. In the parameter interval judgment step, an adaptive interval judgment (AIJ) algorithm is designed to determine the sub-interval containing the true Hamiltonian parameter. Numerical results on two typical quantum systems, i.e., two-level quantum systems and three-level quantum systems, demonstrate the effectiveness and superior performance of the proposed approach for quantum Hamiltonian identification.

Accurate wind speed forecasting is crucial for efficiently integrating of wind power into the electrical grid and ensuring a stable power supply. However, wind speed is inherently noisy and unpredictable, making it challenging to forecast accurately. This study investigates the effects of hyperparameters of Discrete Wavelet Transform (DWT) and Singular Spectrum Analysis (SSA) on the accuracy of wind speed forecasting using various prediction models. Our proposed method focuses on the optimisation of hyperparameters within the existing models, suggesting that significant untapped potential remains. Our study examines a wide range of wavelet function orders for DWT and varying trend ratio parameter for SSA, and evaluates their impact on the prediction accuracy using real data from thirteen locations in Jordan. Particularly, our investigation reveals that high-order Daubechies wavelets in DWT outperform low-order wavelets. The study also illustrates that optimal hyperparameters must be modified when changing the prediction model and the combination of DWT and SSA enhances prediction performance when the trend ratio is set to 90%. Our results demonstrate that fine-tuning data preprocessing techniques is essential for accurate wind speed prediction since hyperparameter tuning results in greater improvements in prediction accuracy than sophisticated prediction models alone. Our findings underscore the importance of leveraging data preprocessing techniques and hyperparameter tuning for accurate wind speed forecasting.

The task to estimate all the parameters of an unknown quantum state, also called quantum state tomography, is essential for characterizing and controlling quantum systems. In this paper, we utilize observable time traces to identify the initial quantum state of a closed quantum system, based on the state space approach in the control theory. In the informationally complete scenario, we show that with a linear regression estimation (LRE), the mean squared error (MSE) scales as O1/N, where Nis the resource number. In the informationally incomplete scenario, we introduce regularization LRE to perform the state tomography task. We employ PBH test to demonstrate that closed quantum systems with only one observable are informationally incomplete and propose using d-1observables, where dis the dimension of the quantum state, for informational completeness. Numerical examples demonstrate the effectiveness of our method.

Fast quantum control helps reduce the influence of unavoided disturbances and hence plays a vital role in practical quantum technology and chemical reactions. Instead of optimizing the terminal cost like standard optimal quantum control methods, this paper formulates the problem as a trajectory optimization problem, and implements the sequential quadratic programming algorithm to search for short control fields. The core idea is to minimize the cumulative intermediate error to incentivize early achievement of the designed gate. The numerical result on the Toffoli gate demonstrates the effectiveness of the proposed method.

Multi-Agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this article, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

The rapid proliferation of learning systems in an arbitrarily changing environment mandates the need to manage tensions between exploration and exploitation. This work proposes a quantum-inspired bandit learning approach for the learning-and-adapting-based offloading problem where a client observes and learns the costs of each task offloaded to the candidate resource providers, e.g., fog nodes. In this approach, a new action update strategy and novel probabilistic action selection are adopted, provoked by the amplitude amplification and collapse postulate in quantum computation theory. We devise a locally linear mapping between a quantum-mechanical phase in a quantum domain, e.g., Grover-type search algorithm, and a distilled probability-magnitude in a value-based decision-making domain, e.g., adversarial multi-armed bandit algorithm. The proposed algorithm is generalized, via the devised mapping, for better learning weight adjustments on favorable/unfavorable actions, and its effectiveness is verified via simulation.

Evolution strategies (ESs), as a family of black-box optimization algorithms, recently emerge as a scalable alternative to reinforcement learning (RL) approaches such as Q-learning or policy gradient and are much faster when many central processing units (CPUs) are available due to better parallelization. In this article, we propose a systematic incremental learning method for ES in dynamic environments. The goal is to adjust previously learned policy to a new one incrementally whenever the environment changes. We incorporate an instance weighting mechanism with ES to facilitate its learning adaptation while retaining scalability of ES. During parameter updating, higher weights are assigned to instances that contain more new knowledge, thus encouraging the search distribution to move toward new promising areas of parameter space. We propose two easy-to-implement metrics to calculate the weights: instance novelty and instance quality. Instance novelty measures an instance’s difference from the previous optimum in the original environment, while instance quality corresponds to how well an instance performs in the new environment. The resulting algorithm, instance weighted incremental evolution strategies (IW-IESs), is verified to achieve significantly improved performance on challenging RL tasks ranging from robot navigation to locomotion. This article thus introduces a family of scalable ES algorithms for RL domains that enables rapid learning adaptation to dynamic environments.

While reinforcement learning (RL) algorithms are achieving state-of-the-art performance in various challenging tasks, they can easily encounter catastrophic forgetting or interference when faced with lifelong streaming information. In this article, we propose a scalable lifelong RL method that dynamically expands the network capacity to accommodate new knowledge while preventing past memories from being perturbed. We use a Dirichlet process mixture to model the nonstationary task distribution, which captures task relatedness by estimating the likelihood of task-to-cluster assignments and clusters the task models in a latent space. We formulate the prior distribution of the mixture as a Chinese restaurant process (CRP) that instantiates new mixture components as needed. The update and expansion of the mixture are governed by the Bayesian nonparametric framework with an expectation maximization (EM) procedure, which dynamically adapts the model complexity without explicit task boundaries or heuristics. Moreover, we use the domain randomization technique to train robust prior parameters for the initialization of each task model in the mixture; thus, the resulting model can better generalize and adapt to unseen tasks. With extensive experiments conducted on robot navigation and locomotion domains, we show that our method successfully facilitates scalable lifelong RL and outperforms relevant existing methods.

Deep reinforcement learning (DRL) has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel DRL approach by constructing a curriculum consisting of a set of intermediate tasks defined by fidelity thresholds, where the tasks among a curriculum can be statically determined before the learning process or dynamically generated during the learning process. By transferring knowledge between two successive tasks and sequencing tasks according to their difficulties, the proposed curriculum-based DRL (CDRL) method enables the agent to focus on easy tasks in the early stage, then move onto difficult tasks, and eventually approaches the final task. Numerical comparison with the traditional methods [gradient method (GD), genetic algorithm (GA), and several other DRL methods] demonstrates that CDRL exhibits improved control performance for quantum systems and also provides an efficient way to identify optimal strategies with few control pulses.

Quantum detector tomography is a fundamental technique for calibrating quantum devices and thus lay foundations for quantum information processing tasks. In this work, we propose a quantum detector tomography method that employs deep neural networks to reconstruct quantum detectors from a set of probe states with high efficiency. Numerical results demonstrate that the proposed method exhibits a significant potential to estimate phase-insensitive detectors.

This article proposes a novel energy management algorithm that controls the battery energy storage system (BESS) and on-grid supply. It employs the deep-Q-network agent with prioritized experience replay, and its efficacy is validated and verified by comparison to a benchmark method for mixed integer linear programming. The grid and energy storage systems are governed by switching operations initiated by BESS controllers via the automatic transfer switch. The primary objective is to accomplish optimal scheduling of batteries one day in advance to reduce electricity costs while maintaining battery health and primary power supply reliability. The methods proposed in this work provide practicable grid and battery operation patterns that test all conceivable planning scenarios for energy storage operation. Finally, a comparative analysis is performed to evaluate the efficacy of the proposed BESS operation scheduling methods.

Quantum state tomography and quantum detector tomography are two main problems in quantum system identification. In this paper, we study state tomography and detector tomography with prior knowledge about the true rank of the unknown state/detector. In this case, we propose a sufficient condition for static (non-adaptive) tomography algorithms to achieve the optimal infidelity scaling O(1/N)on the number of state copies N.Then we present two examples of such static quantum state tomography and quantum detector tomography algorithms. Numerical examples demonstrate the effectiveness of our algorithms.

In this article, adaptive regulation of block-oriented nonlinear systems, i.e., Hammerstein and Wiener systems, with binary-valued measurements of the regulation errors is considered. Compared with the classical framework for stochastic adaptive control, the new feature here is that only binary-valued observations of regulation errors are available to the controller. An adaptive regulator based on the stochastic approximation algorithm is proposed and it is proved that the regulator is optimal in the sense that it minimizes the long-run average of the squared regulation errors almost surely. Numerical examples as well as real applications of the proposed algorithms to automotive engine control are given.

Quantum language models (QLMs) in which words are modeled as a quantum superposition of sememes have demonstrated a high level of model transparency and good post-hoc interpretability. Nevertheless, in the current literature, word sequences are basically modeled as a classical mixture of word states, which cannot fully exploit the potential of a quantum probabilistic description. A quantum-inspired neural network (NN) module is yet to be developed to explicitly capture the nonclassical correlations within the word sequences. We propose a NN model with a novel entanglement embedding (EE) module, whose function is to transform the word sequence into an entangled pure state representation. Strong quantum entanglement, which is the central concept of quantum information and an indication of parallelized correlations among the words, is observed within the word sequences. The proposed QLM with EE (QLM-EE) is proposed to implement on classical computing devices with a quantum-inspired NN structure, and numerical experiments show that QLM-EE achieves superior performance compared with the classical deep NN models and other QLMs on question answering (QA) datasets. In addition, the post-hoc interpretability of the model can be improved by quantifying the degree of entanglement among the word states.

Robustness and reliability are two key requirements for developing practical quantum control systems. The purpose of this article is to design a coherent feedback controller for a class of linear quantum systems suffering from Markovian jumping faults so that the closed-loop quantum system has both fault tolerance and $H^\infty$ disturbance attenuation performance. This article first extends the physical realization conditions from the time-invariant case to the time-varying case for linear stochastic quantum systems. By relating the fault-tolerant $H^\infty$ control problem to the dissipation properties and the solutions of Riccati differential equations, an $H^\infty$ controller for the quantum system is then designed by solving a set of linear matrix inequalities. In particular, an algorithm is employed to introduce additional quantum inputs and to construct the corresponding input matrices to ensure the physical realizability of the quantum controller. Also, we propose a real application of the developed fault-tolerant control strategy to quantum optical systems. A linear quantum system example from quantum optics, where the amplitude of the pumping field randomly jumps among different values due to the fault processes, can be modeled as a linear Markovian jumping system. It is demonstrated that a quantum $H^\infty$ controller can be designed and implemented using some basic optical components to achieve the desired control goal for this class of systems. [ABSTRACT FROM AUTHOR]

A central capability of a long-lived reinforcement learning (RL) agent is to incrementally adapt its behavior as its environment changes and to incrementally build upon previous experiences to facilitate future learning in real-world scenarios. In this article, we propose lifelong incremental reinforcement learning (LLIRL), a new incremental algorithm for efficient lifelong adaptation to dynamic environments. We develop and maintain a library that contains an infinite mixture of parameterized environment models, which is equivalent to clustering environment parameters in a latent space. The prior distribution over the mixture is formulated as a Chinese restaurant process (CRP), which incrementally instantiates new environment models without any external information to signal environmental changes in advance. During lifelong learning, we employ the expectation–maximization (EM) algorithm with online Bayesian inference to update the mixture in a fully incremental manner. In EM, the E-step involves estimating the posterior expectation of environment-to-cluster assignments, whereas the M-step updates the environment parameters for future learning. This method allows for all environment models to be adapted as necessary, with new models instantiated for environmental changes and old models retrieved when previously seen environments are encountered again. Simulation experiments demonstrate that LLIRL outperforms relevant existing methods and enables effective incremental adaptation to various dynamic environments for lifelong learning.

The increasing penetration of renewable energy sources in power systems introduces additional load fluctuations. Lack of awareness of them may lead to a high risk of system failure. Many existing approaches mitigate this issue by considering individual chance constraints (CCs) for the physical limitation of the system. To guarantee that all the operational constraints are satisfied with a predetermined probability, this paper uses joint chance constraints to formulate the optimal power flow (OPF) problem. Additionally, to ensure the scalability, this paper presents an alternating direction method of multipliers (ADMM) with convex optimization subproblems to solve the joint chance-constrained (JCC) OPF, where the computational burden is reduced. At last, to avoid making assumptions about the uncertainties, the CCs are approximated with a sample-based approach.

In this article, we consider the dynamical modeling of a class of quantum network systems consisting of qubits, where information extraction is allowed by performing measurement on several selected qubits of the system. For a variety of applications, a state space model is a useful approach to modeling the system dynamics. To construct a state space model for a quantum network system, the major task is to find an accessible set containing all of the operators coupled to the measurement operators. This article focuses on the generation of a proper accessible set for a given system and measurement scheme. We provide analytic results on simplifying the process of generating accessible sets for systems with a time-independent Hamiltonian. Since the order of elements in the accessible set determines the form of state space matrices, guidance is provided to effectively arrange the ordering of elements in the state vector. Defining a system state according to the accessible set, one can develop a state space model with a special pattern inherited from the system structure. As a demonstration, we specifically consider a typical 1-D-chain system with several common measurements and employ the proposed method to determine its accessible set.

In this letter, we consider a wireless uplink transmission scenario in which an unmanned aerial vehicle (UAV) serves as an aerial base station collecting data from ground users. To optimize the expected sum uplink transmit rate without any prior knowledge of ground users (e.g., locations, channel state information and transmit power), the trajectory planning problem is optimized via the quantum-inspired reinforcement learning (QiRL) approach. Specifically, the QiRL method adopts novel probabilistic action selection policy and new reinforcement strategy, which are inspired by the collapse phenomenon and amplitude amplification in quantum computation theory, respectively. Numerical results demonstrate that the proposed QiRL solution can offer natural balancing between exploration and exploitation via ranking collapse probabilities of possible actions, compared to the traditional reinforcement learning approaches that are highly dependent on tuned exploration parameters.

This work conducts robust $H^{\infty}$ analysis for a class of quantum systems subject to perturbations in the interaction Hamiltonian. A necessary and sufficient condition for the robustly strict bounded real property of this type of uncertain quantum system is proposed. This paper focuses on the study of coherent robust $H^{\infty}$ controller design for quantum systems with uncertainties in the interaction Hamiltonian. The desired controller is connected with the uncertain quantum system through direct and indirect couplings. A necessary and sufficient condition is provided to build a connection between the robust $H^{\infty}$ control problem and the scaled $H^{\infty}$ control problem. A numerical procedure is provided to obtain coefficients of a coherent controller. An example is presented to illustrate the controller design method.

The identifiability of a system is concerned with whether the unknown parameters in the system can be uniquely determined with all the possible data generated by a certain experimental setting. A test of quantum Hamiltonian identifiability is an important tool to save time and cost when exploring the identification capability of quantum probes and experimentally implementing quantum identification schemes. In this article, we generalize the identifiability test based on the similarity transformation approach (STA) in classical control theory and extend it to the domain of quantum Hamiltonian identification. We employ the STA to prove the identifiability of spin-1/2 chain systems with arbitrary dimension assisted by single-qubit probes. We further extend the traditional STA method by proposing a structure preserving transformation (SPT) method for nonminimal systems. We use the SPT method to introduce an indicator for the existence of economic quantum Hamiltonian identification algorithms, whose computational complexity directly depends on the number of unknown parameters (which could be much smaller than the system dimension). Finally, we give an example of such an economic Hamiltonian identification algorithm and perform simulations to demonstrate its effectiveness. [ABSTRACT FROM AUTHOR]

This article develops a quantum projection filtering approach to approximating the quantum filter equation based on It $\hat{\rm o}$ stochastic Taylor expansions and quantum information geometric techniques. The proposed approximation scheme is designed so that the truncated Taylor expansion of the difference between the true quantum trajectory, and its approximation on a lower dimensional differential submanifold is minimized, through an orthogonal projection operation in the quantum Fisher metric. In addition, a convenient design for a special class of open quantum systems is formulated. Simulation results from a numerical example demonstrate the approximation capability of the proposed quantum projection filter. [ABSTRACT FROM AUTHOR]

Controlling coherence and interference of quantum states is one of the central goals in quantum science. Different from energetically discrete quantum states, however, it remains a demanding task to visualize coherent properties of degenerate states (e.g., magnetic sublevels). It becomes further inaccessible in the absence of an external perturbation (e.g., Zeeman effect). Here, we present a theoretical analysis of all-optical control of degenerate magnetic states in the molecular hydrogen ion, $ {\rm H}_2^ + $H2+, by using two time-delayed co- and counterrotating circularly polarized attosecond extreme-ultraviolet (XUV) pulses. We perform accurate simulations to examine this model by solving the three-dimensional time-dependent Schrödinger equation. A counterintuitive phenomenon of quantum interference between degenerate magnetic sublevels appears in the time-dependent electronic probability density, which is observable by using x-ray-induced transient angular and energy-resolved photoelectron spectra. This work provides an insight into quantum interference of electron dynamics inside molecules at the quantum degeneracy level.

The purpose of this paper is to design a coherent feedback controller for a Markovian jump linear quantum system suffering from a fault signal. The control objective is to bound the effect of the disturbance input on the output for the time-varying quantum system. We prove the relation between the H∞control problem, the dissipation properties, and the solutions of Riccati differential equations, by which the H∞controller of the Markovian jump linear quantum system is given by the solutions of Linear Matrix Inequalities (LMIs).

Quantum sensing, utilizing quantum techniques to extract key information of a quantum (or classical) system, is a fundamental area in quantum science and technology. For quantum sensors, a basic capability is to uniquely infer unknown parameters in a system based on measurement data from the sensors. In this paper, we investigate the capability of a class of quantum sensors for a spin-12chain system with unknown parameters. The sensors are composed of qubits which are coupled to the object system and can be initialized and measured. We consider the capability of the single- and two-qubit sensors and show that the capability of single-qubit quantum sensors can be enhanced by adding an extra qubit into the sensor under a certain initialization and measurement setting.

The deployment of advanced information and communication technologies necessitates considering new security threats, such as distributed denial of service attacks and malware, which can fault power generators and feeders and exacerbate power outages in distributed generation systems (DGS). Existing cyber-security studies fail to validate the attacker–defender game model between operators and hackers or provide a DGS model that accounts for realistic characteristics and operations. Furthermore, current game models may be infeasible for large-scale systems and are not robust against uncertainties owing to the use of metaheuristic algorithms. To overcome these gaps, this study quantified the result of a game using the contest success function and estimated the parameters of this function based on real-world evidence: the dataset of cyber crime incidents from Advisen, US. The DGS management was optimized using the power flow model considering the scenario-based uncertainty stemming from cyber-attacks. A three-stage attack+defend–defend–attack framework is proposed to optimize attack–defense resource allocation using the cooperative game and ϵ-subgradient method. The results for IEEE 4, 13, 34, 123 and 342 test node feeders show that the proposed framework is applicable to large-scale systems and robust to various types of cyber-attacks. The proposed model and algorithms further enhance the DGS performance under uncertainties by protecting the entire grid or only critical nodes according to the defenders’ objectives.

In this paper, a new training paradigm is proposed for deep reinforcement learning using self-paced prioritized curriculum learning with coverage penalty. The proposed deep curriculum reinforcement learning (DCRL) takes the most advantage of experience replay by adaptively selecting appropriate transitions from replay memory based on the complexity of each transition. The criteria of complexity in DCRL consist of self-paced priority as well as coverage penalty. The self-paced priority reflects the relationship between the temporal-difference error and the difficulty of the current curriculum for sample efficiency. The coverage penalty is taken into account for sample diversity. With comparison to deep Q network (DQN) and prioritized experience replay (PER) methods, the DCRL algorithm is evaluated on Atari 2600 games, and the experimental results show that DCRL outperforms DQN and PER on most of these games. More results further show that the proposed curriculum training paradigm of DCRL is also applicable and effective for other memory-based deep reinforcement learning approaches, such as double DQN and dueling network. All the experimental results demonstrate that DCRL can achieve improved training efficiency and robustness for deep reinforcement learning. [ABSTRACT FROM AUTHOR]

Quantum Hamiltonian identification (QHI) is important for characterizing the dynamics of quantum systems, calibrating quantum devices, and achieving precise quantum control. In this paper, an effective two-step optimization (TSO) QHI algorithm is developed within the framework of quantum process tomography. In the identification method, different probe states are input into quantum systems and the output states are estimated using the quantum state tomography protocol via linear regression estimation. The time-independent system Hamiltonian is reconstructed based on the experimental data for the output states. The Hamiltonian identification method has computational complexity $O(d^6)$ , where $d$ is the dimension of the system Hamiltonian. An error upper bound $O(\frac{d^3}{\sqrt{N}})$ is also established, where $N$ is the resource number for the tomography of each output state, and several numerical examples demonstrate the effectiveness of the proposed TSO Hamiltonian identification method. [ABSTRACT FROM AUTHOR]

A microgrid, regarded as one of the cornerstones of the future smart grid, uses distributed generations and information technology to create a widely distributed automated energy delivery network. This paper presents a review of the microgrid concept, classification and control strategies. Besides, various prospective issues and challenges of microgrid implementation are highlighted and explained. Finally, the important aspects of future microgrid research are outlined. This study would help researchers, scientists, and policymakers to get in-depth and systematic knowledge on microgrid. It will also contribute to identify the key factors for mobilizing this sector for a sustainable future.

We investigated the real-time pricing demand response management system of multiple microgrids and multiple power users. Accordingly, we have proposed a Stackelberg-evolutionary joint game framework to examine the real-time pricing scheme of multiple microgrids and multiple power users so as to establish equilibrium strategies. Both a non-cooperative game among multiple microgrids and a multi-population evolutionary game among multiple power users were considered. Furthermore, we constructed a Stackelberg game between microgrids and power users to reflect their sequential interaction, wherein the microgrids are leaders, and the power users are followers. We also proved the existence and uniqueness of the Stackelberg equilibrium. Furthermore, we proposed an iterative algorithm to compute the equilibrium strategy and demonstrate the convergence and effectiveness through numerical simulations, which demonstrated that the algorithm could achieve a balance between power supply and demand balance.

This paper considers the consensus behavior of a spatially distributed 3-D dynamical network composed of heterogeneous agents: leaders and followers, in which the leaders have the preferred information about the destination, while the followers do not have. All followers move in a 3-D Euclidean space with a given speed and with their headings updated according to the average velocity of the corresponding neighbors. Compared with the 2-D model, a key point lies in how to analyze the dynamical behavior of a “linear” nonhomogeneous equation where the nonhomogeneous term strongly nonlinearly depends on the states of all agents. Using the network structure and the estimation of some characteristics for the initial states, we present a proper decaying rate for the nonhomogeneous term and then establish lower bounds on the ratio of the number of leaders to the number of followers that is needed for the expected consensus by considering two cases: 1) fixed speed and neighborhood radius and 2) variable speed and neighborhood radius with respect to the population size. Some simulation examples are given to justify the theoretical results.

In this paper, we develop a direct method for the characterization of dark modes. The results can be used to construct a transformation that separates dark and bright modes, through the decomposition of system dynamics. We also study a synthesis problem by engineering the system–environment coupling and Hamiltonian engineering. We apply the theory to investigate an optomechanical dark mode. [ABSTRACT FROM AUTHOR]

Learning control has been recognized as a powerful approach in quantum information technology. In this paper, we extend the application of differential evolution (DE) to design optimal control for various quantum systems. Various DE methods are introduced and analyzed, and EMSDE featuring in equally mixed strategies is employed for quantum control. Two classes of quantum control problems, including control of four-level open quantum ensembles and quantum superconducting systems, are investigated to demonstrate the performance of EMSDE for learning control of quantum systems. Numerical results verify the effectiveness of the EMSDE method for various quantum systems and show the potential for complex quantum control problems.

Quantum state tomography is one of the critical tasks in the development of quantum technology. This paper further investigates the adaptive linear regression estimation (ALRE) method that was developed recently for state reconstruction of qubit systems. In this method, adaptive measurements are used to improve the performance of state reconstruction where current measurement basis is adaptively optimized based on previous knowledge through minimizing an index related to the mean squared error (MSE). The numerical results on single-qubit and two-qubit systems demonstrate the effectiveness of this proposed adaptive estimation method over static one for quantum state reconstruction.

Identification of the Hamiltonian is vital for characterizing the dynamical evolution of a quantum system. The dimension of a multi-qubit system increases exponentially with the qubit number, which usually leads to daunting computational complexity for general Hamiltonian identification algorithms. In this paper, we design an efficient quantum Hamiltonian identification method based on periodical sampling. The computational complexity is O(M2+ MN2), where Mis the number of unknown parameters to be identified in the Hamiltonian and N is the length of the sampling data. Numerical results with different data lengths demonstrate the effectiveness of the proposed identification algorithm.

A filtering problem for a class of quantum systems disturbed by a classical stochastic process is investigated in this paper. The classical disturbance process, which is assumed to be described by a linear stochastic differential equation, is modeled by a quantum cavity model. Then the hybrid quantum-classical system is described by a combined quantum system consisting of two quantum cavity subsystems. Quantum filtering theory and a quantum extended Kalman filter method are employed to estimate the states of the combined quantum system. An estimate of the classical stochastic process is derived from the estimate of the combined quantum system.

In quantum engineering, faults may occur in a quantum control system, which will cause the quantum control system unstable or deteriorate other relevant performance of the system. This note presents an estimator-based fault-tolerant control design approach for a class of linear quantum stochastic systems subject to fault signals. In this approach, the fault signals and some commutative components of the quantum system observables are estimated, and a fault-tolerant controller is designed to compensate the effect of the fault signals. Numerical procedures are developed for controller design and an example is presented to demonstrate the proposed design approach. [ABSTRACT FROM AUTHOR]

Quantum ensemble classification (QEC) has significant applications in discrimination of atoms (or molecules), separation of isotopes, and quantum information extraction. However, quantum mechanics forbids deterministic discrimination among nonorthogonal states. The classification of inhomogeneous quantum ensembles is very challenging, since there exist variations in the parameters characterizing the members within different classes. In this paper, we recast QEC as a supervised quantum learning problem. A systematic classification methodology is presented by using a sampling-based learning control (SLC) approach for quantum discrimination. The classification task is accomplished via simultaneously steering members belonging to different classes to their corresponding target states (e.g., mutually orthogonal states). First, a new discrimination method is proposed for two similar quantum systems. Then, an SLC method is presented for QEC. Numerical results demonstrate the effectiveness of the proposed approach for the binary classification of two-level quantum ensembles and the multiclass classification of multilevel quantum ensembles. [ABSTRACT FROM PUBLISHER]

This technical note extends applications of the quantum small gain and Popov methods from existing results on robust stability to performance analysis results for a class of uncertain quantum systems. This class of systems involves a nominal linear quantum system and is subject to quadratic perturbations in the system Hamiltonian. Based on these two methods, coherent guaranteed cost controllers are designed for a given quantum system to achieve improved control performance. An illustrative example also shows that the quantum Popov approach can obtain less conservative results than the quantum small gain approach for the same uncertain quantum system. [ABSTRACT FROM AUTHOR]

In this paper, we propose and study a master-equation based approach to drive a quantum network with n qubits to a consensus (symmetric) state introduced by Mazzarella et al. The state evolution of the quantum network is described by a Lindblad master equation with the Lindblad terms generated by continuous-time swapping operators, which also introduce an underlying interaction graph. We establish a graphical method that bridges the proposed quantum consensus scheme and classical consensus dynamics by studying an induced graph (with 2^{2n} nodes) of the quantum interaction graph (with n$ qubits). A fundamental connection is then shown that quantum consensus over the quantum graph is equivalent to componentwise classical consensus over the induced graph, which allows various existing works on classical consensus to be applicable to the quantum setting. Some basic scaling and structural properties of the quantum induced graph are established via combinatorial analysis. Necessary and sufficient conditions for exponential and asymptotic quantum consensus are obtained, respectively, for switching quantum interaction graphs. As a quantum analogue of classical synchronization of coupled oscillators, quantum synchronization conditions are also presented, in which the reduced states of all qubits tend to a common trajectory. [ABSTRACT FROM PUBLISHER]

In this paper, a robust H ∞ analysis method is presented for a class of uncertain quantum systems where uncertainties may exist in the system and interaction Hamiltonians, and the coupling operators. We provide a sufficient condition to guarantee this class of uncertain systems robust strictly bounded real with respect to a given disturbance attenuation. Moreover, we propose a robust H ∞ control method for quantum systems with uncertain system Hamiltonian and uncertain coupling operators. The controller and the quantum plant are connected via direct and indirect couplings. This robust H ∞ control problem is shown to have a connection with a scaled H ∞ problem. We propose a numerical procedure to find the corresponding coefficients of the desired H ∞ controller by using an LMI formulation and multi-step optimization. [ABSTRACT FROM AUTHOR]

Designing highly accurate and robust controls for quantum unitary operations is vital for practical quantum computation. In this paper, we demonstrate that the robustness of quantum gate controls can be enhanced by optimizing the sampling-based infidelity variance, which is formulated as a multi-objective optimization task to achieve high robustness while maintaining high gate fidelity. A two-step approach that first optimizes the average fidelity and then turns to the infidelity variance is proposed, where two modified differential evolution (DE) algorithms, i.e., mixed-guided-strategy DE (MGSDE) and multi-objective mixed-strategy DE (MOMSDE), are designed to search fields for the two steps of robust quantum gate control, respectively. Both MGSDE and MOMSDE adopt the mixed strategy, and in particular, MGSDE adopts a guided mutation scheme to accelerate the convergence, and MOMSDE uses an optimal buffer to explore the Pareto front. Numerical results demonstrate that the proposed approach enhances the control robustness and provides an efficient in-situ learning paradigm to tackle the disturbance problem for the control design of quantum gates. [ABSTRACT FROM AUTHOR]

The aim of quantum metrology is to exploit quantum effects to improve the precision of parameter estimation beyond its classical limit. In this paper, we investigate the quantum parameter estimation problem with multiple channels. It is related but not limited to the following two important and practical quantum metrology problems: (i) Quantum enhanced metrology with control, whose aim is to improve the precision of quantum sensing by utilizing feedback or open-loop control; (ii) Practical quantum metrology where the underlying evolution of quantum probes may change from a unitary dynamics to an open system dynamics, owing to the inevitable decoherence during the quantum sensing operation. For various kinds of quantum multiple channels, the corresponding quantum channel Fisher information is derived. To demonstrate the results, some illustrative examples are given.

The balance between exploration and exploitation is a key problem for reinforcement learning methods, especially for Q-learning. In this paper, a fidelity-based probabilistic Q-learning (FPQL) approach is presented to naturally solve this problem and applied for learning control of quantum systems. In this approach, fidelity is adopted to help direct the learning process and the probability of each action to be selected at a certain state is updated iteratively along with the learning process, which leads to a natural exploration strategy instead of a pointed one with configured parameters. A probabilistic Q-learning (PQL) algorithm is first presented to demonstrate the basic idea of probabilistic action selection. Then the FPQL algorithm is presented for learning control of quantum systems. Two examples (a spin-1/2 system and a Λ-type atomic system) are demonstrated to test the performance of the FPQL algorithm. The results show that FPQL algorithms attain a better balance between exploration and exploitation, and can also avoid local optimal policies and accelerate the learning process.

In this paper, we develop a control algorithm to generate entanglement in a quantum system with uncertainties. The system under consideration is an uncertain system of two two-level atoms interacting with each other through a dipole-dipole interaction. The sampling-based learning control (SLC) strategy is employed to find a control law. An SLC strategy contains two steps of training and evaluation. In the training step, we obtain several samples to construct an augmented system by sampling the uncertainties according to a possible distribution of the uncertainty parameters and learn an optimal control law by maximizing the performance index. In the evaluation step, we apply the obtained control law from the training step to additional samples through randomly sampling the uncertainties. Numerical results are presented showing the success of the SLC method in control design for generating entanglement.

This technical note presents a sampled-data approach to the robust control of a single qubit (quantum bit). The required robustness is defined using a sliding mode domain and the control law is designed offline and then utilized online with a single qubit having bounded uncertainties. Two classes of uncertainties are considered involving the system Hamiltonian and the coupling strength of the system-environment interaction. Four cases are analyzed in detail including without decoherence, with amplitude damping decoherence, phase damping decoherence and depolarizing decoherence. Sampling periods are specifically designed for these cases to guarantee the required robustness. Two sufficient conditions are presented for the design of a unitary control for the cases without decoherence and with amplitude damping decoherence. The proposed approach has potential applications in quantum error-correction and in constructing robust quantum gates. [ABSTRACT FROM AUTHOR]

Abstract: As a widely used reinforcement learning method, Q-learning is bedeviled by the curse of dimensionality: The computational complexity grows dramatically with the size of state-action space. To combat this difficulty, an integrated hierarchical Q-learning framework is proposed based on the hybrid Markov decision process (MDP) using temporal abstraction instead of the simple MDP. The learning process is naturally organized into multiple levels of learning, e.g., quantitative (lower) level and qualitative (upper) level, which are modeled as MDP and semi-MDP (SMDP), respectively. This hierarchical control architecture constitutes a hybrid MDP as the model of hierarchical Q-learning, which bridges the two levels of learning. The proposed hierarchical Q-learning can scale up very well and speed up learning with the upper level learning process. Hence this approach is an effective integral learning and control scheme for complex problems. Several experiments are carried out using a puzzle problem in a gridworld environment and a navigation control problem for a mobile robot. The experimental results demonstrate the effectiveness and efficiency of the proposed approach.

In [Dong and Petersen, Sliding Mode Control of Two-Level Quantum Systems, arXiv:1009.0558, quant-ph, 2010], a sliding mode control approach has been proposed for two-level quantum systems to deal with bounded uncertainties in the system Hamiltonian. This paper further extends these results in two directions. One extension is to consider the effect of uncertainties during the process of driving the system's state back to the sliding mode domain from outside and we propose two approaches to accomplish this control task. The other extension generalizes the previous design approach to consider the uncertainties described as perturbations in the free Hamiltonian.

A brand-new paradigm of robots–quantum robots–is proposed through the fusion of quantum theory with robot technology. A quantum robot is essentially a complex quantum system which generally consists of three fundamental components: multi-quantum computing units (MQCU), quantum controller/actuator, and information acquisition units. Corresponding to the system structure, several learning control algorithms, including quantum searching algorithms and quantum reinforcement learning algorithms, are presented for quantum robots. The theoretical results show that quantum robots using quantum searching algorithms can reduce the complexity of the search problem from O($N^2)$ in classical robots to O($N\sqrt N)$. Simulation results demonstrate that quantum robots are also superior to classical robots in efficient learning under novel quantum reinforcement learning algorithms. Considering the advantages of quantum robots, some important potential applications are also analyzed and prospected.