Search

Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes information-theoretic perspective, this can be framed as an inference problem. We propose the Maximum Geometric Quantum Entropy Principle to exploit the notions of Quantum Information Dimension and Geometric Quantum Entropy. These allow us to quantify the entropy of fully arbitrary ensembles and select the one that maximizes it. After formulating the principle mathematically, we give the analytical solution to the maximization problem in a number of cases and discuss the physical mechanism behind the emergence of such maximum entropy ensembles.

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a "next-generation" reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano's inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least ∼60% higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines-specifically, large ϵ -machines-are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

Predictive equivalence in discrete stochastic processes has been applied with great success to identify randomness and structure in statistical physics and chaotic dynamical systems and to inferring hidden Markov models. We examine the conditions under which predictive states can be reliably reconstructed from time-series data, showing that convergence of predictive states can be achieved from empirical samples in the weak topology of measures. Moreover, predictive states may be represented in Hilbert spaces that replicate the weak topology. We mathematically explain how these representations are particularly beneficial when reconstructing high-memory processes and connect them to reproducing kernel Hilbert spaces.

We introduce a fast and highly efficient physically realizable bit swap. Employing readily available and scalable Josephson junction microtechnology, the design implements the recently introduced paradigm of momentum computing. Its nanosecond speeds and sub-Landauer thermodynamic efficiency arise from dynamically storing memory in momentum degrees of freedom. As such, during the swap, the microstate distribution is never near equilibrium and the memory-state dynamics fall far outside of stochastic thermodynamics that assumes detailed-balanced Markovian dynamics. The device implements a bit-swap operation - a fundamental operation necessary to build reversible universal computing. Extensive, physically calibrated simulations demonstrate that device performance is robust and that momentum computing can support thermodynamically efficient, high-speed, large-scale general-purpose computing that circumvents Landauer's bound.

Predictive states for stochastic processes are a nonparametric and interpretable construct with relevance across a multitude of modeling paradigms. Recent progress on the self-supervised reconstruction of predictive states from time-series data focused on the use of reproducing kernel Hilbert spaces. Here, we examine how Wasserstein distances may be used to detect predictive equivalences in symbolic data. We compute Wasserstein distances between distributions over sequences ("predictions") using a finite-dimensional embedding of sequences based on the Cantor set for the underlying geometry. We show that exploratory data analysis using the resulting geometry via hierarchical clustering and dimension reduction provides insight into the temporal structure of processes ranging from the relatively simple (e.g., generated by finite-state hidden Markov models) to the very complex (e.g., generated by infinite-state indexed grammars).

Building on parallels between geometric quantum mechanics and classical mechanics, we explore an alternative basis for quantum thermodynamics that exploits the differential geometry of the underlying state space. We focus on microcanonical and canonical ensembles, looking at the geometric counterpart of Gibbs ensembles for distributions on the space of quantum states. We show that one can define quantum heat and work in an intrinsic way, including single-trajectory work. We reformulate thermodynamic entropy in a way that accords with classical, quantum, and information-theoretic entropies. We give both the first and second laws of thermodynamics and Jarzynki's fluctuation theorem. Overall, this results in a more transparent physics than conventionally available. The mathematical structure and physical intuitions underlying classical and quantum dynamics are seen to be closely aligned. The experimental relevance is brought out via a stochastic model for chiral molecules (in the two-state approximation) and Josephson junctions. Numerically, we demonstrate this invariably leads to the emergence of the geometric canonical ensemble.

Nowcasting is a term originating from economics, finance, and meteorology. It refers to the process of determining the uncertain state of the economy, markets or the weather at the current time by indirect means. In this paper, we describe a simple two-parameter data analysis that reveals hidden order in otherwise seemingly chaotic earthquake seismicity. One of these parameters relates to a mechanism of seismic quiescence arising from the physics of strain-hardening of the crust prior to major events. We observe an earthquake cycle associated with major earthquakes in California, similar to what has long been postulated. An estimate of the earthquake hazard revealed by this state variable time series can be optimized by the use of machine learning in the form of the Receiver Operating Characteristic skill score. The ROC skill is used here as a loss function in a supervised learning mode. Our analysis is conducted in the region of 5° × 5° in latitude-longitude centered on Los Angeles, a region which we used in previous papers to build similar time series using more involved methods (Rundle & Donnellan, 2020, https://doi.org/10.1029/2020EA001097; Rundle, Donnellan et al., 2021, https://doi.org/10.1029/2021EA001757; Rundle, Stein et al., 2021, https://doi.org/10.1088/1361-6633/abf893). Here we show that not only does the state variable time series have forecast skill, the associated spatial probability densities have skill as well. In addition, use of the standard ROC and Precision (PPV) metrics allow probabilities of current earthquake hazard to be defined in a simple, straightforward, and rigorous way.

Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g. hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system’s future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel maximum entropy and maximum caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining both implicit and explicit aspects.

Stochastic thermodynamics has largely succeeded in characterizing both equilibrium and far-from-equilibrium phenomena. Yet many opportunities remain for application to mesoscopic complex systems-especially biological ones-whose effective dynamics often violate detailed balance and whose microscopic degrees of freedom are often unknown or intractable. After reviewing excess and housekeeping energetics-the adaptive and homeostatic components of a system's dissipation-we extend stochastic thermodynamics with a trajectory class fluctuation theorem for nonequilibrium steady-state, nondetailed-balanced complex systems. We then take up the neurobiological examples of voltage-gated sodium and potassium ion channels to apply and illustrate the theory, elucidating their nonequilibrium behavior under a biophysically plausible action potential drive. These results uncover challenges for future experiments and highlight the progress possible understanding the thermodynamics of complex systems-without exhaustive knowledge of every underlying degree of freedom.

Adaptive systems—such as a biological organism gaining survival advantage, an autonomous robot executing a functional task, or a motor protein transporting intracellular nutrients—must somehow embody relevant regularities and stochasticity in their environments to take full advantage of thermodynamic resources. Analogously, but in a purely computational realm, machine learning algorithms estimate models to capture predictable structure and identify irrelevant noise in training data. This happens through optimization of performance metrics, such as model likelihood. If such learning is physically implemented, is there a sense in which computational models estimated through machine learning are physically preferred? We introduce the thermodynamic principle that work production is the most relevant performance measure for an adaptive physical agent and compare the results to the maximum-likelihood principle that guides machine learning. Within the class of physical agents that most efficiently harvest energy from their environment, we demonstrate that an efficient agent’s model explicitly determines its architecture and how much useful work it harvests from the environment. We then show that selecting the maximum-work agent for given environmental data corresponds to finding the maximum-likelihood model. This establishes an equivalence between nonequilibrium thermodynamics and dynamic learning. In this way, work maximization emerges as an organizing principle that underlies learning in adaptive thermodynamic systems.

Geometric quantum mechanics, through its differential-geometric underpinning, provides additional tools of analysis and interpretation that bring quantum mechanics closer to classical mechanics: state spaces in both are continuous and equipped with symplectic geometry. This opens the door to revisiting foundational questions and issues, such as the nature of quantum entropy, from a geometric perspective. Central to this is the concept of a geometric quantum state - the probability measure on a system's space of pure states. This space's continuity leads us to introduce two analysis tools, inspired by Renyi's approach to information theory of continuous variables, to characterize and quantify fundamental properties of geometric quantum states: the quantum information dimension, which is the rate of geometric quantum state compression, and the dimensional geometric entropy that monitors information stored in quantum states. We recount their classical definitions, information-theoretic interpretations, and adapt them to quantum systems via the geometric approach. We then explicitly compute them in various examples and classes of quantum system. We conclude commenting on future directions for information in geometric quantum mechanics.

Landauer's Principle states that the energy cost of information processing must exceed the product of the temperature, Boltzmann's constant, and the change in Shannon entropy of the information-bearing degrees of freedom. However, this lower bound is achievable only for quasistatic, near-equilibrium computations-that is, only over infinite time. In practice, information processing takes place in finite time, resulting in dissipation and potentially unreliable logical outcomes. For overdamped Langevin dynamics, we show that counterdiabatic potentials can be crafted to guide systems rapidly and accurately along desired computational paths, providing shortcuts that allow for the precise design of finite-time computations. Such shortcuts require additional work, beyond Landauer's bound, that is irretrievably dissipated into the environment. We show that this dissipated work is proportional to the computation rate as well as the square of the information-storing system's length scale. As a paradigmatic example, we design shortcuts to create, erase, and transfer a bit of information metastably stored in a double-well potential. Though dissipated work generally increases with operation fidelity, we show that it is possible to compute with perfect fidelity in finite time with finite work. We also show that the robustness of information storage affects an operation's energetic cost-specifically, the dissipated work scales as the information lifetime of the bistable system. Our analysis exposes a rich and nuanced relationship between work, speed, size of the information-bearing degrees of freedom, storage robustness, and the difference between initial and final informational statistics.

Abstract: The earthquake cycle of stress accumulation and release is associated with the elastic rebound hypothesis proposed by H.F. Reid following the M7.9 San Francisco earthquake of 1906. However, observing details of the actual values of time- and space-dependent tectonic stress is not possible at the present time. In two previous papers, we have proposed methods to image the earthquake cycle in California by means of proxy variables. These variables are based on correlations in patterns of small earthquakes that occur nearly continuously in time. The purpose of the present paper is to compare these two methods by evaluating their information content using decision thresholds and Receiver Operating Characteristic methods together with Shannon information entropy. Using seismic data from 1940 to present in California, we find that both methods provide nearly equivalent information on the rise and fall of earthquake correlations associated with major earthquakes in the region. We conclude that the resulting timeseries can be viewed as proxies for the cycle of stress accumulation and release associated with major tectonic activity. Article Highlights: The current state of the earthquake cycle of tectonic stress accumulation and release is unobservableWe review two methods for visualizing the current state of the earthquake cycle from correlation in small earthquake patternsMachine learning techniques indicate that signals in a correlation time series corresponding to future large earthquakes can be detected

Pairwise interactions are fundamental drivers of collective behavior-responsible for group cohesion. The abiding question is how each individual influences the collective. However, time-delayed mutual information and transfer entropy, commonly used to quantify mutual influence in aggregated individuals, can result in misleading interpretations. Here, we show that these information measures have substantial pitfalls in measuring information flow between agents from their trajectories. We decompose the information measures into three distinct modes of information flow to expose the role of individual and group memory in collective behavior. It is found that decomposed information modes between a single pair of agents reveal the nature of mutual influence involving many-body nonadditive interactions without conditioning on additional agents. The pairwise decomposed modes of information flow facilitate an improved diagnosis of mutual influence in collectives.

We merge computational mechanics' definition of causal states (predictively equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation-a finite- or infinite-state kernel ϵ-machine-is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Planck equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high-dimensional data.

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

We generalize the exact predictive regularity of symmetry groups to give an algebraic theory of patterns, building from a core principle of future equivalence. For topological patterns in fully-discrete one-dimensional systems, future equivalence uniquely specifies a minimal semiautomaton. We demonstrate how the latter and its semigroup algebra generalizes translation symmetry to partial and hidden symmetries. This generalization is not as straightforward as previously considered. Here, though, we clarify the underlying challenges. A stochastic form of future equivalence, known as predictive equivalence, captures distinct statistical patterns supported on topological patterns. Finally, we show how local versions of future equivalence can be used to capture patterns in spacetime. As common when moving to higher dimensions, there is not a unique local approach, and we detail two local representations that capture different aspects of spacetime patterns. A previously developed local spacetime variant of future equivalence captures patterns as generalized symmetries in higher dimensions, but we show that this representation is not a faithful generator of its spacetime patterns. This motivates us to introduce a local representation that is a faithful generator, but we demonstrate that it no longer captures generalized spacetime symmetries. Taken altogether, building on future equivalence, the theory defines and quantifies patterns present in a wide range of classical field theories.

We compare and contrast three different, but complementary views of "structure" and "pattern" in spatial processes. For definiteness and analytical clarity, we apply all three approaches to the simplest class of spatial processes: one-dimensional Ising spin systems with finite-range interactions. These noncritical systems are well-suited for this study since the change in structure as a function of system parameters is more subtle than that found in critical systems where, at a phase transition, many observables diverge, thereby making the detection of change in structure obvious. This survey demonstrates that the measures of pattern from information theory and computational mechanics differ from known thermodynamic and statistical mechanical functions. Moreover, they capture important structural features that are otherwise missed. In particular, a type of mutual information called the excess entropy-an information theoretic measure of memory-serves to detect ordered, low entropy density patterns. It is superior in several respects to other functions used to probe structure, such as magnetization and structure factors. ϵ-Machines-the main objects of computational mechanics-are seen to be the most direct approach to revealing the (group and semigroup) symmetries possessed by the spatial patterns and to estimating the minimum amount of memory required to reproduce the configuration ensemble, a quantity known as the statistical complexity. Finally, we argue that the information theoretic and computational mechanical analyses of spatial patterns capture the intrinsic computational capabilities embedded in spin systems-how they store, transmit, and manipulate configurational information to produce spatial structure.

Reservoir computers (RCs) and recurrent neural networks (RNNs) can mimic any finite-state automaton in theory, and some workers demonstrated that this can hold in practice. We test the capability of generalized linear models, RCs, and Long Short-Term Memory (LSTM) RNN architectures to predict the stochastic processes generated by a large suite of probabilistic deterministic finite-state automata (PDFA) in the small-data limit according to two metrics: predictive accuracy and distance to a predictive rate-distortion curve. The latter provides a sense of whether or not the RNN is a lossy predictive feature extractor in the information-theoretic sense. PDFAs provide an excellent performance benchmark in that they can be systematically enumerated, the randomness and correlation structure of their generated processes are exactly known, and their optimal memory-limited predictors are easily computed. With less data than is needed to make a good prediction, LSTMs surprisingly lose at predictive accuracy, but win at lossy predictive feature extraction. These results highlight the utility of causal states in understanding the capabilities of RNNs to predict.

We propose a new machine learning-based method for nowcasting earthquakes to image the time-dependent earthquake cycle. The result is a timeseries that may correspond to the process of stress accumulation and release. The timeseries are constructed by using principal component analysis of regional seismicity. The patterns are found as eigenvectors of the cross-correlation matrix of a collection of seismicity timeseries in a coarse grained regional spatial grid (pattern recognition via unsupervised machine learning). The eigenvalues of this matrix represent the relative importance of the various eigenpatterns. Using the eigenvectors and eigenvalues, we compute the weighted correlation timeseries of the regional seismicity. This timeseries has the property that the weighted correlation generally decreases prior to major earthquakes in the region, and increases suddenly just after a major earthquake occurs. As in a previous paper (Rundle & Donnellan, 2020, https://doi.org/10.1029/2020ea001097), we find that this method produces a nowcasting timeseries that resembles the hypothesized regional stress accumulation and release process characterizing the earthquake cycle. We then address the problem of whether the timeseries contain information regarding future large earthquakes. For this, we compute a receiver operating characteristic and determine the decision thresholds for several future time periods of interest (optimization via supervised machine learning). We find that signals can be detected that can be used to characterize the information content of the timeseries. These signals may be useful in assessing present and near-future seismic hazards.

The ε-machine is a stochastic process's optimal model-maximally predictive and minimal in size. It often happens that to optimally predict even simply defined processes, probabilistic models-including the ε-machine-must employ an uncountably infinite set of features. To constructively work with these infinite sets we map the ε-machine to a place-dependent iterated function system (IFS)-a stochastic dynamical system. We then introduce the ambiguity rate that, in conjunction with a process's Shannon entropy rate, determines the rate at which this set of predictive features must grow to maintain maximal predictive power over increasing horizons. We demonstrate, as an ancillary technical result that stands on its own, that the ambiguity rate is the (until now missing) correction to the Lyapunov dimension of an IFS's attracting invariant set. For a broad class of complex processes, this then allows calculating their statistical complexity dimension-the information dimension of the minimal set of predictive features.

We experimentally demonstrate that highly structured distributions of work emerge during even the simple task of erasing a single bit. These are signatures of a refined suite of time-reversal symmetries in distinct functional classes of microscopic trajectories. As a consequence, we introduce a broad family of conditional fluctuation theorems that the component work distributions must satisfy. Since they identify entropy production, the component work distributions encode the frequency of various mechanisms of both success and failure during computing, as well giving improved estimates of the total irreversibly dissipated heat. This new diagnostic tool provides strong evidence that thermodynamic computing at the nanoscale can be constructively harnessed. We experimentally verify this functional decomposition and the new class of fluctuation theorems by measuring transitions between flux states in a superconducting circuit.

Even simply defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains, the consequences are dramatic. Their predictive models are generically infinite state. Until recently, one could determine neither their intrinsic randomness nor structural complexity. The prequel to this work introduced methods to accurately calculate the Shannon entropy rate (randomness) and to constructively determine their minimal (though, infinite) set of predictive features. Leveraging this, we address the complementary challenge of determining how structured hidden Markov processes are by calculating their statistical complexity dimension-the information dimension of the minimal set of predictive features. This tracks the divergence rate of the minimal memory resources required to optimally predict a broad class of truly complex processes.

Multiregional input-output (MRIO) tables, in conjunction with Leontiefanalysis, are widely-used to assess the geographical distribution of carbonemissions and the economic activities that cause them. Majorization, a tooloriginating in economics that has found utility in statistical mechanics, canprovide insight into how Leontief analysis links disparities in emissions withglobal income inequality. We examine Leontief analysis as a model, drawing outsimilarities with modern nonequilibrium statistical mechanics. Paralleling thephysical concept of thermo-majorization, we define the concept ofeco-majorization and show it is a sufficient condition to determine thedirectionality of embodied emission flows. Surprisingly, relatively small tradedeficits and a geographically heterogeneous emissions-per-dollar ratio greatlyincreases the appearance of eco-majorization, regardless of any further contentin the MRIO tables used. Our results are bolstered by a statistical analysis ofnull models of MRIO tables, based on data provided by the Global TradeAggregation Project9

In quantum mechanics, states are described by density matrices. Though their probabilistic interpretation is rooted in ensemble theory, density matrices embody a known shortcoming. They do not completely express the physical realization of an ensemble. Conveniently, the outcome statistics of projective and positive operator-valued measurements do not depend on the ensemble realization, only on the density matrix. Here, we show how the geometric approach to quantum mechanics tracks ensemble realizations. We do so in two concrete cases of a finite-dimensional quantum system interacting with another one with (i) finite-dimensional Hilbert space, relevant for quantum thermodynamics, and (ii) infinite-dimensional Hilbert space, relevant for state-manipulation protocols.

Practical, useful computations are instantiated via physical processes. Information must be stored and updated within a system's configurations, whose energetics determine a computation's cost. To describe thermodynamic and biological information processing, a growing body of results embraces rate equations as the underlying mechanics of computation. Strictly applying these continuous-time stochastic Markov dynamics, however, precludes a universe of natural computing. Within this framework, operations as simple as a NOT gate (flipping a bit) and swapping two bits, and swapping bits are inaccessible. We show that expanding the toolset to continuous-time hidden Markov dynamics substantially removes the constraints, by allowing information to be stored in a system's latent states. We demonstrate this by simulating computations that are impossible to implement without hidden states. We design and analyze a thermodynamically costless bit flip, providing a counterexample to rate-equation modeling. We generalize this to a costless Fredkin gate - a key operation in reversible computing that is Turing complete (computation universal). Going beyond rate-equation dynamics is not only possible but also necessary if stochastic thermodynamics is to become part of the paradigm for physical information processing.

The $\epsilon$-machine is a stochastic process' optimal model -- maximallypredictive and minimal in size. It often happens that to optimally predict evensimply-defined processes, probabilistic models -- including the$\epsilon$-machine -- must employ an uncountably-infinite set of features. Toconstructively work with these infinite sets we map the $\epsilon$-machine to aplace-dependent iterated function system (IFS) -- a stochastic dynamicalsystem. We then introduce the ambiguity rate that, in conjunction with aprocess' Shannon entropy rate, determines the rate at which this set ofpredictive features must grow to maintain maximal predictive power. Wedemonstrate, as an ancillary technical result which stands on its own, that theambiguity rate is the (until now missing) correction to the Lyapunov dimensionof an IFS's attractor. For a broad class of complex processes and for the firsttime, this then allows calculating their statistical complexity dimension --the information dimension of the minimal set of predictive features.

Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously complicated, however, even if the chain is finite state: no finite expression for their Shannon entropy rate exists, as the set of their predictive features is generically infinite. As such, to date one cannot make general statements about how random they are nor how structured. Here, we address the first part of this challenge by showing how to efficiently and accurately calculate their entropy rates. We also show how this method gives the minimal set of infinite predictive features. A sequel addresses the challenge’s second part on structure.

Nonequilibrium information thermodynamics determines the minimum energy dissipation to reliably erase memory under time-symmetric control protocols. We demonstrate that its bounds are tight and so show that the costs overwhelm those implied by Landauer’s energy bound on information erasure. Moreover, in the limit of perfect computation, the costs diverge. The conclusion is that time-asymmetric protocols should be developed for efficient, accurate thermodynamic computing. And, that Landauer’s Stack—the full suite of theoretically-predicted thermodynamic costs—is ready for experimental test and calibration.

Even simply-defined, finite-state generators produce stochastic processesthat require tracking an uncountable infinity of probabilistic features foroptimal prediction. For processes generated by hidden Markov chains theconsequences are dramatic. Their predictive models are genericallyinfinite-state. And, until recently, one could determine neither theirintrinsic randomness nor structural complexity. The prequel, though, introducedmethods to accurately calculate the Shannon entropy rate (randomness) and toconstructively determine their minimal (though, infinite) set of predictivefeatures. Leveraging this, we address the complementary challenge ofdetermining how structured hidden Markov processes are by calculating theirstatistical complexity dimension -- the information dimension of the minimalset of predictive features. This tracks the divergence rate of the minimalmemory resources required to optimally predict a broad class of truly complexprocesses.

We uncovered an error in 1 that claims the predictive rate-distortion function separates achievable from unachievable sensors.

Power spectral densities are a common, convenient, and powerful way to analyze signals, so much so that they are now broadly deployed across the sciences and engineering - from quantum physics to cosmology and from crystallography to neuroscience to speech recognition. The features they reveal not only identify prominent signal frequencies but also hint at mechanisms that generate correlation and lead to resonance. Despite their near-centuries-long run of successes in signal analysis, here we show that flat power spectra can be generated by highly complex processes, effectively hiding all inherent structure in complex signals. Historically, this circumstance has been widely misinterpreted, being taken as the renowned signature of "structureless"white noise - the benchmark of randomness. We argue, in contrast, to the extent that most real-world complex systems exhibit correlations beyond pairwise statistics their structures evade power spectra and other pairwise statistical measures. As concrete physical examples, we demonstrate that fraudulent white noise hides the predictable structure of both entangled quantum systems and chaotic crystals. To make these words of warning operational, we present constructive results that explore how this situation comes about and the high toll it takes in understanding complex mechanisms. First, we give the closed-form solution for the power spectrum of a very broad class of structurally complex signal generators. Second, we demonstrate the close relationship between eigenspectra of evolution operators and power spectra. Third, we characterize the minimal generative structure implied by any power spectrum. Fourth, we show how to construct arbitrarily complex processes with flat power spectra. Finally, leveraging this diagnosis of the problem, we point the way to developing more incisive tools for discovering structure in complex signals.

Given a classical channel—a stochastic map from inputs to outputs—the input can often be transformed into an intermediate variable that is informationally smaller than the input. The new channel accurately simulates the original but at a smaller transmission rate. Here, we examine this procedure when the intermediate variable is a quantum state. We determine when and how well quantum simulations of classical channels may improve upon the minimal rates of classical simulation. This inverts Holevo’s original question of quantifying the capacity of quantum channels with classical resources: We determine the lowest-capacity quantum channel required to simulate a classical channel. We also show that this problem is equivalent to another, involving the local generation of a distribution from common entanglement.

We merge computational mechanics' definition of causal states(predictively-equivalent histories) with reproducing-kernel Hilbert space(RKHS) representation inference. The result is a widely-applicable method thatinfers causal structure directly from observations of a system's behaviorswhether they are over discrete or continuous events or time. A structuralrepresentation -- a finite- or infinite-state kernel $\epsilon$-machine -- isextracted by a reduced-dimension transform that gives an efficientrepresentation of causal states and their topology. In this way, the systemdynamics are represented by a stochastic (ordinary or partial) differentialequation that acts on causal states. We introduce an algorithm to estimate theassociated evolution operator. Paralleling the Fokker-Plank equation, itefficiently evolves causal-state distributions and makes predictions in theoriginal data space via an RKHS functional mapping. We demonstrate thesetechniques, together with their predictive abilities, on discrete-time,discrete-value infinite Markov-order processes generated by finite-state hiddenMarkov models with (i) finite or (ii) uncountably-infinite causal states and(iii) continuous-time, continuous-value processes generated by thermally-drivenchaotic flows. The method robustly estimates causal structure in the presenceof varying external and measurement noise levels and for very high dimensionaldata.

We merge computational mechanics' definition of causal states(predictively-equivalent histories) with reproducing-kernel Hilbert space(RKHS) representation inference. The result is a widely-applicable method thatinfers causal structure directly from observations of a system's behaviorswhether they are over discrete or continuous events or time. A structuralrepresentation -- a finite- or infinite-state kernel $\epsilon$-machine -- isextracted by a reduced-dimension transform that gives an efficientrepresentation of causal states and their topology. In this way, the systemdynamics are represented by a stochastic (ordinary or partial) differentialequation that acts on causal states. We introduce an algorithm to estimate theassociated evolution operator. Paralleling the Fokker-Plank equation, itefficiently evolves causal-state distributions and makes predictions in theoriginal data space via an RKHS functional mapping. We demonstrate thesetechniques, together with their predictive abilities, on discrete-time,discrete-value infinite Markov-order processes generated by finite-state hiddenMarkov models with (i) finite or (ii) uncountably-infinite causal states and(iii) a continuous-time, continuous-value process generated by athermally-driven chaotic flow. The method robustly estimates causal structurein the presence of varying external and measurement noise levels.

We discuss the Szilard engine whose working fluid consists of a singlequantum-mechanical particle. Following Szilard's original solution of Maxwell'sSecond Law paradox, which turned on physically instantiating the demon (controlsubsystem), the quantum engine's design parallels the classically-chaoticSzilard Map that operates a thermodynamic cycle of measurement, thermal-energyextraction, and memory reset. We analyze in detail the dynamical mechanisms bywhich the quantum engine operates, both its thermodynamic costs and therequired information processing to observe and control the particle, comparingthese in the quantum, semiclassical, and classical limits. We establishLandauer Principles for information-processing-induced thermodynamicdissipation in the quantum and semiclassical regimes.

Local causal states are latent representations that capture organized patternand structure in complex spatiotemporal systems. We expand their functionality,framing them as spacetime autoencoders. Previously, they were only consideredas maps from observable spacetime fields to latent local causal state fields.Here, we show that there is a stochastic decoding that maps back from thelatent fields to observable fields. Furthermore, their Markovian propertiesdefine a stochastic dynamic in the latent space. Combined with stochasticdecoding, this gives a new method for forecasting spacetime fields.

All computation is physically embedded. Reflecting this, a growing body ofresults embraces rate equations as the underlying mechanics of thermodynamiccomputation and biological information processing. Strictly applying theimplied continuous-time Markov chains, however, excludes a universe of naturalcomputing. We show that expanding the toolset to continuous-time hidden Markovchains substantially removes the constraints. The general point is madeconcrete by our analyzing two eminently-useful computations that are impossibleto describe with a set of rate equations over the memory states. We design andanalyze a thermodynamically-costless bit flip, providing a first counterexampleto rate-equation modeling. We generalize this to a costless Fredkin gate---akey operation in reversible computing that is computation universal. Goingbeyond rate-equation dynamics is not only possible, but necessary if stochasticthermodynamics is to become part of the paradigm for physical informationprocessing.

When an experimentalist measures a time series of qubits, the outcomes constitute a classical stochastic process. We show that projective measurement induces high complexity in these processes in two specific senses: They are inherently random (finite Shannon entropy rate) and they require infinite memory for optimal prediction (divergent statistical complexity). We identify nonorthogonality of the quantum states as the mechanism underlying the resulting complexities and examine the influence that measurement choice has on the randomness and structure of measured qubit processes. We introduce quantitative measures of this complexity and provide efficient algorithms for their estimation.

When an experimentalist measures a time series of qubits, the outcomes constitute a classical stochastic process. We show that projective measurement induces high complexity in these processes in two specific senses: They are inherently random (finite Shannon entropy rate) and they require infinite memory for optimal prediction (divergent statistical complexity). We identify nonorthogonality of the quantum states as the mechanism underlying the resulting complexities and examine the influence that measurement choice has on the randomness and structure of measured qubit processes. We introduce quantitative measures of this complexity and provide efficient algorithms for their estimation.

Szilard's now-famous single-molecule engine was only the first of three constructions he introduced in 1929 to resolve several challenges arising from Maxwell's demon paradox. Given that it has been thoroughly analyzed, we analyze Szilard's remaining two demon models. We show that the second one, though a markedly different implementation employing a population of distinct molecular species and semipermeable membranes, is informationally and thermodynamically equivalent to an ideal gas of the single-molecule engines. One concludes that (i) it reduces to a chaotic dynamical system-called the Szilard Map, a composite of three piecewise linear maps and associated thermodynamic transformations that implement measurement, control, and erasure; (ii) its transitory functioning as an engine that converts disorganized heat energy to work is governed by the Kolmogorov-Sinai entropy rate; (iii) the demon's minimum necessary "intelligence" for optimal functioning is given by the engine's statistical complexity; and (iv) its functioning saturates thermodynamic bounds and so it is a minimal, optimal implementation. We show that Szilard's third construction is rather different and addresses the fundamental issue raised by the first two: the link between entropy production and the measurement task required to implement either of his engines. The analysis gives insight into designing and implementing novel nanoscale information engines by investigating the relationships between the demon's memory, the nature of the "working fluid," and the thermodynamic costs of erasure and measurement.

Modern digital electronics support remarkably reliable computing, especially given the challenge of controlling nanoscale logical components that interact in fluctuating environments. However, we demonstrate that the high-reliability limit is subject to a fundamental error-energy-efficiency tradeoff that arises from time-symmetric control: Requiring a low probability of error causes energy consumption to diverge as the logarithm of the inverse error rate for nonreciprocal logical transitions. The reciprocity (self-invertibility) of a computation is a stricter condition for thermodynamic efficiency than logical reversibility (invertibility), the latter being the root of Landauer's work bound on erasing information. Beyond engineered computation, the results identify a generic error-dissipation tradeoff in steady-state transformations of genetic information carried out by biological organisms. The lesson is that computational dissipation under time-symmetric control cannot reach, and is often far above, the Landauer limit. In this way, time-asymmetry becomes a design principle for thermodynamically efficient computing.

Building on parallels between geometric quantum mechanics and classicalmechanics, we explore an alternative basis for quantum thermodynamics thatexploits the differential geometry of the underlying state space. We developboth microcanonical and canonical ensembles, introducing continuous mixedstates as distributions on the manifold of quantum states. We call out theexperimental consequences for a gas of qudits. We define quantum heat and workin an intrinsic way, including single-trajectory work, and reformulatethermodynamic entropy in a way that accords with classical, quantum, andinformation-theoretic entropies. We give both the First and Second Laws ofThermodynamics and Jarzynki's Fluctuation Theorem. The result is a moretransparent physics, than conventionally available, in which the mathematicalstructure and physical intuitions underlying classical and quantum dynamics areseen to be closely aligned.

Density matrices capture all of a quantum system's statistics accessiblethrough projective and positive operator-valued measurements. They do notcompletely determine its state, however, as they neglect the physicalrealization of ensembles. Fortunately, the concept of geometric quantum statedoes properly describe physical ensembles. Here, given knowledge of a densitymatrix, possibly arising from a tomography protocol, we show how to estimatethe geometric quantum state using a maximum entropy principle based on ageometrically-appropriate entropy.

Maxwellian ratchets are autonomous, finite-state thermodynamic engines that implement input-output informational transformations. Previous studies of these "demons"focused on how they exploit environmental resources to generate work: They randomize ordered inputs, leveraging increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir while respecting both Liouvillian state-space dynamics and the second law. However, to date, correctly determining such functional thermodynamic operating regimes was restricted to a very few engines for which correlations among their information-bearing degrees of freedom could be calculated exactly and in closed form, a highly restricted set. Additionally, a key second dimension of ratchet behavior was largely ignored: Ratchets do not merely change the randomness of environmental inputs, their operation constructs and deconstructs patterns. To address both dimensions, we adapt recent results from dynamical-systems and ergodic theories that efficiently and accurately calculate the entropy rates and the rate of statistical complexity divergence of general hidden Markov processes. In concert with the information processing second law, these methods accurately determine thermodynamic operating regimes for finite-state Maxwellian demons with arbitrary numbers of states and transitions. In addition, they facilitate analyzing structure versus randomness tradeoffs that a given engine makes. The result is a greatly enhanced perspective on the information processing capabilities of information engines. As an application, we give a thorough-going analysis of the Mandal-Jarzynski ratchet, demonstrating that it has an uncountably infinite effective state space.

Quantum coherence allows for reduced-memory simulators of classical processes. Using recent results in single-shot quantum thermodynamics, we derive a minimal work cost rate for quantum simulators that is quasistatically attainable in the limit of asymptotically infinite parallel simulation. Comparing this cost with the classical regime reveals that quantizing classical simulators not only results in memory compression but also in reduced dissipation. We explore this advantage across a suite of representative examples.

Quantum coherence allows for reduced-memory simulators of classical processes. Using recent results in single-shot quantum thermodynamics, we derive a minimal work cost rate for quantum simulators that is quasistatically attainable in the limit of asymptotically infinite parallel simulation. Comparing this cost with the classical regime reveals that quantizing classical simulators not only results in memory compression but also in reduced dissipation. We explore this advantage across a suite of representative examples.

Modularity dissipation identifies how locally implemented computation entails costs beyond those required by Landauer's bound on thermodynamic computing. We establish a general theorem for efficient local computation, giving the necessary and sufficient conditions for a local operation to have zero modularity cost. Applied to thermodynamically-generating stochastic processes it confirms a conjecture that classical generators are efficient if and only if they satisfy retrodiction, which places minimum-memory requirements on the generator. This extends immediately to quantum computation: Any quantum simulator that employs quantum memory compression cannot be thermodynamically efficient.

Short-term survival and an exuberant plunge into building our future aregenerating a new kind of unintended consequence -- hidden fragility. This is adirect effect of the sophistication and structural complexity of thesocio-technical systems humans create. It is inevitable. And so the challengeis, How much can we understand and predict about these systems and about thesocial dynamics that lead to their construction?

We implement a thermal-fluctuation-driven logical bit reset on a superconducting flux logic cell. We show that the logical state of the system can be continuously monitored with only a small perturbation to the thermally activated dynamics at 500 mK. We use the trajectory information to derive a single-shot estimate of the work performed on the system per logical cycle. We acquire a sample of 105 erasure trajectories per protocol and show that the work histograms agree with both microscopic theory and global fluctuation theorems. The results demonstrate how to design and diagnose complex, high-speed, and thermodynamically efficient computing using superconducting technology.