Search

We implement a Szilard engine using a 2-bit logical unit consisting of inductively coupled quantum flux parametrons (QFPs) -- Josephson-junction superconducting circuits with applications in both the classical and quantum information processing regimes. Detailed simulations show that it is highly thermodynamically efficient while functioning as a Maxwell demon -- converting heat to work. The physically-calibrated design is targeted to direct experimental exploration. However, variations in Josephson junction fabrication introduce asymmetries that result in energy inefficiency and low operational fidelity. We provide a design solution that mitigates these practical challenges. The resulting platform is ideally suited to probe the thermodynamic foundations of information processing devices far from equilibrium., Comment: 18 pages, 9 figures, 4 tables

Reducing the energy inefficiency of conventional CMOS-based computing devices -- which rely on logically irreversible gates to process information -- remains both a fundamental engineering challenge and a practical social challenge of increasing importance. We extend an alternative computing paradigm that manipulates microstate distributions to store information in the metastable minima determined by an effective potential energy landscape. These minima serve as mesoscopic memories that are manipulated by a dynamic landscape to perform information processing. Central to our results is the control erase (CE) protocol that controls the landscape's metastable minima to determine whether information is preserved or erased. Importantly, successive protocol executions can implement a NAND gate -- a logically-irreversible universal logic gate. We show how to practically implement this in a device created by two inductively-coupled superconducting quantum interference devices (SQUIDs). We identify circuit parameter ranges that give rise to effective CEs and establish the device's robustness against logical errors. These SQUID-based logical devices are capable of operating above GHz frequencies and at the $k_\text{B} T$ energy scale. Due to this, optimized devices and associated protocols provide a universal-computation substrate that is both computationally fast and energy efficient., Comment: 13 pages, 8 figures, 2 tables; https://csc.ucdavis.edu/~cmg/compmech/pubs/cebb.htm

Entropy production (EP) is a central quantity in nonequilibrium physics as it monitors energy dissipation, irreversibility, and free energy differences during thermodynamic transformations. Estimating EP, however, is challenging both theoretically and experimentally due to limited access to the system dynamics. For overdamped Langevin dynamics and Markov jump processes it was recently proposed that, from thermodynamic uncertainty relations (TUR), short-time cumulant currents can be used to estimate EP without knowledge of the dynamics. Yet, estimation of EP in underdamped Langevin systems remains an active challenge. To address this, we derive a modified TUR that relates the statistics of two specific novel currents -- one cumulant current and one stochastic current -- to a system's EP. These two distinct but related currents are used to constrain EP in the modified TUR. One highlight is that there always exists a family of currents such that the uncertainty relations saturate, even for long-time averages and in nonsteady-state scenarios. Another is that our method only requires limited knowledge of the dynamics -- specifically, the damping-coefficient to mass ratio and the diffusion constant. This uncertainty relation allows estimating EP for both overdamped and underdamped Langevin dynamics. We validate the method numerically, through applications to several underdamped systems, to underscore the flexibility in obtaining EP in nonequilibrium Langevin systems., Comment: 15 pages, 5 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/EstimateEP.htm

Efficiently harvesting thermodynamic resources requires a precise understanding of their structure. This becomes explicit through the lens of information engines -- thermodynamic engines that use information as fuel. Maximizing the work harvested using available information is a form of physically-instantiated machine learning that drives information engines to develop complex predictive memory to store an environment's temporal correlations. We show that an information engine's complex predictive memory poses both energetic benefits and risks. While increasing memory facilitates detection of hidden patterns in an environment, it also opens the possibility of thermodynamic overfitting, where the engine dissipates additional energy in testing. To address overfitting, we introduce thermodynamic regularizers that incur a cost to engine complexity in training due to the physical constraints on the information engine. We demonstrate that regularized thermodynamic machine learning generalizes effectively. In particular, the physical constraints from which regularizers are derived improve the performance of learned predictive models. This suggests that the laws of physics jointly create the conditions for emergent complexity and predictive intelligence.

Recording the undersea three-dimensional bioacoustic sound field in real-time promises major benefits to marine behavior studies. We describe a novel hydrophone array -- the hydroambiphone (HAP) -- that adapts ambisonic spatial-audio theory to sound propagation in ocean waters to realize many of these benefits through spatial localization and acoustic immersion. Deploying it to monitor the humpback whales (Megaptera novaeangliae) of southeast Alaska demonstrates that HAP recording provides a qualitatively-improved experience of their undersea behaviors; revealing, for example, new aspects of social coordination during bubble-net feeding. On the practical side, spatialized hydrophone recording greatly reduces post-field analytical and computational challenges -- such as the "cocktail party problem" of distinguishing single sources in a complicated and crowded auditory environment -- that are common to field recordings. On the scientific side, comparing the HAP's capabilities to single-hydrophone and nonspatialized recordings yields new insights into the spatial information that allows animals to thrive in complex acoustic environments. Spatialized bioacoustics markedly improves access to the humpbacks' undersea acoustic environment and expands our appreciation of their rich vocal lives., Comment: 11 pages, 4 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/whalesinspace.htm

After more than a century of concerted effort, physics still lacks basic principles of spontaneous self-organization. To appreciate why, we first state the problem, outline historical approaches, and survey the present state of the physics of self-organization. This frames the particular challenges arising from mathematical intractability and the resulting need for computational approaches, as well as those arising from a chronic failure to define structure. Then, an overview of two modern mathematical formulations of organization -- intrinsic computation and evolution operators -- lays out a way to overcome these challenges. Together, the vantage point they afford shows how to account for the emergence of structured states via a statistical mechanics of systems arbitrarily far from equilibrium. The result is a constructive path forward to principles of organization that builds on mathematical identification of structure., Comment: 51 pages, 7 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/tsfpoo.htm

Alternative computing paradigms open the door to exploiting recent innovations in computational hardware to probe the fundamental thermodynamic limits of information processing. One such paradigm employs superconducting quantum interference devices (SQUIDs) to execute classical computations. This, though, requires constructing sufficiently complex superconducting circuits that support a suite of useful information processing tasks and storage operations, as well as understanding these circuits' energetics. First-principle circuit design, though, leads to prohibitive algebraic complications when deriving the effective equations of motion -- complications that to date have precluded achieving these goals, let alone doing so efficiently. We circumvent these complications by (i) specializing our class of circuits and physical operating regimes, (ii) synthesizing existing derivation techniques to suit these specializations, and (iii) implementing solution-finding optimizations which facilitate physically interpreting circuit degrees of freedom that respect physically-grounded constraints. This leads to efficient, practical circuit prototyping and access to scalable circuit architectures. The analytical efficiency is demonstrated by reproducing the potential energy landscape generated by the quantum flux parametron (QFP). We then show how inductively coupling two QFPs produces a device that is capable of executing 2-bit computations via its composite potential energy landscape. More generally, the synthesis methods detailed here provide a basis for constructing universal logic gates and investigating their thermodynamic performance., Comment: 9 pages, 3 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/nds.htm

When reformulated as a resource theory, thermodynamics can analyze system behaviors in the single-shot regime. In this, the work required to implement state transitions is bounded by {\alpha}-Renyi divergences and so differs in identifying efficient operations compared to stochastic thermodynamics. Thus, a detailed understanding of the difference between stochastic and resource-theoretic thermodynamics is needed. To this end, we explore reversibility in the single-shot regime, generalizing the two-level work reservoirs used there to multi-level work reservoirs. This achieves reversibility in any transition in the single-shot regime. Building on this, we systematically develop multi-level work reservoirs in the nondissipation regime with and without catalysts. The resource-theoretic results show that two-level work reservoirs undershoot Landauer's bound, misleadingly implying energy dissipation during computation. In contrast, we demonstrate that multilevel work reservoirs achieve Landauer's bound while producing arbitrarily low entropy., Comment: 23 pages, 8 figures, 7 tables; https://csc.ucdavis.edu/~cmg/compmech/pubs/eqwratn.htm

Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically-grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice. Its building blocks are spacetime lightcones that embody how information propagates across a system through local interactions. We show that predictive equivalence classes of lightcones -- local causal states -- capture organized behaviors and coherent structures in complex spatiotemporal systems. Employing an unsupervised physics-informed machine learning algorithm and a high-performance computing implementation, we demonstrate automatically discovering coherent structures in two real world domain science problems. We show that local causal states identify vortices and track their power-law decay behavior in two-dimensional fluid turbulence. We then show how to detect and track familiar extreme weather events -- hurricanes and atmospheric rivers -- and discover other novel coherent structures associated with precipitation extremes in high-resolution climate data at the grid-cell level.

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a "next-generation" reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano's inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least ~ 60% higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines -- specifically, large $\epsilon$-machines -- are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures., Comment: 10 pages, 5 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/ngrc.htm

Stationary quantum information sources emit sequences of correlated qudits -- that is, structured quantum stochastic processes. If an observer performs identical measurements on a qudit sequence, the outcomes are a realization of a classical stochastic process. We introduce quantum-information-theoretic properties for separable qudit sequences that serve as bounds on the classical information properties of subsequent measured processes. For sources driven by hidden Markov dynamics we describe how an observer can temporarily or permanently synchronize to the source's internal state using specific positive operator-valued measures or adaptive measurement protocols. We introduce a method for approximating an information source with an independent and identically-distributed, Markov, or larger memory model through tomographic reconstruction. We identify broad classes of separable processes based on their quantum information properties and the complexity of measurements required to synchronize to and accurately reconstruct them., Comment: 39 pages, 25 figures; 1 appendix, 5 pages; https://csc.ucdavis.edu/~cmg/compmech/pubs/eparable_qudit_processes.html

Over several days in early August 2021, while at sea in Chatham Strait, Southeast Alaska, aboard M/Y Blue Pearl, an online twitch.tv stream broadcast in real-time humpback whale vocalizations monitored via hydrophone. Dozens on mainland North American and around the planet listened in and chatted via the stream. The webcasts demonstrated a proof-of-concept: only relatively inexpensive commercial-off-the-shelf equipment is required for remote mobile streaming at sea. These notes document what was required and make recommendations for higher-quality and larger-scale deployments. One conclusion is that real-time, automated audio documenting whale acoustic behavior is readily accessible and, using the cloud, it can be directly integrated into behavioral databases -- information sources that now often focus exclusively on nonreal-time visual-sighting narrative reports and photography., Comment: 6 pages, 3 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/whalecasting.html

The averaged steady-state surprisal links a driven stochastic system's information processing to its nonequilibrium thermodynamic response. By explicitly accounting for the effects of nonequilibrium steady states, a decomposition of the surprisal results in an information processing First Law that extends and tightens -- to strict equalities -- various information processing Second Laws. Applying stochastic thermodynamics' integral fluctuation theorems then shows that the decomposition reduces to the second laws under appropriate limits. In unifying them, the First Law paves the way to identifying the mechanisms by which nonequilibrium steady-state systems extract work from information-bearing degrees of freedom. To illustrate, we analyze an autonomous Maxwellian information ratchet that tunably violates detailed balance in its effective dynamics. This demonstrates how the presence of nonequilibrium steady states qualitatively alters an information engine's allowed functionality., Comment: 15 pages, 9 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/ipfl.htm

Thermodynamic uncertainty relations (TURs) express a fundamental tradeoff between the precision (inverse scaled variance) of any thermodynamic current by functionals of the average entropy production. Relying on purely variational arguments, we significantly extend these inequalities by incorporating and analyzing the impact of higher statistical cumulants of entropy production within a general framework of time-symmetrically controlled computation. This allows us to derive an exact expression for the current that achieves the minimum scaled variance, for which the TUR bound tightens to an equality that we name Thermodynamic Uncertainty Theorem (TUT). Importantly, both the minimum scaled variance current and the TUT are functionals of the stochastic entropy production, thus retaining the impact of its higher moments. In particular, our results show that, beyond the average, the entropy production distribution's higher moments have a significant effect on any current's precision. This is made explicit via a thorough numerical analysis of swap and reset computations that quantitatively compares the TUT against previous generalized TURs. Our results demonstrate how to interpolate between previously-established bounds and how to identify the most relevant TUR bounds in different nonequilibrium regimes.

Quantum Darwinism builds on decoherence theory to explain the emergence of classical behavior in a fundamentally quantum universe. Within this framework we prove two crucial insights about the emergence of classical phenomenology, centered around quantum discord as the measure of quantumness of correlations. First, we show that the so-called branching structure of the joint state of the system and environment is the only one compatible with zero discord. Second, we prove that for small but nonzero discord and for good but not perfect decoherence, the structure of the globally pure state must be arbitrarily close to the branching form, with each branch exhibiting low entanglement. Our results significantly improve on previous bounds and reinforce the existing evidence that this class of branching states is the only one compatible with the emergence of classical phenomenology, as described by Quantum Darwinism., Comment: 14 pages, 3 figures

Yes. Interval statistics have been used to conclude that major earthquakes are random events in time and cannot be anticipated or predicted. Machine learning is a powerful new technique that enhances our ability to understand the information content of earthquake catalogs. We show that catalogs contain significant information on current hazard and future predictability for large earthquakes., Comment: Preprint, submitted to Earth and Space Science

The Trajectory Class Fluctuation Theorem (TCFT) substantially strengthens the Second Law of Thermodynamics -- that, in point of fact, can be a rather weak bound on resource fluxes. Practically, it improves empirical estimates of free energies, a task known to be statistically challenging, and has diagnosed successful and failed information processing in experimentally-implemented Josephson-junction information engines. The development here justifies that empirical analysis, explicating its mathematical foundations. The TCFT reveals the thermodynamics induced by macroscopic system transformations for each measurable subset of system trajectories. In this, it directly combats the statistical challenge of extremely rare events that dominate thermodynamic calculations. And, it reveals new forms of free energy -- forms that can be solved for analytically and practically estimated. Conceptually, the TCFT unifies a host of previously-established fluctuation theorems, interpolating from Crooks' Detailed Fluctuation Theorem (single trajectories) to Jarzynski's Equality (trajectory ensembles)., Comment: 23 pages, 2 figures, 3 appendices; http://csc.ucdavis.edu/~cmg/compmech/pubs/tcft.htm

We generalize the exact predictive regularity of symmetry groups to give an algebraic theory of patterns, building from a core principle of future equivalence. For topological patterns in fully-discrete one-dimensional systems, future equivalence uniquely specifies a minimal semiautomaton. We demonstrate how the latter and its semigroup algebra generalizes translation symmetry to partial and hidden symmetries. This generalization is not as straightforward as previously considered. Here, though, we clarify the underlying challenges. A stochastic form of future equivalence, known as predictive equivalence, captures distinct statistical patterns supported on topological patterns. Finally, we show how local versions of future equivalence can be used to capture patterns in spacetime. As common when moving to higher dimensions, there is not a unique local approach, and we detail two local representations that capture different aspects of spacetime patterns. A previously-developed local spacetime variant of future equivalence captures patterns as generalized symmetries in higher dimensions, but we show this representation is not a faithful generator of its spacetime patterns. This motivates us to introduce a local representation that is a faithful generator, but we demonstrate that it no longer captures generalized spacetime symmetries. Taken altogether, building on future equivalence, the theory defines and quantifies patterns present in a wide range of classical field theories.

Predictive states for stochastic processes are a nonparametric and interpretable construct with relevance across a multitude of modeling paradigms. Recent progress on the self-supervised reconstruction of predictive states from time-series data focused on the use of reproducing kernel Hilbert spaces. Here, we examine how Wasserstein distances may be used to detect predictive equivalences in symbolic data. We compute Wasserstein distances between distributions over sequences ("predictions"), using a finite-dimensional embedding of sequences based on the Cantor for the underlying geometry. We show that exploratory data analysis using the resulting geometry via hierarchical clustering and dimension reduction provides insight into the temporal structure of processes ranging from the relatively simple (e.g., finite-state hidden Markov models) to the very complex (e.g., infinite-state indexed grammars)., Comment: 9 pages, 4 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/PredStateCantorWass.htm

Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system's future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel Maximum Entropy and Maximum Caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining implicit and explicit aspects.

Stochastic thermodynamics has largely succeeded in characterizing both equilibrium and far-from-equilibrium phenomena. Yet many opportunities remain for application to mesoscopic complex systems -- especially biological ones -- whose effective dynamics often violate detailed balance and whose microscopic degrees of freedom are often unknown or intractable. After reviewing excess and housekeeping energetics -- the adaptive and homeostatic components of a system's dissipation -- we extend stochastic thermodynamics with a trajectory class fluctuation theorem for nonequilibrium steady-state, nondetailed-balanced complex systems. We then take up the neurobiological examples of voltage-gated sodium and potassium ion channels to apply and illustrate the theory, elucidating their nonequilibrium behavior under a biophysically plausible action potential drive. These results uncover challenges for future experiments and highlight the progress possible understanding the thermodynamics of complex systems -- without exhaustive knowledge of every underlying degree of freedom., Comment: 17 pages, 9 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/toic.htm

We introduce a fast and highly-efficient physically-realizable bit swap. Employing readily available and scalable Josephson junction microtechnology, the design implements the recently introduced paradigm of momentum computing. Its nanosecond speeds and sub-Landauer thermodynamic efficiency arise from dynamically storing memory in momentum degrees of freedom. As such, during the swap, the microstate distribution is never near equilibrium and the memory-state dynamics fall far outside of stochastic thermodynamics that assumes detailed-balanced Markovian dynamics. The device implements a bit-swap operation -- a fundamental operation necessary to build reversible universal computing. Extensive, physically-calibrated simulations demonstrate that device performance is robust and that momentum computing can support thermodynamically-efficient, high-speed, large-scale general-purpose computing that circumvents Landauer's bound., Comment: 18 pages, 11 figures, 5 appendices; http://csc.ucdavis.edu/~cmg/compmech/pubs/gslmc.htm

Geometric quantum mechanics, through its differential-geometric underpinning, provides additional tools of analysis and interpretation that bring quantum mechanics closer to classical mechanics: state spaces in both are equipped with symplectic geometry. This opens the door to revisiting foundational questions and issues, such as the nature of quantum entropy, from a geometric perspective. Central to this is the concept of geometric quantum state -- the probability measure on a system's space of pure states. This space's continuity leads us to introduce two analysis tools, inspired by Renyi's information theory, to characterize and quantify fundamental properties of geometric quantum states: the quantum information dimension that is the rate of geometric quantum state compression and the dimensional geometric entropy that monitors information stored in quantum states. We recount their classical definitions, information-theoretic meanings, and physical interpretations, and adapt them to quantum systems via the geometric approach. We then explicitly compute them in various examples and classes of quantum system. We conclude commenting on future directions for information in geometric quantum mechanics., Comment: 19 pages, 11 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/qidge.htm

Predictive equivalence in discrete stochastic processes have been applied with great success to identify randomness and structure in statistical physics and chaotic dynamical systems and to inferring hidden Markov models. We examine the conditions under which they can be reliably reconstructed from time-series data, showing that convergence of predictive states can be achieved from empirical samples in the weak topology of measures. Moreover, predictive states may be represented in Hilbert spaces that replicate the weak topology. We mathematically explain how these representations are particularly beneficial when reconstructing high-memory processes and connect them to reproducing kernel Hilbert spaces., Comment: 16 pages, 4 figures; http://csc.ucdavis.edu/~cmg/papers/gartsp.pdf

Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequality. We examine Leontief analysis as a model, drawing out similarities with modern nonequilibrium statistical mechanics. Paralleling the physical concept of thermo-majorization, we define the concept of eco-majorization and show it is a sufficient condition to determine the directionality of embodied emission flows. Surprisingly, relatively small trade deficits and a geographically heterogeneous emissions-per-dollar ratio greatly increases the appearance of eco-majorization, regardless of any further content in the MRIO tables used. Our results are bolstered by a statistical analysis of null models of MRIO tables, based on data provided by the Global Trade Aggregation Project9, Comment: 14 pages, 5 figures; 1 appendix; http://csc.ucdavis.edu/~cmg/compmech/pubs/netacam.htm

The $\epsilon$-machine is a stochastic process' optimal model -- maximally predictive and minimal in size. It often happens that to optimally predict even simply-defined processes, probabilistic models -- including the $\epsilon$-machine -- must employ an uncountably-infinite set of features. To constructively work with these infinite sets we map the $\epsilon$-machine to a place-dependent iterated function system (IFS) -- a stochastic dynamical system. We then introduce the ambiguity rate that, in conjunction with a process' Shannon entropy rate, determines the rate at which this set of predictive features must grow to maintain maximal predictive power. We demonstrate, as an ancillary technical result which stands on its own, that the ambiguity rate is the (until now missing) correction to the Lyapunov dimension of an IFS's attractor. For a broad class of complex processes and for the first time, this then allows calculating their statistical complexity dimension -- the information dimension of the minimal set of predictive features., Comment: 13 pages, 9 figures; supplementary material, 3 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/arhmp.htm

While Landauer's Principle sets a lower bound for the work required for a computation, that work is recoverable for efficient computations. However, practical physical computers, such as modern digital computers or biochemical systems, are subject to constraints that make them inefficient -- irreversibly dissipating significant energy. Recent results show that the dissipation in such systems is bounded by the nonreciprocity of the embedded computation. We investigate the consequences of this bound for different types of memory, showing that different memory devices are better suited for different computations. This correspondence comes from the time-reversal symmetries of the memory, which depend on whether information is stored positionally or magnetically. This establishes that the time symmetries of the memory device play an essential roll in determining energetics. The energetic consequences of time symmetries are particularly pronounced in nearly deterministic computations, where the cost of computing diverges as minus log of the error rate. We identify the coefficient of that divergence as the dissipation divergence. We find that the dissipation divergence may be zero for a computation when implemented in one type of memory while it's maximal when implemented with another. Given flexibility in the type of memory, the dissipation divergence is bounded below by the average state compression of the computation. Moreover, we show how to explicitly construct the memory to achieve this minimal dissipation. As a result, we find that logically reversible computations are indeed thermodynamically efficient, but logical irreversibility comes at a much higher cost than previously anticipated.

Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are dramatic. Their predictive models are generically infinite-state. And, until recently, one could determine neither their intrinsic randomness nor structural complexity. The prequel, though, introduced methods to accurately calculate the Shannon entropy rate (randomness) and to constructively determine their minimal (though, infinite) set of predictive features. Leveraging this, we address the complementary challenge of determining how structured hidden Markov processes are by calculating their statistical complexity dimension -- the information dimension of the minimal set of predictive features. This tracks the divergence rate of the minimal memory resources required to optimally predict a broad class of truly complex processes., Comment: 16 pages, 6 figures; Supplementary Material, 6 pages, 2 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/icfshmp.htm

Pairwise interactions between individuals are taken as fundamental drivers of collective behavior responsible for group cohesion and decision-making. While an individual directly influences only a few neighbors, over time indirect influences penetrate a much larger group. The abiding question is how this spread of influence comes to affect the collective. One or a few individuals are often identified as leaders, being more influential than others. Transfer entropy and time-delayed mutual information are used to identify underlying asymmetric interactions, such as leader-follower classification in aggregated individuals--cells, birds, fish, and animals. However, these conflate distinct functional modes of information flow between individuals. Computing information measures conditioning on multiple agents requires the proper sampling of a probability distribution whose dimension grows exponentially with the number of agents being conditioned on. Employing simple models of interacting self-propelled particles, we examine the pitfalls of using time-delayed mutual information and transfer entropy to quantify the strength of influence from a leader to a follower. Surprisingly, one must be wary of these pitfalls even for two interacting particles. As an alternative we decompose transfer entropy and time-delayed mutual information into intrinsic, shared, and synergistic modes of information flow. The result not only properly reveals the underlying effective interactions, but also facilitates a more detailed diagnosis of how individual interactions lead to collective behavior. This exposes the role of individual and group memory in collective behaviors. In addition, we demonstrate in a multi-agent system how knowledge of the decomposed information modes between a single pair of agents reveals the nature of many-body interactions without conditioning on additional agents.

Nonequilibrium information thermodynamics determines the minimum energy dissipation to reliably erase memory under time-symmetric control protocols. We demonstrate that its bounds are tight and so show that the costs overwhelm those implied by Landauer's energy bound on information erasure. Moreover, in the limit of perfect computation, the costs diverge. The conclusion is that time-asymmetric protocols should be developed for efficient, accurate thermodynamic computing. And, that Landauer's Stack -- the full suite of theoretically-predicted thermodynamic costs -- is ready for experimental test and calibration., Comment: 12 pages, 6 figures, 1 table; http://csc.ucdavis.edu/~cmg/compmech/pubs/tsperase.htm. arXiv admin note: substantial text overlap with arXiv:1909.06650

We merge computational mechanics' definition of causal states (predictively-equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely-applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation -- a finite- or infinite-state kernel $\epsilon$-machine -- is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Plank equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably-infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally-driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high dimensional data., Comment: 23 pages, 11 figures, 64 citations; https://team.inria.fr/comcausa/continuous-causal-states/

We analyze an engine whose working fluid consists of a single quantum particle, paralleling Szilard's construction of a classical single-particle engine. Following his resolution of Maxwell's Second Law paradox using the latter, which turned on physically instantiating the demon (control subsystem), the quantum engine's design mirrors the classically-chaotic Szilard Map that operates a thermodynamic cycle of measurement, thermal-energy extraction, and memory reset. Focusing on the thermodynamic costs to observe and control the particle and comparing these in the quantum and classical limits, we detail the thermodynamic tradeoffs behind Landauer's Principle for information-processing-induced thermodynamic dissipation in the quantum and classical regimes. In particular, and as found with the classical engine, we show that the sum of the thermodynamic costs over a cycle obeys a generalized Landauer Principle, exactly balancing energy extraction from the heat bath. Thus, the quantum engine obeys the Second Law. However, the quantum engine does so via substantially different mechanisms: classically measurement and erasure determine the thermodynamics, while in the quantum implementation the cost of partition insertion is key., Comment: 10 pages; 9 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/qse_letter.htm

Local causal states are latent representations that capture organized pattern and structure in complex spatiotemporal systems. We expand their functionality, framing them as spacetime autoencoders. Previously, they were only considered as maps from observable spacetime fields to latent local causal state fields. Here, we show that there is a stochastic decoding that maps back from the latent fields to observable fields. Furthermore, their Markovian properties define a stochastic dynamic in the latent space. Combined with stochastic decoding, this gives a new method for forecasting spacetime fields.

All computation is physically embedded. Reflecting this, a growing body of results embraces rate equations as the underlying mechanics of thermodynamic computation and biological information processing. Strictly applying the implied continuous-time Markov chains, however, excludes a universe of natural computing. We show that expanding the toolset to continuous-time hidden Markov chains substantially removes the constraints. The general point is made concrete by our analyzing two eminently-useful computations that are impossible to describe with a set of rate equations over the memory states. We design and analyze a thermodynamically-costless bit flip, providing a first counterexample to rate-equation modeling. We generalize this to a costless Fredkin gate---a key operation in reversible computing that is computation universal. Going beyond rate-equation dynamics is not only possible, but necessary if stochastic thermodynamics is to become part of the paradigm for physical information processing., Comment: 6 pages, 3 figures; Supplementary Material, 1 page; http://csc.ucdavis.edu/~cmg/compmech/pubs/cbdb.htm

Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously complicated, however, even if the chain is finite state: no finite expression for their Shannon entropy rate exists, as the set of their predictive features is generically infinite. As such, to date one cannot make general statements about how random they are nor how structured. Here, we address the first part of this challenge by showing how to efficiently and accurately calculate their entropy rates. We also show how this method gives the minimal set of infinite predictive features. A sequel addresses the challenge's second part on structure., Comment: 11 pages, 4 figures; supplementary material 10 pages, 7 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/serhmp.htm

Building on parallels between geometric quantum mechanics and classical mechanics, we explore an alternative basis for quantum thermodynamics that exploits the differential geometry of the underlying state space. We develop both microcanonical and canonical ensembles, introducing continuous mixed states as distributions on the manifold of quantum states. We call out the experimental consequences for a gas of qudits. We define quantum heat and work in an intrinsic way, including single-trajectory work, and reformulate thermodynamic entropy in a way that accords with classical, quantum, and information-theoretic entropies. We give both the First and Second Laws of Thermodynamics and Jarzynki's Fluctuation Theorem. The result is a more transparent physics, than conventionally available, in which the mathematical structure and physical intuitions underlying classical and quantum dynamics are seen to be closely aligned., Comment: 10 pages, 1 figure; Supplementary Material: 7 pages; http://csc.ucdavis.edu/~cmg/compmech/pubs/gqt.htm

Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally suitable, possibilities. Following Jaynes' information-theoretic perspective, this can be framed as an inference problem. We propose the Maximum Geometric Quantum Entropy Principle to exploit the notions of Quantum Information Dimension and Geometric Quantum Entropy. These allow us to quantify the entropy of fully arbitrary ensembles and select the one that maximizes it. After formulating the principle mathematically, we give the analytical solution to the maximization problem in a number of cases and discuss the physical mechanism behind the emergence of such maximum entropy ensembles., Comment: 11 pages, no figures; Supplementary Material: 2 appendices; http://csc.ucdavis.edu/~cmg/compmech/pubs/qgse.htm

A quantum system's state is identified with a density matrix. Though their probabilistic interpretation is rooted in ensemble theory, density matrices embody a known shortcoming. They do not completely express an ensemble's physical realization. Conveniently, when working only with the statistical outcomes of projective and positive operator-valued measurements this is not a hindrance. To track ensemble realizations and so remove the shortcoming, we explore geometric quantum states and explain their physical significance. We emphasize two main consequences: one in quantum state manipulation and one in quantum thermodynamics., Comment: 7 pages, 1 figure; Supplementary Material: 2 pages; http://csc.ucdavis.edu/~cmg/compmech/pubs/rgqs.htm

Many natural, engineered, and social systems can be represented using the framework of a layered network, where each layer captures a different type of interaction between the same set of nodes. The study of such multiplex networks is a vibrant area of research. Yet, understanding how to quantify the correlations present between pairs of layers, and more so present in their co-evolution, is lacking. Such methods would enable us to address fundamental questions involving issues such as function, redundancy and potential disruptions. Here we show first how the edge-set of a multiplex network can be used to construct an estimator of a joint probability distribution describing edge existence over all layers. We then adapt an information-theoretic measure of general correlation called the conditional mutual information, which uses the estimated joint probability distribution, to quantify the pairwise correlations present between layers. The pairwise comparisons can also be temporal, allowing us to identify if knowledge of a certain layer can provide additional information about the evolution of another layer. We analyze datasets from three distinct domains---economic, political, and airline networks---to demonstrate how pairwise correlation in structure and dynamical evolution between layers can be identified and show that anomalies can serve as potential indicators of major events such as shocks., Comment: Accepted to Journal of Complex Networks

Short-term survival and an exuberant plunge into building our future are generating a new kind of unintended consequence -- hidden fragility. This is a direct effect of the sophistication and structural complexity of the socio-technical systems humans create. It is inevitable. And so the challenge is, How much can we understand and predict about these systems and about the social dynamics that lead to their construction?, Comment: 11 pages; http://csc.ucdavis.edu/~cmg/compmech/pubs/focs.htm

Szilard's now-famous single-molecule engine was only the first of three constructions he introduced in 1929 to resolve several paradoxes arising from Maxwell's demon. We analyze Szilard's remaining two demon models. We show that the second one, though a markedly different implementation employing a population of distinct molecular species and semi-permeable membranes, is informationally and thermodynamically equivalent to an ideal gas of the single-molecule engines. Since it is a gas of noninteracting particles one concludes, following Boyd and Crutchfield, that (i) it reduces to a chaotic dynamical system---called the Szilard Map, a composite of three piecewise linear maps that implement the thermodynamic transformations of measurement, control, and erasure; (ii) its transitory functioning as an engine that converts disorganized heat energy to work is governed by the Kolmogorov-Sinai entropy rate; (iii) the demon's minimum necessary "intelligence" for optimal functioning is given by the engine's statistical complexity, and (iv) its functioning saturates thermodynamic bounds and so it is a minimal, optimal implementation. We show that Szilard's third model is rather different and addresses the fundamental issue, raised by the first two, of measurement in and by thermodynamic systems and entropy generation. Taken together, Szilard's suite of constructions lays out a range of possible realizations of Maxwellian demons that anticipated by almost two decades Shannon's and Wiener's concept of information as surprise and cybernetics' notion of functional information. This, in turn, gives new insight into engineering implementations of novel nanoscale information engines that leverage microscopic fluctuations and into the diversity of thermodynamic mechanisms and intrinsic computation harnessed in physical, molecular, biochemical, and biological systems., Comment: 15 pages, 14 figures; Supplementary Materials, pages; http://csc.ucdavis.edu/~cmg/compmech/pubs/voadt.htm

Maxwellian ratchets are autonomous, finite-state thermodynamic engines that implement input-output informational transformations. Previous studies of these "demons" focused on how they exploit environmental resources to generate work: They randomize ordered inputs, leveraging increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir while respecting both Liouvillian state-space dynamics and the Second Law. However, to date, correctly determining such functional thermodynamic operating regimes was restricted to a very few engines for which correlations among their information-bearing degrees of freedom could be calculated exactly and in closed form---a highly restricted set. Additionally, a key second dimension of ratchet behavior was largely ignored---ratchets do not merely change the randomness of environmental inputs, their operation constructs and deconstructs patterns. To address both dimensions, we adapt recent results from dynamical-systems and ergodic theories that efficiently and accurately calculate the entropy rates and the rate of statistical complexity divergence of general hidden Markov processes. In concert with the Information Processing Second Law, these methods accurately determine thermodynamic operating regimes for finite-state Maxwellian demons with arbitrary numbers of states and transitions. In addition, they facilitate analyzing structure versus randomness trade-offs that a given engine makes. The result is a greatly enhanced perspective on the information processing capabilities of information engines. As an application, we give a thorough-going analysis of the Mandal-Jarzynski ratchet, demonstrating that it has an uncountably-infinite effective state space., Comment: 16 pages, 7 figures; supplemental materials, 8 pages, 3 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/ftamr.htm

Modularity dissipation identifies how locally-implemented computation entails costs beyond those required by Landauer's bound on thermodynamic computing. We establish a general theorem for efficient local computation, giving the necessary and sufficient conditions for a local operation to have zero modularity cost. Applied to thermodynamically-generating stochastic processes it confirms a conjecture that classical generators are efficient if and only if they satisfy retrodiction, which places minimal memory requirements on the generator. This extends immediately to quantum computation: Any quantum simulator that employs quantum memory compression cannot be thermodynamically efficient., Comment: 12 pages, 4 figures; supplementary material 18 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/telc.htm

The hardware and software foundations laid in the first half of the 20th Century enabled the computing technologies that have transformed the world, but these foundations are now under siege. The current computing paradigm, which is the foundation of much of the current standards of living that we now enjoy, faces fundamental limitations that are evident from several perspectives. In terms of hardware, devices have become so small that we are struggling to eliminate the effects of thermodynamic fluctuations, which are unavoidable at the nanometer scale. In terms of software, our ability to imagine and program effective computational abstractions and implementations are clearly challenged in complex domains. In terms of systems, currently five percent of the power generated in the US is used to run computing systems - this astonishing figure is neither ecologically sustainable nor economically scalable. Economically, the cost of building next-generation semiconductor fabrication plants has soared past $10 billion. All of these difficulties - device scaling, software complexity, adaptability, energy consumption, and fabrication economics - indicate that the current computing paradigm has matured and that continued improvements along this path will be limited. If technological progress is to continue and corresponding social and economic benefits are to continue to accrue, computing must become much more capable, energy efficient, and affordable. We propose that progress in computing can continue under a united, physically grounded, computational paradigm centered on thermodynamics. Herein we propose a research agenda to extend these thermodynamic foundations into complex, non-equilibrium, self-organizing systems and apply them holistically to future computing systems that will harness nature's innate computational capacity. We call this type of computing "Thermodynamic Computing" or TC., Comment: A Computing Community Consortium (CCC) workshop report, 36 pages

Quantum coherence allows for reduced-memory simulators of classical processes. Using recent results in single-shot quantum thermodynamics, we derive a minimal work cost rate for quantum simulators that is quasistatically attainable in the limit of asymptotically-infinite parallel simulation. Comparing this cost with the classical regime reveals that quantizing classical simulators not only results in memory compression but also in reduced dissipation. We explore this advantage across a suite of representative examples., Comment: 7 pages, 3 figures; Supplementary material: 8 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/qmcate.htm

We implement a thermal-fluctuation driven logical bit reset on a superconducting flux logic cell. We show that the logical state of the system can be continuously monitored with only a small perturbation to the thermally activated dynamics at 500 mK. We use the trajectory information to derive a single-shot estimate of the work performed on the system per logical cycle. We acquire a sample of $10^5$ erasure trajectories per protocol, and show that the work histograms agree with both microscopic theory and global fluctuation theorems. The results demonstrate how to design and diagnose complex, high-speed, and thermodynamically efficient computing using superconducting technology., Comment: 9 pages, 7 figures

Extracting actionable insight from complex unlabeled scientific data is an open challenge and key to unlocking data-driven discovery in science. Complementary and alternative to supervised machine learning approaches, unsupervised physics-based methods based on behavior-driven theories hold great promise. Due to computational limitations, practical application on real-world domain science problems has lagged far behind theoretical development. We present our first step towards bridging this divide - DisCo - a high-performance distributed workflow for the behavior-driven local causal state theory. DisCo provides a scalable unsupervised physics-based representation learning method that decomposes spatiotemporal systems into their structurally relevant components, which are captured by the latent local causal state variables. Complex spatiotemporal systems are generally highly structured and organize around a lower-dimensional skeleton of coherent structures, and in several firsts we demonstrate the efficacy of DisCo in capturing such structures from observational and simulated scientific data. To the best of our knowledge, DisCo is also the first application software developed entirely in Python to scale to over 1000 machine nodes, providing good performance along with ensuring domain scientists' productivity. We developed scalable, performant methods optimized for Intel many-core processors that will be upstreamed to open-source Python library packages. Our capstone experiment, using newly developed DisCo workflow and libraries, performs unsupervised spacetime segmentation analysis of CAM5.1 climate simulation data, processing an unprecedented 89.5 TB in 6.6 minutes end-to-end using 1024 Intel Haswell nodes on the Cori supercomputer obtaining 91% weak-scaling and 64% strong-scaling efficiency.

Extreme weather is one of the main mechanisms through which climate change will directly impact human society. Coping with such change as a global community requires markedly improved understanding of how global warming drives extreme weather events. While alternative climate scenarios can be simulated using sophisticated models, identifying extreme weather events in these simulations requires automation due to the vast amounts of complex high-dimensional data produced. Atmospheric dynamics, and hydrodynamic flows more generally, are highly structured and largely organize around a lower dimensional skeleton of coherent structures. Indeed, extreme weather events are a special case of more general hydrodynamic coherent structures. We present a scalable physics-based representation learning method that decomposes spatiotemporal systems into their structurally relevant components, which are captured by latent variables known as local causal states. For complex fluid flows we show our method is capable of capturing known coherent structures, and with promising segmentation results on CAM5.1 water vapor data we outline the path to extreme weather identification from unlabeled climate model simulation data.

When an experimentalist measures a time series of qubits, the outcomes generate a classical stochastic process. We show that measurement induces high complexity in these processes in two specific senses: they are inherently unpredictable (positive Shannon entropy rate) and they require an infinite number of features for optimal prediction (divergent statistical complexity). We identify nonunifilarity as the mechanism underlying the resulting complexities and examine the influence that measurement choice has on the randomness and structure of measured qubit processes. We introduce new quantitative measures of this complexity and provide efficient algorithms for their estimation., Comment: 6 pages, 3 figures; supplemental material, 5 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/qdic_prl.htm

We experimentally demonstrate that highly structured distributions of work emerge during even the simple task of erasing a single bit. These are signatures of a refined suite of time-reversal symmetries in distinct functional classes of microscopic trajectories. As a consequence, we introduce a broad family of conditional fluctuation theorems that the component work distributions must satisfy. Since they identify entropy production, the component work distributions encode both the frequency of various mechanisms of success and failure during computing, as well giving improved estimates of the total irreversibly-dissipated heat. This new diagnostic tool provides strong evidence that thermodynamic computing at the nanoscale can be constructively harnessed. We experimentally verify this functional decomposition and the new class of fluctuation theorems by measuring transitions between flux states in a superconducting circuit., Comment: 13 pages, 3 figures, supplementary material; http://csc.ucdavis.edu/~cmg/compmech/pubs/tcft.htm