Search

Information is not a monolithic entity, but can be decomposed into synergistic, unique, and redundant components. Relative predominance of synergy and redundancy in the human brain follows a unimodal–transmodal organisation and reflects underlying structure, neurobiology, and dynamics. Brain regions navigate trade-offs between these components to combine the flexibility of synergy for higher cognition and the robustness of redundancy for key sensory and motor functions. Redundancy appears stable across primate evolution, whereas synergy is selectively increased in humans and especially in human-accelerated regions. Computational studies offer new insights into the causal relationship between synergy, redundancy, and cognitive capabilities. To explain how the brain orchestrates information-processing for cognition, we must understand information itself. Importantly, information is not a monolithic entity. Information decomposition techniques provide a way to split information into its constituent elements: unique, redundant, and synergistic information. We review how disentangling synergistic and redundant interactions is redefining our understanding of integrative brain function and its neural organisation. To explain how the brain navigates the trade-offs between redundancy and synergy, we review converging evidence integrating the structural, molecular, and functional underpinnings of synergy and redundancy; their roles in cognition and computation; and how they might arise over evolution and development. Overall, disentangling synergistic and redundant information provides a guiding principle for understanding the informational architecture of the brain and cognition. [ABSTRACT FROM AUTHOR]

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

High-level brain functions are widely believed to emerge from the orchestrated activity of multiple neural systems. However, lacking a formal definition and practical quantification of emergence for experimental data, neuroscientists have been unable to empirically test this long-standing conjecture. Here we investigate this fundamental question by leveraging a recently proposed framework known as “Integrated Information Decomposition,” which establishes a principled information-theoretic approach to operationalise and quantify emergence in dynamical systems — including the human brain. By analysing functional MRI data, our results show that the emergent and hierarchical character of neural dynamics is significantly diminished in chronically unresponsive patients suffering from severe brain injury. At a functional level, we demonstrate that emergence capacity is positively correlated with the extent of hierarchical organisation in brain activity. Furthermore, by combining computational approaches from network control theory and whole-brain biophysical modelling, we show that the reduced capacity for emergent and hierarchical dynamics in severely brain-injured patients can be mechanistically explained by disruptions in the patients’ structural connectome. Overall, our results suggest that chronic unresponsiveness resulting from severe brain injury may be related to structural impairment of the fundamental neural infrastructures required for brain dynamics to support emergence.

A growing number of papers on style transfer for texts rely on information decomposition. The performance of the resulting systems is usually assessed empirically in terms of the output quality or requires laborious experiments. This paper suggests a straightforward information theoretical framework to assess the quality of information decomposition for latent representations in the context of style transfer. Experimenting with several state-of-the-art models, we demonstrate that such estimates could be used as a fast and straightforward health check for the models instead of more laborious empirical experiments. [ABSTRACT FROM AUTHOR]

The integrated information theory of consciousness (IIT) is divisive: while some believe it provides an unprecedentedly powerful approach to address the 'hard problem', others dismiss it on grounds that it is untestable. We argue that the appeal and applicability of IIT can be greatly widened if we distinguish two flavours of the theory: strong IIT, which identifies consciousness with specific properties associated with maxima of integrated information; and weak IIT, which tests pragmatic hypotheses relating aspects of consciousness to broader measures of information dynamics. We review challenges for strong IIT, explain how existing empirical findings are well explained by weak IIT without needing to commit to the entirety of strong IIT, and discuss the outlook for both flavours of IIT. The integrated information theory of consciousness (IIT) is unprecedentedly ambitious in that it proposes a universal mathematical formula, derived from fundamental properties of conscious experience, to describe the quality and quantity of consciousness for any physical system that possesses it. IIT proponents believe it may solve the 'hard problem' of consciousness of why and how physical processes can be accompanied by subjective experience. However, in the current formulation, IIT formulae are not always well-defined and current empirical evidence does not support the level of specificity present in the theory. At the same time, available empirical evidence does support a weaker, less prescriptive version of the theory. We argue that distinguishing a 'weak' from a 'strong' flavour of IIT can provide a useful theoretical umbrella for ongoing empirical work, widening the overall appeal and applicability of the theory. [ABSTRACT FROM AUTHOR]

Emergence is a profound subject that straddles many scientific disciplines, including the formation of galaxies and how consciousness arises from the collective activity of neurons. Despite the broad interest that exists on this concept, the study of emergence has suffered from a lack of formalisms that could be used to guide discussions and advance theories. Here, we summarize, elaborate on, and extend a recent formal theory of causal emergence based on information decomposition, which is quantifiable and amenable to empirical testing. This theory relates emergence with information about a system's temporal evolution that cannot be obtained from the parts of the system separately. This article provides an accessible but rigorous introduction to the framework, discussing the merits of the approach in various scenarios of interest. We also discuss several interpretation issues and potential misunderstandings, while highlighting the distinctive benefits of this formalism. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'. [ABSTRACT FROM AUTHOR]

This paper investigates the relationship between audit fees and MD&A readability in China. We find that audit fees generally are higher when MD&A readability is lower. However, we find Big4 auditors and industry specialist auditors do not increase audit fees when MD&As are less readable while the other auditors do. We also find that it is the content unique to the reporting firm, rather than standard industry and market content in a MD&A that affects auditor fees. Our results also support that audit risk and audit efforts are two channels through which MD&A readability affects audit fees. [ABSTRACT FROM AUTHOR]

A growing number of papers on style transfer for texts rely on information decomposition. The performance of the resulting systems is usually assessed empirically in terms of the output quality or requires laborious experiments. This paper suggests a straightforward information theoretical framework to assess the quality of information decomposition for latent representations in the context of style transfer. Experimenting with several state-of-the-art models, we demonstrate that such estimates could be used as a fast and straightforward health check for the models instead of more laborious empirical experiments.

The brain routes and integrates information from many sources during behavior. A number of models explain this phenomenon within the framework of mixed selectivity theory, yet it is difficult to compare their predictions to understand how neurons and circuits integrate information. In this work, we apply time-series partial information decomposition [PID] to compare models of integration on a dataset of superior colliculus [SC] recordings collected during a multi-target visual search task. On this task, SC must integrate target guidance, bottom-up salience, and previous fixation signals to drive attention. We find evidence that SC neurons integrate these factors in diverse ways, including decision-variable selectivity to expected value, functional specialization to previous fixation, and code-switching (to incorporate new visual input).

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning., Competing Interests: EC, RB, MG, SR, PK, JB, AB No competing interests declared, (© 2023, Combrisson et al.)

We study the unique information function UI(T: X n Y) defined by Bertschinger et al. [4] within the framework of information decompositions. In particular, we study uniqueness and support of the solutions to the convex optimization problem underlying the definition of UI. We identify suficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of T, X and Y. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of UI(T: X n Y), most notably when T is binary. Optima in the relative interior of the optimization domain are solutions of linear equations if T is binary. In the all binary case, we obtain a complete picture of where the optimizing probability distributions lie. [ABSTRACT FROM AUTHOR]

Understanding how different networks relate to each other is key for understanding complex systems. We introduce an intuitive yet powerful framework to disentangle different ways in which networks can be (dis)similar and complementary to each other. We decompose the shortest paths between nodes as uniquely contributed by one source network, or redundantly by either, or synergistically by both together. Our approach considers the networks' full topology, providing insights at multiple levels of resolution: from global statistics to individual paths. Our framework is widely applicable across scientific domains, from public transport to brain networks. In humans and 124 other species, we demonstrate the prevalence of unique contributions by long-range white-matter fibers in structural brain networks. Across species, efficient communication also relies on significantly greater synergy between long-range and short-range fibers than expected by chance. Our framework could find applications for designing network systems or evaluating existing ones., Competing Interests: The authors declare no competing interests., (© 2024 The Authors.)

Autonomic nervous system (ANS) activity and imbalance between its sympathetic and parasympathetic components are important factors contributing to the initiation and progression of many cardiovascular disorders related to obesity. The results on respiratory sinus arrhythmia (RSA) magnitude changes as a parasympathetic index were not straightforward in previous studies on young obese subjects. Considering the potentially unbalanced ANS regulation with impaired parasympathetic control in obese patients, the aim of this study was to compare the relative contribution of baroreflex and non-baroreflex (central) mechanisms to the origin of RSA in obese vs. control subjects. To this end, we applied a recently proposed information-theoretic methodology – partial information decomposition (PID) – to the time series of heart rate variability (HRV, computed from RR intervals in the ECG), systolic blood pressure (SBP) variability, and respiration (RESP) pattern measured in 29 obese and 29 age- and gender-matched non-obese adolescents and young adults monitored in the resting supine position and during postural and cognitive stress evoked by head-up tilt and mental arithmetic. PID was used to quantify the so-called unique information transferred from RESP to HRV and from SBP to HRV, reflecting, respectively, non-baroreflex and RESP-unrelated baroreflex HRV mechanisms, and the redundant information transferred from (RESP, SBP) to HRV, reflecting RESP-related baroreflex RSA mechanisms. Our results suggest that obesity is associated: (i) with blunted involvement of non-baroreflex RSA mechanisms, documented by the lower unique information transferred from RESP to HRV at rest; and (ii) with a reduced response to postural stress (but not to mental stress), documented by the lack of changes in the unique information transferred from RESP and SBP to HRV in obese subjects moving from supine to upright, and by a decreased redundant information transfer in obese compared to controls in the upright position. These findings were observed in the presence of an unchanged RSA magnitude measured as the high frequency (HF) power of HRV, thus suggesting that the changes in ANS imbalance related to obesity in adolescents and young adults are subtle and can be revealed by dissecting RSA mechanisms into its components during various challenges.

In computer science, we can theoretically neatly separate transmission and processing of information, hardware and software, and programs and their inputs. This is much more intricate in biology. Nevertheless, I argue that Shannon's concept of information is useful in biology, although its application is not as straightforward as many people think. In fact, the recently developed theory of information decomposition can shed much light on the complementarity between coding and regulatory, or internal and environmental information. The key challenge that we formulate in this contribution is to understand how genetic information and external factors combine to create an organism, and conversely how the genome has learned in the course of evolution how to harness the environment, and analogously how coding, regulation and spatial organization interact in cellular processes. [ABSTRACT FROM AUTHOR]

Despite the near universal assumption of individuality in biology, there is little agreement about what individuals are and few rigorous quantitative methods for their identification. Here, we propose that individuals are aggregates that preserve a measure of temporal integrity, i.e., "propagate" information from their past into their futures. We formalize this idea using information theory and graphical models. This mathematical formulation yields three principled and distinct forms of individuality—an organismal, a colonial, and a driven form—each of which varies in the degree of environmental dependence and inherited information. This approach can be thought of as a Gestalt approach to evolution where selection makes figure-ground (agent–environment) distinctions using suitable information-theoretic lenses. A benefit of the approach is that it expands the scope of allowable individuals to include adaptive aggregations in systems that are multi-scale, highly distributed, and do not necessarily have physical boundaries such as cell walls or clonal somatic tissue. Such individuals might be visible to selection but hard to detect by observers without suitable measurement principles. The information theory of individuality allows for the identification of individuals at all levels of organization from molecular to cultural and provides a basis for testing assumptions about the natural scales of a system and argues for the importance of uncertainty reduction through coarse-graining in adaptive systems. [ABSTRACT FROM AUTHOR]

Autonomic nervous system (ANS) activity and imbalance between its sympathetic and parasympathetic components are important factors contributing to the initiation and progression of many cardiovascular disorders related to obesity. The results on respiratory sinus arrhythmia (RSA) magnitude changes as a parasympathetic index were not straightforward in previous studies on young obese subjects. Considering the potentially unbalanced ANS regulation with impaired parasympathetic control in obese patients, the aim of this study was to compare the relative contribution of baroreflex and non-baroreflex (central) mechanisms to the origin of RSA in obese vs. control subjects. To this end, we applied a recently proposed information-theoretic methodology – partial information decomposition (PID) – to the time series of heart rate variability (HRV, computed from RR intervals in the ECG), systolic blood pressure (SBP) variability, and respiration (RESP) pattern measured in 29 obese and 29 age- and gender-matched non-obese adolescents and young adults monitored in the resting supine position and during postural and cognitive stress evoked by head-up tilt and mental arithmetic. PID was used to quantify the so-called unique information transferred from RESP to HRV and from SBP to HRV, reflecting, respectively, non-baroreflex and RESP-unrelated baroreflex HRV mechanisms, and the redundant information transferred from (RESP, SBP) to HRV, reflecting RESP-related baroreflex RSA mechanisms. Our results suggest that obesity is associated: (i) with blunted involvement of non-baroreflex RSA mechanisms, documented by the lower unique information transferred from RESP to HRV at rest; and (ii) with a reduced response to postural stress (but not to mental stress), documented by the lack of changes in the unique information transferred from RESP and SBP to HRV in obese subjects moving from supine to upright, and by a decreased redundant information transfer in obese compared to controls in the upright position. These findings were observed in the presence of an unchanged RSA magnitude measured as the high frequency (HF) power of HRV, thus suggesting that the changes in ANS imbalance related to obesity in adolescents and young adults are subtle and can be revealed by dissecting RSA mechanisms into its components during various challenges. [ABSTRACT FROM AUTHOR]

Abstract Accurate seafloor maps are critical in offshore petroleum projects. Almost all subsurface modeling workflows use these maps: for example, pore pressure models and reservoir geomechanics. These studies generally involve sums of properties along the vertical, consequently, noise or acquisition artifacts at the seafloor map tend to propagate and even amplify in the subsequent models. Seafloor maps are normally acquired by sound-sensing technology and related procedures, which expectedly introduce noise and artifacts. Moving window filters, such as average and median filters, are easy to use and fast to compute, hence they are used very often by interpreters to make geologically sound horizons. However such filters are unable to discern noise, artifacts and signal, resulting in the compromise of actual information. Filtering in the frequency domain can pose difficulties when noise, artifacts and geological information overlap in the frequency domain, as the total energy tend to concentrate in the lowest frequencies. Factorial kriging (FK) is a spatial filter that yields separate estimations, each corresponding to a variogram nested structure. FK relies on the principle that structurally complex information is a sum of independent information components while the variogram is a clear and reliable way to select the frequency ranges of each component. Hence, this method can effectively separate acquisition artifacts from seafloor information based on their different spatial structures modeled with variograms. This work investigates an application of FK for filtering of hard-edged (broadband) artifacts in post-processed interpreted seismic data. Differently from random noise, artifacts are spatially correlated information components mixed with actual geologic information. Initially, removal of the bathymetric trend makes the case stationary for a high-resolution variogram map computed in the frequency domain via fast Fourier transform (FFT), which enables a full 2D variogram modeling. Theoretical variogram surfaces are fit to the experimental variogram map, enabling the identification and modeling of a set of nested variograms. The authors envisaged search strategies to select a small number of informative samples to reduce kriging runtime and, at the same time, cover the necessary spatially correlated variability. A workflow on a large seismic-derived seafloor 2D map is demonstrated, detailing each step with emphasis on the sensitive variogram modeling step, followed by search strategy definition and choice of the geology-bearing factors for seafloor restoration. Filtering in frequency domain was performed for comparison. Results showed that broadband artifacts require a non-traditional approach to variogram modeling, which was performed in a second FK run with a wholly new variogram model. The filtering operations revealed seafloor features, which were previously unseen in the original map, and the final result was similar to that obtained with FFT. FK is a lossless filtering technique that is a highly effective method to remove acquisition footprints in final grids and volumes in which acquisition geometry information is not available. 1 1 FFT – fast Fourier transform 2 2 FK – Factorial Kriging 3 3 KT – Kriging with a trend 4 4 TCL – Tool Command Language Highlights • Variographic modeling of broadband artifacts require multiple nested structures. • FFT-computed experimental variogram allows the observation of variographic structures with small contributions. • Search strategy with sample density control to reduce estimation runtime and cover the necessary variability. • Separating large-scale trends facilitates the identification of small variographic structures. • A full 2D variogram modeling approach allows accurate separation of information components with different spatial structures. • Comparison with filtering in the frequency domain and the variographic modeling of a broadband structure. [ABSTRACT FROM AUTHOR]

If regularity in data takes the form of higher-order functions among groups of variables, models which are biased towards lower-order functions may easily mistake the data for noise. To distinguish whether this is the case, one must be able to quantify the contribution of different orders of dependence to the total information. Recent work in information theory attempts to do this through measures of multivariate mutual information (MMI) and information decomposition (ID). Despite substantial theoretical progress, practical issues related to tractability and learnability of higher-order functions are still largely unaddressed. In this work, we introduce a new approach to information decomposition—termed Neural Information Decomposition (NID)—which is both theoretically grounded, and can be efficiently estimated in practice using neural networks. We show on synthetic data that NID can learn to distinguish higher-order functions from noise, while many unsupervised probability models cannot. Additionally, we demonstrate the usefulness of this framework as a tool for exploring biological and artificial neural networks.

Neural systems are composed of many local processors that generate an output given their many inputs as specified by a transfer function. This paper studies a transfer function that is fundamentally asymmetric and builds on multi-site intracellular recordings indicating that some neocortical pyramidal cells can function as context-sensitive two-point processors in which some inputs modulate the strength with which they transmit information about other inputs. Learning and processing at the level of the local processor can then be guided by the context of activity in the system as a whole without corrupting the message that the local processor transmits. We use a recent advance in the foundations of information theory to compare the properties of this modulatory transfer function with that of the simple arithmetic operators. This advance enables the information transmitted by processors with two distinct inputs to be decomposed into those components unique to each input, that shared between the two inputs, and that which depends on both though it is in neither, i.e., synergy. We show that contextual modulation is fundamentally asymmetric, contrasts with all four simple arithmetic operators, can take various forms, and can occur together with the anatomical asymmetry that defines pyramidal neurons in mammalian neocortex.

The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of multivariate information content that can be accurately depicted using Venn diagrams for any number of random variables. These measures complement the existing measures of multivariate mutual information and are constructed by considering the algebraic structure of information sharing. It is shown that the distinct ways in which a set of marginal observers can share their information with a non-observing third party corresponds to the elements of a free distributive lattice. The redundancy lattice from partial information decomposition is then subsequently and independently derived by combining the algebraic structures of joint and shared information content.

Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians.

Ecohydrologic fluxes within atmosphere, vegetation, and soil systems exhibit a joint variability that arises from forcing and feedback interactions. These interactions cause fluctuations to propagate between variables at many time scales. In an ecosystem, this connectivity dictates responses to climate change, landcover change, and weather events and must be characterized to understand resilience and sensitivity. We use an information theory-based approach to quantify connectivity in the form of information flow associated with the propagation of fluctuations between variables. We apply this approach to study ecosystems that experience changes in dry-season moisture availability due to rainfall and drought conditions. We use data from two transects with flux towers located along elevation gradients and quantify redundant, synergistic, and unique flow of information between lagged sources and targets to characterize joint asynchronous time dependencies. At the Reynolds Creek Critical Zone Observatory in Idaho, a dry-season rainfall pulse leads to increased connectivity from soil and atmospheric variables to heat and carbon fluxes. At the Southern Sierra Critical Zone Observatory in California, separate sets of dominant drivers characterize two sites at which fluxes exhibit different drought responses. For both cases, our information flow-based connectivity characterizes dominant drivers and joint variability before, during, and after disturbances. This approach to gauge the responsiveness of ecosystem fluxes under multiple sources of variability furthers our understanding of complex ecohydrologic systems. [ABSTRACT FROM AUTHOR]

We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X, Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.

Heart rate variability (HRV; variability of the RR interval of the electrocardiogram) results from the activity of several coexisting control mechanisms, which involve the influence of respiration (RESP) and systolic blood pressure (SBP) oscillations operating across multiple temporal scales and changing in different physiological states. In this study, multiscale information decomposition is used to dissect the physiological mechanisms related to the genesis of HRV in 78 young volunteers monitored at rest and during postural and mental stress evoked by head-up tilt (HUT) and mental arithmetics (MA). After representing RR, RESP and SBP at different time scales through a recently proposed method based on multivariate state space models, the joint information transfer T RESP , SBP → RR is decomposed into unique, redundant and synergistic components, describing the strength of baroreflex modulation independent of respiration ( U SBP → RR ), nonbaroreflex ( U RESP → RR ) and baroreflex-mediated ( R RESP , SBP → RR ) respiratory influences, and simultaneous presence of baroreflex and nonbaroreflex respiratory influences ( S RESP , SBP → RR ), respectively. We find that fast (short time scale) HRV oscillations—respiratory sinus arrhythmia—originate from the coexistence of baroreflex and nonbaroreflex (central) mechanisms at rest, with a stronger baroreflex involvement during HUT. Focusing on slower HRV oscillations, the baroreflex origin is dominant and MA leads to its higher involvement. Respiration influences independent on baroreflex are present at long time scales, and are enhanced during HUT.

Secret sharing is a cryptographic discipline in which the goal is to distribute information about a secret over a set of participants in such a way that only specific authorized combinations of participants together can reconstruct the secret. Thus, secret sharing schemes are systems of variables in which it is very clearly specified which subsets have information about the secret. As such, they provide perfect model systems for information decompositions. However, following this intuition too far leads to an information decomposition with negative partial information terms, which are difficult to interpret. One possible explanation is that the partial information lattice proposed by Williams and Beer is incomplete and has to be extended to incorporate terms corresponding to higher-order redundancy. These results put bounds on information decompositions that follow the partial information framework, and they hint at where the partial information lattice needs to be improved. [ABSTRACT FROM AUTHOR]

In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to describe the relationships among all PID lattices. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system's entropy, hence it characterizes completely all the interdependencies in the system. [ABSTRACT FROM AUTHOR]

We consider the problem of quantifying the information shared by a pair of random variables X1, X2 about another variable S. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about S is bounded from below by the information shared about f (S) for any function f . We show that our measure leads to a new nonnegative decomposition of the mutual information I(S; X1 X2 ) into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic. [ABSTRACT FROM AUTHOR]

• Relationship between emergent dynamics, functional hierarchy, and controllability. • DOC patients exhibit less hierarchical and emergent neural dynamics. • DOC patients' structural connectomes exhibit compromised modal controllability. • Whole-brain model based on patient connectomes recapitulates functional alterations. High-level brain functions are widely believed to emerge from the orchestrated activity of multiple neural systems. However, lacking a formal definition and practical quantification of emergence for experimental data, neuroscientists have been unable to empirically test this long-standing conjecture. Here we investigate this fundamental question by leveraging a recently proposed framework known as "Integrated Information Decomposition," which establishes a principled information-theoretic approach to operationalise and quantify emergence in dynamical systems — including the human brain. By analysing functional MRI data, our results show that the emergent and hierarchical character of neural dynamics is significantly diminished in chronically unresponsive patients suffering from severe brain injury. At a functional level, we demonstrate that emergence capacity is positively correlated with the extent of hierarchical organisation in brain activity. Furthermore, by combining computational approaches from network control theory and whole-brain biophysical modelling, we show that the reduced capacity for emergent and hierarchical dynamics in severely brain-injured patients can be mechanistically explained by disruptions in the patients' structural connectome. Overall, our results suggest that chronic unresponsiveness resulting from severe brain injury may be related to structural impairment of the fundamental neural infrastructures required for brain dynamics to support emergence. [ABSTRACT FROM AUTHOR]

Information is often described as a reduction of uncertainty associated with a restriction of possible choices. Despite appearing in Hartley’s foundational work on information theory, there is a surprising lack of a formal treatment of this interpretation in terms of exclusions. This paper addresses the gap by providing an explicit characterisation of information in terms of probability mass exclusions. It then demonstrates that different exclusions can yield the same amount of information and discusses the insight this provides about how information is shared amongst random variables—lack of progress in this area is a key barrier preventing us from understanding how information is distributed in complex systems. The paper closes by deriving a decomposition of the mutual information which can distinguish between differing exclusions; this provides surprising insight into the nature of directed information.

The formulation of the Partial Information Decomposition (PID) framework by Williams and Beer in 2010 attracted a significant amount of attention to the problem of defining redundant (or shared), unique and synergistic (or complementary) components of mutual information that a set of source variables provides about a target. This attention resulted in a number of measures proposed to capture these concepts, theoretical investigations into such measures, and applications to empirical data (in particular to datasets from neuroscience). In this Special Issue on “Information Decomposition of Target Effects from Multi-Source Interactions” at Entropy, we have gathered current work on such information decomposition approaches from many of the leading research groups in the field. We begin our editorial by providing the reader with a review of previous information decomposition research, including an overview of the variety of measures proposed, how they have been interpreted and applied to empirical investigations. We then introduce the articles included in the special issue one by one, providing a similar categorisation of these articles into: i. proposals of new measures; ii. theoretical investigations into properties and interpretations of such approaches, and iii. applications of these measures in empirical studies. We finish by providing an outlook on the future of the field.

The interactions between three or more random variables are often nontrivial, poorly understood and, yet, are paramount for future advances in fields such as network information theory, neuroscience and genetics. In this work, we analyze these interactions as different modes of information sharing. Towards this end, and in contrast to most of the literature that focuses on analyzing the mutual information, we introduce an axiomatic framework for decomposing the joint entropy that characterizes the various ways in which random variables can share information. Our framework distinguishes between interdependencies where the information is shared redundantly and synergistic interdependencies where the sharing structure exists in the whole, but not between the parts. The key contribution of our approach is to focus on symmetric properties of this sharing, which do not depend on a specific point of view for differentiating roles between its components. We show that our axioms determine unique formulas for all of the terms of the proposed decomposition for systems of three variables in several cases of interest. Moreover, we show how these results can be applied to several network information theory problems, providing a more intuitive understanding of their fundamental limits. [ABSTRACT FROM AUTHOR]

Secret sharing is a cryptographic discipline in which the goal is to distribute information about a secret over a set of participants in such a way that only specific authorized combinations of participants together can reconstruct the secret. Thus, secret sharing schemes are systems of variables in which it is very clearly specified which subsets have information about the secret. As such, they provide perfect model systems for information decompositions. However, following this intuition too far leads to an information decomposition with negative partial information terms, which are difficult to interpret. One possible explanation is that the partial information lattice proposed by Williams and Beer is incomplete and has to be extended to incorporate terms corresponding to higher-order redundancy. These results put bounds on information decompositions that follow the partial information framework, and they hint at where the partial information lattice needs to be improved.

In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to describe the relationships among all PID lattices. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system’s entropy, hence it characterizes completely all the interdependencies in the system.

We consider the problem of quantifying the information shared by a pair of random variables X 1 , X 2 about another variable S. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about S is bounded from below by the information shared about f ( S ) for any function f. We show that our measure leads to a new nonnegative decomposition of the mutual information I ( S ; X 1 X 2 ) into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic.

Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians. [ABSTRACT FROM AUTHOR]

To determine the periodicity of a DNA sequence, different spectral approaches are applied (discrete Fourier transform (DFT), autocorrelation (CORR), information decomposition (ID), hybrid method (HYB), concept of spectral envelope for spectral analysis (SE), normalized autocorrelation (CORR_N) and profile analysis (PA). In this work, we investigated the possibility of finding the true period length, by depending on the average number of accumulated changes in DNA bases (PM) for the methods stated above. The results show that for periods with short length (≤4 b.p), it is possible to use the hybrid method (HYB), which combines properties of autocorrelation, Fourier transform, and information decomposition (ID). For larger period lengths (>4) with values of point mutation (PM) equal to 1.0 or more per one nucleotide, it is preferable to use information of decomposition method (ID), as the other spectral approaches cannot achieve correct determination of the period length present in the analyzed sequence. [ABSTRACT FROM AUTHOR]

The interactions between three or more random variables are often nontrivial, poorly understood and, yet, are paramount for future advances in fields such as network information theory, neuroscience and genetics. In this work, we analyze these interactions as different modes of information sharing. Towards this end, and in contrast to most of the literature that focuses on analyzing the mutual information, we introduce an axiomatic framework for decomposing the joint entropy that characterizes the various ways in which random variables can share information. Our framework distinguishes between interdependencies where the information is shared redundantly and synergistic interdependencies where the sharing structure exists in the whole, but not between the parts. The key contribution of our approach is to focus on symmetric properties of this sharing, which do not depend on a specific point of view for differentiating roles between its components. We show that our axioms determine unique formulas for all of the terms of the proposed decomposition for systems of three variables in several cases of interest. Moreover, we show how these results can be applied to several network information theory problems, providing a more intuitive understanding of their fundamental limits.

We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y,Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X,Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures. [ABSTRACT FROM AUTHOR]

Abstract: We present MMsat—a database of DNA sequences from GenBank possessing the latent periodicity at high level of statistical significance and having the period length in a range from 2 to 100 bases. The periodicity was found by analytical method of information decomposition. These sequences can be considered as potential micro- and minisatellites and thus can be useful for PCR analysis and evolutional studies. Distribution, properties, and potential functions of periodicity are discussed. Availability: http://victoria.biengi.ac.ru/mmsat [Copyright &y& Elsevier]

We used the method of Information Decomposition developed by us to identify the latent dinucleotide periodicity regions in bacterial genomes. The number of potential minisatellite sequences obtained at high level of statistical significance was 454. Then we classified the periodicity matrices and obtained 45 classes. We used the other new method developed by us—Modified Profile Analysis—to reveal more periodic sequences in the presence of indels using the classes obtained. The number of sequences found by combination of these two methods was 3949. Most of them cannot be revealed by other methods including dynamic programming and Fourier transformation. [ABSTRACT FROM PUBLISHER]

For detection of the latent periodicity of the protein families responsible for various biological functions, methods of information decomposition, cyclic profile alignment, and the method of noise decomposition have been used. The latent periodicity, being specific to a particular family, is recognized in 94 of 110 analyzed protein families. Family specific periodicity was found for more than 70% of amino acid sequences in each of these families. Based on such sequences the characteristic profile of the latent periodicity has been deduced for each family. Possible relationship between the recognized latent periodicity, evolution of proteins, and their structural organization is discussed. [ABSTRACT FROM AUTHOR]

A program package has been developed to search for hidden tandem repeats of any specified type in the protein sequence databases. The applied algorithm of the locally optimal cyclic alignment is able to find subsequences possessing a certain profile-based periodicity type when no appreciable homology between periods is observed, as well as in the presence of arbitrary insertions/deletions. The profile can be adjusted to search for the periodicity types structurally and functionally important. The Swiss-Prot database has been analyzed to reveal the periodicities undetectable earlier that are caused by the secondary and super-secondary structure regularities of the NAD-binding sites. In particular, a significant periodicity of 24 aa was found to be characteristic of the absolute majority of domains possessing the Rossman (or Rossman-like) fold and displaying apparent regularity in their secondary structures, not being obvious at the primary structure level. [ABSTRACT FROM AUTHOR]

A central question in neuroscience concerns the relationship between consciousness and its physical substrate. Here, we argue that a richer characterization of consciousness can be obtained by viewing it as constituted of distinct information-theoretic elements. In other words, we propose a shift from quantification of consciousness-viewed as integrated information-to its decomposition. Through this approach, termed Integrated Information Decomposition (ΦID), we lay out a formal argument that whether the consciousness of a given system is an emergent phenomenon depends on its information-theoretic composition-providing a principled answer to the long-standing dispute on the relationship between consciousness and emergence. Furthermore, we show that two organisms may attain the same amount of integrated information, yet differ in their information-theoretic composition. Building on ΦID's revised understanding of integrated information, termed Φ R , we also introduce the notion of Φ R -ing ratio to quantify how efficiently an entity uses information for conscious processing. A combination of Φ R and Φ R -ing ratio may provide an important way to compare the neural basis of different aspects of consciousness. Decomposition of consciousness enables us to identify qualitatively different 'modes of consciousness', establishing a common space for mapping the phenomenology of different conscious states. We outline both theoretical and empirical avenues to carry out such mapping between phenomenology and information-theoretic modes, starting from a central feature of everyday consciousness: selfhood. Overall, ΦID yields rich new ways to explore the relationship between information, consciousness, and its emergence from neural dynamics., (© The Author(s) 2021. Published by Oxford University Press.)