Search

Laboratory performance testing of the phase angle of asphalt concrete (AC) mixtures is often expensive and requires enormous human effort and time. To circumvent this problem, several regression-based methods have been proposed in the literature to model the phase angle behavior of AC mixtures using various approaches. However, these methods impose strict assumptions on the underlying relationship between phase angle and its corresponding covariates as well as how well and accurately these covariates are measured, restricting us from fully analyzing the predictive capability of any modeling method. To this end, this study proposed an alternative approach for modeling the phase angle characteristics of AC mixtures based on a recurrent neural network (RNN) that inherently and implicitly captures the effects of covariates. This approach is suitable to model the sequential nature of data recorded in laboratory testing where phase angle testing was repeated for a set of six loading frequencies forming a recurrent pattern. The proposed RNN model (P-RNN) was applied separately to wearing and base course mixtures by considering the historical values of phase angle as input and to predict its value for the next loading frequency, keeping temperature as a constant. To demonstrate the superiority of the proposed approach, the P-RNN model is compared with other competing models from the literature, and the results reveal superior performance of the P-RNN model.

Many online fighting games use rollback netcode in order to compensate for network delay. Rollback netcode allows players to experience the game as having reduced delay. A drawback of this is that players will sometimes see the game quickly ”jump” to a different state to adjust for the the remote player’s actions. Rollback netcode implementations require a method for predicting the remote player’s next button inputs. Current implementations use a naive repeatlastframe policy for such prediction. There is a possibility that alternative methods may lead to improved user experience. This project examines the problem of improving input prediction in fighting games. It details the development of a new prediction model based on recurrent neural networks. The model was trained and evaluated using a dataset of several thousand recorded player input sequences. The results show that the new model slightly outperforms the naive method in prediction accuracy, with the difference being greater for longer predictions. However, it has far higher requirements both in terms of memory and computation cost. It seems unlikely that the model would significantly improve on current rollback netcode implementations. However, there may be ways to improve predictions further, and the effects on user experience remains unknown., Många online fightingspel använder rollback netcode för att kompensera för nätverksfördröjning. Rollback netcode låter spelare uppleva spelet med mindre fördröjning. En nackdel av detta är att spelare ibland ser spelet snabbt ”hoppa” till ett annat tillstånd för att justera för motspelarens handlingar. Rollback netcode implementationer behöver en policy för att förutsäga motspelarens nästa knapptryckningar. Nuvarande implementationer använder en naiv repetera-senaste-frame policy för förutsägelser. Det finns en möjlighet att alternativa metoder kan leda till förbättrad användarupplevelse. Det här projektet undersöker problemet att förbättra förutsägelser av knapptryckningar i fightingspel. Det beskriver utvecklingen av en ny förutsägelsemodell baserad på rekursiva neuronnät. Modellen tränades och evaluerades med ett dataset av flera tusen inspelade knappsekvenser. Resultaten visar att den nya modellen överträffar den naiva metoden i noggrannhet, med större skillnad för längre förutsägelser. Dock har den mycket högre krav i både minne och beräkningskostad. Det verkar osannolikt att modellen skulle avsevärt förbättra nuvarande rollback netcode implementationer. Men det kan finnas sätt att förbättra förutsägelser ytterligare, och påverkan på användarupplevelsen förblir okänd.

This work proposes a low-power high-accuracy embedded hand-gesture recognition algorithm targeting battery-operated wearable devices using low-power short-range RADAR sensors. A 2-D convolutional neural network (CNN) using range-frequency Doppler features is combined with a temporal convolutional neural network (TCN) for time sequence prediction. The final algorithm has a model size of only 46 thousand parameters, yielding a memory footprint of only 92 KB. Two data sets containing 11 challenging hand gestures performed by 26 different people have been recorded containing a total of 20'210 gesture instances. On the 11 hand gesture data set, accuracies of 86.6% (26 users) and 92.4% (single user) have been achieved, which are comparable to the state of the art, which achieves 87% (10 users) and 94% (single user), while using a TCN-based network that is $7500\times $ smaller than the state of the art. Furthermore, the gesture recognition classifier has been implemented on a parallel ultralow power processor, demonstrating that real-time prediction is feasible with only 21 mW of power consumption for the full TCN sequence prediction network, while a system-level power consumption of less than 120 mW is achieved. We provide open-source access to example code and all data collected and used in this work on tinyradar.ethz.ch. © 2014 IEEE., Cited By :1; Export Date: 20 September 2021; Article; Correspondence Address: Scherer, M.; Department of Information Technology and Electrical Engineering, Switzerland; email: scheremo@iis.ee.ethz.ch

With the rapid development of artificial intelligence and industrial Internet of Things (IIoT) technologies, intelligent predictive maintenance (IPdM) has received considerable attention from researchers and practitioners. To efficiently predict impending failures and mitigate unexpected downtime, while satisfying the instant maintenance demands of industrial facilities is very important for improving the production efficiency. In this article, a self-attention based "Perception and Prediction" framework, called DeepHealth, is proposed for the instant IPdM. Specifically, the framework is composed of two submodels (i.e., DH-1 and DH-2), which are respectively utilized to perform the health perception and sequence prediction. By operating the framework, the proposed models can predict the health conditions via predicting the future signal samples, thereby completing the instant IPdM. Considering the potential temporal correlation in time series, we deploy an enhanced attention mechanism to capture global dependencies from the vibration signals, and leverage the long- and short-term sequence prediction of sensor signals to support instant maintenance decision-making. On this basis, we conduct a destructive experiment based on the IIoT-enabled rotating machinery and construct a balanced industrial dataset for model evaluations. Extensive experiment results show that the proposed solution achieves good prediction accuracy for instant IPdM on the automatic washing equipment and Case Western Reserve University datasets.

Many online fighting games use rollback netcode in order to compensate for network delay. Rollback netcode allows players to experience the game as having reduced delay. A drawback of this is that players will sometimes see the game quickly ”jump” to a different state to adjust for the the remote player’s actions. Rollback netcode implementations require a method for predicting the remote player’s next button inputs. Current implementations use a naive repeatlastframe policy for such prediction. There is a possibility that alternative methods may lead to improved user experience. This project examines the problem of improving input prediction in fighting games. It details the development of a new prediction model based on recurrent neural networks. The model was trained and evaluated using a dataset of several thousand recorded player input sequences. The results show that the new model slightly outperforms the naive method in prediction accuracy, with the difference being greater for longer predictions. However, it has far higher requirements both in terms of memory and computation cost. It seems unlikely that the model would significantly improve on current rollback netcode implementations. However, there may be ways to improve predictions further, and the effects on user experience remains unknown., Många online fightingspel använder rollback netcode för att kompensera för nätverksfördröjning. Rollback netcode låter spelare uppleva spelet med mindre fördröjning. En nackdel av detta är att spelare ibland ser spelet snabbt ”hoppa” till ett annat tillstånd för att justera för motspelarens handlingar. Rollback netcode implementationer behöver en policy för att förutsäga motspelarens nästa knapptryckningar. Nuvarande implementationer använder en naiv repetera-senaste-frame policy för förutsägelser. Det finns en möjlighet att alternativa metoder kan leda till förbättrad användarupplevelse. Det här projektet undersöker problemet att förbättra förutsägelser av knapptryckningar i fightingspel. Det beskriver utvecklingen av en ny förutsägelsemodell baserad på rekursiva neuronnät. Modellen tränades och evaluerades med ett dataset av flera tusen inspelade knappsekvenser. Resultaten visar att den nya modellen överträffar den naiva metoden i noggrannhet, med större skillnad för längre förutsägelser. Dock har den mycket högre krav i både minne och beräkningskostad. Det verkar osannolikt att modellen skulle avsevärt förbättra nuvarande rollback netcode implementationer. Men det kan finnas sätt att förbättra förutsägelser ytterligare, och påverkan på användarupplevelsen förblir okänd.

With an increasing demand for bandwidth and lower latency in mobile communication networks it becomes gradually more important to improve current mobile network management solutions using available network data. To improve the network management it can for instance be of interest to infer future available bandwidth to the end user of the network. This can be done by utilizing the current knowledge of real-time user equipment (UE) behaviour in the network. In the scope of this thesis interest lies in, given a set of visited radio access points (cells), to predict what the next one is going to be. For this reason the aim is to investigate the prediction performance when utilizing the All-K-Order Markov (AKOM) model, with some added variations, on collected data generated from train trajectories. Moreover a method for testing the suitability of modeling the sequence of cells as a time-homogeneous Markov chain is proposed, in order to determine the goodness-of- t with the available data. Lastly, the elapsed time in each cell is attempted to be predicted using linear regression given the prior history window of previous cell and elapsed times pairs. The results show that moderate to good prediction accuracy on the upcoming cell can be achieved with AKOM and associated variations. For predicting the upcoming sojourn time in future cells the results reveal that linear regression does not yield satisfactory results and possibly another regression model should be utilized., Med en ökande efterfrågan på banbredd och kortare latens i mobila nätverk har det gradvis blivit viktigare att förbättra nuvarande lösningar för hantering av nätverk genom att använda tillgänglig nätverksdata. Specifikt är det av intresse att kunna dra slutsatser kring vad framtida bandbredsförhållanden kommer vara, samt övriga parametrar av intresse genom att använda tillgänglig information om aktuell mobil användarutrustnings (UE) beteende i det mobila nätverket. Inom ramen av detta masterarbete ligger fokus på att, givet tidigare besökta radio accesspunkter (celler), kunna förutspå vilken nästkommande besökta cell kommer att vara. Av denna anledning är målet att undersöka vilken prestanda som kan uppnås när All-$K$-Order Markov (AKOM) modellen, med associerade varianter av denna, används på samlad data från tågfärder. Dessutom ges det förslag på test som avgör hur lämpligt det är att modelera observerade sekvenser av celler som en homogen Markovkedja med tillgänglig data. Slutligen undersöks även om besökstiden i en framtida cell kan förutspås med linjär regression givet ett historiskt fönster av tidigare cell och besökstids par. Erhållna resultat visar att måttlig till bra prestanda kan uppnås när kommande celler förutspås med AKOM modellen och associerade variationer. För prediktering av besökstid i kommande cell med linjär regression erhålles det däremot inte tillfredsställande resultat, vilket tyder på att en alternativ regressionsmetod antagligen är bättre lämpad för denna data.

With an increasing demand for bandwidth and lower latency in mobile communication networks it becomes gradually more important to improve current mobile network management solutions using available network data. To improve the network management it can for instance be of interest to infer future available bandwidth to the end user of the network. This can be done by utilizing the current knowledge of real-time user equipment (UE) behaviour in the network. In the scope of this thesis interest lies in, given a set of visited radio access points (cells), to predict what the next one is going to be. For this reason the aim is to investigate the prediction performance when utilizing the All-K-Order Markov (AKOM) model, with some added variations, on collected data generated from train trajectories. Moreover a method for testing the suitability of modeling the sequence of cells as a time-homogeneous Markov chain is proposed, in order to determine the goodness-of- t with the available data. Lastly, the elapsed time in each cell is attempted to be predicted using linear regression given the prior history window of previous cell and elapsed times pairs. The results show that moderate to good prediction accuracy on the upcoming cell can be achieved with AKOM and associated variations. For predicting the upcoming sojourn time in future cells the results reveal that linear regression does not yield satisfactory results and possibly another regression model should be utilized., Med en ökande efterfrågan på banbredd och kortare latens i mobila nätverk har det gradvis blivit viktigare att förbättra nuvarande lösningar för hantering av nätverk genom att använda tillgänglig nätverksdata. Specifikt är det av intresse att kunna dra slutsatser kring vad framtida bandbredsförhållanden kommer vara, samt övriga parametrar av intresse genom att använda tillgänglig information om aktuell mobil användarutrustnings (UE) beteende i det mobila nätverket. Inom ramen av detta masterarbete ligger fokus på att, givet tidigare besökta radio accesspunkter (celler), kunna förutspå vilken nästkommande besökta cell kommer att vara. Av denna anledning är målet att undersöka vilken prestanda som kan uppnås när All-$K$-Order Markov (AKOM) modellen, med associerade varianter av denna, används på samlad data från tågfärder. Dessutom ges det förslag på test som avgör hur lämpligt det är att modelera observerade sekvenser av celler som en homogen Markovkedja med tillgänglig data. Slutligen undersöks även om besökstiden i en framtida cell kan förutspås med linjär regression givet ett historiskt fönster av tidigare cell och besökstids par. Erhållna resultat visar att måttlig till bra prestanda kan uppnås när kommande celler förutspås med AKOM modellen och associerade variationer. För prediktering av besökstid i kommande cell med linjär regression erhålles det däremot inte tillfredsställande resultat, vilket tyder på att en alternativ regressionsmetod antagligen är bättre lämpad för denna data.

The most important requirements for a video surveillance system are efficiency and effectiveness. In fact, it has to be fast in detecting a potentially dangerous event in real time, but it has also not to miss any of them. However, it would be even better if a system could detect dangerous events even before they actually occur. For that reason, in this paper we propose a very fast approach for learning and predicting event sequences in a surveillance context, that can also be applied to a robotic platform for improving the whole monitoring process. Preliminary experiments confirm that the proposed approach is very promising.

Дана робота використовує результати дослідження біологічних нейронних мереж для вдосконалення попередніх та побудови нових моделей просторо-часової асоціативної пам’яті. Запропоновано нову модель просторової асоціативної пам’яті на основі сігма-пай нейрона, що враховує дендритні обчислення та має високу ємність пам’яті, що за певними показниками перевершує класичну модель Вілшоу. Показано, що використання нейрона сігма-пай та розрідженої активації покращує на порядок ємність пам’яті переходів послідовності відомої моделі ієрархічно темпоральної пам’яті (HTM) при однаковій кількості зв’язків. Запропоновано нову модель представлення послідовності в бінарне розріджено-розподілене представлення, яка включає здатність нейрона до активації і якісно відтворює біологічні ефекти, такі як збереження схожості, чутливість до порядку, доповнення послідовності та часову схожість. Представлені моделі просторово-часової асоціативної пам’яті показують гірші результати розпізнавання образів для задач робототехніки в порівняні з підходом глибоких нейронних мереж, проте вони біологічно подібні, мають привабливі обчислювальні властивості і тому варті подальших досліджень.

Дана робота використовує результати дослідження біологічних нейронних мереж для вдосконалення попередніх та побудови нових моделей просторо-часової асоціативної пам’яті. Запропоновано нову модель просторової асоціативної пам’яті на основі сігма-пай нейрона, що враховує дендритні обчислення та має високу ємність пам’яті, що за певними показниками перевершує класичну модель Вілшоу. Показано, що використання нейрона сігма-пай та розрідженої активації покращує на порядок ємність пам’яті переходів послідовності відомої моделі ієрархічно темпоральної пам’яті (HTM) при однаковій кількості зв’язків. Запропоновано нову модель представлення послідовності в бінарне розріджено-розподілене представлення, яка включає здатність нейрона до активації і якісно відтворює біологічні ефекти, такі як збереження схожості, чутливість до порядку, доповнення послідовності та часову схожість. Представлені моделі просторово-часової асоціативної пам’яті показують гірші результати розпізнавання образів для задач робототехніки в порівняні з підходом глибоких нейронних мереж, проте вони біологічно подібні, мають привабливі обчислювальні властивості і тому варті подальших досліджень., This work presents biologically plausible models of spatio-temporal association memory based on recent ideas and discoveries from neuroscience. The architectures of most artificial neural networks, like convolutional neural networks, are based on biological ideas dated back to the 60-80s of the previous century. However, since then our understanding of neural computation changed dramatically and new evidence should be formalized, investigated and incorporated into new algorithms. The work starts with the presentation of a two-dimensional bidirectional plasticity model in a simple spiking neural network. The model includes triplet spike-time-dependent plasticity rule, fast homeostatic plasticity rule and reinforcement signal that significantly speeds up learning and stabilizes the weighs. It is based on the findings that molecular mechanisms in the dendritic spines work on different time scales and is similar to eligibility traces. The model showі learning of associative connections that are stable with time, and importantly, new experience does not significantly disrupt previously learned weights. The model has a possible biological interpretation, where the fast weight represents the number and efficiency of AMPA channels and slow weight represents slow molecular mechanisms, possibly CAMKII based, and surface effects on a membrane that determines the stabilization of AMPA receptors. However, it is argued that spiking neural networks are still inapplicable for a practical application like for robotics, so the rest of the work continues with more mathematically abstract, but computationally trackable models. The next model takes into account dendritic computation, particularly, synaptic clustering. Recently, it was shown that synapses from active input neurons group locally on the dendritic branch, so that later reactivation can cause supralinear summation and dendritic spike. Such coincident detection nature reflected in the sigma-pi neuron model that in this work u, Данная работа использует результаты исследования биологических нейронных сетей для совершенствования предыдущих и построения новых моделей пространственно-временной ассоциативной памяти. Предложена новая модель пространственной ассоциативной памяти на основе сигма-пай нейрона, что учитывает дендритные исчисления и имеет высокую емкость памяти, которая по определенным показателям превосходит классическую модель Вилшоу. Показано, что использование нейрона сигма-пай и разреженной активации улучшает на порядок емкость памяти переходов последовательности известной модели иерархически темпоральной памяти (HTM) при одинаковом количестве связей. Предложена новая модель представления последовательности в бинарное разреженно-распределенное представление, что включает способность нейрона к активации и качественно воспроизводит биологические эффекты, такие как сохранение сходства, чувствительность к порядку, дополнения последовательности и временное сходство. Представленные модели пространственно-временной ассоциативной памяти показывают худшие результаты распознавания образов для задач робототехники в сравнении с подходом глубоких нейронных сетей, однако они биологически подобные, имеют привлекательные вычислительные свойства и интересны для дальнейших исследований.

Predicting future behavior and positions of other traffic participants from observations is a key problem that needs to be solved by human drivers and automated vehicles alike to safely navigate their environment and to reach their desired goal. In this paper, we expand on previous work on an automotive environment model based on vector symbolic architectures (VSAs). We investigate a vector-representation to encapsulate spatial information of multiple objects based on a convolutive power encoding. Assuming that future positions of vehicles are influenced not only by their own past positions and dynamics (e.g., velocity and acceleration) but also by the behavior of the other traffic participants in the vehicle's surroundings, our motivation is 3-fold: we hypothesize that our structured vector-representation will be able to capture these relations and mutual influence between multiple traffic participants. Furthermore, the dimension of the encoding vectors remains fixed while being independent of the number of other vehicles encoded in addition to the target vehicle. Finally, a VSA-based encoding allows us to combine symbol-like processing with the advantages of neural network learning. In this work, we use our vector representation as input for a long short-term memory (LSTM) network for sequence to sequence prediction of vehicle positions. In an extensive evaluation, we compare this approach to other LSTM-based benchmark systems using alternative data encoding schemes, simple feed-forward neural networks as well as a simple linear prediction model for reference. We analyze advantages and drawbacks of the presented methods and identify specific driving situations where our approach performs best. We use characteristics specifying such situations as a foundation for an online-learning mixture-of-experts prototype, which chooses at run time between several available predictors depending on the current driving situation to achieve the best possible forecast., QC 20200406

Predictive business process monitoring methods exploit logs of completed cases of a process in order to make predictions about running cases thereof. Existing methods in this space are tailor-made for specific prediction tasks. Moreover, their relative accuracy is highly sensitive to the dataset at hand, thus requiring users to engage in trial-and-error and tuning when applying them in a specific setting. This paper investigates Long Short-Term Memory (LSTM) neural networks as an approach to build consistently accurate models for a wide range of predictive process monitoring tasks. First, we show that LSTMs outperform existing techniques to predict the next event of a running case and its timestamp. Next, we show how to use models for predicting the next task in order to predict the full continuation of a running case. Finally, we apply the same approach to predict the remaining time, and show that this approach outperforms existing tailor-made methods.

Protein X-ray crystallography images are usually studied for the purpose of identifying the three-dimensional structure of proteins at atomic resolution. These experiments produce hundreds and thousands of experimental results, leading to the need for automatic rather than manual analysis. The experiments are also time dependent and require monitoring over a long period of time, beginning from an hour of starting the experiment to a couple of months. The time gap between the imaged instances of these experiments also varies from frame to frame.This thesis introduces the use of difference images of an x-ray crystallography sequence in classifying the protein x-ray crystallography images, and exploits their various features along with the features of the original images. By further experiments using several combinations of difference images along with the original images, the significance of difference images in accurate classification of x-ray crystallography is established.At this stage, two independent feature sets for protein x-ray crystallography images are available. Based on these two feature sets, multi-view learning is employed for the purpose of classification of x-ray crystallography images, and random forests is shown to be a good base classifier for both views. During multi-view learning, different co-training methods are experimented with to determine the best possible classification result for protein x-ray crystallography images.After establishing by manual analysis that the same outcomes can be obtained via several different pathways in a crystallography time sequence, the results obtained from single frame classification experiments are used in the classification of protein x-ray crystallography sequences. Finally, the time varying nature of the x-ray crystallography experiments is further exploited together with the results of the previous experiments, to predict the final outcomes of crystallography experiments based on the first few frames. This p

Understanding inductive reasoning is a problem that has engaged mankind for thousands of years. This problem is relevant to a wide range of fields and is integral to the philosophy of science. It has been tackled by many great minds ranging from philosophers to scientists to mathematicians, and more recently computer scientists. In this article we argue the case for Solomonoff Induction, a formal inductive framework which combines algorithmic information theory with the Bayesian framework. Although it achieves excellent theoretical results and is based on solid philosophical foundations, the requisite technical knowledge necessary for understanding this framework has caused it to remain largely unknown and unappreciated in the wider scientific community. The main contribution of this article is to convey Solomonoff induction and its related concepts in a generally accessible form with the aim of bridging this current technical gap. In the process we examine the major historical contributions that have led to the formulation of Solomonoff Induction as well as criticisms of Solomonoff and induction in general. In particular we examine how Solomonoff induction addresses many issues that have plagued other inductive systems, such as the black ravens paradox and the confirmation problem, and compare this approach with other recent approaches.

Many learning tasks can be viewed as sequence prediction problems. For example, online classification can be converted to sequence prediction with the sequence being pairs of input/target data and where the goal is to correctly predict the target data given input data and previous input/target pairs. Solomonoff induction is known to solve the general sequence prediction problem, but only if the entire sequence is sampled from a computable distribution. In the case of classification and discriminative learning though, only the targets need be structured (given the inputs). We show that the normalised version of Solomonoff induction can still be used in this case, and more generally that it can detect any recursive sub-pattern (regularity) within an otherwise completely unstructured sequence. It is also shown that the unnormalised version can fail to predict very simple recursive sub-patterns.

The Minimum Description Length (MDL) principle selects the model that has the shortest code for data plus model. We show that for a countable class of models, MDL predictions are close to the true distribution in a strong sense. The result is completely general. No independence, ergodicity, stationarity, identifiability, or other assumption on the model class need to be made. More formally, we show that for any countable class of models, the distributions selected by MDL (or MAP) asymptotically predict (merge with) the true measure in the class in total variation distance. Implications for non-i.i.d. domains like time-series forecasting, discriminative learning, and reinforcement learning are discussed.

Suppose we are given two probability measures on the set of one-way infinite finite-alphabet sequences. Consider the question of when one of the measures predicts the other, that is, when conditional probabilities converge (in a certain sense), if one of the measures is chosen to generate the sequence. This question may be considered a refinement of the problem of sequence prediction in its most general formulation: for a given class of probability measures, does there exist a measure which predicts all of the measures in the class? To address this problem, we find some conditions on local absolute continuity which are sufficient for prediction and generalize several different notions that are known to be sufficient for prediction. We also formulate some open questions to outline a direction for finding the conditions on classes of measures for which prediction is possible.

Solomonoff’s central result on induction is that the prediction of a universal semimeasure M converges rapidly and with probability 1 to the true sequence generating predictor µ, if the latter is computable. Hence, M is eligible as a universal sequence predictor in the case of unknown µ. Despite some nearby results and proofs in the literature, the stronger result of convergence for all (Martin-Lof) random sequences remained open. Such a convergence result would be particularly interesting and natural, ¨ since randomness can be defined in terms of M itself. We show that there are universal semimeasures M which do not converge to µ on all µ-random sequences, i.e. we give a partial negative answer to the open problem. We also provide a positive answer for some non-universal semimeasures. We define the incomputable measure D as a mixture over all computable measures and the enumerable semimeasure W as a mixture over all enumerable nearly measures. We show that W converges to D and D to µ on all random sequences. The Hellinger distance measuring closeness of two distributions plays a central role.

The Bayesian framework is a well-studied and successful framework for inductive reasoning, which includes hypothesis testing and confirmation, parameter estimation, sequence prediction, classification, and regression. But standard statistical guidelines for choosing the model class and prior are not always available or can fail, in particular in complex situations. Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. I discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. I show that Solomonoff’s model possesses many desirable properties: strong total and future bounds, and weak instantaneous bounds, and, in contrast to most classical continuous prior densities, it has no zero p(oste)rior problem, i.e. it can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.

Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. We discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. We show that Solomonoff’s model possesses many desirable properties: Fast convergence and strong bounds, and in contrast to most classical continuous prior densities has no zero p(oste)rior problem, i.e. can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.

We outline a method for quantifying the error of a sequence prediction. With sequence predictions represented by semimeasures $ u(x)$ we define their error to be $-log_2 u(x)$. We note that enumerable semimeasures are those which model the sequence as the output of a computable system given unknown input. Using this we define the simulation complexity of a computable system $C$ relative to another $U$ giving an emph{exact} bound on their difference in error. This error in turn gives an exact upper bound on the number of predictions $ u$ gets incorrect.

We address the problem of sequence prediction for nonstationary stochastic processes. In particular, given two measures on the set of one-way infinite sequences over a finite alphabet, consider the question whether one of the measures predicts the other. We find some conditions on local absolute continuity under which prediction is possible.

Minimum description length (MDL) is an important principle for induction and prediction, with strong relations to optimal Bayesian learning. This paper deals with learning processes which are not necessarily independent and identically distributed, by means of two-part MDL, where the underlying model class is countable. We consider the online learning framework, i.e., observations come in one by one, and the predictor is allowed to update its state of mind after each time step.We identify two ways of predicting by MDL for this setup, namely, a static and a dynamic one. (A third variant, hybrid MDL, will turn out inferior.)We will prove that under the only assumption that the data is generated by a distribution contained in the model class, the MDL predictions converge to the true values almost surely. This is accomplished by proving finite bounds on the quadratic, the Hellinger, and the KullbackLeibler loss of the MDL learner, which are, however, exponentially worse than for Bayesian prediction. We demonstrate that these bounds are sharp, even for model classes containing only Bernoulli distributions. We show how these bounds imply regret bounds for arbitrary loss functions. Our results apply to a wide range of setups, namely, sequence prediction, pattern classification, regression, and universal induction in the sense of algorithmic information theory among others., Erratum published: IEEE Transactions on Information Theory. Volume 52, Issue 3, March 2006, p.1279, https://doi.org/10.1109/TIT.2006.869753

Minimum description length (MDL) is an important principle for induction and prediction, with strong relations to optimal Bayesian learning. This paper deals with learning processes which are not necessarily independent and identically distributed, by means of two-part MDL, where the underlying model class is countable. We consider the online learning framework, i.e., observations come in one by one, and the predictor is allowed to update its state of mind after each time step.We identify two ways of predicting by MDL for this setup, namely, a static and a dynamic one. (A third variant, hybrid MDL, will turn out inferior.)We will prove that under the only assumption that the data is generated by a distribution contained in the model class, the MDL predictions converge to the true values almost surely. This is accomplished by proving finite bounds on the quadratic, the Hellinger, and the KullbackLeibler loss of the MDL learner, which are, however, exponentially worse than for Bayesian prediction. We demonstrate that these bounds are sharp, even for model classes containing only Bernoulli distributions. We show how these bounds imply regret bounds for arbitrary loss functions. Our results apply to a wide range of setups, namely, sequence prediction, pattern classification, regression, and universal induction in the sense of algorithmic information theory among others., Erratum published: IEEE Transactions on Information Theory. Volume 52, Issue 3, March 2006, p.1279, https://doi.org/10.1109/TIT.2006.869753

We study the properties of the MDL (or maximum penalized complexity) estimator for Regression and Classification, where the underlying model class is countable. We show in particular a finite bound on the Hellinger losses under the only assumption that there is a ``true'' model contained in the class. This implies almost sure convergence of the predictive distribution to the true one at a fast rate. It corresponds to Solomonoff's central theorem of universal induction, however with a bound that is exponentially larger.

Erratum published: IEEE Transactions on Information Theory. Volume 52, Issue 3, March 2006, p.1279, Minimum description length (MDL) is an important principle for induction and prediction, with strong relations to optimal Bayesian learning. This paper deals with learning processes which are not necessarily independent and identically distributed, by means of two-part MDL, where the underlying model class is countable. We consider the online learning framework, i.e., observations come in one by one, and the predictor is allowed to update its state of mind after each time step.We identify two ways of predicting by MDL for this setup, namely, a static and a dynamic one. (A third variant, hybrid MDL, will turn out inferior.)We will prove that under the only assumption that the data is generated by a distribution contained in the model class, the MDL predictions converge to the true values almost surely. This is accomplished by proving finite bounds on the quadratic, the Hellinger, and the KullbackLeibler loss of the MDL learner, which are, however, exponentially worse than for Bayesian prediction. We demonstrate that these bounds are sharp, even for model classes containing only Bernoulli distributions. We show how these bounds imply regret bounds for arbitrary loss functions. Our results apply to a wide range of setups, namely, sequence prediction, pattern classification, regression, and universal induction in the sense of algorithmic information theory among others., https://doi.org/10.1109/TIT.2006.869753

Minimum description length (MDL) is an important principle for induction and prediction, with strong relations to optimal Bayesian learning. This paper deals with learning processes which are not necessarily independent and identically distributed, by means of two-part MDL, where the underlying model class is countable. We consider the online learning framework, i.e., observations come in one by one, and the predictor is allowed to update its state of mind after each time step.We identify two ways of predicting by MDL for this setup, namely, a static and a dynamic one. (A third variant, hybrid MDL, will turn out inferior.)We will prove that under the only assumption that the data is generated by a distribution contained in the model class, the MDL predictions converge to the true values almost surely. This is accomplished by proving finite bounds on the quadratic, the Hellinger, and the KullbackLeibler loss of the MDL learner, which are, however, exponentially worse than for Bayesian prediction. We demonstrate that these bounds are sharp, even for model classes containing only Bernoulli distributions. We show how these bounds imply regret bounds for arbitrary loss functions. Our results apply to a wide range of setups, namely, sequence prediction, pattern classification, regression, and universal induction in the sense of algorithmic information theory among others., Erratum published: IEEE Transactions on Information Theory. Volume 52, Issue 3, March 2006, p.1279, https://doi.org/10.1109/TIT.2006.869753

We study the properties of the Minimum Description Length principle for sequence prediction, considering a two-part MDL estimator which is chosen from a countable class of models. This applies in particular to the important case of universal sequence prediction, where the model class corresponds to all algorithms for some fixed universal Turing machine (this correspondence is by enumerable semimeasures, hence the resulting models are stochastic). We prove convergence theorems similar to Solomonoff’s theorem of universal induction, which also holds for general Bayes mixtures. The bound characterizing the convergence speed for MDL predictions is exponentially larger as compared to Bayes mixtures. We observe that there are at least three different ways of using MDL for prediction. One of these has worse prediction properties, for which predictions only converge if the MDL estimator stabilizes. We establish sufficient conditions for this to occur. Finally, some immediate consequences for complexity relations and randomness criteria are proven.