Search

Based on recent advancements in natural language processing, computer vision and robotics, a growing number of researchers and traders attempt to predict future asset prices using deep learning techniques. Typically, the goal is to find a profitable and at the same time low-risk trading strategy. However, it is not straightforward to evaluate a found trading strategy. Evaluating solely on historic price data neglects important factors arising in real markets. In this paper, we analyze the impact of real-world market conditions in terms of trading fees, borrow interests, slippage and spreads on trading returns. For that, we propose a deep learning trading bot based on Temporal Convolutional Networks, which is deployed to a real cryptocurrency exchange. We compare the results obtained in the real market with simulated returns and investigate the impact of the different real-world market conditions. Our results show that besides trading fees (which have the biggest impact on returns), factors like slippage and spread also affect the returns of the trading strategy.

Experiments for the subjective evaluation of multimedia presentations and content are traditionally conducted in a laboratory environment. In this respect common procedures for the evaluation of teleconference systems are no different. The strictly controlled laboratory environment, however, often gives a rather poor representation of the actual use case. Therefore in this study we crowdsourced the evaluation of a teleconference system to perform the evaluation in a real-life environment. Moreover, we used the unique possibilities of crowdsourcing to employ two different demographics by hiring workers from Germany on the one hand and the US and Great Britain on the other hand. The goal of this experiment was to assess the perceived Quality of Experience (QoE) during a listening test and compare the results to results from a similar listening test conducted in the controlled laboratory environment. In doing so, we observed not only intriguing differences in the collected QoE ratings between the results of laboratory and crowdsourcing experiments, but also between the different worker demographics in terms of reliability, availability and efficiency.

In this work a novel approach to Transfer Learning for the use in Deep Reinforcement Learning is introduced. The agent is realized as an actor-critic framework, namely the Deep Deterministic Policy Gradient algorithm. The Q-function and the policy are represented as deep feed-forward networks, that are trained by minimizing the mean squared Bellman error and by maximizing the expected reward, respectively. For Transfer Learning, the actor is modified with a new regularization term, called the knowledge regularizer. It allows to include prior knowledge in from of an existing policy in the learning process. The knowledge regularizer shifts the current weight vector during the gradient descent step towards a region of the weight space, that is centered around the existing policy. Because neural networks are universal and smooth function approximators, the weights of the existing policy and the new ones have to lie close to each other in the weight space. Solving a task therefore benefits from the prior knowledge, when it is used to manipulate the gradient given by the critic. We could experimentally verify, that the knowledge regularizer results in a higher performance achieved by the agent and in a reduction of the learning time. Furthermore, the knowledge regularizer can be used as a replacement for labeled training data, which renders it especially useful for physical applications.

Numerous dynamic influences affect producing companies and require a continuous adaptation of the production. At present, modeling these influences and their interdependencies is mainly limited to quantitative factors. In this paper, an approach is proposed that allows to model the missing qualitative influences and their interdependencies via recurrent fuzzy systems (RFS), where the transitions between the state variables are additionally weighted (transition adaptation). This allows an easy adaption of a production environment's behavior while maintaining the interpretability of the model and the model-based analysis results. To handle the complexity of the resulting models, a structured way to simplify a large fuzzy rule base, to reduce the number of required weighting coefficients, and to merge the state variable onto a single production effectiveness value is shown. All of this is directly illustrated by exemplarily modeling the influence of some relevant qualitative factors onto a production environment.

Quality of experience (QoE) is becoming increasingly important in signal processing applications. In taking inspiration from chemometrics, we provide an introduction to the design of video quality metrics by using data analysis methods, which are different from traditional approaches. These methods do not necessitate a complete understanding of the human visual system (HVS). We use multidimensional data analysis, an extension of well-established data analysis techniques, allowing us to better exploit higher-dimensional data. In the case of video quality metrics, it enables us to exploit the temporal properties of video more properly; the complete three-dimensional structure of the video cube is taken into account in metrics' design. Starting with the well-known principal component analysis and an introduction to the notation of multiway arrays, we then present their multidimensional extensions, delivering better quality prediction results. Although we focus on video quality, the presented design principles can easily be adapted to other modalities and to even higher dimensional data sets as well.

The open access application format is an integrated exchange format that improves the management and exchange of content. It can enable Web sites, archives, or search engines to be able to perform automatic processing, indexing, and presentation of content. This method also allows the interoperable exchange of content between providers or between users. For the scientific community, this capability would enhance the visibility and comparability of the content and thus improve the organization and exchange of scientific literature. This would be helpful not only for the scientific community, but also other communities and organizations whose goal is to efficiently distribute and exchange open content.

Fluctuations of video quality over time can have a significant influence on the overall perceived quality as represented by the QoE. Existing methodologies for subjective video quality assessment, however, are often not suitable for the evaluation of these quality fluctuations, especially if they occur within very small time frames. In this contribution, we therefore propose a new method, VIQPAC, which addresses this shortcoming by using a pattern categorisation approach. Instead of requiring the subjects to provide a continuous quality evaluation, the subjects assess the overall quality impression and the strength of the quality fluctuation, combined with a categorisation of the encountered fluctuation pattern. This allows us to determine the fluctuation dependent temporal changes in the quality. The results show that VIQPAC is able to capture the pattern and strength of quality fluctuations, allowing for a proper description of the temporal quality changes within a video sequence.

Quality of Experience (QoE) in multimedia applications is closely linked to the end users' perception and therefore its assessment requires subjective user studies in order to evaluate the degree of delight or annoyance as experienced by the users. QoE crowdtesting refers to QoE assessment using crowdsourcing, where anonymous test subjects conduct subjective tests remotely in their preferred environment. The advantages of QoE crowdtesting lie not only in the reduced time and costs for the tests, but also in a large and diverse panel of international, geographically distributed users in realistic user settings. However, conceptual and technical challenges emerge due to the remote test settings. Key issues arising from QoE crowdtesting include the reliability of user ratings, the influence of incentives, payment schemes and the unknown environmental context of the tests on the results. In order to counter these issues, strategies and methods need to be developed, included in the test design, and also implemented in the actual test campaign, while statistical methods are required to identify reliable user ratings and to ensure high data quality. This contribution therefore provides a collection of best practices addressing these issues based on our experience gained in a large set of conducted QoE crowdtesting studies. The focus of this article is in particular on the issue of reliability and we use video quality assessment as an example for the proposed best practices, showing that our recommended two-stage QoE crowdtesting design leads to more reliable results.

In previous publications it has been shown that no-reference video quality metrics based on a data analysis approach rather than on modeling the human visual system lead to very promising results and outperform many well-known full-reference metrics. Furthermore, the results improve when taking the temporal structure of the video sequence into account by using multiway analysis methods. This contribution shows a way of refining these multiway quality metrics in order to make them more suitable for real-life applications and maintaining the performance at the same time. Additionally, our results confirm the validity of H.264/AVC bitstream no-reference quality metrics using multiway PLSR by evaluating this concept on an additional dataset.

In this paper, we discuss an approach to evaluate decisions made during a multi-armed bandit learning experiment. Usually, the results of machine learning algorithms applied on multi-armed bandit scenarios are rated in terms of earned reward and optimal decisions taken. These criteria are valuable for objective comparison in finite experiments. But learning algorithms used in real scenarios, for example in robotics, need to have instantaneous criteria to evaluate their actual decisions taken. To overcome this problem, in our approach each decision updates the Zurich model which emulates the human sense of feeling secure and aroused. Combining these two feelings results in an emotional evaluation of decision policies and could be used to model the emotional state of an intelligent agent.

This work examines how adaptive control can learn to point a camera at the active speaker in a conversation by using a Reinforcement Learning approach with audio and video data. A motivating scenario for this problem is a robotic platform that interacts with people around its environment. Using Reinforcement Learning, the task is specified with an observable objective referred to as the reward signal. Specifying this task with a reward signal enables an adaptive controller to improve its performance with experience. The reward for this task is generated by a visual feedback from the conversation participants that is detected by the robot's camera system. Multiple experiments have been performed on a robot system with audiovisual data to examine the feasibility and potential of this approach. Our experimental results demonstrate that the system learns very fast to identify the active speakers. Furthermore, our approach inherently learns how to deal with egonoise that originates from the robot's motor or background noise from the environment.

Video quality evaluation with subjective testing is both time consuming and expensive. A promising new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the Internet. The advantages of this approach are not only the access to a larger and more diverse pool of test subjects, but also the significant reduction of the financial burden. Recent contributions have also shown that crowd-based video quality assessment can deliver results comparable to traditional testing in some cases. In general, however, new problems arise, as no longer every test detail can be controlled, resulting in less reliable results. Therefore we will discuss in this contribution the conceptual, technical, motivational and reliability challenges that need to be addressed, before this promising approach to subjective testing can become a valid alternative to the testing in standardized environments.

The research on video quality metrics depends on the results from subjective testing for both the design and development of metrics, but also for their verification. As it is often too cumbersome to conduct subjective tests, freely available data sets that include both mean opinion scores and the distorted videos are becoming ever more important. While many datasets are already widely available, the majority of these data sets focus on smaller resolutions. We therefore present in this contribution the TUM high definition datasets that include videos in both 1080p25 and 1080p50, encoded with different coding technologies and settings, H.264/AVC and Dirac, but also different presentation devices from reference monitors to home-cinema projectors. Additionally a soundtrack is provided for the home-cinema scenario. The datasets are made freely available for download under a creative commons license.

This paper presents an overview of the new version of our multi-model communication face “Mask-Bot”, a rear-projected animated robotic head, including our display system, face animation, speech communication and sound localization.

Video quality assessment with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. We therefore propose in this contribution the QualityCrowd framework to effortlessly perform subjective quality assessment with crowdsourcing. QualityCrowd allows codec independent quality assessment with a simple web interface, usable with common web browsers. We compared the results from an online subjective test using this framework with the results from a test in a standardized environment. This comparison shows that QualityCrowd delivers equivalent results within the acceptable inter-lab correlation. While we only consider video quality in this contribution, QualityCrowd can also be used for multimodal quality assessment.

Today in many homes big TV screens and hifi systems are common. But is the perception of subjective video quality under professional test conditions the same as at home? Therefore we examined two things: How large is the influence of the presentation device but also the influence of the soundtrack, both in HDTV (1080p50). Previous work has shown that a difference is noticeable, but there have not been studies with consumer devices, yet. It was also shown that there is an influence of the soundtrack, but only in SDTV or lower resolutions. Therefore we conducted subjective video tests: One test with different devices, a 23-inch-reference monitor, a high quality 56-inch-LCD-TV and an HD-projector, and one test in which we presented a soundtrack on a 7.1-channel hifi system in addition to the HD-projector. The results show two things: First the test subjects had a higher quality of experience with the consumer devices than with the reference monitor, although the video quality of the reference monitor itself was rated better in an additional questionnaire and the mean opinion score (MOS). The second result is that there is no significant difference in the MOS between showing the videos on the projector with or without sound.

Video quality evaluation with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. The QualityCrowd framework allows codec independent, crowd-based video quality assessment with a simple web interface, usable with common web browsers. However, due to its codec independent approach, the framework can pose high bandwidth requirements on the coordinating server. We therefore propose in this contribution a cloud-based extension of the QualityCrowd framework in order to perform subjective quality evaluation as a cloud application. Moreover, this allows us to access an even larger pool of potential participants due to the improved connectivity. We compare the results from an online subjective test using this framework with the results from a test in a standardized environment. This comparison shows that QualityCrowd delivers equivalent results within the acceptable inter-lab correlation.

This paper describes an integration of a 3D sound convolution engine into a telephone conference server software. Our system virtually places participants of a telephone conference to different positions around the listeners by using Head Related Transfer Functions (HRTFs), which helps to identify active speakers in an audio conference. Intelligibility of communication is also increased by making use of the cocktail party effect which helps to concentrate on one specific participant, even if there are more simultaneously active speakers. To support regular telephone systems, we focus in particular on compatibility to existing Voice over IP (VoIP) infrastructure by using standardized protocols like the Session Initiation Protocol (SIP) and the Real-Time Transport Protocol (RTP). Both traditional telephone devices and software clients are put into one heterogeneous conference. While the conference server transmits the down-mixed, mono single-channel conference signal to regular phones, the software clients can receive a stereo-signal enabling full 3D audio.

High definition television (HDTV) has become quite common in many homes. Still, there are two different formats used currently in commercial broadcasting: one interlaced format, 1080i50/60, and one progressive format, 720p50/60.There have already been quite a few contributions comparing the visual quality of these formats subjectively under common standard conditions. These conditions, however, dont necessarily represent the viewing conditions in the real-life consumer environment. In this contribution we therefore decided to do a comparison under conditions more representative of the consumer environment with respect to display and viewing conditions. Furthermore, we decided to select not specially prepared test sequences, but real-life content and coding conditions. As we were not interested in the influence of the transmission errors, we captured the sequences directly in the play-out centre of a cable network provider in both 1080i50 and 720p50. Also we captured for comparison the same content in digital PAL-SDTV. We conducted extensive subjective tests with overall 25 test subjects and a modified SSIS method. The results show that both HDTV formats outperform SDTV significantly. Although 720p50 is perceived to have a better quality than 1080i50, this difference is not significant in a statistical sense. This supports the validity of previous contributions results, gained in standard conditions, also for the real-life consumer environment.

No-reference video quality metrics are becoming ever more popular, as they are more useful in real-life applications compared to full-reference metrics. Many proposed metrics extract features related to human perception from the individual video frames. Hence the video sequences have to be decoded first, before the metrics can be applied. In order to avoid the decoding just for quality estimation, we therefore present in this contribution a no-reference metric for HDTV that uses features directly extracted from the H.264/AVC bitstream. We combine these features with the results from subjective tests using a data analysis approach with partial least squares regression to gain a prediction model for the visual quality. For verification, we performed a cross validation. Our results show that the proposed no-reference metric outperforms other metrics and delivers a correlation between the quality prediction and the actual quality of 0.93.

This paper describes an integration of a Head Related Transfer Function (HRTF)-based 3D sound convolution engine into the open-source VoIP conferencing software Mumble. Our system allows to virtually place audio contributions of conference participants to different positions around a listener, which helps to overcome the problem of identifying active speakers in an audio conference. Furthermore, using HRTFs to generate 3D sound in virtual 3D space, the listener is able to make use of the cocktail party effect in order to differentiate between several simultaneously active speakers. As a result intelligibility of communication is increased.

High definition video over IP based networks (IPTV) has become a mainstay in today’s consumer environment. In most applications, encoders conforming to the H.264/AVC standard are used. But even within one standard, often a wide range of coding tools are available that can deliver a vastly different visual quality. Therefore we evaluate in this contribution different coding technologies, using different encoder settings of H.264/AVC, but also a completely different encoder like Dirac. We cover a wide range of different bitrates from ADSL to VDSL and different content, with low and high demand on the encoders. As PSNR is not well suited to describe the perceived visual quality, we conducted extensive subject tests to determine the visual quality. Our results show that for currently common bitrates, the visual quality can be more than doubled, if the same coding technology, but different coding tools are used.

The collaborative research center SFB453 (www.sfb453.de) aims to realize high-fidelity telepresence and teleaction systems.

Vehicles will exchange much information in the future in order to efficiently maintain their inner model of the environment. Before they can belief received pieces of information, they must evaluate their reliability. Trust is a mechanism to estimate this reliability based on the sender. As cars often drive the same route, they meet each other again and again. They can establish friendship-like relations and thus are embedded in a social structure. A trust model depends on this social structure. For this reason, we simulate the driving pattern of a small town. Within this simulation, all cars are equipped with a trust model that continuously monitors the experiences made with others. The developed model focuses on direct experiences of the individual and not on a system-wide reputation which would depend on a central unit. It continuously evaluates the performance and reputation of other cars and includes a feedback loop to faster adapt to changes in the other's behaviour. To make a decision out of the collected data, the model uses the capacity of the binary error and erasure channel from information theory. This capacity provides a better decision criterion than the traditional expectation value. The proposed trust model is an individual-level model; nonetheless it can be connected to a system-wide reputation mechanism.

The most important step in the development process of a video quality metric is its verification with regards to the subjective quality experience. Even though guidelines in the form of standards and recommendations are well known, there are still quite often shortcomings in the verification process of many metrics. In this contribution we revisit these rules, point out important details and review contributions to video quality metrics for typical shortcomings. We will highlight in detail five steps that should be followed in order to improve the overall quality of the verification process of video quality metrics: using a large and diverse data base, planing and conducting subjective tests carefully, using different data for calibration and verification of a metric, avoiding unnecessary data fitting steps, and a clear and meaningful presentation of the results. Also we will provide short examples how an improper verification might affect the results of a video quality metric.

An algorithm for real time humanoid sound localization and tracking using only two microphones in a highly reverberant environment is proposed. Several recently developed 3D humanoid sound localization algorithms require the environment to be anechoic. Also, the resolution of front-back ambiguity problem during sound localization requires the knowledge about the reference signals. Using HRTF (head related transfer functions) based sound localization together with extended Kalman filtering, we are able to accurately track moving sound sources in real time in a highly reverberant environment. This algorithm uses only two microphones and requires no prior knowledge of the reference signals.

One of the key features of the human auditory system, is its nearly constant omni-directional sensitivity, e.g., the system reacts to alerting signals coming from a direction away from the sight of focused visual attention. In many surveillance situations where visual attention completely fails since the robot cameras have no direct line of sight with the sound sources, the ability to estimate the direction of the sources of danger relying on sound becomes extremely important. We present in this paper a novel method for sound localization in azimuth and elevation based on a humanoid head. The method was tested in simulations as well as in a real reverberant environment. Compared to state-of-the-art localization techniques the method is able to localize with high accuracy 3D sound sources even in the presence of reflections and high distortion.

We present a new binaural sound-source separation and localization technique for the underdetermined case where the present sound sources outnumber the available microphones. The proposed technique has access to a generic set of head-related transfer functions (HRTFs) and processes input signals obtained from two small microphones placed inside the ear canals of a robot humanoid head equipped with artificial ears and mounted on a torso. By exploiting sparse representations of the ear input signals, the 3D position of three concurrent sound sources is extracted by identifying the HRTFs that have filtered the sound signals. The sought HRTFs are estimated using a well-known self-splitting competitive learning technique for clustering. Simulation results demonstrated the performance of the new technique in localizing both azimuth and elevation angles for three concurrently active sound sources. The proposed method relies purely on auditive cues and provides an easy implementation on robotic platforms.

In this paper, we combine blind sound separation (BSS) and binaural localization for fast tracking of two concurrent sound sources. Using only two microphones placed inside the ear canals of a robot dummy head mounted on a torso and equipped with artificial ears, a generic set of Head-Related Transfer Functions (HRTFs) is used for accurate 3D source localization. This novel approach of combining the two BSS and system identification processes, and executing them simultaneously, is optimized for moving sound sources. Simulation results demonstrated precise localization of both elevation and azimuth angles of the two concurrent sound sources. The proposed method relies purely on auditive cues and uses a simple algorithm which, compared to the current localization algorithms, provides an easy implementation on robotic platforms and allows accurate as well as fast localization of concurrent sources.

The industry is currently starting to use MPEG-4 compressed digital video for surveillance applications. The transition from analog to digital video raises difficulties for using surveillance video in court. Since it is fairly easy to make hard-to-detect modifications to the video stream captured by the cameras, e.g. mask out a specific event or person, a system for proving authenticity and integrity of video streams is needed. This paper presents such a system based on digital signatures embedded in the video stream. Our concept provides a means for proving authenticity and integrity of MPEG-4 digital video streams, while leaving compatibility with standard media players untouched.

MPEG standards, instrumental in shaping the multimedia landscape since the early 1990's, are taking another step forward with MPEG-A, which specifies multimedia application formats. MAFs are essentially superformats that combine selected technology components from MPEG (and other) standards to provide greater application interoperability, which helps satisfy users' growing need for better-integrated multimedia solutions.