Search

Current progress in the artificial intelligence domain has led to the development of various types of AI-powered dementia assessments, which can be employed to identify patients at the early stage of dementia. It can revolutionize the dementia care settings. It is essential that the medical community be aware of various AI assessments and choose them considering their degrees of validity, efficiency, practicality, reliability, and accuracy concerning the early identification of patients with dementia (PwD). On the other hand, AI developers should be informed about various non-AI assessments as well as recently developed AI assessments. Thus, this paper, which can be readable by both clinicians and AI engineers, fills the gap in the literature in explaining the existing solutions for the recognition of dementia to clinicians, as well as the techniques used and the most widespread dementia datasets to AI engineers. It follows a review of papers on AI and non-AI assessments for dementia to provide valuable information about various dementia assessments for both the AI and medical communities. The discussion and conclusion highlight the most prominent research directions and the maturity of existing solutions., Comment: 49 pages

The recent wave of AI research has enabled a new brand of synthetic media, called deepfakes. Deepfakes have impressive photorealism, which has generated exciting new use cases but also raised serious threats to our increasingly digital world. To mitigate these threats, researchers have tried to come up with new methods for deepfake detection that are more effective than traditional forensics and heavily rely on deep AI technology. In this paper, following up on encouraging prior work for deepfake detection with attribution and ensemble techniques, we explore and compare multiple designs for ensemble detectors. The goal is to achieve robustness and good generalization ability by leveraging ensembles of models that specialize in different manipulation categories. Our results corroborate that ensembles can achieve higher accuracy than individual models when properly tuned, while the generalization ability relies on access to a large number of training data for a diverse set of known manipulations., Comment: 16 pages, 4 figures

The evolution of Knowledge Graphs (KGs), during the last two decades, has encouraged developers to create more and more context related KGs. This advance is extremely important because Artificial Intelligence (AI) applications can access open domain specific information in a semantically rich, machine understandable format. In this paper, we present the XR4DRAMA KG which can represent information for disaster management. More specifically, the XR4DRAMA KG can represent information about: (a) Observations and Events (e.g., data collection of biometric sensors, information in photos and text messages), (b) Spatio-temporal (e.g., highlighted locations and timestamps), (c) Mitigation and response plans in crisis (e.g., first responder teams). Moreover, we offer a mechanism that can create or update Points-Of-Interest (POIs), based on a visual or textual messages received from users.

The document reflects on the lack of specific policy recommendations regarding the use of AI applications and tools in the media sector. This absence of guidance creates important challenges for accomplishing the EU's vision on trustworthy AI. The deliverable investigates how this could be fixed. It is divided into three main parts. Section 3 identifies challenges for the use of AI applications in the media sector. Section 4 provides the initial policy recommendations addressing these challenges. Section 5 assesses the feasibility of adopting the European Digital Media Code of Conduct to mitigate some of the challenges faced by the media sector. The final section offers concluding remarks and explains next steps planned by the AI4Media consortium towards the final policy recommendations due in month 48 of the project. ispartof: Pilot Policy Recommendations for the use of AI in the Media Sector status: Published online

Attack pattern identification is a significant step for protecting organisations from cyber-threats, as it can be used to reveal valuable patterns, enabling the better detection and analysis of the respective attacks that can be leveraged for the development of effective and efficient Intrusion Detection Systems. In this work, Association Rule Learning (ARL), a data mining technique, is used for the identification of attack patterns from data collected from a public honeypot. Using the FP-Growth ARL algorithm, we identified different patterns of attacks and correlated the respective commands executed by various attackers. To our knowledge, this is the first time ARL has been used to extract attack patterns from commands run by the attackers using real-world log data collected at the host level., This is the accepted version of the paper. The final version of the paper can be found at https://ieeexplore.ieee.org/xpl/conhome/9850275/proceeding

Satellite images play a crucial role in monitoring Earth’s oceans, especially when it comes to oil spills. Traditionally, detection methods use Synthetic Aperture Radar (SAR) images that allow the detection of oil spills independent of clouds or daylight. However, SAR based methods are limited by wind conditions, as well as, look-alikes. Multispectral satellite images are perfect tools to fill this gap given that they allow the detection of pollution when weak or strong winds do not allow the use of SAR images. For this, a case of oil spill contamination is investigated in an inland lake in northern Greece using Sentinel-2 and PlanetScope multi-spectral images. This case is characterized by a small sample of known oil spills, making this study even more challenging. First, we implement different atmospheric corrections to acquire the remote sensing reflectance for the multispectral bands. Our sensitivity analysis shows that the detection capability for oil spills in not constrained only to the visible (VIS) part of the spectrum, but also extends to the Near Infrared (NIR), as well as, the Short Wavelength Infrared (SWIR). Among these, the NIR (833 nm) and Narrow NIR (865 nm) seem to have the largest sensitivity to fresh water oil spills. Additionally, the oil spills investigated tend to enhance the remote sensing reflectance for the NIR and SWIR part of the spectrum, but reduce it for the VIS bands, with the exception of Red (665 nm), which has a more ambiguous behavior. Given the reduced known oil spill cases (just two) for this study a pixel based machine learning approach is implemented instead of an object based one. Furthermore, the size of the oil spills will determine the choice of the bands, given that low resolution bands tend to reduce the pixel sample, and high resolution bands are limited by their availability (only four are available). Finally, the chosen bands are feed into a Deep Neural Network with two hidden layers for processing and the optimal hyperparameters are investigated. Despite the limited oil spill sample, the results are encouraging, showing a good detection capability.

Cloud cover is a common issue with optical satellite images, severely affecting the quality and the spatiotemporal availability of surface observations. Clouds can critically impact the utility of satellite images by completely covering the ground below, or distorting the measurements collected. While a generally known practice is cloud identification and removal of affected areas, there is also a growing interest in filling the gaps created with the use of powerful data-driven deep learning methods. Images produced by the Sentinel-2 mission come with such cloud occlusions and a common alternative is the use of Synthetic Aperture Radar (SAR) data, as they are nearly independent of the atmospheric conditions and solar illumination. However, SAR data share entirely different characteristics compared to optical data. Even though Sentinel-1 has the ability to provide continuous, day-and-night observations and to overcome various kinds of bad weather conditions or poor air quality (e.g., clouds, rain, fog and smoke), the information captured by this mission is less descriptive and more complex to interpret than that of optical images. In that case, Generative Adversarial Networks (GANs) are used to translate SAR to optical imagery, while many researchers have recently proposed the information fusion of SAR (Sentinel-1) and optical (Sentinel-2) images with different motives. &nbsp

The frequency of wildfires is growing day by day due to vastly climate changes. Forest fires can have a severe impact on human lives and the environment, which can be minimised if the population has early and accurate warning mechanisms. To date, social media are able to contribute to early warning with the additional, crowd-sourced information they can provide to the emergency response workers during a crisis event. Nevertheless, the detection of real-world fire incidents using social media data, while filtering out the unavoidable noise, remains a challenging task. In this paper, we present an alert framework for the real-time detection of fire events and we propose a novel multimodal event detection model, which fuses both probabilistic and graph methodologies and is evaluated on the largest fires in Spain during 2019.

Cyberattacks are becoming more sophisticated, posing even greater challenges to traditional intrusion detectionEngl methods. Failure to prevent the intrusions could jeopardise security services’ credibility, including data confidentiality, integrity, and availability. Anomaly-based Intrusion Detection Systems and Signature-based Intrusion Detection Systems are two types of systems that have been proposed in the literature to detect security threats. In the current work, a taxonomy of current IDSs is presented, a review of recent works is performed, and we discuss some of the most common datasets used for evaluation. Finally, the survey concludes with a discussion of future IDS research directions and broader observations.

During the last decades, massive amounts of satellite images are becoming available that can be enriched with semantic annotations for the creation of value-added Earth Observation products. One challenge is to extract knowledge from the raw satellite data in an automated way and to effectively manage the extracted information in a semantic way, to allow fast and accurate decisions of spatio-temporal nature in a real operational scenario. In this work we present a framework that combines supervised learning for crop type classification on satellite imagery time-series with Semantic Web and Linked Data technologies to assist in the implementation of rule sets by the European Common Agricultural Policy (CAP). The framework collects georeferenced data that are available online and satellite images from the Sentinel-2 mission. We analyze image time-series that cover the entire cultivation period and link each parcel with a specific crop. On top of that, we introduce a semantic layer to facilitate a knowledge-driven management of the available information, capitalizing on ontologies for knowledge representation and semantic rules, to identify possible farmers noncompliance according to the Greening 1 (Crop Diversification) and SMR 1 rule (protection of waters against pollution caused by nitrates) rules of the CAP. Experiments show the effectiveness of the proposed integrated approach in three different scenarios for crop type monitoring and consistency-checking for noncompliance to the CAP rules; the smart sampling of on-thespot-checks; the automatic detection of CAP’s Greening 1 rule; and the automatic detection of susceptible parcels according to the CAP’s SMR 1 rule.

The necessity of organising big streams of Earth Observation (EO) data induces the efficient clustering of image patches, deriving from satellite imagery, into groups. Since the different concepts of the satellite image patches are not known a priori, DBSCAN-Martingale can be applied to estimate the number of the desired clusters. In this paper we provide a parallel version of the DBSCAN-Martingale algorithm and a framework for clustering EO data in an unsupervised way. The approach is evaluated on a benchmark dataset of Sentinel-2 images with ground-truth annotation and is also implemented on High Performance Computing (HPC) infrastructure to demonstrate its scalability. Finally, a cost-benefit analysis is conducted to find the optimal selection of reserved nodes for running the proposed algorithm, in relation to execution time and cost.

This paper presents VERGE, an interactive video search engine that integrates multiple retrieval methodologies and also combines them with reranking and fusion techniques. Moreover, a user interface, implemented as a Web application, enables users to formulate queries, view the top retrieved shots and watch the respective videos, before submitting a shot to a VBS task, all in an ecient and easy manner.

In this paper we introduce InDistill, a model compression approach that combines knowledge distillation and channel pruning in a unified framework for the transfer of the critical information flow paths from a heavyweight teacher to a lightweight student. Such information is typically collapsed in previous methods due to an encoding stage prior to distillation. By contrast, InDistill leverages a pruning operation applied to the teacher's intermediate layers reducing their width to the corresponding student layers' width. In that way, we force architectural alignment enabling the intermediate layers to be directly distilled without the need of an encoding stage. Additionally, a curriculum learning-based training scheme is adopted considering the distillation difficulty of each layer and the critical learning periods in which the information flow paths are created. The proposed method surpasses state-of-the-art performance on three standard benchmarks, i.e. CIFAR-10, CUB-200, and FashionMNIST by 3.08%, 14.27%, and 1% mAP, respectively, as well as on more challenging evaluation settings, i.e. ImageNet and CIFAR-100 by 1.97% and 5.65% mAP, respectively.

Most language learning applications are aimed at students or people who already know a language and want to improve their skills, or want to learn a new language. These applications, while seeking to be interactive, are not aimed at immigrants, refugees or asylum seekers, since the latter have different needs and interests from casual learners, opting for language skills that will allow them to function independently in the host society. This research is part of the European project WELCOME, which seeks to use state-of-the-art technologies, such as Virtual Reality (VR) apps and dialogue agents, to support the reception and integration of Third Country Nationals (TCNs) in Europe. The platform will be tested in three languages (Catalan, German and Greek), in real situations that the TCNs, mostly immigrants, refugees and asylum seekers, face, combining linguistic activities, where aspects related to language and culture are worked on. Furthermore, a Visual Analytics Component (VAC) leverages authority (NGOs/State institutions) users’ perceptual and cognitive abilities by employing interactive visualisations as interfaces between users and learning analytics outcomes generated by amassed data. The goal is to find patterns within the characteristics of TCNs, and thus help language teachers adapt the content and tools to TCNs, contributing to greater personalisation in learning.

Web bots vary in sophistication based on their purpose, ranging from simple automated scripts to advanced web bots that have a browser fingerprint, support the main browser functionalities, and exhibit a humanlike behaviour. Advanced web bots are especially appealing to malicious web bot creators, due to their browser-like fingerprint and humanlike behaviour which reduce their detectability. This work proposes a web bot detection framework that comprises two detection modules: (i) a detection module that utilises web logs, and (ii) a detection module that leverages mouse movements. The framework combines the results of each module in a novel way to capture the different temporal characteristics of the web logs and the mouse movements, as well as the spatial characteristics of the mouse movements. We assess its effectiveness on web bots of two levels of evasiveness, (a) moderate web bots that have a browser fingerprint and (b) advanced web bots that have a browser fingerprint and also exhibit a humanlike behaviour. We show that combining web logs with visitors’ mouse movements is more effective and robust towards detecting advanced web bots that try to evade detection, as opposed to using only one of those approaches.

CALLISTO is a project funded by the European Union’s Horizon 2020 Research and Innovation Programme under the topic “Big data technologies and Artificial Intelligence for Copernicus” in 2020 and has started in January 2021, with a duration of three years. In this work we introduce the CALLISTO project and describe its concept and approach, the use cases of application and the expected impact.

This paper presents VERGE, an interactive video search engine that supports efficient browsing and searching into a collection of images or videos. The framework involves a variety of retrieval approaches as well as reranking and fusion capabilities. A Web application enables users to create queries and view the results in a fast and friendly manner.

This paper provides an overview of the runs submitted to TRECVID 2020 by ITI-CERTH. ITI-CERTH participated in the Ad-hoc Video Search (AVS), Disaster Scene Description and Indexing (DSDI) and Activities in Extended Video (ActEV) tasks. Our AVS task participation is based on an attention-based cross-modal deep network method for retrieving video shots relevant to ad-hoc textual queries. The DSDI task is performed by implementing a multi-label image classification model, trained on all humanly annotated images and estimating the final classes on averaging the predictions on the keyframes of the video shots. For the ActEV task, we deploy an object detection algorithm and then convert the individual detected objects to activities by following an object tracking technique in order to detect human and vehicle-related activities.

Graph filters are an emerging paradigm that systematizes information propagation in graphs as transformation of prior node values, called graph signals, to posterior scores. In this work, we study the problem of mitigating disparate impact, i.e. posterior score differences between a protected set of sensitive nodes and the rest, while minimally editing scores to preserve recommendation quality. To this end, we develop a scheme that respects propagation mechanisms by editing graph signal priors according to their posteriors and node sensitivity, where a small number of editing parameters can be tuned to constrain or eliminate disparate impact. We also theoretically explain that coarse prior editing can locally optimize posteriors objectives thanks to graph filter robustness. We experiment on a diverse collection of 12 graphs with varying number of nodes, where our approach performs equally well or better than previous ones in minimizing disparate impact and preserving posterior AUC under fairness constraints., Comment: 40 pages, 4 figures, 9 tables

In what has arguably been one of the most troubling periods of recent medical history, with a global pandemic emphasising the importance of staying healthy, innovative tools that shelter patient well-being gain momentum. In that view, we propose a framework that leverages multimodal data, namely inertial and depth sensor-originating data, is integrated in a health care oriented platform, and tackles the crucial issue of detecting patient actions, such as walking, standing and jogging, or even patient falls. To analyse person movement and consequently assess the patient's condition, an efficient methodology is presented that is two-fold: initially a sophisticated Kinect-based methodology is presented that exploits 3DHOG depth features and the descriptive power of a Fisher encoding scheme. This is complemented by wearable sensor data analysis using time domain features and a robust fusion strategy that provides an effective and reliable recognition methodology. The classification accuracy reported in a well-known benchmark dataset proves that the presented approach achieves competitive results and validates the applicability and efficiency of our human action recognition (HAR) methodology., This research has been financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH - CREATE - INNOVATE (T1EDK-00686).

Water scarcity and water stress issues pose a serious threat to the global population. Climate change, drought, population growth and consolidation in urban centres have all been increasing the pressure on water service providers to deploy more sustainable approaches in urban water management. Real-time monitoring and control of water consumption are key ingredients for a smart water management system which will raise consumers’ environmental awareness and reduce costs. This paper introduces a smart infrastructure system which enables remote telemetry and control of water consumption via a web application. The bidirectional and reliable communication between terminal devices (smart meters and valves) and the end-user (consumers and water utility operators) is realized through a fixed hybrid network which uses multiple telecommunication protocols. The metering data is collected and further processed through modern data analytics tools to give a deeper insight on water consumption and behavioral patterns.

Ontologies play a key role in the Semantic Web, providing the machine-interpretable semantic vocabulary and serving as the knowledge representation and exchange vehicle. On top of ontologies, various functionalities can be supported, such as semantic integration, enrichment and reasoning to either further enhance or enrich them with additional information, or to deduct implicit knowledge out of the already annotated information. This paper presents a holistic semantic model employed within the CONNEXIONs EU-funded project that is aimed at semantically representing and reasoning about all pertinent notions derived from the analysis of high volumes of heterogeneous data with the goal to ultimately improve the capabilities of Law Enforcement Agencies in their fight against crime and terrorism. The proposed model enables the compound of various important aspects that resolve around several sources of information considered important in this context, including online sources that are often adversely exploited, as well information produced by Internet of Thing devices, such as sensors and cameras.

This paper demonstrates VERGE, an interactive video retrieval engine for browsing a collection of images or videos and searching for specific content. The engine integrates a multitude of retrieval methodologies that include visual and textual searches and further capabilities such as fusion and reranking. All search options and results appear in a web application that aims at a friendly user experience.

Recent advances in content generation technologies (widely known as DeepFakes) along with the online proliferation of manipulated media content render the detection of such manipulations a task of increasing importance. Even though there are many DeepFake detection methods, only a few focus on the impact of dataset preprocessing and the aggregation of frame-level to video-level prediction on model performance. In this paper, we propose a pre-processing step to improve the training data quality and examine its effect on the performance of DeepFake detection. We also propose and evaluate the effect of video-level prediction aggregation approaches. Experimental results show that the proposed pre-processing approach leads to considerable improvements in the performance of detection models, and the proposed prediction aggregation scheme further boosts the detection efficiency in cases where there are multiple faces in a video.

In this paper we examine the task of automatic detection of speech without microphones, using an overhead camera and wearable accelerometers. For this purpose, we propose the extraction of hand-crafted appearance and optical flow features from the video modality, and time-domain features from the accelerometer data. We evaluate the performance of the separate modalities in a large dataset of over 25 hours of standing conversation between multiple individuals. Finally, we show that applying a multimodal late fusion technique can lead to a performance boost in most cases.

Recognition of daily actions is an essential part of Ambient Assisted Living (AAL) applications and still not fully solved. In this work, we propose a novel framework for the recognition of actions of daily living from depth-videos. The framework is based on low-level human pose movement descriptors extracted from 3D joint trajectories as well as differential values that encode speed and acceleration information. The joints are detected using a depth sensor. The low-level descriptors are then aggregated into discriminative high-level action representations by modeling prototype pose movements with Gaussian Mixtures and then using a Fisher encoding schema. The resulting Fisher vectors are suitable to train Linear SVM classifiers so as to recognize actions in pre-segmented video clips, alleviating the need for additional parameter search with non-linear kernels or neural network tuning. Experimental evaluation on two well-known RGB-D action datasets reveal that the proposed framework achieves close to state-of-the-art performance whilst maintaining high processing speeds., This research has been co‐financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH - CREATE - INNOVATE (T1EDK-00686) and the EC funded project V4Design (H2020-779962).

In this paper, we present work in progress on the development of a smart monitoring framework to support people with motor disabilities and their caregivers in clinical and non-clinical rehabilitation and care environments. The innovation of the platforms lies in the combination of smart monitoring solutions, such as activity recognition and lifestyle tracking, with an intelligent virtual agent that aims to empower and motivate people in need through personalized feedback and responses, as well as to assist caregivers and clinicians to easily collect information about the patients. The proposed system exploits and combines state-of-the-art technologies in speech recognition and synthesis, knowledge representation and reasoning, dialogue management and sensor data analysis, infusing clinical knowledge and patient history Aiming for a practical, acceptable solution, the proposed system takes into account aspects of integration, security and privacy.

Assisted living technologies have become increasingly popular in healthcare with the aim to improve patients’ quality of life. REA is a work in progress research project whose main innovation lies in the combination of smart monitoring solutions, such as activity recognition and physiological/lifestyle tracking, with an intelligent, socially competent virtual agent that aims to empower and motivate people in need through personalized feedback and responses, as well as to assist caregivers and clinicians to easily collect information about the patients.

Video content can be annotated with semantic information such as simple concept labels that may refer to objects (e.g., “car” and “chair”), activities (e.g., “running” and “dancing”), scenes (e.g., “hills” and “beach”), etc.; or more complex (or highlevel) events that describe the main action that takes place in the complete video. An event may refer to complex activities, occurring at specific places and times, which involve people interacting with other people and/or object(s), such as “changing a vehicle tire”, “making a cake”, or “attempting a bike trick”, etc. Concept-based and event-based video search refers to the retrieval of videos/video fragments (e.g., keyframes) that present specific simple concept labels or more complex events from large-scale video collections, respectively. To deal with concept-based video search, video concept detection methods have been developed that automatically annotate video-fragments with semantic labels (concepts). Then, given a specific concept a ranking component retrieves the top related video fragments for this concept. While significant progress has been made during the last years in video concept detection, it continues to be a difficult and challenging task. This is due to the diversity in form and appearance exhibited by the majority of semantic concepts and the difficulty to express them using a finite number of representations. A recent trend is to learn features directly from the raw keyframe pixels using deep convolutional neural networks (DCNNs). Other studies focus on combining many different video representations in order to capture different perspectives of the visual information. Finally, there are studies that focus on multi-task learning in order to exploit concept model sharing, and methods that look for existing semantic relations e.g., concept correlations. In contrast to concept detection, where we most often can use annotated training data for learning the detectors, in the problem of video event detection we can distinguish two different but equally important cases: when a number of positive examples, or no positive examples at all (“zero-example” case), are available for training. In the first case, a typical video event detection framework includes a feature extraction and a classification stage, where an event detector is learned by training one or more classifiers for each event class using available features (sometimes similarly to the learning of concept detectors), usually followed by a fusion approach in order to combine different modalities. In the latter case, where solely a textual description is available for each event class, the research community has directed its efforts towards effectively combining textual and visual analysis techniques, such as using text analysis techniques, exploiting large sets of DCNN-based concept detectors and using various re-ranking methods, such as pseudo-relevance feedback, or self-paced re-ranking. In this chapter, we survey the literature and we present our research efforts towards improving concept- and event-based video search. For concept-based video search, we focus on i) feature extraction using hand-crafted and DCNN-based descriptors, ii) dimensionality reduction using accelerated generalised subclass discriminant analysis (AGSDA), iii) cascades of hand-crafted and DCNN-based descriptors, iv) multi-task learning (MTL) to exploit model sharing and v) stacking architectures to exploit concept relations. For video event detection, we focus on methods which exploit positive examples, when available, again using DCNN-based features and AGSDA, and we also develop a framework for zero-example event detection that associates the textual description of an event class with the available visual concepts in order to identify the most relevant concepts regarding the event class. Additionally, we present a pseudorelevant feedback mechanism that relies on AGSDA.

This paper presents VERGE, an interactive video retrieval engine that enables browsing and searching into video content. The system implements various retrieval modalities, such as visual or textual search, concept detection and clustering, as well as a multimodal fusion and a reranking capability. All results are displayed in a graphical user interface in an efficient and friendly manner.

EOPEN (https://eopen-project.eu/) is a project which has received funding from the European Union’s Horizon 2020 research and innovation programme under the topic EO Big Data Shift in 2017 and has a duration of 3 years, starting from November 2017. In this work, we present the concept of the project, its objectives and the lessons learnt after almost one year of project lifetime, as a follow-up to our previous project presentation at ESA BiDS’17 in Toulouse, France.

In this paper, we investigate the connection between how people understand speech and how speech is understood by a deep neural network. A naïve, general feed-forward deep neural network was trained for the task of vowel/consonant classification. Subsequently, the representations of the speech signal in the different hidden layers of the DNN were visualized. The visualizations allow us to study the distance between the representations of different types of input frames and observe the clustering structures formed by these representations. In the different visualizations, the input frames were labeled with different linguistic categories: sounds in the same phoneme class, sounds with the same manner of articulation, and sounds with the same place of articulation. We investigate whether the DNN clusters speech representations in a way that corresponds to these linguistic categories and observe evidence that the DNN does indeed appear to learn structures that humans use to understand speech without being explicitly trained to do so.

This paper presents the algorithms that CERTH-ITI team deployed so as to deal with flood detection and road passability from social media and satellite data. Computer vision and deep learning techniques are combined so as to analyze social media and satellite images, while word2vec is used to analyze textual data. Multimodal fusion is also deployed in CERTH-ITI framework, both in early and late stage, by combining deep representation features in the former and semantic logic in the latter so as to provide a deeper and more meaningful understanding of the flood events.

The overall objective of EOPEN is to provide a platform targeting non-expert EO data users (non-traditional user communities), experts and the SME community that reveals and makes Copernicus data and services easy to use for Big Data applications by providing EO data analytics services, decision making and infrastructure to support the Big Data processing life-cycle allowing the chaining of value adding activities across multiple platforms.

A novel work for Ambient Assisted Living applications is presented here. More specifically, this paper focuses on activity recognition from recordings of daily living captured by wearable cameras. It constructs a discriminant object centric motion descriptor for representing the micro-actions within the viewpoint of the action maker so as to later define the activity that he/she performs. The accumulation of these activities build patterns over time that can be used to study the behavior of the end-users, which is very useful for health application and monitoring of patients inside their own dwellings or their behavior inside a controlled environment.

This paper presents VERGE interactive video retrieval engine, which is capable of browsing and searching into video content. The system integrates several content-based analysis and retrieval modules including concept detection, clustering, visual and textual similarity search, query analysis and reranking, as well as multimodal fusion.

During climate-related crises, vast volumes of heterogeneous multimodal information are generated. Meaningfully processing and communicating this information for efficient decision support is a key challenge. The paper describes applying Semantic Web technologies for decision support during such crises. We are proposing the application of these technologies in the whole "sensor to decision chain". This approach is being tested within the beAWARE EU project, with contributions by domain experts.

The identification of flooded areas over Earth Observation (EO) satellite images has paved the way to monitor damaged areas and take effective actions. Classifying all pixels of a satellite image as a flooded area or not allows for creating maps which are then used by civil protection agencies and first responders. In this work, a method, firstly implemented for Emergency Management Service (e.g. Copernicus), will be applied for the first time to the detection of changes of surface water bodies, based on water volumes data available from the Moroccan Demonstration Area of the H2020 MOSES (Managing crOp water Saving with Enterprise Services) Project. The novel method is based on the combination of Mahalanobis Distance-based classification for flood mask creation and morphological post-processing for flood mask correction. The classification is performed by using four-dimensional classification features derived directly from the image pixels (namely R, G, B and Near-Infrared channel). The mask correction consists of three steps. The first step relies on the assumption that if the percentage of pixels classified as flooded in an image is very small, and then probably these pixels are misclassified samples. So if the percentage of flooded pixels in an image is less than a preset threshold, they are set to non-flooded (the threshold has been empirically set to 5% after examining classification results of a training set). The second post-processing step aims at eliminating small flooded areas (groups of a few pixels, usually 1 to 10, classified as flooded in non-flooded areas), which are potentially false positives, by applying connected-component analysis: the algorithm counts the number of pixels in such areas and in case it is less than a threshold (10 pixels) marks them as non-flooded. Finally the third post-processing step, aims at eliminating small non-flooded areas inside flooded area, which are probably false negatives, by applying image dilation and erosion. Experiments have been performed on satellite images, collected from PlanetLabsand have been provided in the context of MediaEval2017 - Multimedia Satellite Task (http://www.multimediaeval.org/mediaeval2018). In this work we also examine the performance of the method in the context of water body detection over irrigated agricultural areas. The current technique is going to be extended in the future by considering synthetic aperture radar (SAR) data fusion and additional features in the context of H2020-EOPEN (opEn interOperable Platform for unified access and analysis of Earth observatioN data).

We collect Twitter posts to monitor flood events in Italy during a period of one year. We analyse the statistics of the collected content. We extract word clouds, popular keywords and we demonstrate events in a one-year timeline. A large part of the collection has been annotated by Italian experts on the relevance or not to flood events.

AimsThe association of low endothelial shear stress (ESS) with high-risk plaque (HRP) has not been thoroughly investigated in humans. We investigated the local ESS and lumen remodelling patterns in HRPs using optical coherence tomography (OCT), developed the shear stress score, and explored its association with the prevalence of HRPs and clinical outcomes.Methods and resultsA total of 35 coronary arteries from 30 patients with stable angina or acute coronary syndrome (ACS) were reconstructed with three dimensional (3D) OCT. ESS was calculated using computational fluid dynamics and classified into low, moderate, and high in 3-mm-long subsegments. In each subsegment, (i) fibroatheromas (FAs) were classified into HRPs and non-HRPs based on fibrous cap (FC) thickness and lipid pool size, and (ii) lumen remodelling was classified into constrictive, compensatory, and expansive. In each artery the shear stress score was calculated as metric of the extent and severity of low ESS. FAs in low ESS subsegments had thinner FC compared with high ESS (89 ± 84 vs.138 ± 83 µm, P < 0.05). Low ESS subsegments predominantly co-localized with HRPs vs. non-HRPs (29 vs. 9%, P < 0.05) and high ESS subsegments predominantly with non-HRPs (9 vs. 24%, P < 0.05). Compensatory and expansive lumen remodelling were the predominant responses within subsegments with low ESS and HRPs. In non-stenotic FAs, low ESS was associated with HRPs vs. non-HRPs (29 vs. 3%, P < 0.05). Arteries with increased shear stress score had increased frequency of HRPs and were associated with ACS vs. stable angina.ConclusionLocal low ESS and expansive lumen remodelling are associated with HRP. Arteries with increased shear stress score have increased frequency of HRPs and propensity to present with ACS.

We present a dataset that combines multimodal biosignals and eye tracking information gathered under a human-computer interaction framework. The dataset was developed in the vein of the MAMEM project that aims to endow people with motor disabilities with the ability to edit and author multimedia content through mental commands and gaze activity. The dataset includes EEG, eye-tracking, and physiological (GSR and Heart rate) signals collected from 34 individuals (18 able-bodied and 16 motor-impaired). Data were collected during the interaction with specifically designed interface for web browsing and multimedia content manipulation and during imaginary movement tasks. The presented dataset will contribute towards the development and evaluation of modern human-computer interaction systems that would foster the integration of people with severe motor impairments back into society.

The emergence of social media as news sources has led to the rise of clickbait posts attempting to attract users to click on article links without informing them on the actual article content. This paper presents our efforts to create a clickbait detector inspired by fake news detection algorithms, and our submission to the Clickbait Challenge 2017. The detector is based almost exclusively on text-based features taken from previous work on clickbait detection, our own work on fake post detection, and features we designed specifically for the challenge. We use a two-level classification approach, combining the outputs of 65 first-level classifiers in a second-level feature vector. We present our exploratory results with individual features and their combinations, taken from the post text and the target article title, as well as feature selection. While our own blind tests with the dataset led to an F-score of 0.63, our final evaluation in the Challenge only achieved an F-score of 0.43. We explore the possible causes of this, and lay out potential future steps to achieve more successful results., Comment: Clickbait Challenge 2017

This paper presents VERGE interactive video retrieval engine, which is capable of browsing and searching into video content. The system integrates several content-based analysis and retrieval modules including concept detec-tion, clustering, visual similarity search, object-based search, query analysis and multimodal and temporal fusion.

Nowadays there is an important need by journalists and media monitoring companies to cluster news in large amounts of web articles, in order to ensure fast access to their topics or events of interest. Our aim in this work is to identify groups of news articles that share a common topic or event, without a priori knowledge of the number of clusters. The estimation of the correct number of topics is a challenging issue, due to the existence of “noise”, i.e. news articles which are irrelevant to all other topics. In this context, we introduce a novel density-based news clustering framework, in which the assignment of news articles to topics is done by the well-established Latent Dirichlet Allocation, but the estimation of the number of clusters is performed by our novel method, called “DBSCAN-Martingale”, which allows for extracting noise from the dataset and progressively extracts clusters from an OPTICS reachability plot. We evaluate our framework and the DBSCAN-Martingale on the 20newsgroups-mini dataset and on 220 web news articles, which are references to specific Wikipedia pages. Among twenty methods for news clustering, without knowing the number of clusters k, the framework of DBSCAN-Martingale provides the correct number of clusters and the highest Normalized Mutual Information. This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.

The notion of signal sparsity has been gaining increasing interest in information theory and signal processing communities. As a consequence, a plethora of sparsity metrics has been presented in the literature. The appropriateness of these metrics is typically evaluated against a set of objective criteria that has been proposed for assessing the credibility of any sparsity metric. In this paper, we propose a Generalised Differential Sparsity (GDS) framework for generating novel sparsity metrics whose functionality is based on the concept that sparsity is encoded in the differences among the signal coefficients. We rigorously prove that every metric generated using GDS satisfies all the aforementioned criteria and we provide a computationally efficient formula that makes GDS suitable for high-dimensional signals. The great advantage of GDS is its flexibility to offer sparsity metrics that can be well-tailored to certain requirements stemming from the nature of the data and the problem to be solved. This is in contrast to current state-of-the-art sparsity metrics like Gini Index (GI), which is actually proven to be only a specific instance of GDS, demonstrating the generalisation power of our framework. In verifying our claims, we have incorporated GDS in a stochastic signal recovery algorithm and experimentally investigated its efficacy in reconstructing randomly projected sparse signals. As a result, it is proven that GDS, in comparison to GI, both loosens the bounds of the assumed sparsity of the original signals and reduces the minimum number of projected dimensions, required to guarantee an almost perfect reconstruction of heavily compressed signals. The superiority of GDS over GI in conjunction with the fact that the latter is considered as a standard in numerous scientific domains, prove the great potential of GDS as a general purpose framework for measuring sparsity., Comment: 14 pages, 4 figures, The abstract field cannot be longer than 1,920 characters: The abstract appearing here is slightly shorter than the one in the pdf

Publication in the conference proceedings of EUSIPCO, Poznan, Poland, 2007

Publication in the conference proceedings of EUSIPCO, Rhodes, Greece, 1998

Comunicació presentada a Video Browser Showdown (VBS'17), a 23rd International Conference on MultiMedia Modeling (MMM'17), celebrat el 4 de gener de 2017 a Reykjavik, Islàndia. This paper presents VERGE interactive video retrieval engine, which is capable of browsing and searching into video content. The system integrates several content-based analysis and retrieval modules including concept detec-tion, clustering, visual similarity search, object-based search, query analysis and multimodal and temporal fusion. This work was supported by the EU’s Horizon 2020 research and innovation programme under grant agreements H2020-687786 InVID, H2020-693092 MOVING, H2020-645012 KRISTINA and H2020-700024 TENSOR.

In every disaster and crisis, incident time is the enemy, and getting accurate information about the scope, extent, and impact of the disaster is critical to creating and orchestrating an effective disaster response and recovery effort. Decision Support Systems (DSSs) for disaster and crisis situations need to solve the problem of facilitating the broad variety of sensors available today. This includes the research domain of the Internet of Things (IoT) and data coming from social media. All this data needs to be aggregated and fused, the semantics of the data needs to be understood and the results must be presented to the decision makers in an accessible way. Furthermore, the interaction and integration with existing risk and crisis management systems are necessary for a better analysis of the situation and faster reaction times. This paper provides an insight into the sensor to decision chain and proposes solutions and technologies for each step.