Search

When changes happen to big data analytics (BDA) applications in the Cloud at runtime, the affected BDA applications have to be re-deployed to accommodate the changes. Deciding the most suitable deployment is critical and complicated. Although there have been various research studies working on BDA application management, autonomic deployment decision making is still an open research issue. This paper proposes a deployment decision making solution for BDA applications in the Cloud: first, we propose a novel language, named DepPolicy, to specify runtime deployment information as policies; second, we model the deployment decision making problem as a constraint programming problem using MiniZinc; third, we propose a decision making algorithm that can make different deployment decisions for different jobs in a way that maximises overall utility while satisfying all given constraints (e.g., cost limit); fourth, we design and implement a decision making middleware, named DepWare, for BDA application deployment in the Cloud. The proposed solution is evaluated in terms of feasibility, functional correctness, performance and scalability. [ABSTRACT FROM AUTHOR]

With the rapid growth of video data from various sources, like security and transportation surveillance, there arise requirements for both online real-time analysis and offline batch processing of large-scale video data. Existing video processing systems fall short in addressing many challenges in large-scale video processing, for example performance, data storage, and fault tolerance. The emerging cloud computing and big data techniques shed lights to intelligent processing for large-scale video data. This paper proposes a general cloud-based architecture and platform that can provide a robust solution to intelligent analysis and storage for video data, which is named as BiF (Batch processing Integrated with Fast processing) architecture. We have implemented the BiF architecture using both Hadoop platform and Storm platform, which are typical offline batch processing cloud platform and online real-time processing cloud platform, respectively. The proposed architecture can handle continual surveillance video data effectively, where real-time analysis, batch processing, distributed storage and cloud services are seamlessly integrated to meet the requirements of video data processing and management. The evaluations show that the proposed approach is efficient in terms of performance, storage, and fault tolerance. [ABSTRACT FROM AUTHOR]