Search

Stream workflow is a network of big data streaming applications that acts as key enabler for real-time analysis from Internet of Things data. Smart traffic management and smart grid are examples of stream workflow. The focus of existing work is on streaming operator graphs which differs from stream workflow and handling data fluctuations without significant consideration of different dynamic forms that could happen in the structure of data pipelines. This paper investigates the scheduling problem of stream workflow to support runtime alterations of stream workflow deployment, so that scheduling plans will be revised to handle stream workflow applications with continuously changing characteristics. It proposes a pluggable dynamic scheduling technique that accepts user-defined algorithms to handle stream workflow runtime changes. It also presents three different plug-in algorithms and methods to enable auto-scaling of this workflow in a Multicloud environment. The experimental results of the quality of the solution showed that the proposed plug-in optimisation technique is more efficient than baseline and dynamic fair-share techniques to handle runtime changes. Refereed/Peer-reviewed

Software-defined networking (SDN) has evolved as an approach that allows network administrators to program and initialize, control, change and manage networking components (mostly at L2-L3 layers) of the OSI model. SDN is designed to address the programmability shortcomings of traditional networking architectures commonly used in cloud datacenters (CDC). Deployment of SDN solutions have demonstrated significant improvements in areas such as flow optimization and bandwidth allocation in a CDC. However, the benefits are significantly less explored when considering Software-Defined Wide Area Networks (SD-WAN) architectures in the context of delivering solutions by networking multiple CDCs. To support the testing and bench-marking of data-driven applications that rely on data ingestion and processing (e.g., Smart Energy Cloud, Content Delivery Networks) across multiple cloud datacenters, this paper presents the simulator, IoTSim-SDWAN. To the best of our knowledge, IoTSim-SDWAN is the first simulator that facilitates the modeling, simulating, and evaluating of new algorithms, policies, and designs in the context of SD-WAN ecosystems and SDN-enabled multiple cloud datacenters. Finally, IoTSim-SDWAN simulator is evaluated for network performance and energy to illustrate the difference between classical WAN and SD-WAN environments. The obtained results show that SD-WAN surpasses the classical WAN in terms of accelerating traffic flows and reducing power consumption. Refereed/Peer-reviewed

In the era of big data, the high velocity of data imposes the demand for processing such data in real-time to gain real-time insights. Various real-time big data platforms/services (i.e. Apache Storm, Amazon Kinesis) allow to develop real-time big data applications to process continuous data to get incremental results. Composing those applications to form a workflow that is designed to accomplish certain goal is the becoming more important nowadays. However, given the current need of composing those applications into data pipelines forming stream workflow applications (aka stream graph applications) to support decision making, a simulation toolkit is required to simulate the behaviour of this graph application in Cloud computing environment. Therefore, in this paper, we propose an IoT Simulator for Stream processing on the big data (named IoTSim-Stream) that offers an environment to model complex stream graph applications in Multicloud environment, where the large-scale simulation-based studies can be conducted to evaluate and analyse these applications. The experimental results show that IoTSim-Stream is effective in modelling and simulating different structures of complex stream graph applications with excellent performance and scalability. Refereed/Peer-reviewed