Search

Container based virtualization is an innovative technology that accelerates software development by providing portability and maintainability of applications. Recently, a growing number of workloads such as high performance computing (HPC) and Deep Learning(DL) are deployed in the container based environment. However, GPU resource management issues especially the GPU memory over subscription issue in container-based clusters, which brings substantial performance loss, is still challenging. This paper proposes an adaptive fair-share method to share effectively in container-based virtualization environment as well as an execution rescheduling method to manage the execution order of each container for acquiring maximum performance gain. We also proposed a checkpoint based mechanism especially for DL workload running with TensorFlow, which can efficiently solve the GPU memory over subscription problem. We demonstrate that our approach contributes to overall performance improvement as well as higher resource utilization compared to default and static fair-share methods with homogeneous and heterogeneous workloads. Compared to two other conditions, their results show that the proposed method reduces by 16.37%, 15.61% in average execution time and boosts approximately by 52.46%, 10.3% in average GPU memory utilization, respectively. We also evaluated our checkpoint based mechanism by running multiple CNN workloads with TensorFlow at the same time and the result shows our proposed mechanism can ensure each workload executing safely without out of memory (OOM) error occurs.

Natural language processing is a fast-growing area of data science for the finance industry. The authors demonstrate how an applied linguistics expert system may be used to parse corporate email content and news to assess factors that predict escalating risk or the gradual shifting of other critical characteristics within the firm before they manifest in observable data and financial outcomes. The authors find that email content and news articles meaningfully predict increased risk and potential malaise. The authors also find that other structural characteristics, such as average email length, are strong predictors of risk and subsequent performance. Implementations of three spatial analyses of internal corporate communication, (i.e., email networks, vocabulary trends, and topic analysis) are presented. The authors propose a regulatory technology solution to systematically and effectively detect escalating risk or potential malaise without the need to manually read individual employee emails.

In this chapter, we provide an overview of the Bitcoin blockchain and we describe the underlying consensus protocol (a.k.a., the Nakamoto consensus), which has now been used across many other distributed ledger solutions following the advent of Bitcoin. We also discuss other choices in ledger design (e.g., public versus private, blockchain versus directed acyclic graph), and we discuss other types of consensus protocols. Overall, our purpose is to provide a simple but robust explanation of blockchain’s first widespread use case and to dispel common misunderstandings surrounding Bitcoin in particular, and distributed ledgers in general.

Hybrid fuzzing, which combines fuzzing and concolic execution, is promising in light of the recent performance improvements in concolic engines. We have observed that there is room for further improvement: symbolic emulation is still slow, unnecessary constraints dominate solving time, resources are overly allocated, and hard-to-trigger bugs are missed. To address these problems, we present a new hybrid fuzzer named Intriguer. The key idea of Intriguer is field-level constraint solving, which optimizes symbolic execution with field-level knowledge. Intriguer performs instruction-level taint analysis and records execution traces without data transfer instructions like mov. Intriguer then reduces the execution traces for tainted instructions that accessed a wide range of input bytes, and infers input fields to build field transition trees. With these optimizations, Intriguer can efficiently perform symbolic emulation for more relevant instructions and invoke a solver for complicated constraints only. Our evaluation results indicate that Intriguer outperforms the state-of-the-art fuzzers: Intriguer found all the bugs in the LAVA-M(5h) benchmark dataset for ground truth performance, and also discovered 43 new security bugs in seven real-world programs. We reported the bugs and received 23 new CVEs.

The OS kernel, which has entire system privileges, is an attractive target of attackers. To reduce this threat, we need to find security bugs in the kernel prior to the attackers, and system call fuzzing is a widely used technique for this purpose. However, many system call fuzzers have not been evaluated for coverage performance which is an important indicator in fuzzing. In this poster, we propose a methodology to evaluate the code coverage performance of system call fuzzers with a strategy that combines virtualization and Intel Processor Trace (PT). First, we extract all the functions in the kernel that can be executed by system calls. Then we perform fuzzing with the target system call fuzzer on the guest OS, and record coverage information by leveraging the Intel PT. Finally, we evaluate system call fuzzers by comparing the list of functions related to system calls with the executed functions logged by Intel PT while fuzzing.

As the rapid advancement and diversity in the computing systems, it is demanding to take the most robust scheduling algorithm that guarantee an optimized performance to execute manifold applications on large-scale heterogeneous computing environments. This paper present an adaptive application-aware job scheduling optimization strategy for large-scale high throughput computing in heterogeneous infrastructures. The proposed scheduling optimization method is built on two main concepts. First, it provides application-aware job distribution weights through empirical data in large-scale heterogeneous infrastructures. Here we adopt the concept of weight, which represent the ratio of tasks that will be computed on each resource. The weights can vary in terms of application type and are optimized until the system get in steady status. Secondly, it offers an adaptive control phase that is invoked by the weight adjustment and resource scaling feature. The feedback data from monitoring module is forwarded to the control phase in order to adjust weights and over-provisioning ratio, and result in enhancing overall balance between performances and utilization of system. The experimental evaluation with the four realistic workload patterns demonstrates that, when compared to the core-based scheme which distributes tasks in proportions of each resource's number of cores, the use of our optimization method can achieve 62 % better average throughput, 43 % shorter average queueing time, and 38 % better average utilization of the entire resources in diverse infrastructure environments by harnessing our adaptive module.

This paper presents a MOHA(Many-Task Computing on Hadoop) framework which aims to effectively apply the Many-Task Computing(MTC) technologies originally developed for high-performance processing of many tasks, to the existing Big Data processing platform Hadoop. We present basic concepts, motivation, preliminary results of PoC based on distributed message queue, and future research directions of MOHA. MTC applications may have relatively low I/O requirements per task. However, a very large number of tasks should be efficiently processed with potentially heavy inter-communications based on files. Therefore, MTC applications can show another pattern of data-intensive workloads compared to existing Hadoop applications, typically based on relatively large data block sizes. Through an effective convergence of MTC and Big Data technologies, we can introduce a new MOHA framework which can support the large-scale scientific applications along with the Hadoop ecosystem, which is evolving into a multi-application platform.

Virtualization is an innovative technology that accelerates software development by providing portability and maintainability of applications. However, it often leads underperformance especially caused by overheads from managing virtual machines. To address the limitation of virtual machines, container technology has emerged to deploy and operate distributed applications without launching entire virtual machines. Unfortunately, resources contention issues in container-based clusters, bringing substantial performance loss are still challenging. This paper proposes an adaptive fair-share method to share effectively in container-based virtualization environment. In particular, we focus on enabling GPU sharing between multiple concurrent containers without lack of GPU memory. We demonstrate that our approach contributes to overall performance improvement as well as higher resource utilization compared to default and static fair-share methods with homogeneous and heterogeneous workloads. Compared to two other conditions, their results show that the proposed method reduces by 16.37%, 15.61% in average execution time and boosts approximately by 52.46%, 10.3% in average GPU memory utilization, respectively.

Since its first widespread implementation in 2009, distributed ledgers in general, and blockchain technology in particular, have rapidly become a part of the FinTech vernacular. In this paper, we provide an overview of the history of trade settlement and discuss this nascent technology that may now transform traditional methods of verifying and settling transactions. In so doing, we discuss current and potential use cases of this technology and provide a business-oriented framework for proper as well as improper implementations and applications of blockchains and distributed ledgers.

Cloud computing provides on-demand resource provisioning and scalable resources dynamically for the efficient use of computing resources. Scientific applications recently need a very large number of loosely coupled tasks to be handled efficiently. In response, current computing environments often consist of heterogeneous resources such as cloud computing. To effectively use cloud resources, auto-scaling methods that consider diverse metrics such as CPU utilization and costs of resource usage have been studied widely. However it still remains a challenge to automatically and timely allocate resources such that deadline violation and application types are considered. In this paper, we propose auto-scaling methods that consider specific conditions such as application types, task dependency, user-defined deadlines and data transfer times within a hybrid computing infrastructure. Our hybrid computing infrastructure consists of local cluster and cloud resources using HTCaaS. We observe noticeable improvements in performance when our auto-scaling methods for bag-of-tasks and workflow applications is applied.

In this paper, we demonstrate how an applied linguistics platform may be used to parse corporate email content and news to assess factors predicting escalating risk or the gradual shifting of other critical characteristics within the firm before they are eventually manifested in observable data and financial outcomes. We find that email content and news articles meaningfully predict increased risk and potential malaise. We also find that other structural characteristics, such as the average email length, are strong predictors of risk and subsequent performance. We present implementations of three spatial analyses of internal corporate communication, i.e., email networks, vocabulary trends, and topic analysis. Overall, we propose a RegTech solution by which to systematically and effectively detect escalating risk or potential malaise without the need to manually read individual employee emails.

Numerous scientists who are engaged in compute-intensive researches require more computing facilities than before, while the computing resource and techniques are increasingly becoming more advanced. For this reason, many works for e-Science environment have been actively invested and established around the world, but still the scientists look for an intuitive experimental environment, which is guaranteed the improved environmental facilities without additional configurations or installations. In this paper, we present an integrated scientific workflow execution environment for Scientific applications supporting workflow design with high performance computing infrastructure and accessibility for web browser. This portal supports automated consecutive execution of computation jobs in order of the form defined by workflow design tool and execution service concerning characteristics of each job to batch over distributed grid resources. Workflow editor of the portal presents a high-level frontend and easy-to-use interface with monitoring service, which shows the status of workflow execution in real time so that user can check the intermediate data during experiments. Therefore, the scientists can take advantages of the environment to improve the productivity of study based on HTC.

Recent cloud computing enables numerous scientists to earn advantages by serving on-demand and elastic resources whenever they desire computing resources. This science cloud paradigm has been actively developed and investigated to satisfy requirements of the scientists such as performance, feasibility and so on. However, effective allocation and provisioning virtual machines on clouds are still considered as a challenging issue in scientists using high throughput computing, since it determines whether they can earn benefits from economy of scale in clouds or not. Moreover, allocating the "right" provisioned cloud resources on an optimal data center is very important as performance can vary widely depending on where and under what circumstances it actually runs. In these reasons, it is required that an appropriate and suitable model for science cloud to support increasing scientists and computations.In this paper, we present an allocation and provisioning model of science cloud, especially for high throughput computing applications. In this model, we utilize job traces where statistical method is applied to pick the most influential features for improving application performance. With the feature, the system determines where VM is deployed (allocation) and which instance type is proper (provisioning). An adaptive evaluation step which is subsequent to the job execution enables our model to adapt to dynamical computing environments. We show performance achievements as comparing the proposed model with other policies through experiments. Finally, we expect that improvement on performance as well as reduction of cost from resource consumption through our model.

We present the HTCaaS (High-Throughput Computing as a Service) system which aims to provide researchers with ease of exploring large-scale and complex HTC problems by leveraging Supercomputers, Grids, and Cloud. HTCaaS can hide heterogeneity and complexity of harnessing different types of computing infrastructures from users, and efficiently submit a large number of jobs at once by effectively managing and exploiting of all available computing resources. Our system has been effectively integrated with national Supercomputers in Korea, international computational Grids, and Amazon EC2 resulting in combining a vast amount of computing resources to support most challenging scientific problems.

As computing technologies develop, various studies for e-Science have been actively developed for many years. The advent of Cloud computing, for example, enables scientists to expand their research environments over supercomputers to on-demand and scalable resources. However, several performance drawbacks on the cloud computing can cause considerable obstacles in scientific domains, despite its strong merits. A hybrid infrastructure which consists of the existing and cloud architecture thus can produce a synergy effect for utilizing resources efficiently. In this paper, we proposed hybrid infrastructure-supported a scientific workflow environment for Aerodynamics design and demonstrated its superiority. Hybrid infrastructure covers grid and private cloud computing in this paper. Especially, we focused on improving performance by supporting hybrid infrastructure and efficient usages of physical resources. Since it offers diverse types of computing infrastructures including cloud computing, it can serve elastic resources regardless of the number of tasks for experiments or limitations of space and can reduce costs of time as well as budget during simulations.

The rapid advent of computing technology enables scientists to expand their research infrastructure over supercomputer-on-demand using a resource leasing service on pay-per-use basis. This infrastructure service is called as Science cloud which provides uniform user interface to scientific experiments over large-scale heterogeneous resources. In spite of the strength of conveniences, it is difficult to manage the experiments to guarantee optimal performance of jobs since the execution environment is based on virtualization technology which has additional performance overheads. This paper proposes a cloud resource provisioning model using statistical analysis of job profiles for computational science. In this model, we utilize job profiles which are generated from application executions and identify features of application by applying statistical analysis. The analysis is performed using PCA and the most effective factors are inferred from profiles. The effective factors are used for picking a job profile being referred and then VM type is determined using the most effective job's property. An application is executed on VM of the chosen cluster and its performance result is incorporated into job profile set with purpose of evaluating profile's credit. Our simulation results show this model enables scientists to take advantages of cloud computing without performance degradation.

The advent of cloud computing makes scientists to extend their research environments over supercomputers to on-demand and dynamically scalable resources. Science cloud becomes a trend in various scientific domains these days. However, it is difficult to provide optimal job execution environment rapidly and dynamically depending on user's demands. Therefore, it is very important to predict user's requirements and to prepare execution environment in advance. In addition, it needs scheduling mechanisms for virtual machines to provide some level of guaranteed performance of a user application. In this paper, we propose a cloud resource provisioning model using statistical analysis of job history. In this model, we use job history which is generated from many application executions and identifies characteristics of an application by applying statistical analysis. We utilize a statistical technique, PCA (Principal Component Analysis), to analyze execution history of applications and to extract the factors which contribute much to execution time. The effective factors are used for selecting reference job profile and then VM is deployed on the selected node based on the reference profile. An application is executed on chosen nodes and its performance result is incorporated into job history with the purpose of evaluating profile's credit. As a result, this model can provide efficient management of cloud resource for a service provider and reduce management overhead on cloud.

Due to diverse characteristics of application domains, most experts dealing with scientific computational simulations require a specialized and customized Problem Solving Environment (PSE) with easy-to-use graphical user interfaces integrated with large-scale computing resources for their experiments. It is also important to provide adequate middleware support to drag out best performance of an application from the environment. To fulfill these requirements, we propose a workflow-enabled integrated e-Science portal system supporting interactivities for aerodynamic computing simulations, which are driven to diverse multiple execution stages depending on convergence status of variables. The portal allows scientists to manage whole life-cycle of experiments from designing, deploying, executing, and monitoring a scientific application in aerodynamic research on e-Science environment. Scientists easily extend various types of their experiments with less risk of design errors and save their time for learning curve to handle computing resources to optimize the performance of experiments. Workflow editor presents interactive, high-level web-based front end to scientist and monitoring service shows the step of the execution of workflow in real-time so that scientists in any level of expertise can check the intermediate data during experiments.

As one of IaaS (Infrastructure-as-a-Service), it is beneficial to arrange virtual machines dynamically to applications based on resource provisioning mechanism. However, it is challenging to apply scheduling scheme to utilize resources efficiently when many tasks require a lot of resources at the same time. Especially, scientific applications, which require large-scale computing resource for long term execution period, need more dynamic scheduling to occupy resource appropriately. Resource virtualization, manipulating several virtual machines (VMs) over physical resource gives good opportunities to enhance the performance of applications and resources. In this paper, we conducted experiments on adaptable scheduling schemes with scientific applications which need a lot of resources for long execution time period on cloud computing environment by distributing jobs to VMs. We provide cloud computing infrastructure by using OpenNebula virtual infrastructure engine and Haziea scheduler. Moreover, we verified the improvement of the execution time of applications and whole resource usage by scheduling VMs according to their priorities.

Cloud computing has recently paid attention as a way to share the resources to provide scalable and on-demand services. Across various authorities, Cloud resources should be configured to a virtual organization(VO) according to user's requirements. Ontology-based representation of Cloud computing environment would be able to conceptualize common attributes among Cloud resources and to represent semantic relations among them. However, mutual compatibility among different VOs is limited because a method applying ontology to Cloud is in progress. We propose to introduce a resource virtualization method using virtual ontology. A new Virtual Ontology (VOn) is configured dynamically based on requirement of users, and the VOn is mapped to actual Cloud resources. Our service uses a Map/Reduce model for rapid and efficient merging a number of ontology. The execution environment is orchestrated with selected resources mapped to the VOn, which is generated by Ontology Merge engine. research is at an early stage and is hard to provide interoperability among organizations because of merging resources from only static concepts in existed researches. In the paper (9), they propose a Semantically-Enhanced Resource Allocator (SERA) which is a scheduling system using customer requests and provides the ability of re-scheduling requests based on their priorities and considering advanced reservations. Nevertheless, the experiment is too limited to prove the proposed concept. The characteristics of target applications are not considered. In this paper, we propose an Ontology-based Resource Selection Service (OReSS) which provides a method of resource virtualization using merged ontologies which is interoperable among virtual organizations and then selects resources based on the VOn. OReSS on Cloud computing, especially, selects ontology candidates by calculating a degree of similarity function based on user's requirements. Selected ontologies are merged and provided to a user as a new VOn.