Back to Search
Start Over
DtCraft: A High-Performance Distributed Execution Engine at Scale.
- Source :
-
IEEE Transactions on Computer-Aided Design of Integrated Circuits & Systems . Jun2019, Vol. 38 Issue 6, p1070-1083. 14p. - Publication Year :
- 2019
-
Abstract
- Recent years have seen rapid growth in data-driven distributed systems, such as Hadoop MapReduce, Spark, and Dryad. However, the counterparts for high-performance or compute-intensive applications including large-scale optimizations, modeling, and simulations are still nascent. In this paper, we introduce DtCraft, a modern C++ based distributed execution engine to streamline the development of high-performance parallel applications. Users need no understanding of distributed computing and can focus on high-level developments, leaving difficult details, such as concurrency controls, workload distribution, and fault tolerance handled by our system transparently. We have evaluated DtCraft on both micro-benchmarks and large-scale optimization problems, and shown the promising performance from single multicore machines to clusters of computers. In a particular semiconductor design problem, we achieved ${30\times }$ speedup with 40 nodes and ${15\times }$ less development efforts over hand-crafted implementation. [ABSTRACT FROM AUTHOR]
- Subjects :
- *GROWTH
*PHYSIOLOGY
*DATA
*DATA analysis
*PARALLEL programming
*COMPUTER programming
Subjects
Details
- Language :
- English
- ISSN :
- 02780070
- Volume :
- 38
- Issue :
- 6
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Computer-Aided Design of Integrated Circuits & Systems
- Publication Type :
- Academic Journal
- Accession number :
- 136543539
- Full Text :
- https://doi.org/10.1109/TCAD.2018.2834422