Back to Search
Start Over
Collectives in hybrid MPI+MPI code: Design, practice and performance
- Source :
- Parallel Computing
- Publication Year :
- 2020
-
Abstract
- The use of hybrid scheme combining the message passing programming models for inter-node parallelism and the shared memory programming models for node-level parallelism is widely spread. Existing extensive practices on hybrid Message Passing Interface (MPI) plus Open Multi-Processing (OpenMP) programming account for its popularity. Nevertheless, strong programming efforts are required to gain performance benefits from the MPI+OpenMP code. An emerging hybrid method that combines MPI and the MPI shared memory model (MPI+MPI) is promising. However, writing an efficient hybrid MPI+MPI program -- especially when the collective communication operations are involved -- is not to be taken for granted. In this paper, we propose a new design method to implement hybrid MPI+MPI context-based collective communication operations. Our method avoids on-node memory replications (on-node communication overheads) that are required by semantics in pure MPI. We also offer wrapper primitives hiding all the design details from users, which comes with practices on how to structure hybrid MPI+MPI code with these primitives. The micro-benchmarks show that our collectives are comparable or superior to those in pure MPI context. We have further validated the effectiveness of the hybrid MPI+MPI model (which uses our wrapper primitives) in three computational kernels, by comparison to the pure MPI and hybrid MPI+OpenMP models.<br />Comment: 14 pages. Accepted for publication in Parallel Computing
- Subjects :
- Scheme (programming language)
FOS: Computer and information sciences
Computer Networks and Communications
Semantics (computer science)
Computer science
Message Passing Interface
Context (language use)
010103 numerical & computational mathematics
Parallel computing
Software_PROGRAMMINGTECHNIQUES
01 natural sciences
Theoretical Computer Science
Artificial Intelligence
Synchronization (computer science)
0101 mathematics
computer.programming_language
Shared memory model
Message passing
Computer Graphics and Computer-Aided Design
010101 applied mathematics
ComputingMilieux_GENERAL
Computer Science - Distributed, Parallel, and Cluster Computing
Shared memory
Hardware and Architecture
Programming paradigm
Distributed, Parallel, and Cluster Computing (cs.DC)
computer
Software
Subjects
Details
- ISSN :
- 01678191
- Database :
- OpenAIRE
- Journal :
- Parallel Computing
- Accession number :
- edsair.doi.dedup.....9b012b4670762a5f6e1e4ec317dca564
- Full Text :
- https://doi.org/10.1016/j.parco.2020.102669