Search

Your search keyword '"Dubois, Lionel"' showing total 35 results
Start Over Author "Dubois, Lionel" Publication Year Range Last 50 years Publisher ieee
35 results on '"Dubois, Lionel"'

4. MadPipe: Memory Aware Dynamic Programming Algorithm for Pipelined Model Parallelism

Author

Beaumont, Olivier, Eyraud-Dubois, Lionel, Shilova, Alena, Beaumont, Olivier, High-End Parallel Algorithms for Challenging Numerical Simulations (HiePACS), Laboratoire Bordelais de Recherche en Informatique (LaBRI), Université de Bordeaux (UB)-Centre National de la Recherche Scientifique (CNRS)-École Nationale Supérieure d'Électronique, Informatique et Radiocommunications de Bordeaux (ENSEIRB)-Université de Bordeaux (UB)-Centre National de la Recherche Scientifique (CNRS)-École Nationale Supérieure d'Électronique, Informatique et Radiocommunications de Bordeaux (ENSEIRB)-Inria Bordeaux - Sud-Ouest, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Université de Bordeaux (UB)-École Nationale Supérieure d'Électronique, Informatique et Radiocommunications de Bordeaux (ENSEIRB)-Centre National de la Recherche Scientifique (CNRS)-Université de Bordeaux (UB)-École Nationale Supérieure d'Électronique, Informatique et Radiocommunications de Bordeaux (ENSEIRB)-Centre National de la Recherche Scientifique (CNRS)-Inria Bordeaux - Sud-Ouest, Scool (Scool), Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 (CRIStAL), and Centrale Lille-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Centrale Lille-Université de Lille-Centre National de la Recherche Scientifique (CNRS)

Subjects
[INFO.INFO-DC] Computer Science [cs]/Distributed, Parallel, and Cluster Computing [cs.DC], [INFO.INFO-DC]Computer Science [cs]/Distributed, Parallel, and Cluster Computing [cs.DC]
Abstract

International audience; The training phase in Deep Neural Networks (DNNs) is very computationally intensive and is nowadays often performed on parallel computing platforms, ranging from a few GPUs to several thousand GPUs. The strategy of choice for the parallelization of training is the so-called data parallel approach, based of the parallel training of the different inputs (typically images) and a the aggregation of network weights with collective communications (AllReduce). The scalability of this approach is limited both by the memory available on each node and the networking capacities for collective operations. Recently, a parallel model approach, in which the network weights are distributed and images are trained in a pipeline/stream manner over the computational nodes has been proposed (Pipedream, Gpipe). In this paper, we formalize in detail the optimization problem associated with the placement of DNN layers onto computation resources when using pipelined model parallelism, and we derive a dynamic programming based heuristic, MadPipe, that allows to significantly improve the performance of the parallel model approach compared to the literature.

Published
2022

5. TEXTAROSSA: Towards EXtreme scale Technologies and Accelerators for euROhpc hw/Sw Supercomputing Applications for exascale

Author

Agosta, Giovanni, primary, Cattaneo, Daniele, additional, Fornaciari, William, additional, Galimberti, Andrea, additional, Massari, Giuseppe, additional, Reghenzani, Federico, additional, Terraneo, Federico, additional, Zoni, Davide, additional, Brandolese, Carlo, additional, Celino, Massimo, additional, Iannone, Francesco, additional, Palazzari, Paolo, additional, Zummo, Giuseppe, additional, Bernaschi, Massimo, additional, D'Ambra, Pasqua, additional, Saponara, Sergio, additional, Danelutto, Marco, additional, Torquati, Massimo, additional, Aldinucci, Marco, additional, Arfat, Yasir, additional, Cantalupo, Barbara, additional, Colonnelli, Iacopo, additional, Esposito, Roberto, additional, Martinelli, Alberto R., additional, Mittone, Gianluca, additional, Beaumont, Olivier, additional, Bramas, Berenger, additional, Eyraud-Dubois, Lionel, additional, Goglin, Brice, additional, Guermouche, Abdou, additional, Namyst, Raymond, additional, Thibault, Samuel, additional, Filgueras, Antonio, additional, Vidal, Miquel, additional, Alvarez, Carlos, additional, Martorell, Xavier, additional, Oleksiak, Ariel, additional, Kulczewski, Michal, additional, Lonardo, Alessandro, additional, Vicini, Piero, additional, Cicero, Francesca Lo, additional, Simula, Francesco, additional, Biagioni, Andrea, additional, Cretaro, Paolo, additional, Frezza, Ottorino, additional, Paolucci, Pier Stanislao, additional, Turisini, Matteo, additional, Giacomini, Francesco, additional, Boccali, Tommaso, additional, Montangero, Simone, additional, and Ammendola, Roberto, additional

Published
2021
Full Text
View/download PDF

15. Recent Advances in Matrix Partitioning for Parallel Computing on Heterogeneous Platforms.

Author

Beaumont, Olivier, Becker, Brett A., DeFlumere, Ashley, Eyraud-Dubois, Lionel, Lambert, Thomas, and Lastovetsky, Alexey

Subjects
CLOUD computing, APPROXIMATION algorithms, MATRICES (Mathematics), TOPOLOGY, COMMUNICATION
Abstract

The problem of partitioning dense matrices into sets of sub-matrices has received increased attention recently and is crucial when considering dense linear algebra and kernels with similar communication patterns on heterogeneous platforms. The problem of load balancing and minimizing communication is traditionally reducible to an optimization problem that involves partitioning a square into rectangles. This problem has been proven to be NP-Complete for an arbitrary number of partitions. In this paper, we present recent approaches that relax the restriction that all partitions be rectangles. The first approach uses an original mathematical technique to find the exact optimal partitioning. Due to the complexity of the technique, it has been developed for a small number of partitions only. However, even at a small scale, the optimal partitions found by this approach are often non-rectangular and sometimes non-intuitive. The second approach is the study of approximate partitioning methods utilizing recursive partitioning algorithms. In particular we use the work on optimal partitioning to improve pre-existing algorithms. In this paper we discuss the different perspectives this approach opens and present two algorithms, SNRPP which is a $\sqrt{\frac{3}{2}}$ approximation, and NRPP which is a $\frac{2}{\sqrt{3}}$ approximation. While sub-optimal, the NRRP approach works for an arbitrary number of partitions. We use the first exact approach to analyse how close to the known optimal solutions the NRRP algorithm is for small numbers of partitions. [ABSTRACT FROM AUTHOR]

Published
2019
Full Text
View/download PDF

34. Broadcasting on Large Scale Heterogeneous Platforms under the Bounded Multi-Port Model.

Author

Beaumont, Olivier, Bonichon, Nicolas, Eyraud-Dubois, Lionel, Uznanski, Przemyslaw, and Agrawal, Shailesh Kumar

Subjects
HETEROGENEOUS computing, DISTRIBUTED computing, COMPUTER scheduling, COMPUTER algorithms, COMPUTER simulation
Abstract

We consider the classical problem of broadcasting a large message at an optimal rate in a large scale distributed network under the multi-port communication model. In this context, we are interested in both building an overlay network and providing an explicit algorithm for scheduling the communications. From an optimization point of view, we aim both at maximizing the throughput (i.e., the rate at which nodes receive the message) and minimizing the degree of the participating nodes, i.e., the number of TCP connections they must handle simultaneously. The main novelties of our approach are the introduction of this degree constraint and the classification of the set of participating nodes into two parts: open nodes that stay in the open-Internet and “guarded” nodes that lie behind firewalls or NATs. Two guarded nodes cannot communicate directly, but rather need to use an open node as a gateway for transmitting a message. In the case without guarded nodes, we prove that it is possible to reach the optimal throughput, at the price of a quasi-optimal (up to a small additive increase) degree of the participating nodes. In presence of guarded nodes, our main contributions are a closed form formula for the optimal cyclic throughput and the proof that the optimal solution may require arbitrarily large degrees. In the acyclic case, we propose an algorithm that reaches the optimal throughput with low degree. Then, we prove a worst case ratio between the optimal acyclic and cyclic throughput and show through simulations that this ratio is on average very close to 1, what makes acyclic solutions efficient both in terms of throughput maximization and degree minimization. [ABSTRACT FROM AUTHOR]

Published
2014
Full Text
View/download PDF

35. Heterogeneous Resource Allocation under Degree Constraints.

Author

Beaumont, Olivier, Eyraud-Dubois, Lionel, Thraves Caro, Christopher, and Rejeb, Hejer

Subjects
*RESOURCE allocation, *CLIENT/SERVER computing, *APPROXIMATION algorithms, *CLOUD computing, *BIN packing problem, *COMPUTER scheduling
Abstract

In this paper, we consider the problem of assigning a set of clients with demands to a set of servers with capacities and degree constraints. The goal is to find an allocation such that the number of clients assigned to a server is smaller than the server's degree and their overall demand is smaller than the server's capacity, while maximizing the overall throughput. This problem has several natural applications in the context of independent tasks scheduling or virtual machines allocation. We consider both the offline (when clients are known beforehand) and the online (when clients can join and leave the system at any time) versions of the problem. We first show that the degree constraint on the maximal number of clients that a server can handle is realistic in many contexts. Then, our main contribution is to prove that even if it makes the allocation problem more difficult (NP-Complete), a very small additive resource augmentation on the servers degree is enough to find in polynomial time a solution that achieves at least the optimal throughput. After a set of theoretical results on the complexity of the offline and online versions of the problem, we propose several other greedy heuristics to solve the online problem and we compare the performance (in terms of throughput) and the cost (in terms of disconnections and reconnections) of all proposed algorithms through a set of extensive simulation results. [ABSTRACT FROM AUTHOR]

Published
2013
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results