Abstract The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers. Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation. Introduction Over the past decade, significant advances have been made in the computing power available for reservoir simulation. Computer speeds have roughly doubled every two years for the past forty years. This increased computing power has made possible the practical implementation of more advanced reservoir simulator models. For example, use of the time consuming fully implicit method is now a practical alternative to the Implicit Pressure, Explicit Saturation (IMPES) method. Clearly, the advances in computing power are directly related to both the quality of the results obtained and the number of different scenarios that can be considered. In view of these changes, understanding and applying the new generation of supercomputers is of great importance to reservoir simulation. Recent advances in computing power have to a large degree been brought about by "supercomputer" vector machines such as the CRAY and the Control Data Corporation CYBER. There is increasing interest in producing even faster computers which will be highly parallel in nature. Even CRAY Inc. is following this trend by the introduction of it's XM/P computers which offer parallel central processing units. Highly parallel MIMD computers have been studied in research labs over the last dozen years and a few have been made available commercially. These parallel machines, while already in the supercomputer class, can be considered precursors to the next generation of supercomputers. An understanding of the basic capabilities and properties of these parallel machines is crucial to their appropriate application to reservoir simulation. The intent of this paper is to show basic methods of applying parallel MIMD computers to reservoir simulator problems. Also, it is hoped that the paper will demonstrate the great potential these machines hold for reservoir simulation. First the basics of parallel computers and parallel programming will be introduced. Parallel programming concepts are then applied to two of the most time consuming tasks in reservoir simulation: forming matrix coefficients and sparse matrix solution. The forming of matrix coefficients is considered for the single and multiphase cases using black-oil and compositional fluid models. Parallel matrix solution is addressed using D4 ordered Gaussian elimination, multigrid, conjugate gradient, and SOR methods. P. 281^