Designing and Building Parallel Programs

At last, a practitioner's guide to parallel programming! Students and professionals who use parallel or distributed computer systems will be able to solve real problems with Designing and Building Parallel Programs. This book provides a comprehensive introduction to parallel algorithm design, performance analysis, and program construction. It describes the tools needed to write parallel programs and provides numerous examples.

A unique feature is the companion on-line version, accessible via the World Wide Web using browsers such as Mosaic. This provides a convenient hypertext version of the text with pointers to programming tools, example programs, and other resources on parallel and distributed computing.

Features:

• Shows how to design parallel programs in a methodical fashion and how to analyze and improve program performance.
• Introduces modern parallel languages, including Compositional C++ and Fortran M, that enable you to develop programs easily and efficiently.
• Describes the parallel programming standards High Performance Fortran (HPF) and Message Passing Interface (MPI).
• Presents different tools for collecting and analyzing parallel program performance data.
• Provides numerous bibliographic references so that you can further explore special areas of interest.
• Explains how parallel programming can assist in a broad range of applications, from circuit design to climate modeling.
• Includes many examples and exercises to expand upon concepts introduced in the book.

1. Concepts
1. Parallel Computers and Computation
1. Parallelism and Computing
2. A Parallel Machine Model
3. A Parallel Programming Model
4. Parallel Algorithm Examples
5. Summary
2. Designing Parallel Algorithms
1. Methodical Design
2. Partitioning
3. Communication
4. Agglomeration
5. Mapping
6. Case Study: Atmosphere Model
7. Case Study: Floorplan Optimization
8. Case Study: Computational Chemistry
9. Summary
3. A Quantitative Basis for Design
1. Defining Performance
2. Approaches to Performance Modeling
3. Developing Models
4. Scalability Analysis
5. Experimental Studies
6. Evaluating Implementations
7. A Refined Communication Cost Model
8. Input/Output
9. Case Study: Shortest-Path Algorithm
10. Summary 
4. Putting Components Together
1. Modular Design Review
2. Modularity and Parallel Computing
3. Performance Analysis
4. Case Study: Convolution
5. Case Study: Tuple Space
6. Case Study: Matrix Multiplication
7. Summary
2. Tools
5. Compositional C++
1. C++ Review
2. C++ Introduction
3. Concurrency
4. Locality
5. Communication
6. Asynchronous Communication
7. Determinism
8. Mapping
9. Modularity
10. Performance Issues
11. Case Study: Channel Library
12. Case Study: Fock Matrix Construction
13. Summary
6. Fortran M
1. FM Introduction
2. Concurrency
3. Communication
4. Unstructured Communication
5. Asynchronous Communication
6. Determination
7. Argument Passing
8. Mapping
9. Modularity
10. Performance Issues
11. Case Study: Fock Matrix Construction
12. Summary
7. High Performance Fortran
1. Data Parallelism
2. Fortran 90
3. Data Distribution
4. Concurrency
5. Dummy Arguments and Modularity
6. Other HPF Features
7. Performance Issues
8. Case Study: Gaussian Elimination
9. Summary
8. Message Passing Interface
1. The MPI Programming Model
2. MPI Basics
3. Global Operations
4. Asynchronous Communication
5. Modularity
6. Other MPI Features
7. Performance Issues
8. Case Study: Earth System Model
9. Summary
9. Performance Tools
1. Performance Analysis
2. Data Collection
3. Data Transformation and Visualization
4. Tools
5. Summary
3. Resources
10. Random Numbers
1. Sequential Random Numbers
2. Parallel Random Numbers
3. Distributed Random Numbers
4. Summary
11. Hypercube Algorithms
1. The Hypercube Template
2. Vector reduction
3. Matrix Transposition
4. Mergesort
5. Summary
12. Further Reading
13. About the Online Version
1. Obtaining a Web Browser
2. Web Tours

A good, but extremely boring, introduction to parallel programming.