Advisor(s)
Thomas Gilray
Committee Member(s)
John Johnstone
Sidharth Kumar
School
College of Arts and Sciences
Document Type
Thesis
Department (new version)
Computer and Information Sciences
Date of Award
9-9-2024
Abstract
The complex nature of parallel algorithms in the Message Passing Interface (MPI) standard challenges the ability of trivially visualizing them. Within MPI libaries (e.g., MPICH, Open-MPI), there are many collectives that offer different solutions for communicating data between processes. The collective MPI_Alltoall uses various techniques such as the spread-out and Bruck algorithms to have processes transfer data between all processes. Each technique within MPI_Alltoall is best suited for different scenarios. The spread-out algorithm uses a linear number of iterations, in process count P, while the Bruck algorithm is logarithmic. The Bruck algorithm transfers more data overall, but with fewer communication steps, and is thus better suited for smaller sized (latency-dominated) messages. The Bruck algorithm can be modified to take a parameterized input for the log base (radix). By increasing this radix from the original log base of 2, blocks transferred will decrease while communication steps will increase. MPI libraries use a decision tree to determine which algorithm to use in collectives such as MPI_Alltoall. This thesis details and implements an interactive web-based visualization tool to help a user, such as graduate students studying topics in High Performance Computing (HPC), understand the relation between the log-step Bruck, linear-step spread-out, and log-step tunable radix algorithm (TRA) in the collective MPI_Alltoall. The tool visually illustrates and animates the three algorithms, pointing out key differences such as number of iterations, communication patterns, and whether they are in-place. This work lays the groundwork for developing ways to visualize all algorithms in MPI collectives.
ProQuest ID
Recommended Citation
Netterville, Naeris, "A Visual Guide to Balancing Latency and Bandwidth in MPI All-to-all" (2024). All ETDs from UAB. 7638.
https://digitalcommons.library.uab.edu/etd-collection/7638
