Objectives:
- Use High Performance Linpack and other common benchmarks to explore fundamental concepts in parallel computing:
- Strong scaling & Amdahl’s law
- Weak scaling & Gustafson's law
- Theoretical performance & efficiency
- “Memory wall” and arithmetic intensity
- Bandwidth & latency
- Gain practical experience with parallel computing software and tools:
- MPI & OpenMP
- BLAS libraries
- Linux perf counters
- Understand the impact of system and processor architecture on performance:
- P-states
- Hyperthreading
- NUMA
- SSE/FMA instructions
- Processor cache hierarchy