2022 High Performance Computing Lecture 9 Accelerators and Graphical Processing Units Part2 💻

Lecture 9 - Accelerators and Graphical Processing Units 🖥️ - Part Two Advanced Scientific Computing 16 university lectures with additional practical lectures for hands-on exercises in context The University of Iceland, School of Engineering and Natural Sciences Faculty of Industrial Engineering, Mechanical Engineering and Computer Science Spring 2022 Course Outline 1. High Performance Computing 2. Parallel Programming with MPI 3. Parallelization Fundamentals 4. Advanced MPI Techniques 5. Parallel Algorithms & Data Structures 6. Parallel Programming with OpenMP 7. Hybrid Programming & Patterns 8. Debugging & Profiling & Performance Analysis 9. Accelerators & Graphical Processing Units 10. Parallel & Scalable Machine & Deep Learning 11. HPC in Health & Neurosciences 12. Computational Fluid Dynamics & Finite Elements 13. Systems Biology & Bioinformatics 14. Molecular Systems & Material Sciences 15. Terrestrial Systems & Climate 16. Epilogue Lecture Outline Part One: General Purpose Graphical Processing Units (GPGPUs) Multi-core CPUs vs. Many-core GPUs Revisited & TOP500 Impact Terminology & NVIDIA Architectures (Kepler, Pascal, Volta, Ampere) Understanding Node Architectures with GPUs & Summit HPC System Interconnecting GPUs with NVLink/NVSwitch in ‘islands‘ per Nodes GPUDirect in Modular Supercomputing & Garpur Iceland HPC System Part Two: GPU Libraries & Programming Models Data Science with Deep Learning & Multi GPU Horovod Interconnect Simulation Sciences via Libraries (e.g., cuBLAS & basic linear algebra) NVIDIA GPUs & Compute Unified Device Architecture (CUDA) New Vendors & AMD Instinct in Frontier & LUMI Supercomputers OpenACC & HIP Standard Programming Models