一、課程說明(Course Description)
熟悉平行程式語言及平行程式設計
二、指定用書(Text Books)
1. Parallel Programming – Techniques and applications Using Networked
Workstations and
Parallel Computers, Barry Wilkinson and Michael Allen, Prentice Hall, 1999.
2. Parallel Programming in C with MPI and OpenMP, Michael J. Quinn, McGraw-
Hill, 2003.
3. Intel Multi-Core Programming
三、參考書籍(References)
1. Documentation (PVM, MPI, Cilk, Pthread, TreadMark, SAM).
2. Designing and Building Parallel Programs, Ian Foster, Addison Wesley, 1995.
四、教學方式(Teaching Method)
上課
五、教學進度(Syllabus)
Lectures:
Part I: Introduction
- Introduction to Parallel Computers
- Introduction to Parallel Computing
Part II: Parallel Programming
- Message-Passing Programming (MPI)
- Shared Memory Programming (Pthread and OpenMP)
Part III: Parallel Computing Techniques
- Embarrassingly Parallel Computations
- Partitioning and Divide-and-Conquer Strategies
- Pipelined Computations
- Synchronous Computations
- Load Balancing and Termination Detection
Part IV: GPU Programming
- Heterogeneous computing
- CUDA programming model
- GPU Architecture & Multi-GPU
- Advanced CUDA Programming & Optimization
Part V: Distributed Programming
- Big Data frameworks: MapReduce, Spark
- Distributed Deep Learning
六、成績考核(Evaluation)
5 Programming assignment - 100%
(1) MPI: basic parallel programming
(2) OpenMP + MPI: load balancing
(3) Pthread & MPI: synchronous computing
(4) CUDA: GPU Programming
(5) Final Project