...
The desired learning outcomes fall into three major areas (course modules):
1) Fundamentals of Parallelism: creation and coordination of parallelism (async, finish), abstract performance metrics (work, critical paths), Amdahl's Law, weak vs. strong scaling, data races and determinism, data race avoidance (immutability, futures, accumulators, dataflow), deadlock avoidance, abstract vs. real performance (granularity, scalability), collective & point-to-point synchronization (phasers, barriers), parallel algorithms, systolic algorithms.
2) Fundamentals of Concurrency: critical sections, atomicity, isolation, high level data races, nondeterminism, linearizability, liveness/progress guarantees, actors, request-response parallelism, Java Concurrency, locks, condition variables, semaphores, memory consistency models.
3) Fundamentals of Distributed-Memory ParallelismLocality & Distribution: memory hierarchies, locality, cache affinity, data movement, message-passing (MPI), communication overheads (bandwidth, latency), MapReduce, accelerators, GPGPUs, CUDA, OpenCL, energy efficiency, resilience.
To achieve these learning outcomes, each class period will include time for both instructor lectures and in-class exercises based on assigned reading and videos. The lab exercises will be used to help students gain hands-on programming experience with the concepts introduced in the lectures.
...
There are no required textbooks for the class. Instead, lecture handouts are provided for each module as follows:
- Module 1 handout (Deterministic Shared-Memory Parallelism)
- Module 2 handout (Nondeterministic Shared-Memory Parallelism and Concurrency)
- Module 3 handout (Distributed-Memory Parallelism Distribution and Locality)
You are expected to read the relevant sections in each lecture handout before coming to the lecture. We will also provide a number of references in the slides and handouts.
...