COMP 322: Fundamentals of Parallel Programming (Spring
...
2021)
Instructor: | Mackale Joyner, DH 2063 | Head TAs: | Jonathan Cai (hw), Paul Jiang (lab 1pm), William Su (lab 4pm) |
---|---|---|---|
Admin Assistant: | Annepha Hurlock, annepha@rice.edu, DH 3122, 713-348-5186 | Undergraduate TAs: | Tory Songyang, Zishi Wang |
Piazza site: | https://piazza.com/configure-classesrice/spring2020spring2021/comp322 (Piazza is the preferred medium for all course communications, but you can also send email to comp322-staff at rice dot edu if needed) | Cross-listing: | ELEC 323 |
Lecture location: | Sewell Hall 301Fully Online | Lecture times: | MWF 1:00pm 30pm - 12:50pm25pm |
Lab locations: | Sewell Hall 301Fully Online | Lab times: | Thursday, 1Tu 1:00pm 30pm - 12:50pm25pm, Th 4:00pm 50pm - 45:50pm45pm |
Course Syllabus
A summary PDF file containing the course syllabus for the course can be found here. Much of the syllabus information is also included below in this course web site, along with some additional details that are not included in the syllabus.
...
There are no required textbooks for the class. Instead, lecture handouts are provided for each module as follows. You are expected to read the relevant sections in each lecture handout before coming to the lecture. We will also provide a number of references in the slides and handouts.The links to the latest versions of the lecture handouts are included below:
- Module 1 handout (Parallelism)
- Module 2 handout (Concurrency)
- There is no lecture handout for Module 3 (Distribution and Locality). The instructors will refer you to optional resources to supplement the lecture slides and videos.
...
Lecture Schedule
Week | Day | Date (20202021) | Lecture | Assigned Reading | Assigned Videos (see Canvas site for video links) | In-class Worksheets | Slides | Work Assigned | Work Due | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Mon | Jan 1325 | Lecture 1: Task Creation and Termination (Async, Finish) | Module 1: Section 1.1 | worksheet1 | lec1-slides |
|
| ||||||||||||||||
| Wed | Jan 1527 | Lecture 2: Computation Graphs, Ideal Parallelism | Module 1: Sections 1.2, 1.3 | Topic 1.2 Lecture, Topic 1.2 Demonstration, Topic 1.3 Lecture, Topic 1.3 Demonstration | worksheet2 | lec2-slides | Homework 1 |
| |||||||||||||||
Fri | Jan 1729 | Lecture 3: Abstract Performance Metrics, Multiprocessor Scheduling | Module 1: Section 1.4 | Topic 1.4 Lecture, Topic 1.4 Demonstration | worksheet3 | lec3-slides |
| |||||||||||||||||
2 | Mon | Jan 20 | No lecture, School Holiday (Martin Luther King, Jr. Day) |
| Wed | Jan 22 | Feb 01 | Lecture 4: Parallel Speedup and Amdahl's Law | Module 1: Section 1.5 | Topic 1.5 Lecture, Topic 1.5 Demonstration | worksheet4 | lec4-slides | Quiz for Unit 1 | |||||||||||
| FriWed | Jan 24Feb 03 | Lecture 5: Future Tasks, Functional Parallelism ("Back to the Future") | Module 1: Section 2.1 | Topic 2.1 Lecture, Topic 2.1 Demonstration | worksheet5 | lec5-slides | |||||||||||||||||
3 | MonFri | Jan 27Feb 05 | Lecture 6: Finish Accumulators | Module 1: Section 2.3 | Topic 2.3 Lecture, Topic 2.3 Demonstration | worksheet6 | lec6-slides | Quiz for Unit 1 | ||||||||||||||||
3 | WedMon | Jan 29Feb 08 | Lecture 7: Map Reduce | Module 1: Section 2.4 | Topic 2.4 Lecture, Topic 2.4 Demonstration | worksheet7 | lec7-slides | Homework 1 | ||||||||||||||||
| FriWed | Jan 31Feb 10 | Lecture 8: Data Races, Functional & Structural Determinism | Module 1: Section 2.5, 2.6 | Topic 2.5 Lecture, Topic 2.5 Demonstration, Topic 2.6 Lecture, Topic 2.6 Demonstration | worksheet8 | lec8-slides |
| Quiz for Unit Homework 2 | Homework 1 | ||||||||||||||
4 | MonFri | Feb 0312 | Lecture 9: Java’s Fork/Join Library | Module 1: Sections 2.7, 2.8 | Topic 2.7 Lecture, Topic 2.8 Lecture | worksheet9 | lec9-slides | Quiz for Unit 2 | ||||||||||||||||
4 | WedMon | Feb 0515 | Lecture 10: Loop-Level Parallelism, Parallel Matrix Multiplication | Module 1: Sections 3.1, 3.2 | Topic 3.1 Lecture , Topic 3.1 Demonstration , Topic 3.2 Lecture, Topic 3.2 Demonstration | worksheet10 | lec10-slides | |||||||||||||||||
Wed | Feb 17 | Spring "Sprinkle" Day (no class) | ||||||||||||||||||||||
| Fri | Feb 0719 | Lecture 11: Iteration Grouping (Chunking), Barrier Synchronization | Module 1: Sections 3.3, 3.4 | Topic 3.3 Lecture , Topic 3.3 Demonstration, Topic 3.4 Lecture , Topic 3.4 Demonstration | worksheet11 | lec11-slides | Quiz for Unit 2 | ||||||||||||||||
5 | Mon | Feb 1022 | Lecture 12: Parallelism in Java Streams, Parallel Prefix Sums | Module 1: Section 3.7 | Topic Topic 3.7 Java Streams, Topic 3.7 Java Streams Demonstration | worksheet12 | lec12-slides | Quiz for Unit 2 | ||||||||||||||||
Wed | Feb 1224 | Lecture 13: Iterative Averaging Revisited, SPMD pattern | Module 1: Sections 3.5, 3.6 | Topic 3.5 Lecture , Topic 3.5 Demonstration , Topic 3.6 Lecture, Topic 3.6 Demonstration | worksheet13 | lec13-slides | Homework 3 (includes 2 one intermediate checkpointscheckpoint) Quiz for Unit 3 | Homework 2 | ||||||||||||||||
- | Fri | Feb 14 | Spring Recess | 6 | Mon | Feb 17 | 26 | Lecture 14: Data-Driven Tasks | Module 1: Sections 4.5 | Topic 4.5 Lecture Topic 4.5 Demonstration | worksheet14 | lec14-slides | ||||||||||||
6 | Mon | Mar 01 | Spring "Sprinkle" Day (no class) | |||||||||||||||||||||
| WedFeb | 19Mar 03 | Lecture 15: Point-to-point Synchronization with Phasers | Module 1: Section 4.2, 4.3 | Topic 4.2 Lecture , Topic 4.2 Demonstration, Topic 4.3 Lecture, Topic 4.3 Demonstration | worksheet15 | lec15-slides | |||||||||||||||||
| FriFeb | 21Mar 05 | Lecture 16: Pipeline Parallelism, Signal Statement, Fuzzy Barriers | Module 1: Sections 4.4, 4.1 | Topic 4.4 Lecture , Topic 4.4 Demonstration, Topic 4.1 Lecture, Topic 4.1 Demonstration | worksheet16 | lec16-slides | Quiz for Unit 4 | Quiz for Unit 3 | |||||||||||||||
7 | MonFeb | 24Mar 08 | Lecture 17: Midterm Review | lec17-slides | ||||||||||||||||||||
| WedFeb | 26Mar 10 | Lecture 18: Abstract vs. Real Performance | worksheet18 | lec18-slides | |||||||||||||||||||
| FriFeb | 28Mar 12 | Lecture 19: Critical Sections, Isolated construct (start of Module 2) | Module 2: Sections 5.1, 5.2, 5.6, | Topic 5.1 Lecture, Topic 5.1 Demonstration, Topic 5.2 Lecture, Topic 5.2 Demonstration, Topic 5.6 Lecture, Topic 5.6 Demonstration | worksheet19 | lec19-slides | Homework 3, Checkpoint-1 | ||||||||||||||||
8 | Mon | Mar 0215 | Lecture 20: Parallel Spanning Tree algorithm, Atomic variables | Module 2: Sections 5.3, 5.4, 5.5 | Topic 5.3 Demonstration, Topic 5.4 Lecture, Topic 5.4 Demonstration, Topic 5.5 Lecture, Topic 5.5 Demonstration | worksheet20 | lec20-slides |
| ||||||||||||||||
| Wed | Mar 0417 | Lecture 21: Actors | Module 2: 6.1, 6.2 | Topic 6.1 Lecture , Topic 6.1 Demonstration , Topic 6.2 Lecture, Topic 6.2 Demonstration | worksheet21 | lec21-slides |
| ||||||||||||||||
| Fri | Mar 0619 | Lecture 22: Actors (contd) | Module 2: 6.3, 6.4, 6.5 | Topic 6.3 Lecture, Topic 6.3 Demonstration, Topic 6.4 Lecture , Topic 6.4 Demonstration, Topic 6.5 Lecture, Topic 6.5 Demonstration | worksheet22 | lec22-slides | Quiz for Unit 4 | ||||||||||||||||
9 | Mon | Mar 09 | No class | Quiz for Unit 5 |
| |||||||||||||||||||
| Wed | Mar 11 | No class |
|
| |||||||||||||||||||
| Fri | Mar 13 | No class | |||||||||||||||||||||
- | M-F | Mar 16 - Mar 20 | Spring Break | |||||||||||||||||||||
10 | Mon | Mar 23 | 22 | Lecture 23: Actors (contd) | Module 2: 6.6 | Topic 6.6 Lecture, Topic 6.6 Demonstration | lec23-slides | Quiz for Unit 5 |
| |||||||||||||||
Wed | Mar 2524 | Lecture 24: Java Threads, Java synchronized statement | Module 2: 7.1, 7.2 | Topic 7.1 Lecture, Topic 7.2 Lecture | lec24-slides |
| ||||||||||||||||||
Fri | Mar | 2726 | Spring "Sprinkle" Day (no class) | |||||||||||||||||||||
10 | Mon | Mar 29 | Lecture 25: Java Threads, Java synchronized statement (contd), wait/notify | Module 2: 7.1, 7.2 | Topic 7.1 Lecture, Topic 7.2 Lecture | lec25-slides | Homework 3, Checkpoint-2 | |||||||||||||||||
11 | MonWed | Mar 3031 | Lecture 26: Java Threads (exercise) | lec26-handout | Quiz for Unit 6 | Quiz for Unit 5 | Homework 3 (all) | |||||||||||||||||
| WedFri | Apr 0102 | Lecture 27: Java Locks | Module 2: 7.3 | Topic 7.3 Lecture | lec27-slides |
| Quiz for Unit 6 | Quiz for Unit 5 | |||||||||||||||
11 | FriMon | Apr 0305 | Lecture 28: Linearizability of Concurrent Objects | Module 2: 7.4 | Topic 7.4 Lecture | lec28-slides |
Homework 4 (includes one intermediate checkpoint) |
Homework 3 (all) | ||||||||||||||||
12 | Wed | Mon | Apr 0607 | Lecture 29: Java Locks (exercise) | lec29-handout | Quiz for Unit 6 | ||||||||||||||||||
| WedFri | Apr 0809 | Lecture 30: Safety and Liveness Properties, Java Synchronizers, Dining Philosophers Problem | Module 2: 7.5, 7.6 | Topic 7.5 Lecture, Topic 7.6 Lecture | lec30-slides | Quiz for Unit 7 | Quiz for Unit 6 | ||||||||||||||||
12 | FriMon | Apr 1012 | Lecture 31: Message Passing Interface (MPI), (start of Module 3) | Topic 8.1 Lecture, Topic 8.2 Lecture, Topic 8.3 Lecture | lec31-slides |
| ||||||||||||||||||
13 | MonWed | Apr 1314 | Lecture 32: Message Passing Interface (MPI, contd) | Topic 8.4 Lecture | lec32-slides | Homework 4 Checkpoint-1 | ||||||||||||||||||
| WedFri | Apr 1516 | Lecture 33: Message Passing Interface (MPI, contd) | Topic 8.5 Lecture, Topic 8 Demonstration Video | lec33-slides |
|
| |||||||||||||||||
13 | FriMon | Apr 1719 | Lecture 34: Task Affinity with Places | lec34-slides | Quiz for Unit 8 | Quiz for Unit 7 | ||||||||||||||||||
14 | MonWed | Apr 2021 | Lecture 35: Eureka-style Speculative Task Parallelism | lec35-slides |
| |||||||||||||||||||
| WedFri | Apr 2224 | Lecture 36: Algorithms based on Parallel Prefix (Scan) operations | lec36-slides |
| Homework 4 (all) |
| Fri | ||||||||||||||||
14 | Mon | Apr 24Lecture 37: Course Review (Lectures 19-34)26 | TBD | lec37-slides | Quiz for Unit 8 |
| - | |||||||||||||||||
Wed | Apr 28 | Lecture 38: Course Review (Lectures 19-34) | lec38-slides | Homework 4 (all) | ||||||||||||||||||||
Fri | Apr 30 | TBD | Quiz for Unit 8 |
Lab Schedule
Lab # | Date (20202021) | Topic | Handouts | Examples | ||
---|---|---|---|---|---|---|
0 | Infrastructure Setup | lab0-handout | ||||
1 | Jan 1626 | Async-Finish Parallel Programming with abstract metrics | lab1-handout | |||
- | Feb 02 | No lab this week | ||||
2 | Jan 30Feb 09 | Futures | lab2-handout | |||
3 | Feb 0616 | Cutoff Strategy and Real World Performance | lab3-handout | |||
- |
| No lab this week - Spring Recess | ||||
4 | Feb 2023 | DDFs | lab4-handout handout | |||
- | Feb 27Mar 02 | No lab this week | (midterm exam)||||
5 | Mar 0509 | Loop-level Parallelism | lab5-handout handout | lab5-intro | ||
- |
| Mar 16 | No lab this week (Spring "Sprinkle" Day) | |||
- |
| Isolated Statement and Atomic Variables | ||||
- | Actors | |||||
- |
| Java Threads, Java Locks | ||||
- |
| Message Passing Interface (MPI) | ||||
- |
| Apache Spark | ||||
- |
| Eureka-style Speculative Task Parallelism | ||||
- | Java's ForkJoin Framework |
...
Grading will be based on your performance on five homeworks four homework assignments (weighted 40% in all), two exams (weighted 40% in all), weekly lab exercises (weighted 10% in all), online quizzes (weighted 5% in all), and in-class worksheets (weighted 5% in all).
The purpose of the homeworks homework is to give you practice in solving problems that deepen your understanding of concepts introduced in class. Homeworks are Homework is due on the dates and times specified in the course schedule. No late submissions (other than those using slip days mentioned below) will be accepted.
The slip day policy for COMP 322 is similar to that of COMP 321. All students will be given 3 slip days to use throughout the semester. When you use a slip day, you will receive up to 24 additional hours to complete the assignment. You may use these slip days in any way you see fit (3 days on one assignment, 1 day each on 3 assignments, etc.). Slip days will be automatically tracked through the Autograder, more details are available later in this document and in the Autograder user guide. using the README.md file. Other than slip days, no extensions will be given unless there are exceptional circumstances (such as severe sickness, not because you have too much other work). Such extensions must be requested and approved by the instructor (via e-mail, phone, or in person) before the due date for the assignment. Last minute requests are likely to be denied.
Labs must be checked off submitted by a TA by the following Monday at 11:59pm. Labs must be checked off by a TA.
Worksheets should be completed in class for full credit. For partial credit, a worksheet can be turned in Canvas before the start of the class following the one in which the worksheet for distributed, following class (for full credit) so that solutions to the worksheets can be discussed in the next class.
You will be expected to follow the Honor Code in all homeworks and homework and exams. The following policies will apply to different work products in the course:
- In-class worksheets: You are free to discuss all aspects of in-class worksheets with your other classmates, the teaching assistants and the professor during the class. You can work in a group and write down the solution that you obtained as a group. If you work on the worksheet outside of class (e.g., due to an absence), then it must be entirely your individual effort, without discussion with any other students. If you use any material from external sources, you must provide proper attribution.
- Weekly lab assignments: You are free to discuss all aspects of lab assignments with your other classmates, the teaching assistants and the professor during the lab. However, all code and reports that you submit are expected to be the result of your individual effort. If you work on the lab outside of class (e.g., due to an absence), then it must be entirely your individual effort, without discussion with any other students. If you use any material from external sources, you must provide proper attribution (as shown here).
- HomeworksHomework: All submitted homeworks are homework is expected to be the result of your individual effort. You are free to discuss course material and approaches to problems with your other classmates, the teaching assistants and the professor, but you should never misrepresent someone else’s work as your own. If you use any material from external sources, you must provide proper attribution.
- Quizzes: Each online quiz will be an open-notes individual test. The student may consult their course materials and notes when taking the quizzes, but may not consult any other external sources.
- Exams: Each exam will be a closedopen-book, closedopen-notes, and closedopen-computer individual written test, which must be completed within a specified time limit. No class notes or external materials may be consulted when taking the exams.
...