Course title

Topics in Distributed Systems: Harnessing Massively Parallel Processors

Instructor: Matei Ripeanu

TA: Samer Al-Kiswany

Schedule: Monday 5:00-8:00

Location: KAIS4018

Announcements

[12/01] Project ideas posted here (please revisit this page as I’ll keep updating it).

[10/01] Subscribe to the class mailing list either by emailing sympa@ece.ubc.ca with “subscribe eece571R@ece.ubc.ca” in the body of the message or by visiting https://oldlists.ece.ubc.ca/sympa

Course description

Virtually all semiconductor market domains, including PCs, game consoles, mobile handsets, servers, and supercomputers, are converging to concurrent platforms. There are two important reasons for this trend. First, these concurrent processors can potentially offer more effective use of chip space and power than traditional monolithic microprocessors for many demanding applications. Second, an increasing number of applications that traditionally used Application Specific Integrated Circuits (ASICs) can be implemented using concurrent processors to improve functionality and reduce engineering cost. The real challenge is to develop applications software that effectively uses these concurrent processors to achieve efficiency and performance goals.

The aim of this course is to provide students with knowledge and hands-on experience in developing applications massively parallel processors as well, an understanding of current research topics in this area, and an opportunity to write and critically review research papers. Many commercial offerings from NVIDIA, AMD, and Intel already offer concurrency at massive scale. Effectively programming these processors requires in-depth knowledge about parallel programming principles, as well as the parallelism models, communication models, and resource limitations of these processors. The target audiences of the course are students who aim to develop exciting applications for these processors, as well as those who aim to develop programming tools and future implementations for these processors.

The course will cover fundamental issues related to architecting and building applications that harness massively parallel processors, with a focus on graphical processing units (GPUs): programming models for massive parallelism, application optimization techniques, performance modeling, auto-tuning, compile and runtime support, as well as case studies for specific application-domains.

Course format

The course is structured to provide (i) an in-depth understanding of current topics in distributed/parallel system research; (ii) experience with reviewing and presenting advanced technical material; (iii) exercising writing and critically reviewing research papers. The class workload has a participation component and a final project.

Participation. In each class we discuss two or more research papers. You will have to read the papers before class (be an efficient reader!) and write a review for each paper that includes the following:

1. Summarize the main contribution of the paper

2. Critique the main contribution.

a. Rate the significance of the paper on a scale of 5 (breakthrough), 4 (significant contribution), 3 (modest contribution), 2 (incremental contribution), 1 (no contribution or negative contribution). More importantly: Motivate your rating in a paragraph or two.

b. Discuss how convincing the methodology is. You may consider some of the following questions (use what is relevant): Do the claims and conclusions follow from the experiments? Are the assumptions realistic? Are the experiments well designed? Are there different experiments that would be more convincing? Are there other alternatives the authors should have considered? And, of course, is the paper free of methodological errors?

c. Discuss the most important limitations of the approach?

3. What are the two strongest and/or most interesting ideas in the paper?

4. What are the two most striking weaknesses in the paper?

5. Name two questions that you would like to ask the authors.

6. Detail an interesting extension of the work not mentioned in the future work section.

7. Optional comments on the paper that you’d like to see discussed in class.

Reviews must be submitted by midnight the day before the class to the relevant ‘rotisserie’ on H2O. Papers and your reviews are then discussed in class. Discussions will be lead by one or more students and may include a brief (10-minute) presentation of the paper. Discussion leaders do not need to submit reviews, but they need to prepare a discussion plan.

Project: The final project is an opportunity for hands-on research in parallel/distributed systems. It involves literature survey, programming, running experiments or analytical modeling, analyzing results and writing an up to 8-page report (IEEE conference format). A list of project ideas will be posted, but students are highly encouraged to propose topics of their own interest that match the course focus. Teams of two students are highly recommended. Please see me if you want to form a larger team. Some past project reports are available here.

Schedule for Spring 2011

Previous years’ course schedules can be found here: 2010-Autonomic Computing, 2009-Massively Distributed/Parallel Computing Platforms; 2008-Quality of Service; 2007-Data-intensive Computing Systems.

Topic / Project steps

Research Papers / Other links

(More to come. To be split as required/optional)

[W1]

01/10

Introduction. Overview of current research problems. Amdahl’s Law. [slides]

• A View of the Parallel Computing Landscape, K. Asanović et al., Communications of the ACM, 2009 [link]

• The Unreasonable Effectiveness of Data, Alon Halevy, Peter Norvig, Fernando Pereira, Communication of the ACM, 2009. [link]

• Amdahl's Law in the Multicore Era, Mark D. Hill, Michael R. Marty, IEEE Computer, July 2008. [pdf]

• Modeling Critical Sections in Amdahl's Law and its Implications for Multicore Design, S. Eyerman, L. Eeckhout, ISCA'10 [pdf]

• Mitigating Amdahl’s Law through EPI Throttling, Murali Annavaram, Ed Grochowski, John Shen, 298-309, ISCA'05 [pdf]

• Balance Principles for Algorithm-Architecture Co-Design, Kent Czechowski, Casey Battaglino, Chris McClanahan, Aparna Chandramowlishwaran, Richard Vuduc, HotPar’11 [link]

[W2]

01/17

Limits of GPU computation.

[slides]

[Project: discussion of project themes.]

Discussion leader: Abdullah

Required

• Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU, ISCA'10, [pdf]