EECE 571J: Trustworthy Machine Learning (Spring 2021)

Contents

  1. Overview
  2. Communication and Links
  3. Topics and Tentative Schedule
  4. Course Format
  5. Grading
  6. Absence and Late Deliverables
  7. University Policies

1 Overview

Machine Learning (ML) is a subfield of Artificial Intelligence where computer algorithms are learning “by example”, using past data. ML now replaces humans at many critical decision points and is used in various applications, such as banking and finance, image and speech processing, healthcare, and more. However, like traditional software, AI systems are often faulty and vulnerable to attacks. For example, Amazon had to scrap an AI-based recruiting tool that showed bias against women while Alexa and Siri were recently manipulated with hidden commands that humans cannot hear.

This seminar-style course will explore different topics in emerging research areas related to security, privacy, explainability, ethics, and fairness in machine learning. Students will learn about quality assurance methods for ML systems, attacks against ML systems, defense techniques to mitigate such attacks, and ethical implications of using ML systems.

The course assumes students already have a basic understanding of machine learning. Most of the course readings will come from both seminal and recent papers in the field. Each student will read, summarize, and present several scientific papers, as well as propose, implement, and present their own original project. As such, the course will also focus on polishing the students’ research, development, communication, and technical presentation skills. 

1.1 Learning Objectives

By the end of the course, students will learn:

1.2 Course Prerequisites

This course does not have formal prerequisites. However, previous programming experience and a basic understanding of machine learning (equivalent to CPSC 340 or ELEC400M/ELEC571M) are necessary.

2 Communication and Important Links

2.1 Support Resources

2.2 Time Zone in Canvas

The course is set to Pacific Time (PT) time zone, where the University of British Columbia Vancouver Campus is located. All due dates are set to PT. Canvas will not automatically change time zones for you. If you want Canvas to display dates in your local time zone, you can go into your settings and adjust to your personal local time zone. Please refer to Canvas guide on how to set a time zone in your user account.

3 Topics and Tentative Schedule

Week

Topic

Major Deadlines
(by the beginning of the class – 2pm, unless stated otherwise)

W1: Jan 13

Introductions; ML application scenarios; what can go wrong; Course logistics

W2: Jan 20

AI Trustworthiness - Overview

Submit 2-3 discussion points for each video

W3: Jan 27

HW1 presentations

HW1

Jan 25-Jan 27, 2pm: select papers you would like to present

W4: Feb 3

Adversarial Robustness

Project M0 (finalize groups and discuss project ideas)

W5: Feb 10

Adversarial Robustness in Software Systems

Project M1 (proposal)

W6: Feb 17

Mid-term Break

 

W7: Feb 24

Project proposal presentations

 

W8: Mar 3

Explainability and Interpretability

 

W9: Mar 10

Fairness

Project M2 (first project report)

W10: Mar 17

Ethics and Law

 

W11: Mar 24

Industrial perspectives

 

W12: Mar 31

(Privacy/Open to topics of interest/Guest lecture)

 

W13: Apr 7

(Open to topics of interest/Guest lecture)

 

W14: Apr 14

Workshop:
project presentations and demos

Project M3 (presentations and demos)

(W15: Apr 21)

 

Final project report

4 Course Format

4.1 Reading Assignments

For weeks 4-5 and 8-12, students will read the assigned research papers (two papers each week). Each student will submit a one-page summary of each paper that describes (a) the technical approach and (b) a critical review of the paper.

For (a), describe, in bullet points, the input and outputs to the approach, its technical novelty, how the approach was evaluated, and what the results show. The description should take about 3/4 of the page. Points will be deduced for explanations that are not clear or not specific to the paper.

For (b), specify, in bullet points, 1-2 main strengths and weaknesses of the paper (not including those listed in the paper) and 1-2 suggestions for improvement and follow-up work. Points will be deduced for unclear statements and for listing non-original strength / weaknesses / suggestions, i.e., those stated in the paper.

A template MS-Word document for paper summaries can be found here.

4.2 Paper Presentations

Each week, a student will present one of the assigned research papers to the class (two students each week). The student should motivate the need for the contribution made by the paper, summarize the proposed technique and its evaluation, discuss the strengths and weaknesses of the approach (beyond those listed in the paper), and lead the discussion on the paper. Depending on the number of course participants, each student will present 1-2 papers. Students do not need to submit summaries of the papers they present.

4.3 Homework

The first and only homework assignment (HW1) is due at the beginning of class on Week 3. The students are expected to implement a simple ML classification algorithm using the Scikit-learn machine learning library, analyze its properties, and describe / demonstrate the result in class. The detailed specification for the assignment will be given in class and be posted on Piazza.

4.4 Project

The project will be performed by groups of 2-3 students. The scope of each group's project should match the number of students involved. The expectation for the project is to deepen the class’s understanding in topics related to trustworthy AI. That can include replication studies of existing techniques, novel applications of these techniques to different scenarios, collection of statistical data on existing vulnerabilities and their impact on the society, or novel literature reviews. Come to talk to the course instructor at least one week before the deadline if you want some ideas for inspiration!

There are five deliverables for the project:

5 Grading

This course does not have a final exam. The grading is based on the following components:

6 Absence and Late Deliverables

7 University Policies

UBC provides resources to support student learning and to maintain healthy lifestyles but recognizes that sometimes crises arise and so there are additional resources to access including those for survivors of sexual violence. UBC values respect for the person and ideas of all members of the academic community. Harassment and discrimination are not tolerated nor is suppression of academic freedom. UBC provides appropriate accommodation for students with disabilities and for religious, spiritual and cultural observances. UBC values academic honesty and students are expected to acknowledge the ideas generated by others and to uphold the highest academic standards in all of their actions. Details of the policies and how to access support are available here.