Instructor

Jian Wu

Email

jwu@cs.odu.edu

Office Location

3202 ECSB

Office Hours

by appointment

Class Time

5:45 pm. -8:25 pm. T

Classroom

DRGS 1102

Course Overview

Over the past two decades, deep learning has shaped the computer and information sciences, enabling significant revolutionary advances on state-of-the-art tasks in traditional and new natural language processing (NLP), computer vision (CV), and related fields, such as biomedical sciences. Although new methods kept being proposed on certain tasks, understanding the fundamental concepts and models is critical to understanding, using, and developing new methods and models, which are based on or derived from existing concepts and models. The revolution of deep learning often inspires methodologies of how to approach challenging problems by converting them into basic and well-understood problems. Deep learning has become a core component of modern artificial intelligence (AI) systems, supporting self-driving cars, people identification, question answering, and search engine systems.  

The class covers the fundamentals of deep learning, starting from the basic neural networks, and gradually getting deeper and broader. To go deeper, it covers topics such as loss function, regularization, how to train, optimize, and avoid overfitting of deep neural networks. To go broader, it covers various types of neural networks including convolutional neural networks, recurrent neural networks, transformers, and attention mechanism. It also includes a lecture to cover advanced deep neural network models such as deep reinforcement learning and graph neural networks. The class expects students to be able to derive or prove mathematically key equations and theorems. The class also expects students to write computer programs using off-the-shelf packages to implement neural networks deployable on commodity hardware. Students are expected to reproduce (obtaining exactly the results using the same datasets and methods) and replicate (obtaining consistent results using similar datasets and methods) results published in papers published in top-tier computer and information science conferences or journals. The class expects students to produce a quality report the structure of which is aligned with the award winning repots of the machine learning reproducibility challenge (https://paperswithcode.com/rc2022). To broaden the view, the course will also invite guest speakers to talk about the application of deep learning in focused domains.

Students will be evaluated on attendance, homework assignments, paper reviews, and research projects.

Course Delivery Method

This course will be delivered face-to-face in the classroom specified in the syllabus. For students who cannot attend physically due to sickness, the instructor can setup zoom sessions depending on the available facilities. Course materials will be available on GitHub. Assignment submissions will be made on canvas.odu.edu.

Required Text

There is no required textbook. One recommended textbook is 

o   Neural Networks and Deep Learning: A Textbook, by Charu C. Aggarwal, 2018. ISBN-13: 978-3319944623. ISBN-10: 3319944622

One reference book is

o   Deep Learning: by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. ISBN-13: 978-0262035613. ISBN-10: 0262035618

Hardware and Software Requirements

Students will need frequent access to a PC (with Windows 10) or a Mac (with MacOS 10.14+) or a Linux (with Ubuntu 20.04 LTS) capable of hosting application development activities or of connecting to remote servers. The students will use the GPUs on the Wahab cluster provided by the university.

Grading Policy

Students are graded based on the following aspects.

Attendance: 5%

Discussion: 5% (at my discretion)

Homework assignments: 30% (6, each 5%)

Paper review presentation: 20% (peer evaluation)

Reproducibility/Research project: 40% (proposal 10%, final project presentation 10%, final report 20%)

Grading Chart

* Graduate students: see the graduate policies and procedures page for specific requirements of grades (https://catalog.odu.edu/graduate/graduatepoliciesandprocedures/)

Attendance Policy

Attendance is required. One unexcused absence causes a deduction of 1% on attendance until all points are deducted in this aspect. If more than 5 absences are observed, the student automatically gets an F. In case of absence due to legitimate reasons, including but not limited to sickness, University-approved curricular and extracurricular activities (such as athletic contests), career interviews, the death of family members, students should be prepared to provide documentation before classes. Makeup classes are not available.

Academic Integrity

Individual assignments must be completed independently. Students are encouraged to form study groups and to learn from their peers. However, discussion on final proposal writing and presentation in the study group should be limited to general approaches to solutions. Specific answers should never be discussed. ODU's policy regarding Academic Integrity must be followed. Students who violate academic integrity will be reported and receive an “F” for this course.

Cheating: Using unauthorized assistance, materials, study aids, or other information in any academic exercise (Examples of cheating include, but are not limited to, the following: using unapproved resources or assistance to complete an assignment, paper, project, quiz or exam; collaborating in violation of a faculty member's instructions; and submitting the same, or substantially the same, paper to more than one course for academic credit without first obtaining the approval of faculty).

Plagiarism: Using someone else's language, ideas, or other original material without acknowledging its source in any academic exercise. 4 Examples of plagiarism include, but are not limited to submitting a research paper obtained from a commercial research service, the Internet, or from another student as if it were original work; or making simple changes to borrowed materials while leaving the organization, content, or phraseology intact. Plagiarism also occurs in a group project if one or more of the members of the group does none of the group's work and participates in none of the group's activities but attempts to take credit for the work of the group.

Fabrication: Inventing, altering or falsifying any data, citation or information in any academic exercise. Examples of fabrication include, but are not limited to, the following: citation of a primary source which the student actually obtained from a secondary source; or invention or alteration of experimental data without appropriate documentation (such as statistical outliers).

Facilitation: Helping another student commit, or attempt to commit, any Academic Integrity violation, or failure to report suspected Academic Integrity violations to a faculty member. An example of facilitation may include circulating course materials when the faculty member has not explicitly authorized their use.

Copyright

All course materials students receive or to which students have online access are protected by copyrights. Students may use course materials and make copies for their own use as needed, but unauthorized distribution and/or uploading of materials without the instructor’s express permission is strictly prohibited.

Disability Accommodations

To receive consideration for reasonable accommodations, you must contact the Office of Educational Accessibility (OEA). OEA will provide you with an accommodation letter. Please share this letter with your instructors and discuss the accommodations with them as early in your courses as possible.  The detail of disability accommodations is documented in ODU policy #4500. 

Discrimination and Harassment

The university is committed to equal access to programs, facilities, admission, and employment for all persons. It is the policy of the university to maintain an environment free of harassment and free of discrimination against any person because of age, race, color, ancestry, national origin, religion, creed, service in the  uniformed services (as defined in state and federal law), veteran status, sex, sexual orientation, marital or family status, pregnancy, pregnancy-related conditions, physical or mental disability, gender, perceived gender, gender identity, genetic information or political ideas. Discriminatory conduct and harassment, as well as sexual misconduct and relationship violence, violates the dignity of individuals, impedes the realization of the university’s educational mission, and will not be tolerated.

Gender-based sexual harassment, including sexual violence, are forms of gender discrimination in that they deny or limit an individual’s ability to participate in or benefit from University programs or activities. These policies shall not be construed to restrict academic freedom at the university, nor shall they be construed to restrict constitutionally protected expression. The policy is coded in University Policy #1005.

1

1/10/2023

Course Introduction, An Introduction to Machine Learning

Chapter 1

2

1/17/2023

An Introduction to Deep Learning,

An Introduction to Keras/TensorFlow

Chapter 1

3

1/24/2023

Loss function and shallow models

Chapter 2

4

1/31/2023

Fully connected neural networks

Chapter 2

5

2/7/2023

Training, optimization, and back propagation

Chapter 3

6

2/14/2023

Presentations: Paper Review Topic and Project Proposal

7

2/21/2023

Regularization and generalization

Chapter 4

8

2/28/2023

Convolutional Neural Networks

Chapter 8

9

3/7/2023

Spring Holidays, no class

10

3/14/2023

Recurrent Neural Networks

Chapter 7

11

3/21/2023

Midterm Progress Presentation, Invited Speaker (TBD)

12

3/28/2023

Attention mechanism and Transformers

Chapter 9

13

4/4/2022

Advanced Topics

Chapter 10

14

4/11/2022

Project time

15

4/18/2022

Paper review presentations

Project report due

* Course schedules are subject to change depending on availability of speakers and the instructor.