NLP with Neural Networks

1. Meetings & staff
2. Course objectives
3. Course expectations
4. Prerequisites
5. Grading
6. Course policies
7. Student support, inclusion and wellness
8. Communication with teaching staff and getting help
9. Textbooks and resources

Meetings

Lectures: Mon & Wed, 3:00PM – 4:20PM, WEB L122. See lectures page for more information.

Instructor: Vivek Srikumar

Teaching assistant: Maitrey Mehta

Course objectives

This course will cover recent advances in the use of neural networks for natural language processing. Topics will include word embeddings, recurrent neural networks, transformers, BERTology, LLMs and the use of prompting.

At the end of the course, you should be able to:

1. Define deep neural networks for new NLP problems,

2. Implement, train and use such models using off-the-shelf libraries, and

3. Be able to critically read, evaluate and perhaps replicate current literature in the field.

Course Expectations & Assignments

This is an advanced course that is primarily aimed towards helping you understand recent research to the extent of implementing and using techniques that you may read in papers and, also, hopefully guide you towards becoming a researcher.

Enrolled students are expected to:

• Attend class lectures, participate in the class,

• Complete readings and assignments.

There will be a final exam.

Prerequisites

Students are expected to be familiar with the basics of machine learning and natural language processing to the extent covered in the corresponding courses in the School of Computing.

Of course, we will introduce some of the relevant prerequisite concepts in the lectures as needed, but knowing the topics will significantly help in understanding the new material.

Grading

Your grade will be based on 4-5 assignments, one exams, and class particiation. The different components will be weighted as follows:

Your assignments must be submitted electronically on Canvas by midnight of the due date. Detailed instructions for submission will accompany each assignment. Hand written assignments or printouts will not be accepted or graded.

Information about submitting programming assignments will be available in the homeworks.

Late policy

All assignments must be submitted by the deadline. We will use the timestamp on Canvas as the submission time. Assignments will be accepted up to 24 hours after deadline, but will be assessed a 10% penalty. That is, if your assignment is late and scores 90, then your actual grade will be 81 = 90 - 9.

Assignments will not be accepted 24 hours after the deadline.

We will be strict about this policy: If the deadline is midnight and you submit the assignment at 12:01 AM, you will face the 10% penalty! This may sound harsh, but we have to draw a line somewhere.

To get the best grades possible, we offer the following advice:

• You can submit files as often as you like, so always try to submit something before the due date!

• If you discover a major bug or finish solving a problem within 24 hrs after the due date, and you believe that your new solution is substantially improved over your original solution, then resubmit new files! In this case, you will be assessed the 10% late penalty. But your new solution could earn you a better score, so even with the late penalty you will end up with a higher grade.

Exceptions: All submissions are subject to the late day policy stated here. We understand, however, that certain factors may occasionally interfere with your ability to hand in work on time. If that factor is an extenuating circumstance such as a medical condition, we ask you to provide documentation directly issued by the University, and we will try to work out an agreeable solution with you.

Regrade requests and errors in grading: It is important to us that all assignments are properly graded. If you believe there is an error in your assignment grading, please submit an explanation via canvas within 7 days of receiving the grade. No regrade requests will be accepted orally, and no regrade requests will be accepted more than 7 days after you receive the grade for the assignment.

No double dipping projects across multiple classes

You can not submit the same project to this class and another class that you may be taking at the same time. If you are doing related projects in two different classes, there may be some overlap (e.g. in code libraries, etc.), but they should not be identical. A project that is found to be double-submitted will receive zero credit. If you have questions about this policy, please contact the instructor.

Course Policies

School of Computing Policies and Guidelines

The class operates under the School of Computing’s policies and guidelines. In particular, we will adhere to the school’s academic misconduct policy.

Also see the College of Engineering guidelines for information about appeals procedures, withdrawal procedures, and adding and repeating courses.

Collaboration and Cheating

We encourage collaborate; we will not tolerate cheating. Please do not cheat. It is not worth it, and it is unfair to your peers.

The School of Computing has instituted a “two strikes and you’re out” policy. A strike occurs when you are reported for a major cheating (leading to failing a course), or two comparatively minor cheating instances. If you accumulate two strikes in any SoC courses, you will be unable to register for any future SoC courses. See this document for more details.

You are encouraged to discuss class materials with your peers. If you want, you can form study groups because discussions help understanding. You are also welcome to discuss assignments.

However, you must write your own solutions, proofs and code and submit your own solution. Do not copy or ask for answers to assignment questions from other students or any online sources. Do not let someone else copy your submissions either. Both copying and sharing assignments will count as cheating.

If you are caught cheating once, you will receive a failing grade for that submission and receive a minor sanction. For repeated or systematic cheating, you will fail the class and receive a major sanction.

For projects, you are free to discuss the project with your classmates, but your work should be your own.

For both assignments and the project, you should cite all sources that you refer to. This includes personal communication, books, papers, websites, etc. Doing so reflects academic integrity.

For exams, of course, we will allow neither collaboration nor cheating!

Student support, inclusion and wellness

Students with Disabilities

The University of Utah seeks to provide equal access to its programs, services and activities for people with disabilities. If you will need accommodations in the class, reasonable prior notice needs to be given to the Center for Disability and Access, 162 Olpin Union Building, 581-5020. CDA will work with you and the instructor to make arrangements for accommodations. Accommodations cannot be given without paperwork from this office.

University Safety Statement

The University of Utah values the safety of all campus community members. To report suspicious activity or to request a courtesy escort, call campus police at 801-585-COPS (801-585-2677). You will receive important emergency alerts and safety messages regarding campus safety via text message. For more information regarding safety and to view available training resources, including helpful videos, visit safeu.utah.edu.

Addressing Sexual Misconduct

Title IX makes it clear that violence and harassment based on sex and gender (which Includes sexual orientation and gender identity/expression) is a civil rights offense subject to the same kinds of accountability and the same kinds of support applied to offenses against other protected categories such as race, national origin, color, religion, age, status as a person with a disability, veteran’s status or genetic information.

If you, or someone you know, are harassed or assaulted, you are encouraged to report it to the Title IX Coordinator in the Office of Equal Opportunity and Affirmative Action, 135 Park Building, 801-581-8365, or the Office of the Dean of Students, 270 Union Building, 801-581-7066. For support and confidential consultation, contact the Center for Student Wellness, 426 SSB, 801-581-7776. To report to the police, contact the Department of Public Safety, 801-585-2677(COPS).

Student names and personal pronouns

Class rosters are provided to the instructor with the students legal name as well as Preferred first name (if previously entered by you in the Student Profile section of your CIS account, which managed can be managed at any time). While CIS refers to this as merely a preference, I will honor you by referring to you with the name and pronoun that feels best for you in class or on assignments.

Please advise me of any name or pronoun changes so I can help create a learning environment in which you, your name, and your pronoun are respected. If you need any assistance or support, please reach out to the LGBT Resource Center

Wellness

Personal concerns such as stress, anxiety, relationship difficulties, depression, cross-cultural differences, etc. can interfere with a student’s ability to succeed and thrive at the University of Utah.

For helpful resources, contact the Center for Student Wellness online, or on the phone at 801-581-7776.

Contact the instructor if there is any additional support that would aid in this course.

Diversity

It is our intent that students from all diverse backgrounds and perspectives be well-served by this course, that students’ learning needs be addressed both in and out of class, and that the diversity that the students bring to this class be viewed as a resource, strength and benefit. It is our intent to present materials and activities that are respectful of diversity: gender identity, sexuality, disability, age, socioeconomic status, ethnicity, race, nationality, religion, and culture. Your suggestions are encouraged and appreciated. Please let us know ways to improve the effectiveness of the course for you personally, or for other students or student groups.

COVID guidelines

This course will follow the university guidelines and recommendations as much as possible.

I’d like to emphasize the following: As a health and safety measure, I recommend all students to get a COVID vaccine. For those concerned about your own infection or that of others, N95 (or similar) masks are recommended, but not required.

If you are contagious with COVID (or anything else, in fact), I suggest you do not attend the in-person aspects of class. Reach out to me or the teaching assistants for ways to stay up to date with class, if the answer is not already apparent.

Please don’t hesitate to reach out to me in case of any problems or if you need any accommodations.

Communication with Teaching Staff

Don’t be shy if you don’t understand something: come to office hours, send email, or ask questions in class!

We strongly encourage you to post your questions on the class discussion board on Canvas, unless you have a question that you wish to keep confidential. The instructor and the TA will monitor the forum and will answer questions. Other students may also be able to answer questions on Canvas. Do not post homework answers on the discussion board.

For grading questions, project related issues or simply a more hands-on interaction, we encourage you to attend the office hours or meet the instructor/TAs after class.

Of course, for personal/private questions, do feel free contact the instructor or the TAs by email or in person.

Please note that discussion threads and emails are all considered to be equivalent to the classroom, and your behavior in all these venues should conform to the university’s student code.

Textbooks and resources

See the page on resources