Deep Learning for NLP

Reviews and Presentations

1. Paper reviews
2. Class presentations

Paper Reviews

You can select starred papers on the lecture pages to review. One constraint is that you cannot review a paper that you present in the class. There may be additional restrictions on the choice of the starred papers for review in the class.

You can submit your review any time before the deadline via Canvas.

I am hoping that the reviews will be like peer reviews for conference or journal papers. We will use a Canvas form for this purpose.

For each paper, you should first start off with a one paragraph description of the core contribution of the paper in your own words, followed by a list of pros and cons of the approach. After this, you should be creative and discuss how you would extend or develop this paper, if you had been the author. End the review with questions, doubts, or any thoughts about the paper. You could also list experiments or results that you think would strengthen the paper.

Some tips:

• Do not comment about the grammar and typos. We are reading the paper as computer scientists and are interested in the underlying ideas. Please be constructive and remember that a critical review is meant to critique, not merely criticize.

• Summarize the paper in your own words in a couple of sentences. This will help you understand the paper better.

• Identify the main scientific contribution of the paper

  • A good paper will try to answer a specific question. Try to identify this question. For example, this could be a new idea, a unification of previous ideas into a more general framework, or an implementation of an idea to show that it works.

  • Sometimes the question asked by the paper might be its most important contribution

  • Sometimes the novelty of the approach could be its biggest selling factor

• Identify the problem being solved by the paper

  • Every paper tries to solve a problem. What is it for this one?

  • Was the problem worth solving? Are there simpler solutions that the authors did not consider and might have worked just as well? Does the solution have limitations?

• Identify the positive aspects of the paper

  • What are the good ideas? What is their potential influence?

  • How could you generalize the ideas?

  • If you start research in this direction, what will be your next step?

• Identify the assumptions and claims made by the paper

  • Are they reasonable? Will they ever hold?

  • Given the assumptions, what does the paper claim?

  • Given the assumptions, are the claims of the paper reasonably justified? What is the flow of reasoning?

• If the paper has experiments, identify the value of the experimental evaluation

  • Does the paper use the right set of experiments?

  • Is the experimental setup convincing? What other experiments/results would strengthen the paper?

  • Do the experiments really persuade you about the paper’s claims? Do the results persuade you?

• Always try to think about how this paper would be situated among other relevant papers.

  • Are the ideas novel, or have they appeared before?

Some resources to help you write better reviews:

• How to read a research paper, by Michael Mitzenmacher
• How NOT to review a paper: The tools and techniques of the adversarial reviewer, by Graham Cormode.

---

Class presentations

You can select any of the starred papers on the lecture page to present in class. You can not present a paper that you have chosen to review.

The papers will be assigned on a first-come first-serve basis. If you really feel that you want to present a paper, select it early by emailing me the title of the paper. Class presentations will start after the September 23rd. They will be scheduled so that common topics are grouped together.

Your presentations should be no longer than 15 minutes long. Typically that would be about 15 slides. You can structure it as follows:

• 2-3 slides on motivation/background
• about 6 slides that explains the main idea of the paper
• about 4 slides on how the idea is validated (eg. experimental results)
• 2-4 slides on your thoughts, discussion points, questions, criticism

After you present, please email me the slides (pptx and/or pdf) to be posted on the class website.

Remember: Your presentation should aim to encourage discussion among the students about the topic of the paper. Your grade for the presentation will based on three criteria: quality (slides and presenting), understanding of the content (including the ability to explain and answer any questions) and discussions initiated (points brought up). You are welcome to post messages on the class newsgroup before/after the presentation to engage the class in a discussion

Some tips on presenting:

• Feel free to use examples, pictures, and the white board.

• Remind the audience of the definitions of basic terms

• Do not re-use slides or presentations from presentations by the authors of the paper. The presentation should be your own work.

• Please follow the guidelines for writing good reviews in your presentations
  • Be critical and respectful
  • Try to do justice to the author’s contributions

• Jump to the main point of the paper as soon as possible, within 3-4 slides

• Keep track of time. You are in charge of the room during your presentation and are responsible for managing time and the discussion.

• You can use your own laptop or I can provide mine. If you are using my laptop, please verify that I can go over the slides on my laptop.

Some resources:

• How to give a good research talk
• Some tips for preparing a research presentation