Comp 620: Seminar in Secure Systems

Student Paper Presentations

(adapted from the Comp 520 presentation advice)

One of the main goals of this course - besides learning about secure systems - is to teach you to give good talks. You will present a paper in front of the class, in the hope that this experience will prepare you for giving talks at conferences, interview talks, etc. At the end of the semester, you will also present the results of your group's project to the class.

You should realize that giving a good lecture is hard work and requires extensive preparation. Do not underestimate the amount of time this preparation takes. The following set of recommendations has been distilled from more than five years of experience with this course format, with some additional input from other sources. The same problems occur time and again, and this is an attempt to address some of them ahead of time. Not all recommendations apply equally to all papers and to all speakers, but they provide some guidance in paper presentation that hopefully you will find valuable.

If this is the first time you are giving a technical talk, you will not be able to follow all these recommendations at once. Like with many other things, with talks, practice makes perfect. In order to distinguish the more from the less important, a number of recommendations have been formulated as rules. Try to pay particular attention to these rules in preparing and giving your talk.

As two heads are usually better than one, a second student will be assigned to each paper to help the presenter digest the paper and prepare for the presentation.

Understanding the Paper

The essential ingredient for a good paper presentation is that you thoroughly understand the paper and the points it is trying to make. Nice slides and a polished presentation are important, but cannot make up for a lack of understanding. Therefore, I suggest that you spend a significant amount of time trying to grasp the contents of the paper you are presenting. This may require reading additional papers as well, to get a better understanding of the context. In particular, I expect you to read at least all the other class papers on the same topic.

While it is important to understand the technical details in the paper, the primary goal in this first phase of the preparation is to understand what the key points are that the paper is trying to make. What the key points are is a question you should evaluate in the context of the audience for which you are presenting the paper. Your paper presentation here is part of a course about software for distributed systems. A particular paper that we read may contain an interesting hardware discussion, but, unless that discussion is essential in understanding the software concepts presented in the paper, it is probably not a key point as far as this course is concerned. Also, do not forget that it is a course, and that therefore for many people your presentation is their first exposure to the material. This implies that relatively high-level points are probably of more interest, while low-level details are most likely going to be lost on your audience. Your choice of what topics to cover might be different if you were to present the paper at a conference or as a job interview talk.

Adapting the Paper for Presentation

Experience indicates that the amount of information that people carry away from a lecture or a presentation is relatively small. Therefore, it is essential that you very specifically highlight the key points of the paper. People are most likely to forget the rest anyways. It is also well-known that the audience is most attentive in the beginning of a lecture. Attention then drops until the point where the speaker signals that the end of the talk is near, at which point attention levels rise again.

Rule 1: You should have a slide very early on that states the key points of the paper, and nothing else. You should have a similar slide at the end. You should decide on these slides first before you proceed.

The most common problem with student presentations in previous versions of Comp 620 has been that the student goes over the paper from A to Z, in the same order as the written paper, without adding or deleting anything. This is a very bad idea. A written paper is an archival document, and therefore it tries to be complete. With an oral presentation, one tries to relate the key points of a paper to the audience. That requires highlighting those key points, and only briefly summarizing or deleting lesser points.

Often people are not quite sure what the key points are, or they cannot make up their mind. They then try to bury the slide with the key points into an ``overview'' slide that outlines the paper or the talk. This is not acceptable. Having an overview slide may be a good idea, but it is never a substitute for a slide with the key points of the presentation.

Once you have gotten past this part, it is now time to develop the rest of your presentation.

Rule 2: The rest of the talk should be structured such that it elaborates and clarifies the key points.

If, for instance, the paper claims to provide some functionality not provided by earlier systems, you should specifically state what this functionality is, what it is good for, how it is accomplished, perhaps what the costs are, etc. If the paper does something better or faster than other systems, you should explain what the new concepts are that allow it to do so, and quantify the improvement. Throughout your discussion, you should occasionally return to the key points to make sure that the audience does not lose sight of the overall context.

People are often tempted to budget time to various parts of the talk in a way proportional to the amount of time they spent getting to understand the corresponding part of the paper. This is often a bad idea, because it may lead to a disproportionate amount of time being spent on tricky details that do not contribute to the overall goal of getting your audience to appreciate the paper. Tricky details are far better understood by reading the paper. Your talk should be sufficiently motivating such that people actually want to go read the paper to figure out the details. If you do decide to go into some complicated aspect of the paper, and again, you should only do so if you consider it essential, you have to explain it in real detail and budget enough time to give the audience a chance to absorb the level of detail. There is a big danger here of starting to explain some complicated aspect of the paper and try to hurry through it because it is not very important. Of course, nobody understands what you are trying to do. Attempts like this usually end with the comment ``Well, I know it's complicated, I don't have the time to explain it all in detail, but I hope you got the idea''. Congratulations, you just confused everybody.

Many of the papers that we will read in this course have experiments, measurements, and performance results in them, ``numbers'' as the theoreticians say.

Rule 3: You should fully explain the purpose of the experiments, the experimental setup, the results, and the conclusions to be drawn from these results.

In other words, you should make sure that it does not come across as ``a bunch of numbers'' but as the account of a scientific experiment. There is nothing worse than throwing up a slide with some numbers on, and leave it at that. Even if the numbers are digested into a table or a graph, that does not relieve you of the responsibility to explain how these results were obtained, what they mean, etc. More will be said about the slides used for presenting experimental results below. You should make sure that you budget enough time for this part of your talk, as it is a frequent source of questions from the audience.

A paper presentation in this course is different from a presentation at a conference, in that you are presenting somebody else's work and you may not agree with all of it. Avoid confusion between the paper itself and your opinion of it.

Rule 4: Your talk should consist of two clearly delineated parts, one in which you present the paper as if you were the author, and a second one in which you offer an evaluation of the paper.

The emphasis in the evaluation should be on the contents of the paper and also, but to a lesser extent, on the paper's overall structure and presentation. This is also a good place to make connections with other papers we have read and cast the paper at hand into a wider context.

Slides

You should allow for approximately 45 minutes of presentation. This is just a guideline, not a strict upper or lower limit. In any case, you should allow for questions both during and after the talk, and also for some discussion after the talk. A good rule of thumb is to have about one slide per two minutes of talk, although this is certainly not an universal rule.

If you put something on a slide, it must be that you think it is worthwhile for the audience to read it. From this follows the cardinal rule for making slides, unfortunately also the rule that is most often sinned against.

Rule 5: Use a big letter size.

I cannot stress this rule enough, so just to make sure, I will repeat it.

Rule 6: Use a big letter size or else ...

There is an important corollary to this rule. Occasionally you want to use a figure (or a table) from the paper. You should not do this by copying the figure on to a slide, but by redrawing it on a size that will be readable by your audience. In particular, units on graphs or in tables should be clearly legible. If there is too much in a particular graph in the paper, split it up in two graphs or present only part of it.

The second rule is almost equally obvious, but also often ignored.

Rule 7: Do not clutter your slides.

First, you should leave large margins, both vertical and horizontal. Be especially careful not to go too close to the bottom of the slide, as the audience may not be able to see the bottom of the screen. If you find yourself shifting the slide up and down during your talk, then your margins are not large enough. Second, your slides should just contain the bare essentials: no full sentences, just a few keywords. Never try to cram a lot of material on to a single side, use two or more slides instead. There is nothing wrong with a slide that is three quarters empty.

Rule 8: Your slides should be visually appealing.

A picture is almost always better than text. Long bulleted lists are boring. Try to distinguish major and minor points by using indentation, by using boldface, etc. It is a big plus if you do so consistently throughout your slides. Colors can be very appealing but must be used with care. You must be consistent. If you have two slides with horizontal lines depicting the execution of processes and arrows representing messages between them, use the same color on both slides for the processes and the same color for the messages. Over-use of color can however be distracting. You must also be careful with certain colors. Red and pink are indistinguishable with many projectors.

Rule 9: A single slide should never try to convey more than a single concept.

All slides should have a title that describes this concept. If the explanation of a concept goes on for more than one slide, then repeat the title with some qualifier, or with just (Continued) added to a title. Never start a new idea in the middle of a slide, or try to convey more than one concept in a single side. Use a new slide instead. As said before, there is nothing wrong with a slide that is three quarters empty. It is almost always a bad idea to cover up part of a slide, because it almost always means that you put too much on the slide. Besides, half of the time, the piece of paper that you are using to cover up part of the slide falls off. In the same vein, while overlays can be used to advantage, it is often better to put up a completely new slide, which may include most of what was on the previous slide. Overlays are difficult to line up correctly, and slides taped together tend to fall of the projector. Overlays of more than two or three slides do not project clearly.

If you know ahead of time that you are going to use a slide more than once, you should make as many copies of the slide as you will need and insert them at the appropriate place in your stack of slides. This avoids searching through the slides during the talk.

There is some debate as to whether it is better to typeset slides or to write them by hand. Rumor has it that at theory conferences handwritten slides are the norm. At systems conferences this would be considered very sloppy. You should use Latex or Powerpoint to prepare your slides. Both have reasonably good facilities for doing so. I recommend against handing out copies of your slides before the talk, because it reduces interaction between the speaker and the audience. Make your slides available from your home page, either in Postscript, PDF, or HTML format, so that other students can refer to your slides later on.

The Dry-Run

So now you have all your slides ready.

Rule 10: You should do (at least) a couple of dry runs.

Get some people together, preferably in the room where you are giving the talk, and go through the entire talk, as if you were giving it in class. Ask somebody to time it, and ask people to make notes. It is also a good idea to videotape the talk. We have a video camera available, and you are encouraged to take advantage of this facility. Some people also find it useful to give the talk in an empty room with just the video camera running. The first time you give a talk, you will be surprised to find out that a lot of people will look at you with completely empty faces. Some people will fall asleep. An empty room with a video camera running will prepare you for this situation. You can reserve the video camera with the secretary. When not reserved, it is available on first-come-first-served basis.

You will find that after a first dry-run, you will want to change a lot. In fact, it is not uncommon for people to change all of their slides after a first dry-run.

Rule 11: Avoid premature optimization. Get the contents right first, and then make your slides look pretty.

For a first dry run, you might just have some of the slides hand-written or hand-drawn, as long as they accurately reflect what you are going to have on them for the real talk.

After you have your talk ready, you should spend some time trying to anticipate what questions you might get during and after the talk. If the answer to a particular question you expect is difficult, prepare a slide with the answer. In general, take a few extra blank slides and a suitable pen with you to the talk, so that you can write something down if you need to.

One final rule.

Rule 12: Get some sleep the night before your talk.

The night before your talk is not the right time to do all of the above. Sleepwalkers do not make for exciting speakers.

To Memorize or not to Memorize?

There is often debate on how much of the talk you should memorize. Your presentation should be smooth, but at the same time it should not appear completely scripted. How much of the talk you have to memorize to accomplish this effect differs from person to person. You should definitely not have to consult your notes on a regular basis. Ideally, you should not have to do so at all. It is very useful to completely memorize the first minute of your talk. Many people are quite nervous in the beginning, and memorizing the first minute helps them get off to a smooth start. Along the same lines, you should outline in detail what you are going to say in the technically complicated parts of the talk, if any. Only the most accomplished speakers are capable of improvizing successfully at such times.

You Are on the Stage

Rule 13: Make it look like you are having a good time.

Audiences are far more forgiving if they think you are enjoying yourself. If you look scared or uncertain, it is like letting the wolves smell blood. Show some enthusiasm for the subject. Speak loud and forcefully. Do not end your sentences in a mutter. Do not be afraid to raise and lower your voice, to delineate more or less important points. A moment of silence at the right time can do wonders. If you just said something important or something very complicated, let it sink in for a while before you continue. This technique will work particularly well if your slides are such that each slide introduces a new concept. Do not talk while you are changing slides, and give the audience time to absorb what was on the previous slide.

Rule 14: Make sure that the audience can always see the screen.

Do not stand in front of it, and do not block the projection of the slide on to the screen. Especially after changing slides, people tend to continue to stand right next to the projector. In a level classroom, you should go stand next to the screen after changing slides. Some speakers also move around while they are talking, a good idea when done in moderation. Also, never remove a slide before people have had a reasonable amount of time to read what is on the slide. Never flip back and forth between two or more slides.

Make sure you have a pointer with you to point things out on the slide. Pointing things out with a pen or with your hands obscures the projection of the slide. Do not play with the pointer; put it down when you do not need it. It is a matter of argument whether you should point on the projector or on the screen. Either way can be done right, but pointing to the screen is often better (not always possible, though). If you point on the projector, make sure that you are not obstructing the projection or the view of a large part of the audience, especially the part of the audience sitting on the same side of the projector as where you are standing. If you point on the screen, make sure that you turn back to the audience. Do not stand or talk for an extended period with your back towards the audience. Avoid having to write on a slide during the talk, but have an appropriate pen with you just in case. You will certainly obstruct the projection while you are writing. Make sure to step back and take a short pause afterwards so that the audience can see what you have written before you continue.

Rule 15: Do not try too hard to be funny.

Most people are not very funny, anyways, especially when they are nervous. There is nothing more awkward than somebody who is not funny and who is trying to be. There is nothing more likely to throw you off than to have the audience react stonecold after you tried to make a joke. In general, if something goes wrong during the talk - most likely something will in fact go wrong - do not let it throw you off. Also, people might not take you seriously if your talk turns into a stand-up comic act. This does not mean that you should not be able to see the humor of the situation if something funny happens. But just let it happen, do not try to make it happen. When it does happen and people start laughing, wait until the laughter dies out or nobody will have heard what you said. Also, in some situations you will be introduced before the talk, for instance for an interview talk. Some people have the extremely annoying habit of introducing the speaker by telling some supposedly funny story (In some departments this is a tradition). This is really great, because it guarantees that nobody is listening right from the beginning. If this happens to you, make absolutely sure to wait until the laughter dies out, and be very serious during the first few minutes of your talk to get things back on track.

Interacting with the Audience

Rule 16: Look at the audience and seek eye contact with them.

Do not look at the screen, at your notes, at the slide projector, or away from the audience in any other way. Look at the whole audience, not just your friends, the instructor, the first row. You are much more likely to keep the attention of the audience going this way.

If you are really confident, you should try to engage your audience in actively participating through questions and answers. If you ask a question, you have to be willing to wait until somebody volunteers an answer. If you don't, your question will come across as perfunctory, and you will certainly not get any answers the second time you try. As said before, you have to be pretty confident to pull this off.

An important aspect of giving a talk is dealing with questions. The first thing you should realize here is that questions are a good thing. It means that people are listening. It does not (necessarily) mean that you did a poor job of explaining something. So maintain a positive attitude to questions and the people who are asking them. Allow the person who is asking a question to finish his sentence. Quite often, speakers jump in halfway with their response. It is rude to do so. And you are probably answering the wrong question. It makes people feel good if you ask them if they understood your answer. It is also perfectly acceptable to say that you do not know the answer. Do not try to hide this by not answering the question, and certainly not by making fun of the question or the questioner.

The above not withstanding, it is true that many audiences have a self-appointed designated jerk. Every audience also has people who do not understand a word of what you are talking about. The combination of the two, a person who repeatedly asks stupid questions on a tone of ``what the hell are you talking about'' can be quite deadly. This is one of the most difficult situations to deal with, and there is no universal approach to combat the problem. You are lucky if the audience contains a jerk exterminator. This is the person from the audience who casts an annoyed look over his shoulder, and gives a one-word answer to the question. If that happens, you are safe. Do not rub it in. If not, you must be very firm in not letting this person derail your talk, while remaining polite (if at all possible). One approach is to ask the jerk to defer his questions until after the talk. If he keeps on asking questions, start giving one line answers, and continue immediately without checking if he understood the answer. This situation is unlikely to occur in a course, but in general you should be prepared for it.

The Role of the Instructor

Speakers and the person who helps them prepare their talk should come and see me at any time prior to the talk. Speaking from experience, there seem to be two times at which a meeting is extremely helpful: (1) after you have digested the paper to check if you figured out what the main points are and to clear up technical details about the paper; (2) after you have done a dry-run and fixed up your slides according to the comments of the people who attended. I will go over your slides with you and point out possible improvements.

At lecture time, I will introduce you, and I will lead the discussion after your talk, but you should not count on me helping you out during the talk. Although I may say or ask something, you are in charge and you should be able to handle whatever situation comes up.

Evaluation

After each talk I will hand out evaluation sheets, on which everyone will be asked to evaluate the presentation. I will then summarize the student evaluations, add my own comments, and provide a summary to the speaker. Your evaluations will have no effect on the speaker's grade, so I urge you to be candid and constructive. Your evaluations may however affect your grade, so I encourage you to take them seriously. I will also videotape all student presentations. The tape will be available for some amount of time after the presentation. If you want to keep the tape for an extended amount of time, you should bring your own tape.

The intent of these presentations is not only that you learn the material in each paper, but that you also learn how to make good presentations. These evaluations will allow you to get feedback on how the audience reacts to and perceives your presentation, and the videotape will allow you to see yourself as the audience sees you. Most people seem to have an incredible fear of seeing themselves. It's usually not that terrible. And everybody who has brought up the courage to examine the tape in detail has found it to be extremely helpful.

You should understand that the fact that a paper is presented by a student may occasionally take its toll on the material. The discussions after the paper presentations and the review lectures are meant to smooth out such problems, but they are not a perfect remedy. This is the price that you have to be willing to pay for the opportunity to practice speaking in public.

Dan Wallach, CS Department, Rice University

Last modified: Tue Apr 13 00:37:44 CDT 1999