Course Project

The course project is your chance to gain hands on HRI research experience. You are expected to complete this project in a group of 3. This project must involve conducting a human-subjects study that answers a novel research question focused on the interaction between a physically embodied robot and 1 or more people. This page contains details on the project deliverables, due dates, as well as examples of prior projects (at the bottom of the page).

Course Project

---

• Project Timeline
• Project Deliverables
• Project Grading
• Helpful Information for Running Your Human-Subjects Study
• Example Projects from Prior Topics in HRI Student Teams

Project Timeline

---

Project Deliverables

---

Project Pitches

Before pitching project ideas, read this entire web page thoroughly to get the full picture of what this project will entail.

Although the project will be completed in teams of 3, these pitches are made by individuals. The purpose of these project pitches is to aid in the formation of project teams (your project proposal can and will deviate from these project pitches, again, the project pitches are meant to serve as a starting point for team formation). In this SharePoint document you'll pitch 1-3 project ideas that you'd be interested in pursuing for the course project. It does not need to be fully formed, but enough to give your classmates a good idea of what the project will entail.

Draft Project Proposal

Your draft project proposal should introduce your motivation for your project, the research questions it tries to answer, and proposed methods. We encourage you to use the Lab 4 HRI Study Design Worksheet as a resource and template for the draft project proposal. Your draft proposal should include:

• Motivation & Experimental Design
  • Motivation: One paragraph describing why you're motivated to pursue this project. Why is it important and interesting?
  • Research questions: What research questions does your work try to answer?
  • Hypotheses: Detail your hypotheses.
  • Experimental Conditions: Detail your experimental conditions.
  • Experimental Design: Is your study between or within subjects? How many experimental conditions will a participant experience and in which order?
• Methods
  • Procedure: In 1-2 paragraphs, describe the procedure your human subjects will undergo.
  • Measures: Detail the measures that you will collect. This includes questionnaire measures, behavioral/observational measures, and interview questions. For the draft project proposal, you do not need to attach a Qualtrics survey (but you will for the finalized project proposal), for this deliverable, a bullet-point list will suffice.
  • Sample Size: State your intended sample size, as a reminder, these are the expectations for full credit for the project (and while your group is expected to be 3, there may be groups of 2 or 4, depending on the numbers of students in the class):
    • Group of 2: at least 14 participants
    • Group of 3: at least 20 participants
    • Group of 4: at least 26 participants

Finalized Project Proposal

The finalized project proposal, in addition to what is contained in the draft proposal, should include:

• Timeline: A detailed timeline with, at minimum, weekly project milestones.
• Qualtrics questionnaire: A Qualtrics questionnaire including all of your questionnaire measures (attached to the main document as a PDF).

Before submitting your final project proposal, make sure that you've met with the teaching team to discuss their feedback on your draft project proposal during the designated lab times during week 6 of the semester.

Project Deliverables #1

Project Deliverables: For this first project deliverables submission you are expected to submit (or have already submitted):

• A first draft of your final paper's Introduction and Background sections: You can find the detailed instructions for these sections in the Final Report section below. You are expected to use the the ACM conference template for this submission. You are also expected to include a section on your genAI use for this deliverable, in which you will describe whether you used genAI tools and if so, which tool(s) you used and how you used them.
• Submission of your group's IRB application: Your group is expected to have submitted your IRB application as outlined in the IRB section below before the Project Deliverables #1 deadline. You're more than welcome (and encouraged) to submit earlier!

Project Deliverables #2

Project Deliverables: For this project deliverable, you are expected to submit:

• A first draft of your final paper's Methods section: You can find the detailed instructions for this section in the Final Report section below. We realize that you will not be able (yet) to include all the details required for the final paper (e.g., preregistraion link), however, we encourage you to include as many elements as you can at this deliverable deadline, and at minimum:
  • Hypotheses or research questions
  • Experimental conditions
  • Interaction context/task description
  • Details on robot implementation - including link to your GitHub repository with your robot code
  • Experimental protocol
  • Measures
  You are expected to add the methods section to the the final paper draft you started in the last project deliverable submission (introduction + background). You are also expected to update the section on your genAI use for this deliverable.
• Your robot code GitHub repository link representing a full completed implementation of your robot/system code: You can include the GitHub repository link in your methods section of your paper draft submission. If you are using a private GitHub repository, ensure that the teaching team has access to it. Your GitHub repository should also contain a README file that includes:
  • A brief description of what the code does
  • Instructions for how to run the code
  • An embedded video demonstrating your robot in action

Piloting Your Study

It is incredibly important before you start running real human subjects that you practice, practice, practice executing your human subjects study. We recommend:

1. Running the study several times yourselves from top to bottom, AND
2. Running the study on one or two friends (including having them fill out the questionnaires) and asking them if anything was confusing or if they noticed any typos in your questionnaire.

Piloting is essential to work out any kinks and ensure that your study is set up to adequately answer your research questions.

Additionally, each team is required to pilot their study with a member of the teaching team during lab time in Week 11. This is a mandatory step before you begin running participants. Come to lab that week with your full study ready to run end-to-end (robot interaction + surveys/interviews).

Initial Idea Presentation

During week 8, each project team will give a brief 2-3 minute presentation of their project idea. I will plan to cut you off if your presentation goes beyond 3 minutes, leaving 1-2 minutes for questions and transitioning to the next group. All members of your team are expected to participate in the presentation. Your slides should be uploaded as a PowerPoint presentation to this Initial Idea Presentations SharePoint folder before the class meeting where you will present. Your presentation should cover the following:

• Motivation: Why are you interested in pursuing this project? Why is it an important and interesting question to explore?
• Research Questions & Hypotheses: What are the main research questions your project is trying to answer? What are your hypotheses?
• Methods Overview: Provide a high-level overview of your planned study protocol and experimental design.
• Measures: At a high-level, what measures do you plan to use to answer your research questions and evaluate your hypotheses?

Your presentation will also be graded for its design and style. Please ensure that your slides are NOT too text heavy. If the audience is spending their time trying to read a lot of text on a slide, it's hard for them to pay attention to you. Use visuals and limited text on your slides, describing the details verbally during your presentation.

Midway Presentation

During week 13, each project team will give a 4-6 minute midway progress presentation. I will plan to cut you off if your presentation goes beyond 6 minutes, leaving 1-2 minutes for questions and transitioning to the next group. All members of your team are expected to participate in the presentation. Your slides should be uploaded as a PowerPoint presentation to this Midway Presentations SharePoint folder before the class meeting where you will present. Your presentation should cover the following:

• Motivation & Research Questions/Hypotheses (brief review): Briefly remind us of the motivation behind your project and the research questions and hypotheses you are investigating.
• Background: Describe any relevant background literature and tell us what unique and novel contributions your study is making beyond what already exists in the literature.
• Methods: Spend the majority of your presentation detailing the design of your study: your experimental conditions, study protocol, and interaction context/task.
• Robot Video: Show a video demonstration of your robot in action.
• Measures: Detail the measures you are planning to capture to answer your research questions and evaluate your hypotheses.
• Recruitment & Progress: Report on where you are in terms of participant recruitment and running your study.

As with the initial idea presentation, this presentation will also be graded for its design and style. Please ensure that your slides are NOT too text heavy. If the audience is spending their time trying to read a lot of text on a slide, it's hard for them to pay attention to you. Use visuals and limited text on your slides, describing the details verbally during your presentation.

Final Project Presentation

During the last week of class - week 16, you will present the work that you've done in your course project. You should aim for your presentation to take 8-10 minutes. I will plan to cut you off if your presentation goes beyond 10 minutes, leaving 2-3 minutes for Q&A and transitioning to the next group. All members of your team are expected to participate in the presentation. Your slides should be uploaded as a PowerPoint presentation to this Final Project Presentations SharePoint folder before the class meeting where you will present. Your presentation should highlight the following:

• Motivation, Background, & Research Questions/Hypotheses (brief review): Briefly remind us of the motivation behind your project, what is novel about what you're investigating (compared with prior work), and the research questions and hypotheses you are investigating.
• Methods (brief review): Briefly summarize your study design, experimental conditions, and the measures you used to assess your hypotheses.
• Participant Video: Show video clip(s) of a actual participant(s) in the experiment, showcasing your main experimental manipulation(s) and/or interaction(s) with the robot.
• Results: Report on the results you found. Please include all of the main measures, regardless of whether or not you found significant differences. This section should receive substantial time in your presentation.
• Discussion/Implications: Reflect on what your results mean, how they answer (or don't) your research questions and hypotheses, and the broader implications and takeaways.
• Learning Outcomes: Enumerate some "lessons learned" from this project. What did you lean by going out and actually conducting HRI research, beyond what you've learned from reading the papers in this class?

As with your first two presentations, this presentation will also be graded for its design and style. Please ensure that your slides are NOT too text heavy. If the audience is spending their time trying to read a lot of text on a slide, it's hard for them to pay attention to you. Use visuals and limited text on your slides, describing the details verbally during your presentation.

Final Report

The project report should be written in the style of an academic conference paper (like the papers we've been reading for our class discussions), feel free to use the ACM templates for conferences. The report should be approximately 6-8 pages excluding references and contain the following sections:

• Abstract: A concise (150-250 word) summary of your project's motivation, methods, results, and broader implications.
• Introduction: Motivate why the research you are presenting is both important and novel compared with prior work. Introduce the main concepts that you are investigating with your user study and briefly summarize your approach. All references to prior work should be supported with appropriate citations.
• Background: Review the relevant prior work and briefly highlight how the work you are doing is novel in light of the prior work. For full credit, include at least 15 references to prior work in this section.
• Methods: Include the following details in this section:
  • Link to AsPredicted preregistration
  • UChicago SBS IRB number
  • Link to the UChicago Box folder that contains all of your data (video data, questionnaire data, etc.)
  • Hypotheses or research questions
  • Experimental conditions
  • Interaction context/task description
  • Details on robot implementation (including a link to a public GitHub repo with your robot code)
  • Experimental protocol
  • Measures (remember to cite any measures you adopt from prior work)
  • Participant information (numbers in each condition, demographics)
• Results: Present both quantitative results based on observed participant behavior and/or Likert questionnaire responses as well as qualitative results based on open-ended questionnaire responses and/or interviews. Key results and dependent variables should be displayed visually (e.g., graphs) with readable labels (e.g., axis labels) that are in a large enough font size to easily read. All quantitative results should be accompanied with the test statistic, effect size, and p-value. All qualitative results should report the inter-rater reliability (Cohen's kappa).
• Discussion: Discuss how your findings answer (or not) your research questions and/or hypotheses, the broader implications of your results, and any limitations of your work.
• Conclusion: In 1 paragraph, summarize the main findings and broader implications of your work.
• Contributions: Detail the contributions of each team member, describing which portions of the project were completed by each team member.
• GenAI Use: In one sentence, describe whether you used genAI tools with this reading reflection and if so, which tool(s) you used and how you used them. Please refer to the Use of Generative AI section of the course syllabus for the expectations of genAI use in this course.

Teaming Survey

Due at the same time as your final report, ensure that you've filled out the course project teaming survey. Assuming that all team members make approximately equal contributions on the course project, all team members will receive the same grade. However, if certain team member(s) do not contribute a sufficient amount of work to the course project, each team member may receive a different grade that reflects their individual contributions.

Data Sharing

If you have not done so already, please share all of the data you've collected in your HRI project in a UChicago Box folder and add the teaching team to the shared folder. This includes questionnaire data, video data, and any other data you collected in your project.

Project Grading

---

The grading for the course project will be as follows:

• 25% intermediary deliverables - project pitches, proposals, project deliverables
• 10% final project presentation
• 25% final report
• 40% project work - your grade based on the work that you accomplished during the class project is not dependent on your project being a 'success', your project work will be graded based on the project's scope, novelty, and the investigative work that you put into pursuing your research questions

For further details on project grading, please check out the final project grading rubric.

Helpful Information for Running Your Human-Subjects Study

---

Human Subjects Protection Training

Since you will be conducting a human subjects study in your class project, you are required to complete KU's human subjects training before beginning your study. You will need to submit an IRB (Internal Review Board) Application (see below) and have completed Human Subjects Protection Training (← that link contains all the info you need to complete the training). Before you submit your IRB application, you must have already completed the human subjects protection training.

Internal Review Board (IRB) Application

In order to run human subjects in this course, you must submit and receive approval for an Internal Review Board (IRB) application. This means that BEFORE you run any human subjects, you MUST have your IRB application approved.

The purpose of the IRB is to ensure that research involving human participants is being conducted ethically and with proper respect to those participants. Your IRB application will include:

• a high-level description of the research you plan to conduct
• a protocol specifying what each human participant will do in your study
• a consent form that each participant will read and agree to before beginning your study
• a description of how you plan to protect any identifiable data you collect from human participants and how you plan to store the data
• any questionnaires you plan to administer to the human participants
• any recruitment methods you plan to use (e.g., flyers, social media posts, emails)
• details on how participants are to be paid and how much they will be paid

In this course, you have two options for IRB submission depending on whether you intend to share your findings beyond the classroom:

Option 1: Official KU IRB - Choose this option if you are interested in the possibility of sharing your findings beyond the classroom (e.g., publish a paper, present at a conference). KU's IRB only accepts proposals that are designated as "research", that is, work with the potential to be disseminated beyond the course.

Option 2: "Sarah IRB" - Choose this option if your project is for this course only and you do not plan to share your findings beyond the classroom. In this case, I (Sarah) will serve as a stand-in IRB reviewer. If you choose this option, you must remove the term "research" from all participant-facing materials (consent forms, recruitment materials, etc.), since KU's official IRB does not oversee work that is not designated as research.

Regardless of which option you choose, you will use the forms available at https://research.ku.edu/human-subjects-research-forms to complete your IRB application. When you are ready to submit, email me (Sarah) with the following materials:

• CITI completion certificates for all team members
• KU human research protocol
• All consent forms (e.g., a video consent form if you are recording participants)
• Any recruitment materials you plan to use
• A printable version of your Qualtrics survey (if your study involves a survey)
• A list of any interview questions (if your study involves an interview)

Do not begin running participants until you receive approval from me.

HRI Measurement Scales

HRI researchers at the Naval Research Lab and George Mason University have compiled a database of scales HRI researchers use in their studies. I highly encourage you to visit their website Finding the "Perfect" Scale (especially their database page) where you can find questionnaire scales you can use for your study as well as a rating of the quality of each scale.

Qualtrics

If your study involves asking human participants to fill out a questionnaire, I recommended that you use Qualtrics. You can even put your consent form in Qualtrics, so that participants can read and agree to the consent form before beginning your study. If you are new to Qualtrics, I recommend that you check out their Getting Started Guide and Survey Module Overview.

Recruitment

You will be responsible for recruiting your own participants. You can post on social media, send emails, post on UChicago organization sites, etc. Just be sure that you include all forms of recruitment in your IRB application that you plan to use. We recommend you try to recruit the following minimum number of participants, dependent on your group size:

• Group of 2: at least 14 participants
• Group of 3: at least 20 participants
• Group of 4: at least 26 participants

While your group is expected to be a size of 3, there may be groups of 2 or 4, depending on the numbers of students in the class.

Robot

Payment

For in-person human participants, we will pay them at the rate of $12.00 per hour (the teaching team is still in the process of figuring out how to facilitate this payment). To facilitate this payment, please email the teaching team daily when you are running your study with a list of email addresses that correspond with your study participants from that day. I will ensure that they are sent their Amazon or Tango gift card for participating in your study.

Example Projects from Prior Topics in HRI Student Teams

---