Best Practices ComPAIR Home

Understanding the usefulness and usability of ComPAIR for peer review is an ongoing process, with researchers and developers at the University of British Columbia actively seeking input to determine best practices using a variety of methods.

Collaborative teaching & learning research

Instructor/TA interviews & design input

Student surveys & application usability testing

Findings from research & user feedback

General assignment setup

During the pilot at UBC, instructors, teaching assistants, and students participated in surveys, focus groups, and/or one-on-one interviews, and these revealed common practices that resulted in a more positive learning experience, regardless of discipline. (More detail on the pilot evaluation process and its outcomes is available in our Teaching & Learning Inquiry article .)

As with any learning activity, how assignments were introduced, designed, and integrated in the course significantly impacted the student experience with ComPAIR. The following are the best practices we recommend.

  • Clearly explain the objective
    Students who better understood what underlying skill they were practicing in the application felt they learned more from the assignments. It is key for students to know the end goal of using ComPAIR—how comparing will concretely help their learning—not only how it works.
  • Use detailed criteria/rubrics
    Students given more guidance said they learned more from comparing. Explaining precisely what to look for during comparisons—with multiple descriptive criteria in the application and/or detailed external rubrics—may result in stronger learning and (possibly) richer feedback.
  • Require 2+ comparisons
    Despite low reported confidence giving peer feedback, many students said they learned simply from practicing this with answer pairs. A minimum of two comparisons per assignment is suggested to give students repeated practice to learn from.
  • Build to or from ComPAIR assignments
    Assignments felt more beneficial to students when presented as part of a larger process. It may be more useful for students to use ComPAIR in the context of a bigger course goal (e.g., prep for writing term paper, evaluation of a bigger project or draft), rather than apply it to standalone assignments.

A student handout template (Word document) is available for instructors to download, customize, and use in their courses. This provides a core introduction and instructions recommended for familiarizing students with ComPAIR.

Scoring & grades

How scoring works:

When ComPAIR assignments are set up to select answer pairs adaptively—that is, to pair answers that are closer in quality over time—the application tracks a score for each answer. Every answer starts with a base score and that increases or decreases (based on the Elo rating system ) to reflect the outcome of the comparisons the answer competes in. Basically, after every comparison, the winning answer takes points from the losing one, with the difference between the answer scores at the time of comparison factoring into the total number of points transferred.

Once answers move beyond the base score, higher-rated answers are expected to win more often. So if a higher-rated answer wins (as expected), fewer points are taken from the lower-rated answer's score. Conversely, if a lower-rated answer wins in an upset (unexpectedly), more points will be transferred to the lower-rated answer's score. At the end of this process, if reviewers have generally made the "right" choices for the pairs, stronger answers should end up with higher scores and weaker answers with lower scores.

Given the above functionality, some instructors have chosen to use the scores and the ranking they provide to form all or part of a student's grade for low-stakes assignments, most often by chunking the answer list into rough top, middle, and bottom answer groups.

While anyone is welcome to try the application for crowdsourced grades, we caution against relying on ComPAIR scores without any additional validation.

Our internal research so far indicates that ComPAIR may not reliably map to traditional grading, as it ultimately relies on the skills and training of novices (students) to provide accurate, well-informed rankings of their peers' answers. Many instructors have found it effective to grade students with some combination of their participation in the process (answering, comparing, self-evaluating) and the quality of work (answer, peer feedback) they submit.

Get started »

Get to know ComPAIR

ComPAIR is an open-source peer review application that pairs student answers for deeper learning through comparison of peer work