ComPAIR was first piloted in three courses to test its versatility for peer review in a variety of academic areas. Instructors from three disciplines at the University of British Columbia participated, and their example use cases are presented below.
Students completed two scaffolded ComPAIR assignments. In each, they created a 50- to 75-word critical premise designed to function as the foundation for a 1000-word critical essay responding to a specific prompt. Students were given a 36-hour window in which to submit a text answer, and were then asked to evaluate three pairs of answers over the next two days, using two criteria within the application. After comparing three pairs, students reviewed peer feedback on their own answer and could optionally view all peer answers.
In the final steps, teaching assistants assessed and commented upon each student's answer privately, then used the answers and feedback in a two-step process in the weekly tutorial: they reviewed excellent answers (anonymous and from other tutorial groups) and common pain points, and then used the exercise as the basis for a hands-on essay-development exercise.
Each ComPAIR assignment was assessed on the quality of the answer (2%) and the quality of the feedback given to their peers (2%), with the equal weighting communicating the parallel learning gains available from both individual practice and insightful reflection on what makes classmates' answers more or less effective. The exercise was intentionally kept low-stakes, as part of a scaffolded series of assignments leading to in-class essays and research papers.
"Since the pilot, I have used ComPAIR in all of my larger-format classes. My TA team and I found the app easy to use, and it invited some particularly effective teaching practices in tutorials, as every student had paid very close attention to the assigned reading and was resoundingly prepared to engage and participate. Nearly half of my students listed ComPAIR as the most useful classroom assignment on the year-end course evaluation: they enjoyed the comparison aspect, and they could readily understand how the tool and the assignment were helping them to engage course content and the critical processes behind their assignments and assessment."
– Course instructor Tiffany Potter
Students completed three similar ComPAIR assignments. Each was a part of answering a "Big Picture" question, which involved participating in a series of tasks that mirror Polya's Method for problem solving: 1) understand the question 2) make a plan 3) execute the plan and 4) reflect on your answer. For the first two steps, students collaborated in an external forum. The third and fourth steps took place in ComPAIR. Students executed their plan by submitting a written solution—a combination of calculation and exposition, 3-6 pages long—and reflected on their answer by completing three comparisons and a self-evaluation.
Two external rubrics provided to the students were key to making this process work. The first told them how to assess each other's work during the comparisons, providing clear expectations for the written solutions and how to determine which of two assignments was better. The second rubric was what the instructor or TA used to determine students' grades on the overall assignment. This rubric graded participation in the forum and whether students completed the work in ComPAIR. Most importantly, it graded the quality of the feedback students gave during comparison. This meant students knew grades were not directly linked to the content of their submission; grading was based upon their contributions to the process.
"The 'Big Picture' questions are the heart of this course and it would be impossible to run them without ComPAIR. ComPAIR allows students to read and evaluate their peer's work without the full cognitive load of doing full peer grading. Students can read 6 assignments and get exposure to many viewpoints, while still exercising their peer evaluation skills. The direct comparisons themselves also allow students to more acutely see what separates good work from better work. It's an invaluable tool for getting students to explore the grey areas in science."
– Course instructor James Charbonneau
Students completed two similar ComPAIR assignments at the end of two major units in the course. Each unit developed over a 1-month period and explored a major mathematical tool, and summative assignments asked students to demonstrate their understanding of the relationship between the tool and the geometry it was intended to explore. For each assignment, students had one week to download a template of blank panels to draw figures in and then upload their completed answer. Students completed three comparisons during the following week (using the default criterion) as well as a self-evaluation.
Students received extra participation points for taking part in the assignments, as well as public feedback from the TA (left on their answer in the application, so others could also benefit). The instructor's solution to the assignment was shown and discussed in the lecture following the assignment's completion.
ComPAIR is an open-source peer review application that pairs student answers for deeper learning through comparison of peer work