Collaborative learning is a general concept where two or more people attempt to learn something together. One way in which MOOCs accomplish collaborative learning is through online discussion forums. I am interested in extending the collaborative learning environment to assessments. In the most basic sense, students work together in a MOOC to solve a problem. My hope is that by working together, a higher quality answer is achieved and that each member of the group participates in the development of the answer.
One way in which MOOCs accomplish collaborative learning is through online discussion forums. I am interested in extending the collaborative learning environment to assessments. In the most basic sense, students will work together in a MOOC to solve a problem. I plan to take ideas from Peer Instruction and apply them to the edX platform.
My plan is to replicate the questioning procedure of Peer Instruction on the edX platform. Here are my initial attempts at it…
Peer Instruction is a teaching method developed by Eric Mazur. Questioning students follows a certain procedure, described below:
- Instructor poses question based on students’ responses to their pre-class reading
- Students reflect on the question
- Students commit to an individual answer
- Instructor reviews student responses
- Students discuss their thinking and answers with their peers
- Students then commit again to an individual answer
- The instructor again reviews responses and decides whether more explanation is needed before moving on to the next concept.
Overall I think I am on the right track to replicating this questioning procedure in edX. You be the judge — here is what my attempts looks like on the edX website. I break the presentation of my attempt into how an instructor sets up the problem, how students see the problem, and finally how the instructor could use the results as feedback.
Instructor Workflow — Creating the Problem
The edX platform has a dedicated site for instructors to create course materials (called “Studio”). When an instructor wants to create a new problem, they end up at a page similar to the one below.
When the instructor clicks the “Problem” component to add, they will be presented with a list of options for different problem types. These can be as simple as a text input, or as complicated as a building a circuit schematic or molecular structure. I have extended edX with a new problem type, which I called “Peer Instruction Checkboxes.”
A “Peer Instruction Checkboxes” problem is an extension to the normal Checkboxes problem type. Students can select more than one option presented. But that’s pretty much where the similarities end. Here is what the initial problem looks like.
From there an instructor will, as is the usual workflow in edX, click the “Edit” button and edit the problem. Currently the instructor has to use the advanced editor and cannot use markdown (as they can in the original Checkboxes problem type).
At this point, the instructor is ready to release the problem. Now let’s take a look at what it will look from the student’s perspective.
Student Workflow — Answering the Problem
When a student first sees a peer instruction problem, they will notice it looks quite similar to a normal checkboxes problem. However, instead of seeing a familiar “Check” button near the bottom, they will instead see an option that says “Commit to Initial Response.” Below that gives the feedback: “You will submit an initial response and then be presented with other students’ answers.”
After the student attempts a response, they are presented with information about what all of the students have picked so far. A link also appears which will link them to a discussion forum about the problem. This discussion forum only opens up to them when they make an initial attempt at the problem. Also note that there is no feedback if the student was correct or not. This prevents people from making an argument based on the system telling them they were correct (so as to not stifle discussion).
You might notice that a button has now appeared to the student that says “Commit to Final Response.” So what’s to stop a student from jumping the gun and circumventing discussion to go right to a final answer. I decided to reuse some of edX’s existing features to limit the minimum time between responses (shown below).
Although this does not quite exactly force a student to discuss, it might be enough to get them to at least check out the discussions while they wait. I definitely think there is a lot of interesting implications for this approach, and also even just how much time to limit. For example, a class with very low synchrony might require higher delays between attempts.
Once enough time has passed, (hopefully) more responses have come in and discussions have occurred so that the student may have a more informed opinion. At that point, the student is ready to commit to their final response.
When the student commits to their final response, they will be presented with the option to view the answer as well the grade for their response. Importantly, I chose to open a new discussion forum for those who have finished the problem. They will not stifle conversation of people still “in progress” with the problem (by telling everyone the correct answer with a screenshot). But the discussion doesn’t have to end when you finish the problem. A student might disagree still with what the instructor thinks the correct answer is, for example. I think that this forum would be a nice area to voice that concern.
From the Instructor’s Viewpoint
The final step of the questioning procedure in Peer Instruction is for the instructor to evaluate the responses (ideally they should also look at the first attempts too). This is technically possible through the submission history viewer, but it seems like it might be quite unwieldy to use for a large class. I am still exploring how to exactly surface this information more easily to an instructor. From a more optimistic point of view though, I think the discussion forums could be a valuable source of qualitative feedback for the instructor as it would hopefully show a student’s reasoning as they discuss with each other.
I still am thinking about a lot of things, and the work certainly isn’t complete. Here are a few things I am thinking about:
- The effects of synchrony on peer instruction in a MOOC: Clearly if everyone was in the same class, the questioning procedure in Peer Instruction can be done serially. However in a MOOC that may not be possible as other students may want to move on from the problem as other students are not online at the moment.
- Gatekeeping — when can a student submit their next attempt? In Peer Instruction, it seems as if the instructor should review all answers before allowing another attempt. Perhaps that isn’t a necessary condition in a MOOC (and in fact, my implementation does not have that restriction). It might be useful to leave this as a setting for the instructor. I could see some gatekeeping logic to include: requirement to post on the discussion forum, look at or reply to other student’s posts, time between attempts, and instructor approval.
- Surfacing the Information: I’d like to come up with slick visualizations of all this quantitative and qualitative data. Not only for the instructor to adjust their instruction, but also for the students to see how their responses have changed or what other students are thinking
- Different problem types: Ideally I would have liked a “peer instruction” version of every problem type on edX. As a first attempt, I chose checkboxes for both technical and pragmatic reasons.
- Anonymity in the Discussion Forums: Should students be identified? Should be something pseudorandom — that is, they can be recognized between problems, or a unique identity for every problem. I worry the latter wouldn’t truly be random as people usually have unique writing styles that could be re-identified (see my previous blog post).
- Number of attempts: My code easily generalizes to more than one non-final attempt (in fact it is a setting that the instructor can adjust right now). The question is: would this be useful?
This is my work-in-progress to provide Peer Instruction for the edX platform. It was a lot of fun to hack on the edX code, but I definitely have some gripes about it (perhaps I will post about that in a separate blog post). If you’re interested in seeing the code or trying it out, reach out to me! I’ll see what I can do. This project certainly isn’t over, but I think it’s an exciting first attempt for me at bringing Peer Instruction into a MOOC.