TEA conference presentation sum-up: Scalable peer-feedback design in MOOCs?

TEA conference presentation sum-up: Scalable peer-feedback design in MOOCs?

On Thursday 5 October I got the chance to present our pilot study about scalable peer-feedback design at the TEA conference (International Technology Enhanced Assessment Conference) in Barcelona. In this post, I want to give you a short overview of our study.

Incentive of the study
The motivation behind our study was to investigate whether and how embedded, student-focused peer-feedback design has an impact on student perception regarding peer-feedback. In the literature, peer-feedback is seen as a scalable way to manage large student numbers. Scalability often is approached by a quantitative perspective: “A MOOC is scalable because it provides education to large student numbers.” NO! It would be a shame to pursue a purely quantitative goal in education. What we want is high quality at large scale. That’s why we introduced the term ‘Educational scalability’ which is the capacity of an educational format to maintain high quality despite increasing or large numbers of learners at a stable level of total costs (Kasch, Van Rosmalen, Kalz, 2017, submitted).

Large-scale participation and a big heterogeneity in MOOC students make it impossible for teachers to provide personalized and elaborated feedback on complex learning activities. Think of essays, artworks or design projects which require more than simple correct/incorrect feedback. Although automated feedback systems are getting more and more advanced they are not yet that far to be used in practice on complex learning activities. By implementing peer-feedback teachers and course designers hope to make large-scale participation manageable. However, without a clear, transparent and student-focused design, students will be soon dissatisfied with the entire peer-feedback perception. The literature shows some reasons for student dissatisfaction: unclear design, low quality of the received peer-feedback and a preference for teacher feedback. Any educational design should be based on both qualitative as well as quantitative perspectives. Thus, peer-feedback should be designed as a skill students have to acquire (a learning goal) and an efficient way to reflect on their own and peer’s work.

Sneak Peak Pilot Study
To investigate whether and to what extent peer-feedback training influences student perception regarding peer-feedback we did the following:

First of all, we investigated student’s attitude about peer-feedback and peer-feedback training via a pre-questionnaire. The questionnaire investigated four components of student’s attitude:
(1) their willingness to participate in peer-feedback (training)
(2) their perceived usefulness of peer-feedback (training) in the past
(3) the level of preparedness they felt to provide peer-feedback
(4) their general attitude about peer-feedback (training) in MOOCs

The same components were measured after students did participate in the peer-feedback training and activity to investigate whether and how their attitude had changed.

Preliminary results showed that students’ perception increased in all four aspects (willingness, usefulness, preparedness and general attitude) after having participated in our peer-feedback training & activity. Due to the promising results, we will further investigate the effects of our peer-feedback design on a bigger scale. That way we will not only increase student satisfaction regarding peer-feedback in MOOCs but also will be able to provide design principles that will provide valuable insights to MOOC teachers and designers. More detailed information will be shared after our paper is officially published.

If you are interested in our study you can follow me on twitter @julia_kasch or keep an eye on our website!

Julia, PhD student

CC BY 4.0 This work is licensed under a Creative Commons Attribution 4.0 International License.

Leave a Reply

Your email address will not be published. Required fields are marked *