Scalable Course Design in MOOCs?

We all know that the quality of a course be it a MOOC (Massive Open Online Course) or any other (open online) course, highly depends on the quality of the course design. Quality is a highly discussed topic in the literature and I don’t really want to go into it in depth but a very simple way of “measuring” the quality of a course design is by listening to the students the course is made for and the teachers who are teaching and maybe even did design the course. Students’ (dis)satisfaction regarding the design can tell you a lot about the impact certain design choices have on the learning experience. Are the learning goals clear (enough)? Do students feel prepared for the final assignment? How did they experience the support and feedback they received during the course?
Designing a course takes a lot of time, effort and expertise and it remains a difficult task since the design options often seem to be endless. Sometimes the design options are rather limited by the technical possibilities of the platform or the environment.
To get insight into educational design practices in MOOCs and to identify scalable best practices we developed an instrument called ‘Educational Scalability Analysis Instrument’. The instrument is developed to analyse the course design from a qualitative perspective and it can be used by teachers, designers but also the students themselves. It contains 48 items (open, closed and mixed questions) which are divided among five variables:

Instrument Variables:

(1) Background Information and Complexity Levels
(2) Student-Teacher Interaction
(3) Student-Student Interaction
(4) Student-Content Interaction
(5) Final Questions

How and for whom?

The instrument was specifically designed for the analysis of MOOCs but could also be used for other (open) online courses as long as the course contains any form of formative feedback. Since designing a course that enables scalable interaction is a challenge in open online courses with large-scale participating students we put extra focus on the way interaction is designed. Some questions the instrument contains are: “Which roles do the teacher, student or the course environment have in providing feedback or supporting students”.” How sensitive is the role of the teacher or the environment to student numbers?” “How elaborate is the provided feedback?”
When using the instrument to analyse the design you will get qualitative as well as quantitative information about the educational scalability of the design. It will provide you with descriptive information but also a quality and/or scalability score of each design element. For example: Learning goals and activities that focus on skills and ask students to demonstrate their knowledge score higher on the quality level than learning goals and activities that simply acquire students to show factual knowledge.
The instrument is more than a checklist but can support you in designing a course yourself or getting insight into the scalability and quality of a current course. By using the instrument you can check your own design and the instrument will give you some extra insight into how to improve the design.

Want to use the instrument?

We tested and used the instrument in two pilot studies and it will be accessible for everyone after the corresponding paper is published. I will keep you updated!

Julia, Ph.D student

This work is licensed under a Creative Commons Attribution 4.0 International License.