Originally published in the ACM XRDS blog.
Gamification of University-level courses is becoming a common practice, as many professors decide to try offering their students a more engaging learning environment. Nevertheless, we still do not have a clear idea on how individual students engage differently with a gamified course. But now a detailed, long-term study from the University of Lisbon has presented some insightful observations on this topic.
During the course of their study, the researchers observed three editions of a gamified University of Lisbon course on Multimedia Content Production. The course employed a blended learning method that combined theoretical lectures, lab classes, and an online Moodle component where students engaged in discussions and completed online assignments.
Throughout the years, the researchers have learned from the experience and improved the course’s gameful design. A general observation from the student’s feedback is that they all felt the gamified course was indeed more engaging than the previous non-gamified editions. However, there were some noticeable differences on how individual students engaged with the course, which the researchers sought to investigate.
The research report is a very rich publication, with detailed information on the evolution of the course’s design, results from the collected data, and insights from the student’s evaluations. It is certainly worth the read for more detailed information! Nevertheless, I will summarise some of the most important findings.
First, the researchers observed that differences in gameful design led to relevant differences in the student’s interaction with the course. Throughout the years, they were able to adjust the course’s design based on the feedback from the previous edition and the results showed that the design changes effectively improved student participation. On its core, the course was gamified by shaping activities into meaningful challenges and achievements and used game artifacts such as experience points (XP), badges, and leaderboards to communicate progress and feedback. Students progressed in levels (1-20) as they earned XP, and their final grade for the course corresponded to the level they achieved by the end of the course. The gamified elements were particularly useful to increase student’s engagement with the online component of the course, compared to the observations from the previous non-gamified editions, where students were more engaged with in-person lessons than with online activities. In the second year, the amount of XP allocated to achievements was increased to improve engagement with this game element, which was low in the first year, and a dashboard and skill tree were introduced. In the third year, further adjustments to achievements and challenges were made, an instructor evaluation of student’s post was introduced (to value the quality of posts more than the quantity, which was an issue in the second year), the skill tree was improved, and an avatar world and a collaborative treasure hunt were introduced.
From these iterations, the researchers highlighted the importance of increasing student agency, balancing out competition and collaboration, and providing an untangled gamified experience. Increasing agency had a high impact on student’s motivation, showing the importance of autonomy in learning. It was promoted by distributing the tasks throughout the course, giving students different ways of earning XP, and different ways of recovering lost XP. To balance competition and collaboration, it was necessary to create common goals that required collaborative work (such as the treasure hunt), instead of just passive achievements. Finally, researchers observed that the game needed to be easy to play and with simple, clear rules. Otherwise, complex rules demotivated students to pursue that particular path (like happened with the skill tree in the second year, which was later simplified in the third year, with better results).
Interestingly, the researchers noted that there were clear differences in how students engaged with and performed in the course. To understand these differences, they clustered the students in groups with similar characteristics. Each term, they were able to identify 3-4 clusters: a cluster of top performing students (the Achievers), a cluster of bottom performing students (the Underachievers), and 1-2 clusters of average performing students, with somewhat different characteristics throughout the years. In summary, the student clusters in the last years were:
- Achievers: The top performers and most engaged with the course, they had the largest and steepest XP accumulation, stayed at the top of the leaderboards, completed most of the challenges and achievements, and engaged more often in online discussions.
- Regular students: This was the most common group in the third year. These students also had a good XP accumulation curve, just below the Achievers, and stayed consistently at the top half of the leaderboards.
- Halfhearted students: These students accumulated XP at a lower rate than the Regular students, and performed below them in most aspects of the course.
- Underachievers: The bottom performers and less engaged with the course, these students had the lowest XP accumulation and occupied the bottom of the leaderboards.
The following charts show how these four student clusters differed in terms of leaderboard position and XP accumulation throughout the term. We can clearly see there are visible and well-pronounced differences between the groups.
Average student leaderboard rank in the third year (Barata et al., 2016).
Average accumulated XP per student in the third year (Barata et al., 2016).
This study provides important insights into understanding different student behaviours in gamified education. The clusters should not be taken as absolute categories, as some variation as observed throughout the years, probably as a result of different student characteristics or changes in the course design. However, the researchers observed the consistent existence of a group of Achievers, a group of Underachievers, and one or more groups of average students with slightly different characteristics. These findings contribute to enriching the body of knowledge on gamified education and pose important hypotheses to inform future research and practice in the field.
Original publication: Gabriel Barata, Sandra Gama, Joaquim Jorge, and Daniel Gonçalves. 2016. Studying student differentiation in gamified education: A long-term study. Computers in Human Behavior.