Gamifying a Moodle course. What difference does it make? Week 1

[pulledquote]I announced a few weeks back that I would try and quantify the increase in engagement I have noticed when gamifying my Moodle courses. I have just started teaching two similar groups, and one has been exposed to the regular course, and the other to a gamified version of it.[/pulledquote]

I will post my findings every week, and will draw a summary at the end of the experiment. This post retraces what has happened in the first week of the experiment.

 

Note: Don’t expect a huge study, this is an experiment involving 2 groups of just over 23 students, aged 12 to 13 years old, over an 11 week period. Please keep this in mind when reading my interpretation of the results, and I would recommend you to think twice before quoting it in an essay 😉

Methodology

  • Course was duplicated twice
  • One course per teaching group
  • One group is exposed to the gamified course, the other is exposed to the regular course
  • The resources/activities are exactly the same, are presented in the same order and are available at the same time on both courses
  • The look & feel of both courses is exactly the same
  • In short, both courses are the same but one course is gamified (i.e. students can collect badges when completing activities), and the other isn’t
Gamified courseRegular course
Gamified Moodle courseRegular Moodle course
Full size imageFull size image

Course on the left has badges for students to unlock, course on the right doesn’t

Context

The two teaching groups are relatively similar in terms of gender, age, and academic achievement distribution. I currently work at a non-selective/DSS secondary school in Hong Kong, where most of the 720 students’ first language is not English. All of the students concerned by this experiment have access to a computer at home, connected to high-speed Internet. The experiment is carried out on Form 1 students, which equate to Year 7 in the UK system, or 7th grade in the US system.

Group 1 – Gamified courseGroup 2 – Regular course

23 students (14 boys, 9 girls)

24 students (14 boys, 10 girls)

Findings from week 1 

I am trying to record as much information as possible and, although I have a pretty good idea of what I want to find out, the list of metrics I am analysing is growing organically.  Here are some questions I have asked myself so far, some with early answers.

Do some students cheat?

The age old question! In order to unlock badges, students need to complete activities and view resources. Some of those activities/resources are marked as complete automatically (e.g. when a student views a webpage, a tick is automatically placed next to that resource), or the student needs to mark it as complete themselves. Students can then cheat, ticking an activity as ‘complete’ even though they didn’t even open the activity/resource in question.

Answer: In the first week, 5 students from each group marked activities done, even though they had not even visited the said activities/resources. So yes, some students cheat, whether the course is gamified or not. 

TODO: I need to have a look at the length of time between each click. This leads to a question for the future –> do some students go on a ‘clicking spree’ to mark as many activities complete as possible?

Are some students forgetful?

If some students mark an activity as complete, even though they haven’t completed it, then the reverse must be true. I queried the logs to find out.

Answer:  in group 1, 18 students viewed at least 1 resource and did not mark it as complete. In group 2, 20 students viewed at least 1 resource and did not mark it as complete. This is a lot more than I thought, and again there seems to be little difference as to whether a course is gamified or not.

TODO: I clearly explained to my students that marking an activity as done implies that not only they have viewed it, but also understood it. I need to find out whether students did not mark it done as one of those criteria wasn’t met, or if they simply forgot to tick it as done.

Do students spend more time on the gamified course?

I love statistics so there will be a detailed analysis at the end of the 11 weeks, along with the whole data set in case you want to perform your own analysis. For now, here is a quick breakdown of what happened in week 1, using the ‘Course dedication‘ block.

Time spent on site (hh:mm)Group 1Group 2
Total54:33  44:15
Average02:1601:51
Median02:1301:54
Maximum05:2302:50
Minimum00:2800:00

Answer: as you can see, students who have been exposed to the gamified course have spent longer on the Moodle course than the students exposed to the regular course (+23% on average). It will be interesting to see how this evolves over time.

TODO: I want to have a better overview of what the students are actually doing on the site. Time to find a way to analyse the logs in a meaningful way. 

Do students click on more resources/activities on the gamified course?

In light of the previous answer, the answer to this one might sound obvious but I went to check anyway. I simply looked at the Moodle logs and counted the number of clicks in the course, minus my own and ensured the timeframe was correct.

Number of clicksGroup 1Group 2
Total1,7221,349
Average per student74.956.2
Median7456
Maximum149115
Minimum917

Answer: yes they do, quite a bit more in fact. On average, a student exposed to the gamified course performed 33% more clicks than their counterparts exposed to the regular course.

TODO: again, it would be quite interesting to see what students are doing, more in depth. Also, I’d like to see if this levels off over the next few weeks.  

There is probably a lot more to look at, so please write a comment below if there is anything you’d like to find out before this experiment is over, for example any specific data analysis. If you have any questions for my students, I am happy to ask. 

32 comments

  1. I’m looking forward to reading the next delivery of this amazing experiment. After upgrading to moodle2 I’m ready to perform a similar experiment with my 14-16 years old pupils. Thanks for sharing!

    Toni Soto

    1. Hey Toni, thanks for your comment. I’m looking forward to reading about your findings, please make sure you give us a shout when you’re running your experiment 😉
      Cheers,
      Fred

  2. Fascinating read Fred.

    I would be very interested to see the make up of the course in a slightly bigger picture? What Moodle features are you using and what feature/plugin are you using by way of the badges?

    It must have been quite a challenge to ensure a level playing field and not bias the gameified course in some way eg offering extra incentives to stay on a task such as increasing the badge count/quality if you do a quiz 5 times instead of 3.

    On first reading the obvious difference between the course is the reward of the badge? Are the non-gamified students receiving any traditional rewards.

    Hope that the study progresses well as it may well prove valuable for all of us!

    1. Hi Gideon,

      thank you for your comment. I have added links to the full size screenshots of the courses in their entirety.

      There is a bit more information about how I ‘gamified’ my course, in a previous blog post http://www.iteachwithmoodle.com/2013/01/20/increase-student-engagement-with-moodle-conditional-activities/
      I have sent a presentation proposal for iMoot 2013 to show other teachers how I have done it, fingers crossed it gets accepted. If not, I’ll create a how-to video and place it on Youtube.

      Both sets of students get traditional rewards, in about the same amount. For the gamified course, badges get transformed into ‘house points’ (the school’s reward system) and for the non-gramified course, I encourage students to participate and reward accordingly. I’ll write more on this in the coming weeks.

      Cheers,
      Fred

  3. Fred

    Just noticed that you had some boxes BLUE and some GREEN depending on whether students tick off tasks or are marked automatically. That is very clever!

    Please tell me how it was done… :O)

    1. Hi Gideon,

      I believe this is rendered by the theme (Aardvark Postit) and depends on the option you select in ‘Activity completion > Activity completion’ for each activity/resource, if you select ‘Manually’ it will show up in blue, and if you select ‘When conditions are met’ then it shows up green. In the ‘Standard’ Moodle theme, there are no colours, instead you get different kinds of checkboxes. So this has nothing to do with me 😉
      Cheers,
      Fred

  4. Hi there

    What badge system did you use? The only way I know how to create badges natively in Moodle is via groups and the badges only show up in forums.

    Thanks for the great post!

    KerryJ
    Online Learning Specialist
    Australian Institute of Social Relations

  5. Frederic
    Great Study. Thanks for posting.
    Re Cheating why don’t you remove the students ability to Check an activity has been completed and instead choose “Student must complete this activity to complete it”. For Quizzes and Assignments this even requires a Grade from the teacher. This equally applies to the Forgetful students. This even works with Book where a student has to at least click on every page and wait until it finishes uploading before progressing. If they progress faster than upload or miss a page Moodle says they didn’t do this activity.
    I would be also interested to know if the Gaming students did any better academically? I have pulled out all the bells and whistles I could find to motivate students and while the vibe has usually been better this does not always translate to improved grades.
    Paul

    1. Hi Paul,
      thanks for your message. I will look into that.
      I will post the full results to the study when I find the time. I have started a new job and it’s taking up all of my ‘free’ time at the moment. Stay tuned.
      Cheers,
      Fred

  6. Hi. I have a question and have asked the people in MoodleRooms and they don’t know. Is there a way to attach a link to badges? Say, if a student received a badge and hovered over it, could it connect them to our school linkedin site?
    Thanks

Leave a Reply

Your email address will not be published.