Moodle performance testing: how much more horsepower do each new versions of Moodle require?

Average amount of RAM used for specific activities

Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the server(s) it runs on. I have been asking myself for a while: “Just how much more horsepower do each new versions of Moodle require?” October being Oktobertest for me, I thought I’d investigate further and run some tests of my own.

The results

It is important to note that while I tried my best to compare apples with apples, this test is not perfect and I am sure it could be improved. If you are interested in how I performed the tests, or to judge to what extent these findings are reliable, please scroll all the way down to ‘My Workflow’. One important point to note is that I assume that the ‘Performance information‘ provided by Moodle is accurate.

Throughout the graphs, shorter bars mean better results. You can click on the graphs to reveal full size images.

 

Each new version of Moodle uses more RAM

Average amount of RAM to generate a page

This one is hardly a surprise, but I was shocked to find out that on average Moodle 2.3 requires 3.5 times more RAM than Moodle 1.9 to generate pages – that is a lot! It is also interesting to note that Moodle 2.3 requires 13% more RAM than Moodle 2.2 and 18% more than Moodle 2.1. It will be interesting to see how Moodle 2.4 fares in this test. 

What this means to you: Make sure your Moodle server has plenty of RAM. This also means that unless you are going to run Moodle 2.x with very few concurrent users (a handful), stay away from shared hosting – it simply won’t work well enough, or you’ll end up seeing your account revoked for using too much resources.

 

It takes longer for the server to generate pages

Average time needed to generate a page

On average, it takes 3 to 4 times longer for a Moodle  2.x page to be generated by the server, compared with Moodle 1.9. While a user isn’t likely to notice the difference, it means that Moodle 2.x is more taxing on the hardware. For each Moodle page generated, the server calls files stored on the hard drive, reads/writes database –  this requires CPU and disk input/output.  

What this means to you: you will reach the limits of your hardware more quickly with Moodle 2.x. 

 

An increasing number of files are needed to generate Moodle pages

Average number of files included to generate a page

Under the hood, Moodle’s code is mainly a collection of PHP files saved in folders. With each iteration of Moodle, an increasing number of those files need to be fetched by the server for each page generated. Moodle 2.3 requires the server to fetch 3 times more files than for Moodle 1.9.

What this means to you: using a PHP accelerator for Moodle 1.9 was a recommendation, I would say that for Moodle 2.x it is a requirement. Using a PHP accelerator will save PHP files in your server’s RAM. This is important as RAM is much faster than disk access, meaning your pages will be generated more quickly.

 

Moodle 2.x is more taxing on the CPU

Average CPU load

Warning: this metric is to be taken with a pinch of salt. While I ensured as much as possible that nothing else but Moodle was running while doing the tests, it is possible that there was some background tasks happening on the server during specific tests. I decided to include this graph as I believe fluctuations would be diluted in the amount of server requests that I made.

Moodle 2.3 is almost 60% more demanding on the CPU than Moodle 1.9. Interestingly enough it seems as though Moodle 2.2 was even more demanding than Moodle 2.3, at least in this run of the test.

What this means to you: if you are running CPU demanding activities (e.g. quiz) and you are already reaching the limits of your hardware on 1.9, seriously consider upgrading your hardware before upgrading Moodle to its newest version.

 

Moodle 2.x makes more calls to the database

Average number of database reads

Average number of database writes

This was hinted in one of the points above, here is the proof. On average, Moodle 2.3 makes over twice as many calls to the database as Moodle 1.9 (reads). It is interesting to note that Moodle 2.2 made a lot more calls than Moodle 2.1 or 2.3 during my tests, seemingly due to the ‘forum’ pages. I would need to investigate further to find the cause of the problem, but it is outside the scope of this exercise.

Note that there is no ‘database writes’ data for Moodle 1.9 as it is not provided in the performance information. 

What this means to you: if you run a large Moodle site, you might want to run your database on its own server. If not, you might want to double-check that your database isn’t a bottleneck in your installation.

 

Some activities are much more demanding than others

 

Average amount of RAM used for specific activities

Moodle HQ has always made this known, some activities are much more resource hungry than others. I must admit I was expecting the quiz to be much more demanding than that. As it turns out the ‘Choice’ is a more demanding activity than the quiz.

What this means to you: if you have an ‘underpowered’ server, you might want to steer clear of some activities e.g. quiz, chat

 

Conclusions

  • Moodle 2.x is much more taxing on the hardware than Moodle 1.9 (but so much better in so many ways…)
  • If you are currently running Moodle 1.9 and are going to upgrade to Moodle 2.x, make sure you do plenty of load testing first
  • Have plenty of RAM in your server
  • Use fast disks in your Moodle server
  • Use a PHP accelerator
  • Place your database inside its own server if you run a large site
  • Follow the performance recommendations, they’re awesome
  • If you can afford it, please update to at least Moodle 2.3, your users will thank you for it

 

My Workflow

 

Server

 

Specs

  • VPS 6 cores at 2.4GHZ
  • 2GB of RAM
  • Ubuntu server 10.04LTS 
  • For those who love numbers, you can check the UnixBench results for this server here.

 

Webserver

  • Apache with the following modules loaded
    • core_module (static)
    • log_config_module (static)
    • logio_module (static)
    • mpm_prefork_module (static)
    • http_module (static)
    • so_module (static)
    • alias_module (shared)
    • auth_basic_module (shared)
    • authn_file_module (shared)
    • authz_default_module (shared)
    • authz_groupfile_module (shared)
    • authz_host_module (shared)
    • authz_user_module (shared)
    • autoindex_module (shared)
    • cgi_module (shared)
    • deflate_module (shared)
    • dir_module (shared)
    • env_module (shared)
    • expires_module (shared)
    • headers_module (shared)
    • mime_module (shared)
    • security2_module (shared)
    • negotiation_module (shared)
    • php5_module (shared)
    • reqtimeout_module (shared)
    • rewrite_module (shared)
    • setenvif_module (shared)
    • ssl_module (shared)
    • status_module (shared)
    • unique_id_module (shared)
  • Prefork
  • MySQL database
  • APC
  • Memcached

  

Moodle

 

Versions tested

I installed 4 versions of Moodle on the same server.

  • Moodle 1.9.19+
  • Moodle 2.1.8
  • Moodle 2.2.5
  • Moodle 2.3.1+

All versions of Moodle contain the same amount of courses and users. I did not install any third-party plugins and they all use the ‘Standard’ theme. I added a logo by modifying some of the core code. Authentication is set to ‘Manual accounts’ and sessions are saved on the disk.

 

Course

I created a ‘Performance Testing’ course with a selection of activities/resources. I used exactly the same course on all 4 different versions of Moodle, including blocks and filters. As mentioned above, the same users were enrolled on all 4 versions. Basically, everything was exactly the same on every version. 

Moodle performance testing course

Should you want to try and test your server as well, you can download the courses I used below:

You will also need the users for those courses, download this CSV file to batch upload 50 test users to your Moodle installation. 

 

Test strategy

 

Macros

I created 4 macros using iMacros, which mimic a single user performing the following:

  • Login > View course > View 500KB resource > Logout
  • Login > View course > View directory > Logout
  • Login > View course > Post to forum > Sends choice > Logout
  • Login > View course > Take a 7 question quiz > Logout

I ran each macro one after another 50 times for each version. Altogether, this created over 6,500 Moodle pages. Nothing else was running on the server at the time of testing.

You can download the macros I used here. You will need to change a few things for it to work for you, namely:

  • http://www.somemoodlesite.com/versionxx/ to reflect your actual Moodle installation
  • view.php?id=xxxx to be changed to your course/activities ID’s
  • responses in the macros that deal with quiz. You might want to record a macro using iMacros running the quiz and then edit the macro to check the actual values for your site

 

Recording data

It is possible for a Moodle administrator to show performance data on each page footer. I setup iMacros to scrape that data and save it in a CSV file for every page it visited. 

 

Analysing data

Rather than overcomplicating my iMacros scripts, I decided to manipulate the data post collection using TextWrangler fantastic search & replace multiple files to clean up my data so that it could be analysed in Excel. I then create pivot tables and generated graphs from those.

You can download the raw data here if you would like to play with it. I recorded how long it actually took for the pages to fully load on my computer while running the tests. I decided against mentioning this in the blog post as the metric is subject to be influenced by too many factors (server and computer were not in the same network).

 

Please use the comments sections for questions regarding this test or share similar tests you may have run.

  1. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | tipsmoodle | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  2. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | moodle2 | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  3. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | WEBOLUTION! | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  4. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | mOOdle_ation[s] | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  5. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Moodle News | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  6. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | urgences langues étrangères application TICE | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  7. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Let's Learn IT: Moodle@School | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  8. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Formation au numérique & elearning dans l'Education | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  9. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Moodle Best LMS | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the server(s) it runs on. I have been asking myself for a while: “Just how much more horsepower do each new versions of Moodle require?  [...]

  10. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Moodling | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the … (RT @moodleman: Interested to see how #Moodle's performance stands?  [...]

  11. How to load test your Moodle server using iMacros | I Teach With Moodle | Sharing good practice using Moodle in and out of the classroom

    [...] might also be interested in reading this blog post, where you can find real world examples of iMacros macros [...]

  12. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | E-Learning and Online Teaching | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  13. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | E-Learning and Online Teaching Today

    [...] on http://www.iteachwithmoodle.com Share this:TwitterFacebookPinterestLinkedInTumblrStumbleUponDiggRedditEmailPrintLike this:LikeBe [...]

  14. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | About Moodle | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  15. Moodle 2x and Ram usage test data « Mootmybeer

    [...] moodle 2 and ram usage stats [...]

  16. Moodle performance testing: how much more horsepower do each new versions of Moodle require? | Aprendiendo a Distancia | Scoop.it

    [...] Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the …  [...]

  17. Moodle 2.4 Beta performance test – comparison with Moodle 2.3 | I Teach With Moodle | Sharing good practice using Moodle in and out of the classroom

    [...] 2.4 Beta performance test – comparison with Moodle 2.3 Last month I wrote a blog post comparing different versions of Moodle and the demands they each place on hardware. With Moodle 2.4 [...]

  18. Frankie Kam

    Hi Frederic. Thanks for the Octoberfe…I mean, Oktobertest. Very enlightening. I’m still stuck on Moodle 1.9 due to the sheer number of hacks to my site’s core code, which is more than I have had the discipline to record. Well, I sure am missing out to a ton of features with Moodle 2.x, especially with the impending Moodle 2.4 release. Somehow I think that I can live with it for sometime now, given the interesting comparison graphs that you have so generously shared with us all. For the Moodle 2.4 user with superfast hardware specs, good on ya mate! For others like me, I’m just chugging along fine with the venerable M1.9 pedigree.

  19. John Holmes

    Frederick,
    I’d love to see ver 2.4 tests added to these results. I’m trying to decide if the performance improvement will justify an update from 2.3.3 to 2.4

  20. Moodle 2.4 Beta performance test – comparison with Moodle 2.3 | Moodle בחינוך – יומן מסע טכני

    [...] And also some previous tests, comparing Moodle 2.X to Moodle 1.9.X [...]

Respond