Moodle 2.4 Beta performance test

[pulledquote]Last month I wrote a blog post comparing different versions of Moodle and the demands they each place on hardware. With Moodle 2.4 release just around the corner, I thought I would perform the same series of tests – here are the results.[/pulledquote]

Warning: these tests are not scientific by any standards but I am comparing apples with apples, please read my previous blog post to check my methodology – I did not change anything on my server/Moodle site since my last round of tests. All figures shown below are averages.

Warning 2: Moodle 2.4 is still in its beta stage, I will run the same test once the official Moodle 2.4 stable has been released.

 

 

RAM to generate a page -8.9%

 

Average amount of RAM to generate a Moodle 2.4 page

 

On average, I found that Moodle 2.4 uses 8.9% less RAM to generate pages than Moodle 2.3. This is great news considering that, for a large amount of Moodle servers, is the first thing that needs upgraded when usage increases. Moodle is very ‘RAM hungry’.

 

Time to generate a page -9.1%

 

 

 

On average Moodle 2.4 is 9.1% faster at generating pages than Moodle 2.3. Might not matter much if you only have a few users, but it starts adding up if you have hundreds/thousands of simultaneous users. The less time your CPU/disk spend generating a page, the faster they can move on to their next task.

 

Number of files to generate a page -9.1%

 

Average number of files needed to generate a Moodle 2.4 page

 

On average, it takes 25 fewer files to generate a Moodle 2.4 page compared with Moodle 2.3. This equates to a 9.1% improvement over Moodle 2.3. Each file needs to be collected from the disk before it can be processed by the CPU to generate a page, so the less files to be collected, the better.

 

Average database reads -36.6%

 

Average number of database calls to generate a Moodle 2.4 page

 

This is the big winner and I believe is directly linked to the introduction of the Moodle Universal Cache (MUC from then on) in Moodle 2.4. On average, Moodle 2.4 makes almost 37% fewer calls to the database to generate Moodle pages compared with Moodle 2.3 (39 calls vs. 57). This is huge considering MySQL (if that’s what you use as your database) is often the bottleneck in large-ish Moodle servers. This will have a massive impact when you have a lot of simultaneous users on your Moodle site, less database calls = more simultaneous users who can use Moodle on your current hardware. This is quite impressive considering it is only the first iteration of the MUC.

 

Average CPU load -6.5%

Warning: this is to be taken with a grain of salt, see my previous blog post for reasons why.

 

Average CPU load using Moodle 2.4

 

On average, Moodle 2.4 uses 6.5% less CPU than Moodle 2.3. From experience, CPU is rarely the first thing that needs upgrading on a Moodle server but it’s nice to see an improvement in this area. 

 

Conclusions

We’re still a far cry from the lean-mean Moodle 1.9 but it is great to see that Moodle HQ have managed to keep adding features whilst reducing Moodle’s footprint on hardware. One cannot compare Moodle 2.x with Moodle 1.9.x – there have been too many features added, improvements made for it to be a fair comparison. Moodle relies heavily on calls to the database so the less calls, the better. With a 37% improvement, this alone should prompt you to update to Moodle 2.4 when it comes out.

It is important to note that my server already uses caching techniques that the new Moodle Universal Cache cannot yet leverage (e.g. APC). I fully expect MUC to mature and reduce Moodle’s footprint even further. This is a step in the right direction. Gains brought by the MUC will be more obvious when multiple users are logged on, I will try and test it out soon.

I have another round of tests coming up where we’ll be able to see the difference on a server that does not yet have APC enabled. Stay tuned.

 

Have you run any Moodle performance tests lately? If you have, please don’t be shy and share in the comments section below.

Using Munin to monitor Moodle server

[pulledquote]As part of Oktobertest I am going to share 5 free tools you can use to monitor your Moodle server. The focus of this post is on load testing but these tools can be (and some should be) used as part of your normal monitoring.[/pulledquote]

Note: this post is aimed towards admins who run their Moodle installation on a Ubuntu Linux server. As usual, this is not meant to be a definitive list or how-to, but rather an introduction to what a Moodle administrator can do to ensure their installation is running smoothly

 

 

Monitoring tools

The following tools are listed in increasing order of difficulty of use. I have used them all but I have found that unless I load test, the first 3 tools tend to be enough for me. I ran Zabbix for a while but it requires quite a bit of TLC, which I simply don’t have time for – great tool though. If I had to keep only one tool it would be htop.

 


Moodle server monitoring tool 1: top 

 

Moodle monitoring tool - top

 

What it does

  • By default, the top command lists all processes in decreasing order of amount of CPU they use
  • The top command also shows your memory (RAM) status and your overall load average (CPU usage)
  • Pressing the M key (capital letter) will sort the processes by amount of RAM used (decreasing)
  • All figures are updated in real time, so I usually have it running in a window whenever I load test to see how the server performs under specific loads

 

Installation

  • Top should come packaged up with your Linux server, you shouldn’t need to install anything for it to work.
  • All you need to do to start it up is type the following command in a terminal window:
sudo top

 

If you would like to find out more about all of the options the top command has to offer, read this excellent article.

 


Moodle server monitoring tool 2: atop

  

Using atop to monitor Moodle server

 

What it does

  • atop is very similar to top but gives you access to more data (network, disk, etc.)
  • atop shows usage summaries by all processes e.g. you have access to a summary of all memory, network, CPU (per core) usage

 

Installation

  • If you want to install atop from a package, simply type
sudo apt-get install atop
  • All you need to do to start it up is type the following command in a terminal window:
sudo atop

 

If you would like to find out more about all of the options the atop command has to offer, read this excellent article.

 


Moodle server monitoring tool 3: htop

 

Using htop to monitor Moodle server

  

What it does

  • htop is very similar to top but allows you to sort data more easily
  • htop also shows CPU & memory usage in a more user-friendly way

 

Installation

  • If you want to install htop from a package, simply type
sudo apt-get install htop
  • All you need to do to start it up is type the following command in a terminal window:
sudo htop

 

If you would like to find out more about all of the options the htop command has to offer, read this excellent article.

 


Moodle server monitoring tool 4: Munin

 
Using Munin to monitor Moodle server

 

What it does

  • Munin presents your server data in easy to read graphs
  • Munin can monitor pretty much everything on your server, from CPU usage to MySQL slow queries, etc.
  • Munin data is saved in a flat file and historical data can be accessed in the graphs
  • Good support community

 

Installation

Rather than re-write an install guide for Munin, I have compiled all of the resources I used to get Munin up and running smoothly on my VPS

You might run into some issues (I did) – here is where you can get an easy fix

  

You can see a screenshot of whole Munin report page here (800KB) – possibly the World’s longest screenshot 🙂

 


Moodle server monitoring tool 5: Zabbix

Monitor your Moodle server with Zabbix - IO wait

 

What it does

  • Zabbix is similar to Munin – it displays server vitals in graphs
  • Zabbix also has the ability to send you email alerts should your server reach critical values e.g. disk usage > 80%
  • You have the ability to create your own alerts, with custom thresholds
  • Zabbix is a full fledged server monitoring solution – you can monitor multiple servers
  • Zabbix is best run on its own server as it is MySQL intensive (every piece of data is stored in a database)

 

Installation

 

Monitor your Moodle server with Zabbix - CPU

Monitor your Moodle server with Zabbix - bandwidth

 

 

iMacros Moodle Logos

 

 

 

 

 

This post is part of the Oktobertest series. It is not meant to be a definitive guide on how to use iMacros. Rather it is a quick introduction to get Moodle administrators started with iMacros.

iMacros records your web browsing activity so that you can later simulate the actions of a real Moodle user, all automatically. For example, you start up Firefox and set iMacros to record. You go about your daily business on Moodle as normal (e.g. login, view your course, add an assignment, answer some forum posts and logout), iMacros keeps a record of everything you click on during your session, including all the forms that you fill in until you press ‘Stop recording’. Those steps are saved in a macro, which you can then play back later by opening that macro. This means that you can press the ‘play’ button and Firefox will repeat all of the steps that you did during your session, without any more interactions from your part, thus automatically simulating the web browsing activities of a Moodle user. You can ‘loop’ the action to repeat it automatically as many times as you like.

Before you start following this tutorial, you should download the following:

 You might also be interested in reading this blog post, where you can find real world examples of iMacros macros (!).

 

Step 1: Open iMacros

 
  1. Click on the iMacros icon
  2. Open the webpage where you want to start your test from
  3. Click on the ‘Rec’ icon

 

 

 

Step 2: Start recording your macro

 
  1. Click the ‘Record’ button
  2. As you navigate your Moodle site, you will see that lines of text are added to this box, this is normal

Note: 3. Make sure that the ‘Click mode’ is set to ‘Auto’

 

 

 

Step 3: Stop recording when you’re happy

 
  1. Click ‘Stop’ when you have finished navigating your Moodle site
  2. The macro you have just recorded is called ‘current.iim’. I strongly advise you to rename it to something else, as every time you click the ‘record’ button it overwrites the ‘current.iim’ file (simply right-click on the file name to rename it)

 

 

 

Step 4: Try it

 

Try and play your macro just to make sure it works

  1. Click on the ‘Play’ tab
  2. Click on the ‘Play’ button (make sure the correct macro file is selected)
  3. Sit back and relax

If you want to loop your macro, take a look at this webpage.

 

 

 

Step 5: Load test

 

The macro you recorded will only mimic one user when you play it – hardly useful for load testing. To mimic multiple users at once, you will need to run multiple instances of the macro at the same time. Note that using different tabs will not work, you have to open each new instance of Firefox in its own window. (Ctrl N on Windows/Linux, or Cmd N on Mac). The only limitation is the amount of instances your computer can cope with in terms of CPU/RAM and bandwidth.

It is also possible to run the macro from command line or task scheduler. Procedures differ depending on the platform you use, so instead of me re-inventing the wheel, please check the following links:

 

JMeter and Moodle logos

 

 

 

 

 

Note: this blog post is part of the Oktobertest series

This tutorial is aimed at Moodle administrators who have never used JMeter. It will help you get started but won’t teach you advanced techniques. Should you wish to dig further into this wonderful piece of open-source software, I invite you to read the following:

Before you start this tutorial, you need to download the following:

 

It is possible that you’ll get an error if you’re trying to run this in Moodle 2.2+. All you need to do is open the ‘index.php’ file (for the ‘loadtesting’ report) and search-replace all instances of the entire word ‘mod’ (no quotes) and replace it with ‘module’. This should solve your problem.

 

 

Step 1: Generate your JMeter script

  • Settings > Site Administration > Reports > JMeter loadtesting
  • Use the drop-down box to select the correct category
  • Click ‘Select category’

Note: you must be logged in as an administrator to do this

 

 

 

Most of the options above are self-explanatory, please note the following:

1. This can be a bit confusing. You should see each activity type as a different test. For example, if 10 is entered in this box, then a total of 40 virtual users would be running simultaneously (10 for chat, 10 for forum, 10 for glossary and 10 for quiz)
2. I always tick this, to put as much strain on the disk as possible, and to mimic real world usage as closely as possible
3 & 4. I like to pick and choose activities. The main reason being that students cannot post to the News forums, and also to ensure that the activities I picked are not restricted by time, or by condition (conditional activities), or even available only to particular groupings.
5. I usually tick this option, again to mimic real work usage as closely as possible
6. I like to set this to 10, for no particular good reason, I tend to like even numbers
7. Once you click the button, your Moodle server will generate a zip file, so make sure you have a zipping program installed. It will look like nothing is happening, simply go and check your download folder to see if jmeter.zip is there.

 

 

Step 2: Start JMeter

 

Step 2: Start JMeter

 
  • To open JMeter, you have to visit the JMeter folder you downloaded
  • JMeter > bin > ApacheJMeter.jar
  • Double-click on the ApacheJMeter.jar file

 

 

Step 3: Open the newly generated script

 

 
  • Select the correct file
  • Click ‘Open’

 

 

Step 4: You can now run your script

 

Step 4: You can now run your script!

 
  • Click the play button
  • Look for the numbers at the top right of the window reach the maximum set, and then go back to zero
  • Once your script has looped through the amount defined in your settings, the load test is over

Note: Check out this blog post to find out what tools you could use to monitor your server while load testing

 

 

Step 5: Check the results

 

Step 5: Check the results

 
  • There are several types of reports you can take a look at (see rectangle in picture above). 
  • Not all reports are super user-friendly and might not make a great deal of sense. Further reading on the matter:

See below for more report examples

 

 

 

 

 

 

 

 

Tip 1: Create a ‘Load testing’ category with courses dedicated to load-testing

 

  • If you are planning on enrolling lots of test users to your Moodle courses, you should know that it could have an impact on the performance of the courses involved. 
  • If you create a specific category with dedicated courses, you do not run this risk as the courses are only used once in a while. 
  • You should also reset your courses once in a while

 

 

Tip 2: Generate your script with lots of users

 

The first time you generate a script chances are you’ll have no idea how many users your installation can cope with.
Part of load testing is to find out the limits of your server. Consider this scenario:

  • Generate a script for 50 users
  • Get the script to generate users and enroll them in the course(s)
  • Run the test
  • If that test runs smoothly, you’ll have to create a new script with a new value every time you want to increase the number of users.

Consider this (better) scenario

  • Generate a script for 1,000 users (or large number of users)
  • Get the script to generate users and enroll them in the course(s)
  • Run the test with 50 users (edit the ‘Number of threads’ value)
  • If the test runs smoothly, all you need to do is edit the ‘Number of threads’ value and run the test again (the users are already enrolled)

 

 

Tip 3: Generate your script with lots of activities, and all activity types

 

  • It is possible for you to easily disable activities right in the script after it has been generated.
  • You might want to test the same course under different loads e.g. once with just a forum, once with a forum & a quiz, once with a forum, a quiz and a chat, etc.
  • By creating a test with multiple activities, you can easily re-use the same test by simply disabling/enabling the activities you want to test/not test
  • To disable an activity, you need to right-click on the activity name and then select ‘disable’.
  • All disabled activities show up in grey

 

 

Tip 4: Edit the ramp-up period

 

  • By default, the script generator sets the same value for the ‘Ramp up period’ as the number of users you selected when creating the script. 
  • The ramp-up period determines how long it takes for all virtual users (threads) to start. 
  • For example. if set to 0, all users would start at the same time. If set to 10 (as above), it takes 10 seconds to start all users (1 per second)
  • You can play with different values and monitor how your server reacts to the changes

 

 

Tip 5: Disable some of the reports

 

 

If you are running tests with a lot of virtual users and multiple activities, you might get ‘out of memory’ errors on the computer you are running JMeter. To avoid this:

  • Right-click on a specific report
  • Select ‘Disable’
  • This will not delete the report and you can decide to enable it again later