JMeter and Moodle logos

 

 

 

 

 

Note: this blog post is part of the Oktobertest series

This tutorial is aimed at Moodle administrators who have never used JMeter. It will help you get started but won’t teach you advanced techniques. Should you wish to dig further into this wonderful piece of open-source software, I invite you to read the following:

Before you start this tutorial, you need to download the following:

 

It is possible that you’ll get an error if you’re trying to run this in Moodle 2.2+. All you need to do is open the ‘index.php’ file (for the ‘loadtesting’ report) and search-replace all instances of the entire word ‘mod’ (no quotes) and replace it with ‘module’. This should solve your problem.

 

 

Step 1: Generate your JMeter script

  • Settings > Site Administration > Reports > JMeter loadtesting
  • Use the drop-down box to select the correct category
  • Click ‘Select category’

Note: you must be logged in as an administrator to do this

 

 

 

Most of the options above are self-explanatory, please note the following:

1. This can be a bit confusing. You should see each activity type as a different test. For example, if 10 is entered in this box, then a total of 40 virtual users would be running simultaneously (10 for chat, 10 for forum, 10 for glossary and 10 for quiz)
2. I always tick this, to put as much strain on the disk as possible, and to mimic real world usage as closely as possible
3 & 4. I like to pick and choose activities. The main reason being that students cannot post to the News forums, and also to ensure that the activities I picked are not restricted by time, or by condition (conditional activities), or even available only to particular groupings.
5. I usually tick this option, again to mimic real work usage as closely as possible
6. I like to set this to 10, for no particular good reason, I tend to like even numbers
7. Once you click the button, your Moodle server will generate a zip file, so make sure you have a zipping program installed. It will look like nothing is happening, simply go and check your download folder to see if jmeter.zip is there.

 

 

Step 2: Start JMeter

 

Step 2: Start JMeter

 
  • To open JMeter, you have to visit the JMeter folder you downloaded
  • JMeter > bin > ApacheJMeter.jar
  • Double-click on the ApacheJMeter.jar file

 

 

Step 3: Open the newly generated script

 

 
  • Select the correct file
  • Click ‘Open’

 

 

Step 4: You can now run your script

 

Step 4: You can now run your script!

 
  • Click the play button
  • Look for the numbers at the top right of the window reach the maximum set, and then go back to zero
  • Once your script has looped through the amount defined in your settings, the load test is over

Note: Check out this blog post to find out what tools you could use to monitor your server while load testing

 

 

Step 5: Check the results

 

Step 5: Check the results

 
  • There are several types of reports you can take a look at (see rectangle in picture above). 
  • Not all reports are super user-friendly and might not make a great deal of sense. Further reading on the matter:

See below for more report examples

 

 

 

 

 

 

 

 

Tip 1: Create a ‘Load testing’ category with courses dedicated to load-testing

 

  • If you are planning on enrolling lots of test users to your Moodle courses, you should know that it could have an impact on the performance of the courses involved. 
  • If you create a specific category with dedicated courses, you do not run this risk as the courses are only used once in a while. 
  • You should also reset your courses once in a while

 

 

Tip 2: Generate your script with lots of users

 

The first time you generate a script chances are you’ll have no idea how many users your installation can cope with.
Part of load testing is to find out the limits of your server. Consider this scenario:

  • Generate a script for 50 users
  • Get the script to generate users and enroll them in the course(s)
  • Run the test
  • If that test runs smoothly, you’ll have to create a new script with a new value every time you want to increase the number of users.

Consider this (better) scenario

  • Generate a script for 1,000 users (or large number of users)
  • Get the script to generate users and enroll them in the course(s)
  • Run the test with 50 users (edit the ‘Number of threads’ value)
  • If the test runs smoothly, all you need to do is edit the ‘Number of threads’ value and run the test again (the users are already enrolled)

 

 

Tip 3: Generate your script with lots of activities, and all activity types

 

  • It is possible for you to easily disable activities right in the script after it has been generated.
  • You might want to test the same course under different loads e.g. once with just a forum, once with a forum & a quiz, once with a forum, a quiz and a chat, etc.
  • By creating a test with multiple activities, you can easily re-use the same test by simply disabling/enabling the activities you want to test/not test
  • To disable an activity, you need to right-click on the activity name and then select ‘disable’.
  • All disabled activities show up in grey

 

 

Tip 4: Edit the ramp-up period

 

  • By default, the script generator sets the same value for the ‘Ramp up period’ as the number of users you selected when creating the script. 
  • The ramp-up period determines how long it takes for all virtual users (threads) to start. 
  • For example. if set to 0, all users would start at the same time. If set to 10 (as above), it takes 10 seconds to start all users (1 per second)
  • You can play with different values and monitor how your server reacts to the changes

 

 

Tip 5: Disable some of the reports

 

 

If you are running tests with a lot of virtual users and multiple activities, you might get ‘out of memory’ errors on the computer you are running JMeter. To avoid this:

  • Right-click on a specific report
  • Select ‘Disable’
  • This will not delete the report and you can decide to enable it again later
 

Average amount of RAM used for specific activities

Each new version of Moodle allows teachers to create more engaging online courses, and do so more easily as Moodle becomes more user-friendly. It also puts extra strain on the server(s) it runs on. I have been asking myself for a while: “Just how much more horsepower do each new versions of Moodle require?” October being Oktobertest for me, I thought I’d investigate further and run some tests of my own.

The results

It is important to note that while I tried my best to compare apples with apples, this test is not perfect and I am sure it could be improved. If you are interested in how I performed the tests, or to judge to what extent these findings are reliable, please scroll all the way down to ‘My Workflow’. One important point to note is that I assume that the ‘Performance information‘ provided by Moodle is accurate.

Throughout the graphs, shorter bars mean better results. You can click on the graphs to reveal full size images.

 

Each new version of Moodle uses more RAM

Average amount of RAM to generate a page

This one is hardly a surprise, but I was shocked to find out that on average Moodle 2.3 requires 3.5 times more RAM than Moodle 1.9 to generate pages – that is a lot! It is also interesting to note that Moodle 2.3 requires 13% more RAM than Moodle 2.2 and 18% more than Moodle 2.1. It will be interesting to see how Moodle 2.4 fares in this test. 

What this means to you: Make sure your Moodle server has plenty of RAM. This also means that unless you are going to run Moodle 2.x with very few concurrent users (a handful), stay away from shared hosting – it simply won’t work well enough, or you’ll end up seeing your account revoked for using too much resources.

 

It takes longer for the server to generate pages

Average time needed to generate a page

On average, it takes 3 to 4 times longer for a Moodle  2.x page to be generated by the server, compared with Moodle 1.9. While a user isn’t likely to notice the difference, it means that Moodle 2.x is more taxing on the hardware. For each Moodle page generated, the server calls files stored on the hard drive, reads/writes database –  this requires CPU and disk input/output.  

What this means to you: you will reach the limits of your hardware more quickly with Moodle 2.x. 

 

An increasing number of files are needed to generate Moodle pages

Average number of files included to generate a page

Under the hood, Moodle’s code is mainly a collection of PHP files saved in folders. With each iteration of Moodle, an increasing number of those files need to be fetched by the server for each page generated. Moodle 2.3 requires the server to fetch 3 times more files than for Moodle 1.9.

What this means to you: using a PHP accelerator for Moodle 1.9 was a recommendation, I would say that for Moodle 2.x it is a requirement. Using a PHP accelerator will save PHP files in your server’s RAM. This is important as RAM is much faster than disk access, meaning your pages will be generated more quickly.

 

Moodle 2.x is more taxing on the CPU

Average CPU load

Warning: this metric is to be taken with a pinch of salt. While I ensured as much as possible that nothing else but Moodle was running while doing the tests, it is possible that there was some background tasks happening on the server during specific tests. I decided to include this graph as I believe fluctuations would be diluted in the amount of server requests that I made.

Moodle 2.3 is almost 60% more demanding on the CPU than Moodle 1.9. Interestingly enough it seems as though Moodle 2.2 was even more demanding than Moodle 2.3, at least in this run of the test.

What this means to you: if you are running CPU demanding activities (e.g. quiz) and you are already reaching the limits of your hardware on 1.9, seriously consider upgrading your hardware before upgrading Moodle to its newest version.

 

Moodle 2.x makes more calls to the database

Average number of database reads

Average number of database writes

This was hinted in one of the points above, here is the proof. On average, Moodle 2.3 makes over twice as many calls to the database as Moodle 1.9 (reads). It is interesting to note that Moodle 2.2 made a lot more calls than Moodle 2.1 or 2.3 during my tests, seemingly due to the ‘forum’ pages. I would need to investigate further to find the cause of the problem, but it is outside the scope of this exercise.

Note that there is no ‘database writes’ data for Moodle 1.9 as it is not provided in the performance information. 

What this means to you: if you run a large Moodle site, you might want to run your database on its own server. If not, you might want to double-check that your database isn’t a bottleneck in your installation.

 

Some activities are much more demanding than others

 

Average amount of RAM used for specific activities

Moodle HQ has always made this known, some activities are much more resource hungry than others. I must admit I was expecting the quiz to be much more demanding than that. As it turns out the ‘Choice’ is a more demanding activity than the quiz.

What this means to you: if you have an ‘underpowered’ server, you might want to steer clear of some activities e.g. quiz, chat

 

Conclusions

  • Moodle 2.x is much more taxing on the hardware than Moodle 1.9 (but so much better in so many ways…)
  • If you are currently running Moodle 1.9 and are going to upgrade to Moodle 2.x, make sure you do plenty of load testing first
  • Have plenty of RAM in your server
  • Use fast disks in your Moodle server
  • Use a PHP accelerator
  • Place your database inside its own server if you run a large site
  • Follow the performance recommendations, they’re awesome
  • If you can afford it, please update to at least Moodle 2.3, your users will thank you for it

 

My Workflow

 

Server

 

Specs

  • VPS 6 cores at 2.4GHZ
  • 2GB of RAM
  • Ubuntu server 10.04LTS 
  • For those who love numbers, you can check the UnixBench results for this server here.

 

Webserver

  • Apache with the following modules loaded
    • core_module (static)
    • log_config_module (static)
    • logio_module (static)
    • mpm_prefork_module (static)
    • http_module (static)
    • so_module (static)
    • alias_module (shared)
    • auth_basic_module (shared)
    • authn_file_module (shared)
    • authz_default_module (shared)
    • authz_groupfile_module (shared)
    • authz_host_module (shared)
    • authz_user_module (shared)
    • autoindex_module (shared)
    • cgi_module (shared)
    • deflate_module (shared)
    • dir_module (shared)
    • env_module (shared)
    • expires_module (shared)
    • headers_module (shared)
    • mime_module (shared)
    • security2_module (shared)
    • negotiation_module (shared)
    • php5_module (shared)
    • reqtimeout_module (shared)
    • rewrite_module (shared)
    • setenvif_module (shared)
    • ssl_module (shared)
    • status_module (shared)
    • unique_id_module (shared)
  • Prefork
  • MySQL database
  • APC
  • Memcached

  

Moodle

 

Versions tested

I installed 4 versions of Moodle on the same server.

  • Moodle 1.9.19+
  • Moodle 2.1.8
  • Moodle 2.2.5
  • Moodle 2.3.1+

All versions of Moodle contain the same amount of courses and users. I did not install any third-party plugins and they all use the ‘Standard’ theme. I added a logo by modifying some of the core code. Authentication is set to ‘Manual accounts’ and sessions are saved on the disk.

 

Course

I created a ‘Performance Testing’ course with a selection of activities/resources. I used exactly the same course on all 4 different versions of Moodle, including blocks and filters. As mentioned above, the same users were enrolled on all 4 versions. Basically, everything was exactly the same on every version. 

Moodle performance testing course

Should you want to try and test your server as well, you can download the courses I used below:

You will also need the users for those courses, download this CSV file to batch upload 50 test users to your Moodle installation. 

 

Test strategy

 

Macros

I created 4 macros using iMacros, which mimic a single user performing the following:

  • Login > View course > View 500KB resource > Logout
  • Login > View course > View directory > Logout
  • Login > View course > Post to forum > Sends choice > Logout
  • Login > View course > Take a 7 question quiz > Logout

I ran each macro one after another 50 times for each version. Altogether, this created over 6,500 Moodle pages. Nothing else was running on the server at the time of testing.

You can download the macros I used here. You will need to change a few things for it to work for you, namely:

  • http://www.somemoodlesite.com/versionxx/ to reflect your actual Moodle installation
  • view.php?id=xxxx to be changed to your course/activities ID’s
  • responses in the macros that deal with quiz. You might want to record a macro using iMacros running the quiz and then edit the macro to check the actual values for your site

 

Recording data

It is possible for a Moodle administrator to show performance data on each page footer. I setup iMacros to scrape that data and save it in a CSV file for every page it visited. 

 

Analysing data

Rather than overcomplicating my iMacros scripts, I decided to manipulate the data post collection using TextWrangler fantastic search & replace multiple files to clean up my data so that it could be analysed in Excel. I then create pivot tables and generated graphs from those.

You can download the raw data here if you would like to play with it. I recorded how long it actually took for the pages to fully load on my computer while running the tests. I decided against mentioning this in the blog post as the metric is subject to be influenced by too many factors (server and computer were not in the same network).

 

Please use the comments sections for questions regarding this test or share similar tests you may have run.

LoadStorm Moodle

Note: this post is part of the Oktobertest series

Using this step by step ‘how-to’, you will be able to test how well your Moodle installation would be able to cope with a certain amount of users, using the loadstorm.com service. Whilst this is not a perfect representation of what real life usage of your Moodle installation would be like, it will still give you a very good idea of whether your system is good enough to run Moodle for your expected number of users.

Loadstorm is a web based load testing service. Their business is to simulate multiple virtual users simultaneously navigating a website, following a scenario that you pre-recorded using the loadstorm point and click interface.

This tutorial will show you how to test your Moodle installation with 25 simultaneous users, as you get 25 lifetime free virtual users with loadstorm. It also turns out to be the maximum number of students I have in my classes. If you would like to test with more users, you would need to purchase extra virtual users. I have found the service to be very good value and the support top-notch.

Warning – This tutorial is for Moodle 1.9 but the steps for Moodle 2.x are virtually the same.

 

Before starting this tutorial, you should download the following files:

 

Step 1: Upload test users to Moodle

 

 
  • Note: You must be logged in as an administrator
  • Site Administration > Users > Accounts > Upload users
 
 
  1. Choose file > Select file > Select the mdl_users CSV file you downloaded prior to starting this tutorial
  2. Do not change any of the default values
  3. Click on ‘Upload users’
 
 
  1. Check that the data has been uploaded correctly
  2. Do not change any of the defaults
  3. Click ‘Upload users’
 
 
  • You will get a report of all users that have been created. Click ‘Continue’
  • Note: Don’t forget to enroll those newly created users into the course you want to use for loadtesting.
 
 

Step 2: Upload loadstorm.csv file to your loadstorm account

 

 
 
 
  • Scroll down to the bottom of the page ‘Form Data Sets’
  • Click on ‘Upload data’
 
 
  1. Enter ‘Moodle users’ in the ‘Label’ box
  2. Click ‘Choose’ – Go and select the ‘loadstorm.csv’ file you downloaded prior to starting this tutorial
  3. Click ‘Upload’
 
 
  • Your file is now ready to be used
 
 

Step 3: Build a load test plan

 

 
  • In the ‘Build’ tab, click on ‘Add plan’
  • A ‘plan’ is a collection of ‘scenarios’ in loadstorm. More on this in the steps below
 
 
  1. Use a name that you can easily understand, as you might use many plans in loadstorm
  2. Again, your description should be as detailed as possible
  3. Click ‘Save’ when you’re happy with your form

Note: You can always edit the name & description of your plan

 
 

Step 4: add a scenario to your plan

 

 

A ‘plan’ is a collection of ‘scenarios’. A ‘scenario’ is a series of steps that a virtual user will follow on Moodle. For example, you could have one scenario where a user logs in, looks at the homepage and then logs out (scenario 1); Another scenario could be a user logs in, visits a course, answer a quiz and then logs out (scenario 2). The more scenarios you have in a plan, the more realistic your load testing will be. You can then assign a weight for each scenario so that it further ressembles how Moodle is actually used in your organisation. For example, one could argue that scenario 1 mentioned above would take place a lot more often than scenario 2. You could set a weight of 90 to scenario 1 and 10 to scenario 2 to reflect that.

For the sake of simplicity, we will only create 2 scenarios in this tutorial.

Scenario 1 with a weight of 40:

  • User looks at the homepage
  • Logs in
  • Downloads a 500KB resource
  • Logs out

Scenario 2 with a weight of 60:

  • User looks at the homepage
  • Logs in
  • Downloads a 500KB resource
  • Views a forum
  • Answers a forum post
  • Posts a blog post
  • Logs out

Click ‘Add Scenario’

 
 

1 & 2. Be as descriptive as possible
3. Assign the weighting you have decided to allocate
4 & 5. This is the minimum/maximum amount of time a virtual user would wait between steps (or ‘clicks’ in the real world). The actual time will be picked randomly by loadstorm for every step and every virtual user, between your chosen minimum and maximum values. If you want to put maximum strain on your server, you should try and keep those values as low as possible.
6. Make sure the correct CSV file is selected
7. Leave both boxes ticked so that your test is as close to real life as possible
8. Click ‘Save’

 
 
  • Check that all the options are correct and click ‘Add step’
 
 

Scenario 1, action 1: Visit the homepage

 

 

Warning: If this is your first time using loadstorm you will need to verify your server using the ‘Add a new server’ link. As I have used it before I cannot show you how to do that but there is a good tutorial at http://loadstorm.com/load-test-servers

  1. Leave this as ‘/‘ if your moodle is accessible via an address such as http://www.mymoodle.com. My Moodle installation is accessible at http://www.vleteacher.com/version19 hence the need for me to add ‘/version19’
  2. Select the correct server. Please see the warning note above if it is the first time you are using loadstorm
  3. Click ‘Save’

 

 
 
  • Loadstorm will open the page so that you can check the step was performed correctly
  • Click ‘New step’
 
 

Scenario 1, action 2: Login to Moodle

 

 
  1. The ‘Login’ page of Moodle is located at http://yourmoodleserver.com/login/index.php. Fill in the correct information for your server
  2. Click ‘Save’
  • Click ‘New step’ when the preview shows up
 
 
  • You will now need to tell virtual users what usernames and passwords to use to login. 
  • Check that your values match the ones circled in red above
  • Click ‘Save
 
 
 
 

Scenario 1, action 3: Visit the Load Testing course

 

 
  1. Select ‘Click a link’
  2. Look for and select ‘Load testing course’ (or whatever else course you want the virtual user to access)
  3. Click ‘Save’
 
 
  • Check that the step has been successfully created and click ‘New step’
 
 

Scenario 1, action 4: View a 500KB image

 

 
  • Select ‘Click a link’ and then scroll down to the correct resource
  • Click ‘Save’
 
 
  • Check that the step was successfully created and click ‘New step’
 
 

Scenario 1, action 5: Logout

 

 
  1. Select the ‘Click a link’ option
  2. Select the ‘Logout’ link
  3. Click ‘Save’
 
 
  • Check that the step was successfully created and click on the name of your plan (or the ‘build’ tab).
  • Well done, your first scenario has now been created.
 
 

Step 5: Add scenario 2

 

 
 
 
  • As for scenario 1, make sure you are as descriptive as possible. Note that I have added a weight of 60 to this scenario. Scenario 1 has a weight of 40, therefore the weight of both scenarios equates to 100.
 
 
  • Check that the details are correct and click ‘Add step’
  • Note: We could have copied scenario 1 and edited it, but as this is a tutorial aimed at very beginners, repetition of steps is a good way to learn.
 
 

Scenario 2, action 1: Visit homepage

 

 
  1. Leave this as ‘/’ if your moodle is accessible via an address such as ‘http://www.mymoodle.com
  2. Select the correct server. Please see the warning note above if it is the first time you are using loadstorm
  3. Click ‘Save’
  • Note: After each new step you have created, you will receive a message telling you that the step was created successfully. To keep this tutorial as succinct as possible, I have removed those screenshots for scenario 2. Simply click on ‘New step’ whenever that screen shows up
 
 

Scenario 2, action 2: Login to Moodle

 

 
 
 
  • Make sure your options are similar to the ones shown in the image above
 
 

Scenario 2, action 3: Visit the Load Testing course

 

 
  1. Select ‘Click a link’
  2. Look for and select ‘Load testing course’ (or whatever else course you want the virtual user to access)
  3. Click ‘Save’
 
 
  • Click a link > Choose the right file > Click ‘Save’
 
 

Scenario 2, action 4: Return to course homepage

 

 
 
 

Scenario 2, action 5: View a forum

 

 
  • Click a link > Look for ‘Forum – Standard’ > Click ‘Save’
 
 

Scenario 2, action 6: Post a forum message

 

  • Make sure your details are the same as above
 
 
  • Make sure your details are the same as above
 
 

Scenario 2, action 7: Return to course homepage

 

 
  • The previous step will have returned a message that prevents you from clicking on any links so you’ll have to copy your course URL. The URL to a course is always something like http://www.yourmoodle.com/course/view.php?id=somenumber
  • Simply change ‘somenumber’ to the correct number for your course. Look in your browser address bar to find out what your course ID number.
 
 

Scenario 2, action 8: add a blog post

 

 
  • Click a link > Look for ‘Add a new entry’ > Click ‘Save’
  • Note: you must have the ‘Blog’ block on your course page for that link to show up.
 
 
  • Copy the details as above
 
 

Scenario 2, action 9: Logout

 

 
  • Click a link > Find ‘Logout’ > Click ‘Save’
 
 

Step 6: Add a new load test

 

 
  1. Click on the ‘Build’ tab
  2. When you have checked that the steps are correct, click ‘Add scenario’
 
 
  1. Ensure the right plan is selected
  2. Schedule when your test should be run, or simply tick ‘Start load test as soon as possible’
  3. Be as detailed as possible in your description
  4. These settings are self-explanatory. To replicate real classroom use, I like to use the same figure for ‘Start users’ and ‘Peak users’
  5. Click ‘Save’
 
 

Step 7: Understand the results

 

 

Both graphs show the progression for the entire duration of the test. The example test we ran was 30 minutes long, and this is represented on the X axis.

Graph 1:

  • Throughput: amount of kilobytes per second transferred between your server and the virtual users. Please note that it is bytes, and not bits. This is important as your web hosting company will probably will mention their network throughput in bits, not in bytes. If you would like to convert the throughput shown in this graph against your theoretical server throuput, you would have to multiply the peak number by 8. In this example, the peak throughput was 650KBps or 5200Kbps – which translates as 5.2Mbps.
  • User load: The amount of virtual users completing the scenarios at any given time. Note that there is no scale for this data so you have to over your mouse on the graph to find out the exact data. Our example test started with 15 users and finished with 25 users in increments of 1.
  • Requests per seconds: depending on how you setup your scenarios, virtual users will wait for a random period of time between each ‘click’ (or step). As the time they wait is random, the graph is an easy way to find out how many requests per second your server received during the test. The shorter the wait between steps, the more requests per seconds your server would receive.

Graph 2:

  • Response time, average: shows how long it takes for your server to respond to the request. As far as I understand it, this metric does not reflect how long it takes for a page to fully load.
  • Response time, peak: it won’t always take the same amount of time for your server to respond to requests.
  • Error rate: if a page is not accessible for whatever reason, loadstorm would record it as an error. In this example, there were no errors. Errors should be taken very seriously as a healthy server should never return any errors. Errors could be returned if the server is overloaded. You should download the CSV file for detailed results (see below) and also look at your webserver error logs to find out more.

Note: Click on ‘Click here for details…’ link to get access to useful reports

 
 

Step 8: Dig further

 

Step 9: Dig further
 
  1. When looking at a specific report, you can click on ‘Download as CSV’ – This will show you results for every individual requests
 
 

Looking through the individual results would allow you to zero in on specific issues or find out where your server ‘breaking point’ is at.

 

Free load testing tools for Moodle

[pulledquote]Moodle is becoming increasingly popular at my current school. While this is great, the server is starting to feel sluggish. I wanted to find out how much further I can take my current hardware and how much strain Moodle actually place on my server. This is why I have decided that October would be… Oktobertest for Moodle! Here are 3 ways that will help you find out if your server can cope with the demands you place/want to place on it.[/pulledquote]

Note: This is by no means supposed to be a definitive guide to load testing. It is an introduction to what can be done to ensure a Moodle installation can cope with a specific number of simultaneous users. It is aimed at Moodle admins who haven’t really done load testing before. Be careful, load testing can crash your server and you could get in trouble with your host company – always check with them first.

 

Why is Moodle so demanding? A look under the hood

Warning – what follows is a very simplified version of the truth, but close enough for jazz.

Moodle is a very powerful, but demanding web application. It is basically a collection of files saved on a hard drive and data from a database that get pieced together into a webpage for every click a user performs. For example, if a student visits a Moodle homepage, then a course page, and then posts a forum discussion, a total of 7 webpages will have been created (homepage > login page > course page > forum page > new post > post discussion > return to course page). For every page that is created, it is necessary for the server to go and collect the correct files, call the database multiple times, and also ‘write’ data to the database. The more users visit the site, the more pages need to be generated, the more strain on the server. If you follow the simple example above for a class of 30 students,  210 pages need to be generated in a very short period of time. It is also important to know that not all parts of Moodle were born equal and some of Moodle’s popular modules, such as ‘Quiz’, put more of a strain on the server than other modules such as ‘page’ or ‘URL’; This can be a real problem if your server cannot cope with a class full of students taking a test at the same time.

 

Why you should test

In short, to prevent alienating teachers and students. A lot of people use Moodle for work/school related activities, at my school alone over 800 people. I am prepared to take a risk here and say that most of those people would rather do something else than be on Moodle. As a Moodle administrator, I thrive to ensure the server is as fast and reliable as possible so that people can go about their business swiftly, and this is made possible by testing different scenarios, with specific numbers of users. I test to find out:

  • how long it takes to perform a specific task, for one or multiple concurrent users 
  • how many people can use Moodle at the same time to perform a series of tasks, and still have a usable (read fast) system
  • how much RAM, CPU, bandwidth, etc. is used during a test, and see if I am close to the limits
  • if I need to upgrade my hardware/hosting package and when I might need to upgrade

 

What to test

As you are testing your Moodle installation, you will be recording the vitals of your server, such as CPU usage, RAM, disk input/output, bandwidth, SQL usage, etc. to zero in on possible bottlenecks (check this useful blog post on server monitoring). To get meaningful results from your testing, you’ll want to have a testing strategy that mimics real world usage as much as possible. In order to achieve this, you might want to try and have a look at your Moodle logs. Here are a few rules that I try and stick to when testing:

  • test a mixture of activities
    • some that are database intensive e.g. chat
    • some that are disk intensive e.g. download large files
    • some that are CPU intensive e.g. quiz
  • add randomness to the test, if the tool allows to do so. For example apply some random ‘waiting’ time between clicks
  • test a mixture of accounts (teacher, admin, student) – not always able to do this
  • if on a ‘live’ server, perform the tests when there is little to no use e.g. 3AM for me
  • repeat the same test several times, at different times of the day. This is especially important if you are testing on a ‘live’ server. 

 

Tools to test Moodle

This post offers only a short introduction to the load testing tools and I have listed them in order of difficulty to use.

 


 

Tool 1: iMacros extension

iMacros for Firefox 

iMacros is an automation tool provided free of charge by iOpus. It is not meant to be a load-testing tool but works fine as one. There are free add-ons for Internet Explorer, Firefox and Google Chrome. I personally use the Firefox version, as it’s available both for Windows, Mac and Linux, and is more powerful than the Chrome version. 

iMacros records your web browsing activity so that you can later simulate the actions of a real Moodle user, all automatically. For example, you start up Firefox and set iMacros to record. You go about your daily business on Moodle as normal (e.g. login, view your course, add an assignment, answer some forum posts and logout), iMacros keeps a record of everything you click on during your session, including all the forms that you fill in until you press ‘Stop recording’. Those steps are saved in a macro, which you can then play back later by opening that macro. This means that you can press the ‘play’ button and Firefox will repeat all of the steps that you did during your session, without any more interactions from your part, thus automatically simulating the web browsing activities of a Moodle user. You can ‘loop’ the action to repeat it automatically as many times as you like.

I use iMacros to record simple tasks, for example:

  • Login > View course > View resource > Logout
  • Login > View course > View directory > Logout
  • Login > View course > Post to forum > Choice > Logout
  • Login > View course > Take quiz > Logout
iMacros is meant to only simulate one user at a time, which is hardly load-testing. There is a simple workaround, which is to start multiple instances of Firefox (Ctrl+N for Windows, Cmd+N for Macs) on your PC, you are then only limited by your computer’s ability to run multiple instances of Firefox (RAM, CPU and bandwitdh being the three biggest issues). Each instance of Firefox then simulates one user. On my computer, I can safely run approximately 25 instances of Firefox concurrently (please note that using different tabs will not work). If you have access to a large network of computers, you can save the macros on multiple computers and schedule them to run at set times, automatically using a task scheduler or cron.
 
Difficulty: ★
 
Pros:
  • Very easy to use for simple workflows (e.g. login, view course)
  • Gentle learning curve
  • Exact replica of what a real user’s browsing experience is like
  • Excellent support in the form of a vibrant community and wiki
  • Macros are portable and can be saved/run on multiple computers, from within or outside your network
Cons:
  • More demanding workflows require to dive into the code
  • Limited number of simultaneous users, unless macros are run from multiple computers
 
Tutorial:
 

 

Tool 2: Loadstorm

Loadstorm.com 

Loadstorm is a web based load testing service. Their business is to simulate multiple virtual users simultaneously navigating a website, following a pre-recorded scenario. It is similar to iMacros in that you can record your web activity to play later (scenario), but you have to use Loadstorm’s interface to do it. Although it is not as intuitive as iMacros, it is still simple enough to use to record simple workflows. Loadstorm is free to use for up to 25 simultaneous virtual users (forever, not just once), which is quite handy considering it is quite close to a ‘regular’ classroom size. The 25 user-limit is usually enough to test sites on shared hosting, and bring the server down. You can purchase extra virtual users to test larger sites. To date, I have not yet needed to purchase additional users as Loadstorm often runs promotions whereby you get free users if you follow them on Twitter, or like them on Facebook. I must say that their service is top-notch and will be purchasing when I run out of users.

 

Difficulty: ★★

Pros:

  • Can be used on any device with a web browser and access to the Internet
  • Easy to use for simple workflows
  • Company behind the product provides excellent support
  • Servers based around the World, mimics real-world usage for students accessing Moodle from home
  • Relatively gentle learning curve
Cons:
  • Pay-for service if you want to test more than 25 concurrent users
  • Some more complex workflows are difficult to set up (e.g. multiple quizzes)
  • Doesn’t work behind some firewalls
  • Web based service so there is some extra latency involved
 
Tutorial: 

  

Tool 3: JMeter

Apache JMeter 

JMeter is a fully-fledged load and performance open-source software by the Apache foundation. It is much more powerful than the other tools mentioned (although I suspect Loadstorm is actually based on JMeter) but also has a much steeper learning curve. Luckily, an excellent JMeter script generator was created and shared by the good people of the Open University UK (download it here). Without this tool, it would be rather difficult to create scripts (scenarios) to test your Moodle installation. This tool will help you generate users, and get those users to post to forums, participate in chat sessions and take quizzes. Nothing stops you from adding to the generated script, should you want to add more steps to it. Should you wish to dig further into JMeter, I would strongly recommend you to read the excellent ‘Apache JMeter‘ book by Emily H. Halili.

 

Difficulty: ★★★

Pros:

  • Java based, so can be used on all platforms
  • Scripts are portable – test from within or outside your network
  • Extremely powerful
Cons:

What tools do you use to load-test Moodle? Please feel free to share in the comments section.