Make your Moodle courses faster without fiddling with the server

[pulledquote]Whenever I am asked what do to make a Moodle site faster, I always end up talking about tweaking the server. Often I miss the point as a significant amount of Moodle sites are hosted on servers that cannot be tweaked (e.g. GoDaddy 4GH, etc.), or by educators who simply don’t know how to setup a server.[/pulledquote] In this post, I explore a few simple ideas to help speed up Moodle courses without fiddling with the server, most of which can be done by a teacher. I also find out that some settings make no difference whatsoever. I have ordered my tips in decreasing order of importance i.e. the ones that really matter are at the top. When reading the graphs, make sure you take a look at the scale, as differences can be minimal. It is important to remember that sound pedagogy and good course design are more important than a super fast server. 

 

For the impatient (spoiler)

 

Things that impact performance

  • Amount of blocks displayed on your course pages
  • ‘Show one section only’ in your course settings
  • Images in labels
  • Amount of sections, resources & activities
  • Activity tracking & conditional activities
  • Forum tracking
  • ‘Theme designer’ mode
  • ‘Cache language strings’ option

Things that don’t really impact performance

  • Theme
  • Course format
  • Groups/Groupings 

Note: I ran a series of tests to find out what made a difference when displaying course pages on Moodle, as a student. I did not run tests as a teacher/admin, nor did I test other areas of Moodle. If you want to know more about the methodology for these tests, and for a copy of the raw data I collected, go to the ‘Methodology’ section at the bottom of this page.

 

Blocks

Out of the box, Moodle comes with 38 different blocks, 31 of which can be used on course pages. By default, only the ‘Navigation‘ and ‘Settings‘ blocks appear on course pages. I ran a test to find out to what extent the number of blocks affect the performance of Moodle course pages. 

Q: Should I keep the number of blocks on my course pages to a minimum?

A: Yes, you should only keep the blocks that are absolutely necessary to your teaching & learning experience. A course page with all blocks displayed takes on average 64% longer to generate compared with a page with only the 2 default blocks (0.871 seconds vs. 0.528 seconds). A course page with all blocks will also use 37.5% more RAM than a page with only the ‘Navigation’ and ‘Settings’ blocks displayed (67.62MB vs. 49.9MB).

The amount of blocks displayed on a Moodle course page impacts performance

Click to zoom on any of the images

In my tests, I found out that the ‘Comments‘ block (item 9 on the graph) and the ‘RSS‘ block (item 26 on the graph) are to be avoided if you have an underpowered Moodle server. The ‘Comments’ block needed an extra 7.4MB RAM, and 206 files to display on my test page, while the ‘RSS’ block added an extra 6.2MB RAM.

Recommendation(s):

  • Keep the amount of blocks to a minimum. 
  • If you have third-party blocks (not tested here) you could try and disable blocks one by one and check your server performance. For this, you need to turn on the ‘Performance’ stats in your theme and write down the values when you load pages (Site administration > Development > Debugging > Performance info).
  • Avoid the ‘Comments’ & the ‘RSS’ blocks if your server is underpowered.

  

Number of resources/activities

Adding more resources/activities to a Moodle course makes the page look longer (Scroll of Death), but it is not obvious to a Moodle user whether it affects performance or not. I ran 10 tests, one with the ‘baseline’ amount of activities/resources, and then each subsequent test with an extra ‘baseline’ worth of activities & resources e.g. course 1 has 29 resources & activities (1 of each type), course 2 has 58, course 3 has 87, up to course 10 with 290 resources & activities.

Q: Does the amount of resources/activities on a Moodle course page affect performance?

A: Yes it does, quite a bit but not as much as I thought prior to the tests. On average, adding 29 resources & activities to a course page added an extra 0.5 to 0.6MB of RAM, and the system needed an extra 0.1 to 0.2 seconds to generate the pages (although it was somehow not always the case). Multiplying the amount of activities/resources by 10 (or 1,000%) only slowed down the server by 147%, and increased the amount of RAM needed to generate a course page by 10%.

Amount of RAM used to generate a Moodle page depending on the number of activities and resources

Amount of RAM used to generate a Moodle page depending on the number of activities and resources

Recommendation(s):

  • Use folders if you cannot reduce the amount of resources on your course (for files).
  • Try to minimize the amount of resources you display on a page. It is sometimes possible to use Moodle tools to achieve this e.g. book module, database, lesson, wiki, etc. 
  • Change your course settings to ‘Show one section only‘ if you are on Moodle 2.3 or above.
  • Teach students how to use the ‘highlight’ option to show only 1 section/week at a time (usually a light-bulb icon).

 

Images in labels

If a picture is worth a thousand words, my courses are a few bibles worth. I always encourage teachers to use images on their Moodle courses as  not only does it make the courses look more appealing, it may also help with navigation.

Q: If I add a lot of images on my course pages, will it affect performance?

A: Yes, it will. The number of reads/writes to the database skyrockets when you add images in labels. A page with 100 images requires over 3 times the amount of calls to the database compared with a page with no images in labels. Looking at iMacros loading pages during the tests, I also noticed that images inserted as URL’s (i.e. not uploaded to Moodle) show up much faster in Moodle pages than the images uploaded to Moodle. 

Number of DB reads to generate a Moodle course page - Number of labels (with images)Number of DB writes to generate a Moodle course page - Number of labels (with images)

Recommendation(s): 

  • If you want to show multiple images, perhaps you could look at ‘Folders‘ instead of individual images in labels.
  • Use ‘Pages‘ to add your images to your courses, instead of directly on course pages.
  • Use only images that are useful to your course page e.g. images you cannot really do without.
  • If at all possible, simply link to your images rather than uploading them to Moodle.

 

Course sections

With Moodle 2.3 and above, you are able to ‘Show all sections on one page’ or ‘Show one section per page’ as your course layout (Edit course settings > Course layout)

Q: Does the ‘course layout’ I choose have an impact on Moodle performance?

A: Yes, on average ‘Show one section per page’ made 19% fewer database reads than the default setting. This allowed for marginally faster page generation (-4%). The database is often the first thing to cause a bottleneck on a Moodle server, so you might want to explore this avenue if your server is currently struggling. 

Time to generate a Moodle course page - Course sections optionNumber of DB reads to generate a Moodle course page - Course sections option

Recommendation(s): 

  • If you are running Moodle 2.3 or above and your server is struggling, it might be a good idea to try if ‘Show only one section per page’ makes a difference for you. 
  • It is worth noting the course pages look different with this setting, so it might take your users a while to get used to it.

 

Conditional activities & completion tracking

Conditional activities were introduced in Moodle 2.0 and have been very popular feature. It allows you to create learning pathways, control the flow of a course or even gamify your courses. 

Q: I love the ‘Conditional activities’ features of Moodle 2, should I stop using it on my cheap server?

A: It depends, but probably yes. I found that for courses with few activities & resources, this setting has very little impact (+2%). However, when more activities were involved, the simple fact of turning the ‘Completion tracking’ feature at the site level (Site Administration > Advanced Features > Enable Completion Tracking) and at the course level (Edit course settings > Completion tracking) increased the time needed to generate a page by 29%. When I tested the same course with activities depending on one another (Site Administration > Advanced Features > Enable Completion Tracking > Enable Conditional Access) the time increase was ‘only’ 27%. 

Time to generate a Moodle course page with activity tracking and conditional activities on

Recommendation(s):

  • Design good courses, use conditional activities/resources should it benefit teaching & learning.
  • Again, this feature impacts teaching & learning too much to be left behind, get rid of it only if you must, or if it is completely useless in your context.

 

Forum tracking

It is possible to have Moodle show how many new forum posts were written since you last visited a course page, for each forum. You can switch this option on/off  at the user level (Edit profile > Forum tracking).

Q: I like my students to share things with the forums, should I disable the tracking function?

A: It depends, really does. It mainly depends on how active your forums are. I conducted 3 tests,  one where there were only 10 unread forum posts in 1 forum, another with 500 unread posts, and yet another with a 1,000 unread posts. I found no noticeable difference with only 10 unread posts but a 23% increase in the time needed to generate a Moodle course page with 500 or 1,000 unread posts. Strangely enough there was no noticeable difference between 500 unread and 1,000 posts.

Time to generate a Moodle course page with forum tracking on

Recommendation(s):

  • My tests were run with one forum only. If you have a lot of very active forums, this setting could well cost you quite a bit in terms of performance.

 

Course format

By default, Moodle has 4 different course formats. For the purpose of this test, I only compared the ‘Topics’ and ‘Weekly’ formats, as they are the most widely used (Edit course settings > Format)

Q: Will the course format I choose affect performance? 

A: Very, very little. The difference is minimal, with a slight edge to the ‘Topics’ format being 4.7% faster. It is worth noting that the amount of topics/weeks shown does impact the performance, even if those are empty. A course page with 52 topics took 25% longer to generate than the baseline course page, which has only 3 topics.

Time to generate a Moodle course page depending on course format

I must say I was relieved to see no real difference between the course formats, as the decision to use either ‘Topics’ or ‘Weeks’ is a fundamental one, often you won’t even have the choice and will be forced by your institution. 

Recommendation(s):

  • Choose whichever course format you want!
  • If you use the ‘topics’ format, keep an eye on sections – limit the amount of sections if that doesn’t impact your course design (at the end of the day, it’s all about teaching & learning NOT the performance).

  

Groups & groupings

Teachers have the ability to setup their course with different groups. I ran a test to check if the ‘Separate groups‘ made a difference (Edit course settings > Group mode). I also ran a test to find out whether the ‘Show to groups members only‘ option had an impact on performance (Site administration > Development > Experimental settings > Enable group members only).

Q: Do the different group modes affect performance? 

A: Not really, if anything it has a positive impact on performance. Whilst it might affect other parts of Moodle (not tested), it certainly doesn’t affect performance as far as course pages are concerned – in fact it seems to speed things up marginally.

Time to generate a Moodle course page - Groups: groupings

Recommendation(s):

  • ‘Groups’ are too useful to not be used, it even seems to make things marginally better in terms of performance, so go for it!
  • The ‘Show to group members only’ also has a positive impact on performance.

  

Themes

Q: Does it matter which theme I use? 

A: If you’re using any of the themes that come standard with Moodle, the answer is ‘not really’. All of the themes performed pretty much the same in my tests. ‘Boxxie‘ was the fastest theme with an average of 0.494 seconds to generate a page, when ‘Fusion‘ and ‘Sky high‘ were the slowest at 0.526 seconds on average. It is important to keep in mind that the differences are so minimal that it could be put down to external factors (I/O on the server, etc.), rather than the themes themselves. I did not test any third-party themes

 Time to generate a Moodle course page with all box-standard themes

Warning: Please make sure the ‘Theme designer mode’ is set to ‘OFF’ on your Moodle installation (Site administration > Appearance > Themes > Theme settings > Theme designer mode). This is often the cause of slow Moodle sites. I found that on average it takes at least twice the amount of CPU to run a Moodle site with theme designer mode ON, as shown below.

Load average to generate a Moodle course page - Theme designer mode

Recommendation(s):

  • Pick the theme you like best, it doesn’t ‘cost’ you in terms of server resources
  • If you’re using a custom theme, make sure you compare it with the ‘Standard’ theme, as it’ll give you a good idea of how well it performs

 

Bonus – Cache language strings

Someone recently told me that their site ran faster with the ‘Cache language strings’ option left ‘off’ (Site administration > Language > Language Settings > Cache all language strings). I was intrigued and decided to test it. 

Q: Does the ‘Cache language strings’ option have an effect on Moodle performance?

A: Yes, in my tests course pages were created 64% more slowly with the cache option turned off.

Time to generate a Moodle course page with the cache language strings off

Recommendation(s):

  • This one is obvious – leave it on (it is on by default)

 

Methodology

 

Server and technical

  • 1GB RAM VPS from DigitalOcean (2 Virtual CPU’s) – You can run your own tests using the same server (you pay by the hour) so that you can easily compare with my ‘benchmarks’
  • Ubuntu 12.04LTS
  • As vanilla a LAMP as it gets (no modifications/customisations whatsoever)
  • atop and htop installed

Moodle

  • Moodle 2.4 Beta
  • No customisations
  • No 3rd party plugins
  • All default settings left untouched
  • Only default modules & blocks
  • 200 courses
  • Each course has 1,000 users (students) enrolled through cohort sync
  • Standard theme

Courses

  • All courses share the same backbone i.e. all have an example of each module (1 quiz, 1 page, 1 folder, etc.)
  • Some courses have been tweaked for particular tests e.g. course 10 has twice the amount of activities
  • Some courses have different blocks enabled to allow for the ‘blocks’ tests to be run 
Tests
  • Every test was run between 50 and 100 times (altogether over 20,000 pages were generated)
  • I used iMacros to run the tests and scrape the data 
  • All tests were run as students (50 to 100 different users per test)
  • Browser cache was reset for each user (between 50 to 100 times per test) as to not skew results
Raw data
  • Here is my contribution to big data – raw results in CSV format
  • There are parts of the data that I have not analysed, so have a go if you like
  • If you use the data for anything, please let us know – I’d like to see the results

Have you run any tests of your own? Feel free to share your results in the comments section below. If you want to know more about how I ran individual tests, simply write a comment and I’ll indulge.

Moodle 2.4 Beta performance test

[pulledquote]Last month I wrote a blog post comparing different versions of Moodle and the demands they each place on hardware. With Moodle 2.4 release just around the corner, I thought I would perform the same series of tests – here are the results.[/pulledquote]

Warning: these tests are not scientific by any standards but I am comparing apples with apples, please read my previous blog post to check my methodology – I did not change anything on my server/Moodle site since my last round of tests. All figures shown below are averages.

Warning 2: Moodle 2.4 is still in its beta stage, I will run the same test once the official Moodle 2.4 stable has been released.

 

 

RAM to generate a page -8.9%

 

Average amount of RAM to generate a Moodle 2.4 page

 

On average, I found that Moodle 2.4 uses 8.9% less RAM to generate pages than Moodle 2.3. This is great news considering that, for a large amount of Moodle servers, is the first thing that needs upgraded when usage increases. Moodle is very ‘RAM hungry’.

 

Time to generate a page -9.1%

 

 

 

On average Moodle 2.4 is 9.1% faster at generating pages than Moodle 2.3. Might not matter much if you only have a few users, but it starts adding up if you have hundreds/thousands of simultaneous users. The less time your CPU/disk spend generating a page, the faster they can move on to their next task.

 

Number of files to generate a page -9.1%

 

Average number of files needed to generate a Moodle 2.4 page

 

On average, it takes 25 fewer files to generate a Moodle 2.4 page compared with Moodle 2.3. This equates to a 9.1% improvement over Moodle 2.3. Each file needs to be collected from the disk before it can be processed by the CPU to generate a page, so the less files to be collected, the better.

 

Average database reads -36.6%

 

Average number of database calls to generate a Moodle 2.4 page

 

This is the big winner and I believe is directly linked to the introduction of the Moodle Universal Cache (MUC from then on) in Moodle 2.4. On average, Moodle 2.4 makes almost 37% fewer calls to the database to generate Moodle pages compared with Moodle 2.3 (39 calls vs. 57). This is huge considering MySQL (if that’s what you use as your database) is often the bottleneck in large-ish Moodle servers. This will have a massive impact when you have a lot of simultaneous users on your Moodle site, less database calls = more simultaneous users who can use Moodle on your current hardware. This is quite impressive considering it is only the first iteration of the MUC.

 

Average CPU load -6.5%

Warning: this is to be taken with a grain of salt, see my previous blog post for reasons why.

 

Average CPU load using Moodle 2.4

 

On average, Moodle 2.4 uses 6.5% less CPU than Moodle 2.3. From experience, CPU is rarely the first thing that needs upgrading on a Moodle server but it’s nice to see an improvement in this area. 

 

Conclusions

We’re still a far cry from the lean-mean Moodle 1.9 but it is great to see that Moodle HQ have managed to keep adding features whilst reducing Moodle’s footprint on hardware. One cannot compare Moodle 2.x with Moodle 1.9.x – there have been too many features added, improvements made for it to be a fair comparison. Moodle relies heavily on calls to the database so the less calls, the better. With a 37% improvement, this alone should prompt you to update to Moodle 2.4 when it comes out.

It is important to note that my server already uses caching techniques that the new Moodle Universal Cache cannot yet leverage (e.g. APC). I fully expect MUC to mature and reduce Moodle’s footprint even further. This is a step in the right direction. Gains brought by the MUC will be more obvious when multiple users are logged on, I will try and test it out soon.

I have another round of tests coming up where we’ll be able to see the difference on a server that does not yet have APC enabled. Stay tuned.

 

Have you run any Moodle performance tests lately? If you have, please don’t be shy and share in the comments section below.

Using Munin to monitor Moodle server

[pulledquote]As part of Oktobertest I am going to share 5 free tools you can use to monitor your Moodle server. The focus of this post is on load testing but these tools can be (and some should be) used as part of your normal monitoring.[/pulledquote]

Note: this post is aimed towards admins who run their Moodle installation on a Ubuntu Linux server. As usual, this is not meant to be a definitive list or how-to, but rather an introduction to what a Moodle administrator can do to ensure their installation is running smoothly

 

 

Monitoring tools

The following tools are listed in increasing order of difficulty of use. I have used them all but I have found that unless I load test, the first 3 tools tend to be enough for me. I ran Zabbix for a while but it requires quite a bit of TLC, which I simply don’t have time for – great tool though. If I had to keep only one tool it would be htop.

 


Moodle server monitoring tool 1: top 

 

Moodle monitoring tool - top

 

What it does

  • By default, the top command lists all processes in decreasing order of amount of CPU they use
  • The top command also shows your memory (RAM) status and your overall load average (CPU usage)
  • Pressing the M key (capital letter) will sort the processes by amount of RAM used (decreasing)
  • All figures are updated in real time, so I usually have it running in a window whenever I load test to see how the server performs under specific loads

 

Installation

  • Top should come packaged up with your Linux server, you shouldn’t need to install anything for it to work.
  • All you need to do to start it up is type the following command in a terminal window:
sudo top

 

If you would like to find out more about all of the options the top command has to offer, read this excellent article.

 


Moodle server monitoring tool 2: atop

  

Using atop to monitor Moodle server

 

What it does

  • atop is very similar to top but gives you access to more data (network, disk, etc.)
  • atop shows usage summaries by all processes e.g. you have access to a summary of all memory, network, CPU (per core) usage

 

Installation

  • If you want to install atop from a package, simply type
sudo apt-get install atop
  • All you need to do to start it up is type the following command in a terminal window:
sudo atop

 

If you would like to find out more about all of the options the atop command has to offer, read this excellent article.

 


Moodle server monitoring tool 3: htop

 

Using htop to monitor Moodle server

  

What it does

  • htop is very similar to top but allows you to sort data more easily
  • htop also shows CPU & memory usage in a more user-friendly way

 

Installation

  • If you want to install htop from a package, simply type
sudo apt-get install htop
  • All you need to do to start it up is type the following command in a terminal window:
sudo htop

 

If you would like to find out more about all of the options the htop command has to offer, read this excellent article.

 


Moodle server monitoring tool 4: Munin

 
Using Munin to monitor Moodle server

 

What it does

  • Munin presents your server data in easy to read graphs
  • Munin can monitor pretty much everything on your server, from CPU usage to MySQL slow queries, etc.
  • Munin data is saved in a flat file and historical data can be accessed in the graphs
  • Good support community

 

Installation

Rather than re-write an install guide for Munin, I have compiled all of the resources I used to get Munin up and running smoothly on my VPS

You might run into some issues (I did) – here is where you can get an easy fix

  

You can see a screenshot of whole Munin report page here (800KB) – possibly the World’s longest screenshot 🙂

 


Moodle server monitoring tool 5: Zabbix

Monitor your Moodle server with Zabbix - IO wait

 

What it does

  • Zabbix is similar to Munin – it displays server vitals in graphs
  • Zabbix also has the ability to send you email alerts should your server reach critical values e.g. disk usage > 80%
  • You have the ability to create your own alerts, with custom thresholds
  • Zabbix is a full fledged server monitoring solution – you can monitor multiple servers
  • Zabbix is best run on its own server as it is MySQL intensive (every piece of data is stored in a database)

 

Installation

 

Monitor your Moodle server with Zabbix - CPU

Monitor your Moodle server with Zabbix - bandwidth

 

 

iMacros Moodle Logos

 

 

 

 

 

This post is part of the Oktobertest series. It is not meant to be a definitive guide on how to use iMacros. Rather it is a quick introduction to get Moodle administrators started with iMacros.

iMacros records your web browsing activity so that you can later simulate the actions of a real Moodle user, all automatically. For example, you start up Firefox and set iMacros to record. You go about your daily business on Moodle as normal (e.g. login, view your course, add an assignment, answer some forum posts and logout), iMacros keeps a record of everything you click on during your session, including all the forms that you fill in until you press ‘Stop recording’. Those steps are saved in a macro, which you can then play back later by opening that macro. This means that you can press the ‘play’ button and Firefox will repeat all of the steps that you did during your session, without any more interactions from your part, thus automatically simulating the web browsing activities of a Moodle user. You can ‘loop’ the action to repeat it automatically as many times as you like.

Before you start following this tutorial, you should download the following:

 You might also be interested in reading this blog post, where you can find real world examples of iMacros macros (!).

 

Step 1: Open iMacros

 
  1. Click on the iMacros icon
  2. Open the webpage where you want to start your test from
  3. Click on the ‘Rec’ icon

 

 

 

Step 2: Start recording your macro

 
  1. Click the ‘Record’ button
  2. As you navigate your Moodle site, you will see that lines of text are added to this box, this is normal

Note: 3. Make sure that the ‘Click mode’ is set to ‘Auto’

 

 

 

Step 3: Stop recording when you’re happy

 
  1. Click ‘Stop’ when you have finished navigating your Moodle site
  2. The macro you have just recorded is called ‘current.iim’. I strongly advise you to rename it to something else, as every time you click the ‘record’ button it overwrites the ‘current.iim’ file (simply right-click on the file name to rename it)

 

 

 

Step 4: Try it

 

Try and play your macro just to make sure it works

  1. Click on the ‘Play’ tab
  2. Click on the ‘Play’ button (make sure the correct macro file is selected)
  3. Sit back and relax

If you want to loop your macro, take a look at this webpage.

 

 

 

Step 5: Load test

 

The macro you recorded will only mimic one user when you play it – hardly useful for load testing. To mimic multiple users at once, you will need to run multiple instances of the macro at the same time. Note that using different tabs will not work, you have to open each new instance of Firefox in its own window. (Ctrl N on Windows/Linux, or Cmd N on Mac). The only limitation is the amount of instances your computer can cope with in terms of CPU/RAM and bandwidth.

It is also possible to run the macro from command line or task scheduler. Procedures differ depending on the platform you use, so instead of me re-inventing the wheel, please check the following links: