Welcome

ITU Innovators
IT Kartellet
Talk
Coffee & Networking
ITU Innovators
● Student Entrepreneurial Organization
Driven by volunteers

● Vision
Connecting students with startup companies,
advisors, incubators, events, networking, ...

● Events
Arranging and attending talks, workshops,
competitions, meetings, ...
IT Kartellet
● IT-consultancy
● Software development
● Hosting
● Moodle/Drupal experience
● ASP.NET (MVC)

Copenhagen Tech Talks

Scaling Moodle
● Independent student project fall 2013 under
the supervision of Kasper Østerbye
● Collaboration with the IT department of ITU
(LearnIT)
● Collaboration with ITK (Moodle consulting)
Scalability,
Performance Testing
& Tuning
IT Kartellet and friends
Presented by ITU Innovators
What is Moodle?
● Modular Object-Oriented Dynamic Learning
Environment
● Open source
● Open University - 700.000 users, 6.000
courses
● So it does scale - but how?
 Scalability basics
● Vertical / scaling up
● Horizontal / scaling out
○ X-axis
○ Y-axis
○ Z-axis
● Combining
○ Performance
○ Fault tolerance
○ Fault isolation
Moodle architecture
Minimum Example setup
CBS - Learn
● 20.000 students
● High availability requirements -
no downtime is allowed
● All components should be
fault tolerant
ITU - LearnIT
● 2.500 students
● Permits for small
windows of
downtime
● Should handle
large file uploads
from games and
digital media
students
Moodle as a black box
● ...or a number of black boxes
● Large community
● Developed over many years
● Limited documentation
● How can we make the best out of it?
Testing approach
● Repeatable
● Automated
● Well documented
● Organized
● Comparable
● Targeted
User request simulation
User request simulation
● Simply HTTP Requests
● Realistic input and randomization
● Client-side measurements
● Why jMeter?
Server-side measurements
● Monitor/record during test
● Compare to everyday usage
● Invaluable monitoring tools
○ iostat
○ inotify
○ htop
Server-side measurements - iostat
● Monitoring of cpu/storage activity
● Repeated measure with given interval
● Most relevant columns
○ iowait, avgqu-sz, await
● More details: http://linuxcommand.org/man_pages/iostat1.html
Server-side measurements - inotify
● Monitor all files access, open, write etc.
● More details:
○ inotifywatch http://linux.die.net/man/1/inotifywatch

○ inotify-win https://github.com/thekid/inotify-win / JNotify http://jnotify.sourceforge.net/

Server-side measurements - htop
● When does the server give up - when is it
trashing itself in terms of CPU / memory?
● Adjust the server settings accordingly - max
number of threads, concurrent users etc.
Immediate visualisation
Immediate visualisation
Test and visualisation process

Number of users Status code

List of servers User
Ramp-up Response time Diagram
Shell script JMeter R script

Various
measurements Diagram
Server-side R script
Case example
File storage and cache
File storage and cache
● Moodle requires
shared cache
Data properties
● Shared vs. Local
● Volatile vs. Non-Volatile
Local Strings, plugin data, config,
database meta
Volatile
Htmlpurifier, question data,
Shared
course contacts

Local Update Logs

Non-Volatile

Shared File-store, Temporary files

Baseline (single shared NFS)

NFS
Web server 1
HDD
- File store
Moodle
- Language strings
- Plugins data
- Config
- Htmlpurifier
Web server 2
- Question data
- Course contacts
- Database meta
Moodle
- Temporary files
- Update logs
Local volatile cache

NFS Web server 1

HDD Moodle
- File store Web server 2
- Htmlpurifier
- Question data APC
- Course contacts Moodle - Language
- Temporary files strings
- Update logs - Plugins data
APC - Config
- Language - Database meta
strings
- Plugins data
- Config
- Database meta
Shared volatile cache

Web server 1 Memcached

NFS
Moodle RAM
- Language
HDD strings
- File store Web server 2 - Plugins data
- Temporary files - Config
- Update logs - Database meta
Moodle - Htmlpurifier
- Question data
- Course contacts
Case example
Database
Database load

● Sessions
● User access rights
● Courses
● File meta data
● ...
Single instance
● Easy to set up
● ACID
● Single point of failure
● All load on a single server
● Vertical scaling
Cluster setup
● Redundancy (X-axis)
● Specialisation (sharding)
○ Y-axis
○ Z-axis
Scaling Open Source
● Open source code - not build with scaling in mind - will
only scale “prettily” if redesigned, but we can do it!
○ A lot can be achieved by configuration
● A different approach to contributing to open source!
○ We were imagining to be able to walk right in and
point out where moodle developers were wrong
○ However, we ended up not changing a single line of
moodle code
Questions?
Coffee & Networking