Spicy Moodlebox

The contents below contain partial results for the above titled research project. Please go through its associated research paper and video presentation for more information.

In this work, the performance of a web server running on a Raspberry Pi (RPI) having a 32-bit OS on an SD-Card was evaluated against one running a 64-bit OS with a Solid-State Disk (SSD). Moodle was selected as the testing application.

An existing project labelled Moodlebox by N. Martignoni and team converts Raspberry Pi into a portable Moodle with the capability to support approximately 30 users. Given the impact of this work can have on education in remote areas, it was important to find out if the number of users supported could be increased. By default, Moodle has some predefined test courses and JMETER performance tests that could be generated from the admin panel. These items shall help readers repeat same experiment and compare the results with the ones published here.

A version of Moodlebox was rebuilt for Ubuntu 64-bit OS and labelled as Spicy Moodlebox for this experiment. The experiment was conducted on Pi 32-bit and Ubuntu 64-bit on SD Card and SSD. The test environments were labelled pi-32-SD, pi-32-SSD, ubuntu-64-SD and ubuntu-64-SSD.

RPI version 4 with 4GB memory was used. The CPU speed was 1.5 Ghz. For some tests, the CPU was overclocked to 2.0 and represented below with the wording ‘-OV’ in the test environment label.

Even though the results below are for Moodle, it provides an insight on the performance of the Pi system. Same experiment may be repeated with other applications to evaluate performance of specific applications.

The Application performance Index (APDEX) was measured for different course sizes (small & Medium) against different cohort sizes (10 – 100 users). These were labelled in a specific way to portray the combinations. For example, xx represents an extra small course with 10 users, ss represents small course with a small cohort size (30 users), sm represents a short course with medium cohort size (100 users) and so on.   

Below are the APDEX results when the JMETER throughput was capped to 180/minute. Results from other throughput values were not satisfactory across the tests worth consideration.

Environment/APDEX

xx

ss

sm

m3

m4

m5

m6

m7

m8

m9

mm

pi-32-SD

0.891

0.791

0.659

0.606

0.423

0.398

0.383

0.378

0.371

0.365

0.373

pi-32-SSD

0.928

0.844

0.711

0.695

0.431

0.430

0.442

0.430

0.400

0.400

0.444

pi-32-SSD-OV

0.969

0.935

0.821

0.812

0.630

0.545

0.533

0.524

0.528

0.526

0.557

ubuntu-64-SD

0.919

0.824

0.729

0.707

0.494

0.453

0.465

0.485

0.469

0.482

0.502

ubuntu-64-SSD

0.906

0.853

0.749

0.721

0.537

0.487

0.476

0.473

0.472

0.483

0.498

ubuntu-64-SSD-OV

0.947

0.913

0.814

0.810

0.656

0.551

0.547

0.536

0.537

0.545

0.555

Table 1: APDEX values for different test run with a JMETER throughput of 180/minute.Higher scores are better.(Click on the links for detailed report).

For APDEX the Toleration treshold is set to 500 ms and the Frustration treshold is set to 1500 ms. APDEX values are interpreted and categorised as per the ranges below:

Index Range

Apdex Rating

Symbol

0.94 to 1.00

Excellent

E

0.85 to 0.93

Good

G

0.70 to 0.84

Fair

F

0.50 to 0.69

Poor

P

0.00 to 0.49

Unacceptable

U

Table 2: APDEX Rating