I used misc/load_testing/benchmark_staff.pl
for this load test. My sample data is ~980 biblios and 50 patrons. I tried Koha standard and plack version to measure the performance gain introduced by plack. As said before, the device has a 700MHz ARM11 CPU and 224MB available RAM.
I got some suspicious numbers and so I also tried swap off and swap on (USB stick, 8GB).
Swap off
Standard (no plack) benchmarks (swap off)
————–
Koha STAFF benchmarking utility
————–
Benchmarking with 20 occurences of each operation and 30 concurrent sessions
Step 1: staff client main page | 406251ms | 0.049 pages/sec |
Step 2: catalog detail page | 454814ms | 0.043 biblios/sec |
Step 3: catalogue search | 421513ms | 0.047 biblios/sec |
Step 4: patron detail page | 406234ms | 0.049 borrowers/sec |
Step 5: patron search page | 807744ms | 0.049 borrowers/sec |
Step 6a circulation (checkouts) | 405977ms | 0.049 checkouts/sec |
Step 6b circulation (checkins) | 439515ms | 0.045 checkins/sec |
all transactions at once | 1491201ms | 0.12 operations/sec |
Plack benchmarks (swap off)
————–
Koha STAFF benchmarking utility
————–
Benchmarking with 20 occurences of each operation and 30 concurrent sessions
Step 1: staff client main page | 35817ms | 0.558 pages/sec |
Step 2: catalog detail page | 40279ms | 0.496 biblios/sec |
Step 3: catalogue search | 40394ms | 0.495 biblios/sec |
Step 4: patron detail page | 38110ms | 0.524 borrowers/sec |
Step 5: patron search page | 73556ms | 0.543 borrowers/sec |
Step 6a circulation (checkouts) | 40499ms | 0.493 checkouts/sec |
Step 6b circulation (checkins) | 39898ms | 0.501 checkins/sec |
all transactions at once | 342319ms | 0.525 operations/sec |
Performance gain with Plack
Step 1: staff client main page | 11,34x |
Step 2: catalog detail page | 11,29x |
Step 3: catalogue search | 10,44x |
Step 4: patron detail page | 10,66x |
Step 5: patron search page | 10,98x |
Step 6a circulation (checkouts) | 10,02x |
Step 6b circulation (checkins) | 11,02x |
all transactions at once | 4,36x |
Plack gives an enormous performance boost on this little machine, it is more than ten times faster for each single test. Only the „all transactions at once“ gain is clearly less than that, with a little more than four times the speed of the test without plack. At first I thought that would have to do with more CPU and less RAM needs, but that did not really make sense, even less as it is clearly visible that the last number for no-plack is off: 0,12 pages instead of about 0,05 like in all other tests. Why would Apache suddenly perform better under heavy load? The plack test shows a steady ~0,5 throughout all results.
I ran the tests again to see if the RAM is full at some point. That is not the case for plack, but it seems to be the case for the standard Apache test!
So here are the same tests run with swap enabled.
Swap on
Standard (no plack) benchmarks (swap on)
————–
Koha STAFF benchmarking utility
————–
Benchmarking with 20 occurences of each operation and 30 concurrent sessions
Step 1: staff client main page | 403587ms | 0.049 pages/sec |
Step 2: catalog detail page | 438546ms | 0.045 biblios/sec |
Step 3: catalogue search | 423142ms | 0.047 biblios/sec |
Step 4: patron detail page | 403544ms | 0.049 borrowers/sec |
Step 5: patron search page | 840606ms | 0.047 borrowers/sec |
Step 6a circulation (checkouts) | 407351ms | 0.049 checkouts/sec |
Step 6b circulation (checkins) | 426596ms | 0.046 checkins/sec |
all transactions at once | 12442890ms | 0.014 operations/sec |
Plack benchmarks (swap on)
————–
Koha STAFF benchmarking utility
————–
Benchmarking with 20 occurences of each operation and 30 concurrent sessions
Step 1: staff client main page | 43219ms | 0.462 pages/sec |
Step 2: catalog detail page | 38193ms | 0.523 biblios/sec |
Step 3: catalogue search | 37038ms | 0.539 biblios/sec |
Step 4: patron detail page | 38547ms | 0.518 borrowers/sec |
Step 5: patron search page | 75188ms | 0.531 borrowers/sec |
Step 6a circulation (checkouts) | 41767ms | 0.478 checkouts/sec |
Step 6b circulation (checkins) | 39637ms | 0.504 checkins/sec |
all transactions at once | 344414ms | 0.522 operations/sec |
Performance gain with Plack
Step 1: staff client main page | 9,33x |
Step 2: catalog detail page | 11,48x |
Step 3: catalogue search | 11,42x |
Step 4: patron detail page | 10,47x |
Step 5: patron search page | 11,18x |
Step 6a circulation (checkouts) | 9,75x |
Step 6b circulation (checkins) | 10,76x |
all transactions at once | 361,28x |
In the »all transaction at once« test without plack, we run deeply into swap and get lost there for hours. After about 180 of the almost 210 minutes the test took, top said
Mem: 223348k total, 214792k used, 8556k free, 68k buffers
Swap: 7807552k total, 317044k used, 7490508k free, 3628k cached
I see this as a clear indication that the reason for the good result in the first test was running out of memory and skipping parts of the test, not actually better performance. With plack however we get the same steady results as before and don’t run into swap during the test at all.
Plack gives an amazing performance boost of around ten times the speed of the regular installation on this machine.