Now that we’ve seen how Each caching plugin; Cache Enabler, WP Rocket, WP Super Cache and W3 Total Cache line up with each other during our speed tests using Pingdom HERE, we also wanted to test how each caching plugin does under “stress” using the benchmark and load testing tool called Siege.
Siege is an http load testing and benchmarking utility. It was designed to let people measure their websites / code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP, HTTPS and FTP protocols. It lets its user hit a server with a configurable number of simulated clients. Those clients place the server “under siege.”
When you launch a new WordPress site, you want to have a good idea of how many actual visitors can visit your blog without your site crashing. Siege allows us to figure that out buy using a simple formula which we’ll use for each test. We’ll test each plugin to see just how much it can actually handle under minimal pressure and extreme pressure. When I say pressure, I mean by how many visits PER SECOND can they really hold. Of Course, it has allot to do with the amount of RAM and CPU Cores your environment or VPS has. Each of our test VPS has 768MB total Ram installed and 1 vCPU Core. We’ll also be upgrading each test VPS to use 1GB Ram with 1CPU Core, 2GB Ram with 1CPU Core, 2GB RAM with 2CPU Cores and 4GB Ram with 2CPU Cores. Our first test however, will be with 768MB Ram with 1CPU Core for each test VPS.
Each site is on a different VPS with the same everything. For this first test, we’ll use:
The formula we’ll use ( Example: -c50 -d10 -t3M ) for each test consists of 3 parts; The -c means how many concurrent simulated users/visits per second. The -d option sets a random interval in seconds between 0 and any number, In or case 10 seconds, that each “use/visitor” will sleep for between requests ( for reading and article on your blog/site ) The -t represents time / how long the test will contentiously run for. In our case, we’ll allow the test to run for a duration of 3 minutes. So putting the whole formumla together:
the Concurrent simulated users/visits will change from test to test. We’ll go all the way up to 250 per second but starting with just 50. We’ll also measure how much Load and RAM each VPS is using during each test. The Siege program has been installed on a separate VPS from all the other test VPP’s so we can get, as close to as possible, an accurate result for each test.
So let’s begin the fun!
- TEST FORMULA 1: $ siege -c50 -d10 -t3M http://wphstest1.xyz
- TEST FORMULA 2: $ siege -c100 -d10 -t3M http://wphstest1.xyz
- TEST FORMULA 3: $ siege -c200 -d10 -t3M http://wphstest1.xyz
- TEST FORMULA 4: $ siege -c250 -d10 -t3M http://wphstest1.xyz
TEST FORMULA 1: This test was conducted on the wphstest1.xyz testing site with no caching plugins activated and 50 concurrent users.
It is important to understand the test results shown above so I’ll explain them to you in “laymon’s” terms. In this test, it concluded that 1,555 HITS / active Users ( Transactions ) visited the site within a 3 minute ( 179.03 Sec. ) time frame ( Elapsed Time ), Each transaction / Visit had a 0.69 second ( response time ), 48.07MB of Bandwidth was used ( Data Transfer ) and all 1,555 was able to reach your site without server down errors ( Successful Transactions ) and the VPS was up 100% ( Availability ). The longest a visitor had to wait was 2.00 Seconds ( Longest Transaction ) and the shortest time to wait was 0.42 seconds ( Shortest Transaction ). The other parts of the tests don’t matter as much as the ones I explained to you here so you can just ignore those. Even though the CPU reached 100% several times, the VPS / site never failed.
This animated GIF seen below, was taken during the siege test which shows you the how much CPU was being used, the load average and how much RAM/Swap was being utilized.
TEST FORMULA 2: This test was conducted on the wphstest1.xyz testing site with no caching plugins activated and 100 concurrent users. We allowed the CPU / Load average to go down to normal between each test as a cool off period; about 5 minutes to get the best possible results for each test taken.
Test 2 Conclusion:
100 concurrent transactions / visits / users for 3 minutes. 2065 total visits | 100% VPS / Site availability | 63.83MB Bandwidth used
TEST FORMULA 3: This test was conducted on the wphstest1.xyz testing site with no caching plugins activated and 200 concurrent users. We allowed the CPU / Load average to go down to normal between each test as a cool off period; about 5 minutes to get the best possible results for each test taken.
Test 3 Conclusion:
200 concurrent transactions / visits / users for 3 minutes. 2115 total visits | 100% VPS / Site availability | 65.38MB Bandwidth used
TEST FORMULA 4: This test was conducted on the wphstest1.xyz testing site with no caching plugins activated and 250 concurrent users. We allowed the CPU / Load average to go down to normal between each test as a cool off period; about 5 minutes to get the best possible results for each test taken.
Test 3 Conclusion:
250 concurrent transactions / visits / users for 3 minutes. 2092 total visits | 100% VPS / Site availability | 64.67MB Bandwidth used
The tests above were conducted with no caching plugin using the Siege benchmark / load testing tool. I am a big fan of this tool because it gives us a great deal of critical information regarding max Visits a WordPress based site can handle per CPU/RAM with or without failing, Gives us an insight on how the CPU Core is reacting to the test and how much RAM is being utilized during the test. All of which are extremely important factors when shopping around for a host who can handle small to large spurts of traffic. Even though the CPU was maxed out and hammered for 3 solid minutes in all of the tests, nginx never failed which is a huge positive note in my book. Nginx is built to be able to withstand pressure ( large amount of visits ) and beatings for long periods of time without failing or keeping to a bare minimum.
This post is the first post of multiple series tests. This test was done with no caching plugin installed. Our next test, we will activate the Cache Enabler plugin and see how well it does under pressure. Will there be a difference in using a caching plugin -vs- not using one? Let’s find out!
Click here to go to the next test – Cache Enabler
If you like, you can skip to the different tests: