Olino: Throughout The Peak As Already Seen In The Graph Above The Response Time Went Sky-High

 slashdot effect Mindy and I became friends.

She healed children.

Mindy was BRIGHT. Brilliant. So, not only smart, bright -LIGHT. She was a pediatrician. We talked about how life felt like hell and heaven with not much in between. Mindy and I wrote to one another often. She was a mama to Lana and a wife to Jordan and a chemosabe to Meganand a buddy to everyone else she met. We wrote about illness and healing and pain and parenthood and friendship and marriage and God and how all these things were so completely impossible and miraculous. The overall amount of visitors kept growing any hour.

This translates back to around 39 visitors any minute.

 slashdot effect The amount of visitors per hour went slowly down with a few hundred visitors less any hour until it stabilised at 11 dot 00PM / 05 dot 00PM the server load was already increased to 27, a clear indication that the server is already overloaded, after the peak.

At the peak 03 dot 25PM / 09 dot 25AM, the OliNo site received 2320 visitors per hour.

At 12 dot 00PM / 06 dot 00AM. It’s an interesting fact that the second tip to use a Content Distribution Network is something Amazon can offer with CloudFront. Fact, content is delivered with better possible performance, requests for your objects are automatically routed to the nearest edge location. Anyway, it should also result in a faster loading website for the visitor since the content to load is located more closely to the physical location of the visitor.

 slashdot effect CSS file and theme images are loaded on the CDN this will offload a bunch of HTTP request from the main web server, when the javascript file.

Some digging on the internet shows that using Amazon Cloudfront will double the bandwidth costs for your website.

Amazon CloudFront delivers your content using a global network of edge locations. Throughout the day way more visitors came from different parts of the world.a bunch of visitors came still from the USA. In the evening we have received 27169 visitors from 134 countries/territories. So, our Apache thread uses between the 16 20MB per thread.

 slashdot effect I know that the value ain’t ain’t shown.

Setting this value Besides, the tip of Apache is to divide the available quantity of memory for Apache by the percentage of memory used in Top by a single Apache thread. Notice, it seems we need to reduce the tal number of MaxClients from 150 to 50 for our Apache configuration. Instance is running in the useast1b datacenter of Amazon. With all that said… One EC2 Compute Unit provides the equivalent CPU capacity of a ’02’ GHz 2007 Opteron or 2007 Xeon processor.

OliNo website is using a small instance type. Therefore this instance has 7 memory GB, 1 EC2 Compute Unit, 160 GB of instance storage, ’32 bit’ platform. There’s a perfect document about Apache Performance Tuning on the website of Apache. Besides, the current setting is Setting this value must result in error in the log. Among the first tips is to check the MaxClients option. I’m sure you heard about this. So it is a first step we can take. Our Apache web server ain’t tuned for performance. Often loads of readers will hit the link to read the story, when Slashdot links a site. Doesn’t it sound familiar? The Slashdot website serves around 3 million pages on weekdays, and slightly less on weekends. Whenever causing the site to buckle under the strain, when all those Slashdot readers start crashing the party, it can saturate a smaller website completely. So, this can easily throw thousands of hits at the site in minutes.

I am sure that the site is said to be Slashdotted, when this happens.

It is clearly a proven set up.

All the web server instances connect with a single MySQL database so mostly there’s a single state of the website. For example, this load balancing set up is similar how Slashdot has setup their hardware. On p of this, the loadbalancer will balance the load over the running webserver instances. You should take it into account. The first thing you read about improving the performance of WordPress is to use caching. We use WordPress as blogging software. We use the WP Super Cache plugin.

I had already done tests with it and it seems to improve performance significantly when reloading similar page multiple times.

We never had tested it under Slashdot effect conditions, where thousands of users simultaneous need to access really similar webpage on the website.

Theoretically this cache module will handle this smoothly. Needless to say, perform your load test, find the bottlenecks, try to solve them by adjusting the set up, do your performance tests again. Essentially, the bottleneck can be CPU, IO or network. Nevertheless, repeat this process until you are satisfied with the performance.

We are clearly guessing now, as you can see.

So there’re a couple of options which can be tuned like the Apache webserver, the MySQL database or even the Linux kernel.

Therefore this process is iterative. Only one way to find the real bottleneck your favorite performance tests on an identical non production set up. During these tests the load on the server can be controlled and the server can be monitored closely. I thought I should leave my first comment. I will visit this blog very often. For example, I don’t know what to say except that I have enjoyed reading. Anyways, nice blog. I recently came across your blog and was reading along. With that said, this will not prevent an overload of the server since the Slashdot effect.

I guess this will only work for ‘revisiting’ users. I am sure that the remaining tips like adding Expires header and configure ETags are both cache related. With some having their own Javascript files, so this would mean we need to do plenty of customizing, as we use loads of WordPress plugins. I am wondering if that’s worth the effort with the new WordPress 8 already announced where a bunch of there problems will be solved automatically. So, ‘autoscaling’ will stop a lot of the running instances, when the load drops again as the visitors are gone. Now look. Gether with the announcement of Amazon for their loadbalancer solution they announced a feature called autoscaling. Therefore this could be very effective for surviving the slashdot effect. Now pay attention please. Now this makes sense, as long as with Amazon EC2, you only pay for running instances per hour.

I know that the loadbalancer will balance the load over the running web server instances.

This makes it possible to start / stop instances on the basis of the load of a bunch of instances using a voting mechanism.

Auto scaling could startup more instances to handle the increased load, when there’s a huge peak in visitors. In the course of the peak we got 2320 visitors per hour. Loads of info can be found by going online. This calculates back to 590 KB/s of continuous traffic. Basically, with 914 KB, the webpage is quite large. Consequently, this shouldn’t be the bottleneck. You see, this generates 914 KB x 2320 = 2 GB per hour of data traffic. We can save few processor cycles by disabling it.

In our case we do not use the access log.

This can take up valuable time it’s just a case of writing the information,, albeit Apache keeps the log files open.

Another option we can use it so disable access logging. Writing log information is a time consuming process. Error logging will still be enabled. We simply comment out the log lines in the configuration file, with the intention to do this. Therefore this gives a clear indication that the web site needs a bunch of tweaking to get a better performance. Amid the first things we did is to analyse the website with the very nice Firefox plugin YSlow. Improvements suggestions are sorted on most impact on top. Tip given we have got to combine as much Javascript files and Stylesheets into both single files. It’s also suggested to use minified version for the Javascript files.

Now look, the webpage has 14 external Javascript files, 6 external CSS StyleSheets and 24 CSS background images.

Preferably it’s a good idea to have only one Javascript and one CSS file.

Tip to minimize HTTP requests, is the first we should tackle. Basically, so that’s a cool article and I have posted a link to my twitter account. On p of that, you can check my very own ol CloudBerry Explorer that helps to manage S3 and CloudFront. I will also refer my customers to your AWS experience. Seriously. I know it’s a freeware. We found an appropriate name for our organisation, OliNo, that is a word joke for No Oil! Our organisation will contribute to reach this goal. Of course let start a civilization which provides for its energy using renewable energy. Surely it’s a combination of two Catalan words.Oli and No. Gether they mean Oil No! I changed some MySQL settings. Prior the slashdot effect, the mysql database settings where optimized with the MySQL Performance Tuning Primer script.

See also the results of a recent performance analysis and the current MySQL settings.

a notification is send when the server is unavailable.

With an interval of 5 minutes the response time of the website is measured from the different locations.

Response time of the website is monitored by a global monitoring network service of Pingdom. They have probe servers which are located in the USA, Canada, UK, Sweden and France. OliNo website is hosted in the Amazon Elastic Computer Cloud. To be able to handle a bigger load on the webserver one could start using multiple instances. Until the time comes when Amazon will offer a load balancing service in their EC2 environment, people are forced to use a ‘softwarebased’ load balancing solution.

Did you know that the web traffic could be load balanced over the two web servers.

Effectively quadrupling the current cost price of using a single EC2 instance.

Therefore this effectively means we need at least 4 small EC2 instances. Actually the next step will be running two small instances with the apache webserver with wordpress and a third instance with MySQL. Let me ask you something. Is it faster to keep them on the they are already in the browsers cache of the visitor because Google servers? We use Google Javascripts for showing Ads with Google Adsense and monitoring traffic using Google Analytics. Is it allowed and does it work?

What about Javascript from Google? Does it make sense, to take these scripts and combine them with your personal Javascripts to improve performance? I know that the server load snapshot taken at 08 dot 35AM is a clear indicator the server is overloaded. Looking at the details reveals that user processes is only taking 30 CPU usage and the processor is still 16 dot 4 idle. Whenever throughout the peak the response time went sky high, as already seen in the graph above. With all that said… So this happened at exactly 12 dot 00PM. Explanation may be that at identical time, the Europeans take a lunch break and the east coast of the USA awakens. Result is the perfect storm completely overloading the web server. There is a lot more info about this stuff on this website. The impact of the Slashdot effect is clearly visible in the response time of the web server.

Leave a Comment