Reddit overload

Taking your site down gracefully when the traffic is just too much.

You work for Scoot Industries, LLC, who develops eco-friendly transportation. Your company has just released an affordable consumer car powered by vegetable oil, which gets 300 miles to the gallon and doesn’t pollute. It has the recommendation of the EPA, Greenpeace, the US Department of Transportation and Michael Moore. It truly is the greatest thing since sliced bread.

And because it’s so fantastic, it’s attracted the attention of news sites everywhere – which of course makes for a lot of traffic coming to your website. As a result, several news organizations have linked to your website, and your poor web server, running on an old desktop under your desk, is working its little heart out.

The problem is that your marketing department has placed several large movie files on the site, and they are being requested by just about everybody that’s coming to the site. Your office network is hitting capacity and connections are being refused when people try to hit the site.

Obviously, you don’t want to lose potential customers with a website that’s down, but you don’t want to be redesigning the site to remove access to the movies either. The secret is to use Apache’s mod_rewrite module, to redirect users requesting the videos to a page that explains your website is currently experiencing a heavy amount of traffic, and the videos are not available.

First, I’ve enabled bandwidth throttling through the mod_throttle module, which we discussed in Chapter 5. First, I enabled the module through the LoadModule directive. Next, in the main section, I added a throttling policy that limits the server to 10 requests and 100 kilobytes per second. This will help to alleviate some of the pressure on the server’s resources. I’ll likely keep this in place even after the heavy traffic has dissipated, just in case it happens again.

The rewrite rule is here for a temporary measure, however; I want to remove it once the traffic dies down and things get back to normal. As a result, I’ve placed the rewrite rules in their own section, so I can find it easily later.

The bulk of the rewrite configuration comes in the last three lines of the statement. First, you need to turn on the rewriting engine through the RewriteEngine directive. Next, you have to tell Apache what root URL the redirected file or directory lives in, using the RewriteBase directive. We covered the modules responsible for redirecting content in Chapter 3.

The last directive is the rewrite rule itself. For the source in this example, we’re using a wildcard to specify that all files within the /products/videos directory on the web server will be redirected to the ‘server busy’ page.

Example configuration
# Load basic Apache configuration
Include basic.conf
# Load Modules
Include modules.conf
LoadModule throttle_module /usr/lib/apache/
# Load Logs
Include logfiles.conf
ServerAdmin [email protected]
DocumentRoot /var/www/scootindustries/public
ErrorLog logs/scootindustries-error.log
CustomLog logs/scootindustries-access.log combined
<Directory />
Options Indexes SymLinksIfOwnerMatch MultiViews
    AllowOverride None
<Directory /htdocs/
    Options Indexes Includes FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all
    <IfModule throttle_module>
        ThrottlePolicy request 10 1s
        ThrottlePolicy speed 100 1s
# Redirect users from the videos to a “Sorry” page
<Directory /var/www/scootindustries/public/assets/videos/>
    Options Indexes Includes FollowSymLinks MultiViews ExecCGI
    AllowOverride All
    RewriteEngine on
    RewriteBase /products/scootbot/
    RewriteRule ^videos/.*