bandwidth conundrum

Jake posted on Tuesday, April 20th, 2010 | Blog, Tech

We keep our tech team way too busy for them to blog about the incredible work they’re doing every day, but I’m glad when they find moments to advise producers and stations on questions that come up. Here’s one from today that Robert and Andrew jumped on when a PRX producer with a WordPress site asked for advice on preparing for a potential big spike in traffic from being featured on a major national show.

The question:

Subject: Re: Bandwidth Conundrum
From: Ian Tue, Apr 20, 2010 at 2:19 PM
To: PRX

Hi Andrew and Robert,

Our current monthly bandwidth is pretty low, around 1.7GB out of an 1000GB limit. Our package includes 100GB of storage and we’re using about 57GB currently (the majority of our videos are hosted by Vimeo/Youtube and embedded to lessen our footprint.)

Our concern is the traffic spike we’re expecting with the launch of our new season, particularly when a segment of our show will air nationally.

Even though our usage is low at the moment with that type of exposure and all the content we have available for streaming (audio and video podcasts, short video documentaries, etc.), we’re afraid we could easily eat up our 1000GB (at which point it’s .50 per GB used.) In your experience does that seem likely or are we worried about the sky falling?

and the response:

Subject: Re: Bandwidth Conundrum
From: Andrew Tue, Apr 20, 2010 at 3:08 PM
To: Ian

I would like to be optimistic that you will have the kind of success that will require some changes to handle a large number of concurrent users.

It also is pretty easy to chew up bandwidth if you are serving media files, so I agree, there is a real possibility of using up your TB of bandwidth if you get a hit on your hands.

There are several things to worry about here.

One is stability of the system when lots of folks come to hit your site at once – what is usually called scalability.

WordPress takes more resources to serve than just static webpages, so if 100,000 people all came to your page, you would probably crash if this is a shared server, and maybe even crash a dedicated virtual server, or a small server.

The other problem with lots of users hitting at once is that sites and media serving can get much slower when there are lots of concurrent users. This is usually referred to as performance. You can have a site that scales to 1,000,000 users, but gets very slow, and that is no good either.

After these problems the next issue is bandwidth – assuming you can keep a site up under heavy load, you could conceivably burn through your allotted bandwidth and have to pay exorbitant prices for going beyond your limit, or you could be cut-off (depending on the service – most service would rather charge you for the overage and keep the site running).

Before anything else, you want to make sure you know if your site is getting clobbered.

If you haven’t already, you also should set-up monitoring using pingdom or montastic http://www.montastic.com/ so you know if and when your site is down. Your host may also already provide some tools for monitoring.

For improving both performance and scalability, caching is usually the first and best (and often last) answer.

There are several plugins for wordpress that are good at creating static versions of your pages, and therefore with very little stress to the server. This can dramatically improve performance and stability, and are a great first step. Several of these plugins, such as W3 Total Cache, can also be used in combination with offloading to a CDN for additional scalability/performance.
(SimpleCDN has amazing pricing compared to other CDNs (http://www.simplecdn.com/pricing), often as much as 1/4 the price. We use the upload bucket which is $0.039 per GB, as compared to $0.150 per GB from Amazon’s Cloudfront CDN. MaxCDN is another good lower cost CDN with pay as you go pricing http://www.maxcdn.com/pricing.php).

If it were me, I would go ahead and get caching setup first, and get signed up with a CDN and have that configured for off-loading in case I needed it, so that as you see your popularity climb, you have this ready at the time rather than scrambling when the demand hits.

CDNs provide both dramatically better performance and scalability. Off loading audio serving (as you have with video) will also make your site that much more scalable. Again, pay as you go is a nice way to do this, that way your cost only goes up (and not by nearly as much as your overage charges) when you have a ‘hit’, and in the meantime, you get the benefit of fast media serving.

Here are some good articles on this and other WordPress optimizations.

http://codex.wordpress.org/WordPress_Optimization/Caching
http://codex.wordpress.org/WordPress_Optimization
http://www.ghacks.net/2010/04/13/setting-up-a-content-delivery-network-in-wordpress/

A final note I guess I’ll mention is to keep audio you serve through the web as small as possible. Offering lower bitrate mp3 files, as low as 64kbps joint stereo for example, instead of 128 is another great way to lower bandwidth charges. I think you are using 96kbps now, but you may want to try out 64kbps, and see if how it sounds to you.

Hope that helps,

Andrew