Scaling Servers with the Cloud: Amazon S3

Zach Dunn on November 10th, 2009 Posted in Tutorials

This is the first of two articles focused on using Amazon’s cloud services to boost your server’s performance and capabilities. Read the second part.

Do you live in fear of image size and sudden traffic spikes? With the power of Amazon’s web services, you’ll be able to give your server the added boost it needs to handle heavier loads and reduce load times.

The basic idea of S3 is to free up as much of your primary server’s resources as possible by outsourcing the small and frequent jobs to the cloud. More hands, less work.

This blog has experienced a change in performance after moving almost all of our static theme files to S3. There’s been a significant decrease in server resources and  load times. This has allowed us to handle larger surges in traffic without the usual slowdown at very low cost.

What is a Static File?

Static files are any site assets that remain largely unchanged. This includes images, videos, CSS, and Javascript files. On most sites, static files account for the largest percentage of server load. Reducing the weight of the static files on your primary server will cause everything else to run smoother, because the server is only dedicating itself to things like PHP and not dozens of HTTP requests for images.

Pair this with CSS sprites to lower the total number of HTTP requests, and your site’s load time will drop incredibly fast.

Pennies for Bandwidth

Have you hesitated before using higher quality images on a site/blog because of bandwidth concerns? Sounds like S3 would work well for you. S3, like most of the Amazon Web Services, is built on the “Pay for what you use” model. This means that you’ll be able to keep a much tighter control on both your resources and budget.

I’ve included a screen of rates for the United States storage in November 2009 as a baseline. International pricing is also available from the full description page.

To put this into perspective, it’s entirely possible to run a site integrated with S3 for under $10/month. If a site benefits from higher quality media (e.g. Hi-res photography), that’s money well spent. It’s more economical to put an extra $5 into cloud services than spend $100 more in monthly server costs.

Register an Account with AWS

Head on over to the Amazon Web Services and register an account. This will be free until you start to add and use services.

In order to work with Amazon’s services, you’ll need a compatible client. My personal favorite is the Mac FTP client Transmit, but the S3Fox plugin for Firefox is also popular and is used in this post’s screenshots.

Creating Your First Bucket

A bucket is Amazon’s term for a specific group of content, and is the place where all uploads live. The bucket name also appears in the main domain.

Sample Bucket: buildinternet.s3.amazonaws.com

Once you’ve successfully made an account on AWS and added the service, you’ll need credentials to access S3 directly. When logged in to the AWS dashboard, head to the “Security Credentials” page and scroll down to find your access key. This will be your login.

Take the credentials from this page and open up S3Fox. Select “Manage Accounts” and fill out the fields to continue. Keep in mind that the “Account Name” field is strictly for labeling purposes within the plugin.

Uploading an Image

You can put a file on S3 the same way that you would with a typical FTP connection. Once uploaded, you’ll have a slight moderation to make before being able to use the file.

Locked by Default

Any files uploaded to S3 are only accessible to the owner by default. This means that in order to reference files inside your S3 bucket, you’ll need to change the permissions. In S3Fox this is done by right clicking the bucket name, and selecting “Edit ACL” from the menu. Make sure that “World” is allowed to read the file:

Once you’ve got the permissions straightened out, you’re free to copy the path (Right-click file > Copy URL to Clipboard) and paste as needed.

Test the Results

Now that you’ve started the process of outsourcing HTTP requests to the cloud, it’s a good time to take a look at the results. Pingdom Tools is a great way to see the specific breakdown of file load times. Enter your URL, and then it will give you all the vital load statistics. Based on these results, you’ll be able to more accurately target the bandwidth hogs for special attention.

We’ll see you back here later this week as we use another one of Amazon’s cloud services to speed up delivery time for visitors all around the globe.

Server icon in banner from Nuvola icon set.