Scaling Servers with the Cloud: Amazon S3

Scaling Servers with the Cloud: Amazon S3

This is the first of two articles focused on using Amazon’s cloud services to boost your server’s performance and capabilities. Read the second part.

Do you live in fear of image size and sudden traffic spikes? With the power of Amazon’s web services, you’ll be able to give your server the added boost it needs to handle heavier loads and reduce load times.

The basic idea of S3 is to free up as much of your primary server’s resources as possible by outsourcing the small and frequent jobs to the cloud. More hands, less work.

This blog has experienced a change in performance after moving almost all of our static theme files to S3. There’s been a significant decrease in server resources and  load times. This has allowed us to handle larger surges in traffic without the usual slowdown at very low cost.

What is a Static File?

Static files are any site assets that remain largely unchanged. This includes images, videos, CSS, and Javascript files. On most sites, static files account for the largest percentage of server load. Reducing the weight of the static files on your primary server will cause everything else to run smoother, because the server is only dedicating itself to things like PHP and not dozens of HTTP requests for images.

Pair this with CSS sprites to lower the total number of HTTP requests, and your site’s load time will drop incredibly fast.

Pennies for Bandwidth

Have you hesitated before using higher quality images on a site/blog because of bandwidth concerns? Sounds like S3 would work well for you. S3, like most of the Amazon Web Services, is built on the “Pay for what you use” model. This means that you’ll be able to keep a much tighter control on both your resources and budget.

I’ve included a screen of rates for the United States storage in November 2009 as a baseline. International pricing is also available from the full description page.

To put this into perspective, it’s entirely possible to run a site integrated with S3 for under $10/month. If a site benefits from higher quality media (e.g. Hi-res photography), that’s money well spent. It’s more economical to put an extra $5 into cloud services than spend $100 more in monthly server costs.

Register an Account with AWS

Head on over to the Amazon Web Services and register an account. This will be free until you start to add and use services.

In order to work with Amazon’s services, you’ll need a compatible client. My personal favorite is the Mac FTP client Transmit, but the S3Fox plugin for Firefox is also popular and is used in this post’s screenshots.

Creating Your First Bucket

A bucket is Amazon’s term for a specific group of content, and is the place where all uploads live. The bucket name also appears in the main domain.

Sample Bucket: buildinternet.s3.amazonaws.com

Once you’ve successfully made an account on AWS and added the service, you’ll need credentials to access S3 directly. When logged in to the AWS dashboard, head to the “Security Credentials” page and scroll down to find your access key. This will be your login.

Take the credentials from this page and open up S3Fox. Select “Manage Accounts” and fill out the fields to continue. Keep in mind that the “Account Name” field is strictly for labeling purposes within the plugin.

Uploading an Image

You can put a file on S3 the same way that you would with a typical FTP connection. Once uploaded, you’ll have a slight moderation to make before being able to use the file.

Locked by Default

Any files uploaded to S3 are only accessible to the owner by default. This means that in order to reference files inside your S3 bucket, you’ll need to change the permissions. In S3Fox this is done by right clicking the bucket name, and selecting “Edit ACL” from the menu. Make sure that “World” is allowed to read the file:

Once you’ve got the permissions straightened out, you’re free to copy the path (Right-click file > Copy URL to Clipboard) and paste as needed.

Test the Results

Now that you’ve started the process of outsourcing HTTP requests to the cloud, it’s a good time to take a look at the results. Pingdom Tools is a great way to see the specific breakdown of file load times. Enter your URL, and then it will give you all the vital load statistics. Based on these results, you’ll be able to more accurately target the bandwidth hogs for special attention.

We’ll see you back here later this week as we use another one of Amazon’s cloud services to speed up delivery time for visitors all around the globe.

Server icon in banner from Nuvola icon set.

Posted Tuesday, November 10th, 2009 · Back to Top

SPONSOR

Add Comment

19 Comments 2 Mentions

  1. Andy Author Editor

    Every Amazon S3 related blog gives you an impression that there is only one Amazon S3 client available – S3 fox. However there are many more freeware clients which are much superior. For instance, S3HUB for Mac (http://s3hub.com) and CloudBerry Explorer for Windows (http://s3explorer.cloudberrylab.com/) .

    ·

  2. Kawsar Ali Author Editor

    Great info and detailed. Looking forward for next part.
    .-= Kawsar Ali´s last blog ..25 Interesting Glass Art and Sculpture Design =-.

    ·

  3. Saad Author Editor

    Will you please tell what is the problem in uploading images at Flickr?and then use them at your website?
    .-= Saad´s last blog ..Instant Online Web Messengers to Chat on MSN,Yahoo,AIM,Google,Facebook,MySpace,and ICQ =-.

    ·

  4. S3 Browser Team Author Editor

    Thanks for the article.

    If you will need an Amazon S3 Client for Windows – you can check S3 Browser – http://s3browser.com – fast, powerful and easy to use.
    .-= S3 Browser Team´s last blog ..S3 Browser Version 1.9.7 Released =-.

    ·

  5. joris_lucius Author Editor

    Very nice article, thanks for sharing!
    .-= joris_lucius´s last blog ..Lucius doneert site nationalekledingactie.nl =-.

    ·

  6. Greg Author Editor

    I was investigating this lately as well (not that my pitiful little site would benefit, but because I’m a curious type) and I also noticed Amazon’s “Cloudfront” service.

    As far as I can tell, the “Cloudfront” is like S3 (probably shares the S3 API) but also distributes content to edge servers; in other words, an on-demand “pay for what you use” CDN. Which would be even more useful for large media files when you have international traffic.

    Am I going crazy? Is Cloudfront not a better way to leverage S3 for web-hosting needs, rather than a straight-up S3 bucket?
    .-= Greg´s last blog ..Coles First Halloween =-.

    ·

  7. Jimbob Author Editor

    I would like to look into using the Amazon for hosting various assets eating up HTTP requests. Im just wondering would it be possible to utilise my own custom PHP CMS to upload the files across to the cloud in an automated way. Just don’t want this to be manually done as it would defeat the purpose if that makes sense. If it frees up the resources of my dedicated server for database and PHP I would be over the moon.

    ·

  8. Jannis Gerlinger Author Editor

    yes, nice article.
    I share it.

    ·

  9. Lee Author Editor

    To be honest I haven’t worked out the bandwidth we use on our sites, but I wrote a blog post a while back about EC2 and in reality how expensive it is

    http://www.blogfodder.co.uk/post/Wow-Amazon-EC2-Is-Expensive-For-Start-Ups.aspx

    Amazon do a great job of marketing these ‘cloud’ services – But when you actually sit down and do the maths they tend to work out a lot more expensive than say a 1&1 server/hosting with no bandwidth limits.

    ·

  10. Zach Dunn Author Editor

    @Greg

    CloudFront is an expansion (but not quite a replacement) to S3. Think of S3 like the main warehouse, and CloudFront as the delivery trucks.

    We’ll discuss the benefits of CDN’s more in the followup article.

    ·

  11. Eli Buenaventura Author Editor

    Thanks for this article, I’m now checking if our http://www.kalokohan.com website will benefit from this. Are there any other alternatives like this service? Akamai is a good solution too, but it’s quite pricey. Thanks!

    ·

  12. Shawn B. Author Editor

    I could be totally wrong about this, but from what I understand another big benefit of cloud computing is that people who are accessing your website from very far away, like if your server is based in Washington and the user is sitting in Florida, it can help with load times.

    Is this correct or am I confusing this with a different technology?

    ·

  13. Ian Author Editor

    Has anyone worked out an automated way to keep static file changes (or new files) synced with S3? e.g. you add some new images to your server and they’re on your bucket soon after without intervention?

    ·

  14. Jessica Author Editor

    Wow Really great article.. we are using Amazon for hosting http://www.lexiology.com .

    ·

  15. Dinheiro Na Internet Author Editor

    Hi Zach!

    Congratulations for the article!

    Yes, days will come when quality hosting and bandwidth will be totally free, as lots of services and devices already on the internet.

    I suppose the prices are lower right now, more than one year that this post was written.

    I will check them.

    Thanks for your insights, useful as always.

    ·

  16. Awais Author Editor

    I am not a real tech geek so dealing with server and hosting i always find difficult. I always find your posts as very user friendly. Please write a post on handling the VPS.

    ·

  17. Shiela Bergan Author Editor

    I visited a lot of website but I conceive this one has something extra in it. “Those who can command themselves command others.” by William Hazlitt.

    ·

  18. Bblythe Author Editor

    Many have suffered for talking, none ever suffered for keeping silence.

    ·

  19. Victor Author Editor

    Impressive. The article is great.

    ·

 

Build Internet by One Mighty Roar. Since 2008.