Stuck? Need startup advice?

Complete guide to serving your Rails assets over S3 with asset_sync

Fb613ae74d247c05eba250f575e2c9b0?size=150
by Michiel Sikkes on April 17, 2012 with comments

In the previous blog post I wrote about the first thing that made our blog super fast by using Key-based cache expiration.

The second thing I am going to write about is making your Rails app serve assets from S3 and using a CloudFront distribution to serve those files even faster to visitors based on their geographical location.

On top of that I made sure to circumvent the "maximum browser connection to the same host"-problem by configuring multiple subdomains on my own domain name to point to the CloundFront host.

Read on for the steps you can take to implement these things in your own production Rails app. I am basing these steps on an app that runs on Heroku. However all steps should also work for any hosting environment. If not, please let me know!

Synchronizing assets on deploy using the asset_sync gem

At the end of this section, we will have configured the asset_sync gem to precompile and synchronize all your assets to an S3 bucket when the Rake task rake assets:precompile is run.

First of all, add the asset_sync gem to your Gemfile like so:

gem "asset_sync"

and install it in your bundle:

bundle install

Now, to make syncing to your S3 bucket work on Heroku, you will need to add the following configuration environment variables to your Heroku app:

heroku config:add FOG_PROVIDER=AWS AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy

This will enable your app on Heroku to be able to connect to your AWS account.

Next, create the bucket where you want to synchronize the compiled assets to. For example, for our blog engine Beeblebrox, I am using the bucket beeblebroxblog. This will make my asset host become beeblebroxblog.s3.amazonaws.com.

You can create a bucket by logging into the AWS Management Console, clicking the "S3" tab and by clicking the "Create Bucket" button on the top section of the left sidebar.

Now, configure your Heroku app to use this bucket when generating the assets:

heroku config:add FOG_DIRECTORY=beeblebroxblog

Next, take a look at config/application.rb in your Rails app to make sure the following settings are enabled:

# Enable the asset pipeline
config.assets.enabled = true
config.assets.digest = true

Also, if you have more than one top-level manifest file under your stylesheets, make sure you also add them in config/application.rb or they will not be compiled and synchronized to S3 and they will break your app. This is the line we're using for our blogging engine:

config.assets.precompile += %w( blogs.css firmhouse.css inbound_marketing.css )

We've done a lot of stuff and we are halfway trough so lets see if everything works so far.

First, lets see if the assets do actually precompile and get uploaded to our S3 bucket when we run the right commands locally. To do that, we need to set the same configuration variables we set for the Heroku app locally:

export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=yyy
export FOG_DIRECTORY=beeblebroxblog
export FOG_PROVIDER=AWS

And run the rake task:

bundle exec rake assets:precompile

Now, if you see something like the output below, you know your assets are getting precompiled and uploaded to S3.

iMac-van-Michiel-Sikkes:beeblebrox michiel$ rake assets:precompile
/Users/michiel/.rvm/rubies/ruby-1.9.3-p0/bin/ruby /Users/michiel/.rvm/gems/ruby-1.9.3-p0@global/bin/rake assets:precompile:all RAILS_ENV=production RAILS_GROUPS=assets
AssetSync: using default configuration from built-in initializer
AssetSync: using default configuration from built-in initializer
AssetSync: Syncing.
Using: Directory Search of /Users/michiel/Code/beeblebrox/public/assets
Uploading: assets/application-34cec68e88b51431f93033c2d4d78bfb.css
Uploading: assets/application-34cec68e88b51431f93033c2d4d78bfb.css.gz
Uploading: assets/blogs-d639e608959aa7605de15076f25a40c1.css
Uploading: assets/blogs-d639e608959aa7605de15076f25a40c1.css.gz
Uploading: assets/blogs.css
Uploading: assets/blogs.css.gz
Uploading: assets/firmhouse-6e03d76faec40ab21e83a52a770a197c.css
Uploading: assets/firmhouse-6e03d76faec40ab21e83a52a770a197c.css.gz
Uploading: assets/firmhouse.css
Uploading: assets/firmhouse.css.gz
Uploading: assets/inbound_marketing-9fea544283eba75053027f4bc670eb22.css
Uploading: assets/inbound_marketing-9fea544283eba75053027f4bc670eb22.css.gz
Uploading: assets/inbound_marketing.css
Uploading: assets/inbound_marketing.css.gz
AssetSync: Done.

Before we head on to the CloudFront part, let's make sure serving the assets from S3 works on your live app. For this you'll need to modify config/environments/production.rb and make sure the line with the asset_host looks something like:

Beeblebrox::Application.configure do
  config.action_controller.asset_host = "http://beeblebroxblog.s3.amazonaws.com"
end

Done! Commit your changes and deploy the app on Heroku by pushing to the remote like you would do normally. The deploy task should also run rake assets:precompile which should synchronize all modified and new assets to the S3 bucket.

git commit -am "Added asset_sync gem and configure asset_host to serve from S3 bucket in production"
git push heroku master

Head on to your app in your browser and see if it works. If you have any questions or problems at this point, don't hesitate to get in touch at michiel [at] firmhouse [dot] com or by posting a comment. The rest of this post will depend on what you did so far.

Using CloudFront to geographically distribute assets to visitors

CloudFront is a service by Amazon to deliver content to your visitors. It lets you create a "distribution" based on an S3 bucket. If you point visitors to the automatically generated URL, the requests will automatically be routed to an edge location near your visitor.

So, let's start and create a CloudFront distribution from the S3 bucket we used in the previous section. You can do this very easily by using the "Create Distribution Wizard" under the "CloudFront" tab in your AWS Management Console.

On the first step, leave the Delivery Method to "Download" and select your S3 bucket from the dropdown. Click Continue.

Leave everything in the "Distribution Details" step the default and click Continue.

Check if everything is correct in the "Review" step and click "Create Distribution" to create the distribution.

In the CloudFront overview tab you should now see a new line with your new CloudFront distribution and Status "InProgress". Now, this can take a while to be finished. So, please go and grab a cup of coffee or tea and wait for the Status to turn to "Deployed".

We're almost there. When your CloudFront distribution is Deployed, copy and paste the distribution Domain Name from the AWS Management Console into the asset_host configuration option in config/environments/production.rb like so:

config.action_controller.asset_host = "http://d3m1ms3h9or4yp.cloudfront.net"

Commit & deploy and your assets will now be served even faster from your CloudFront distribution.

git commit -am "Serve assets from CloudFront in production"
git push heroku master

Serving the assets from your own subdomains for speed & looks

Next up is the final step in this guide. This section will show you how to set up your CloudFront distribution to be accessed from your own domain name so you can server assets from http://assets.yourapp.com instead of http://flubbeldubbelbubble.cloudfront.net. You'll also need to modify your own DNS records in this section.

First. In the CloudFront AWS Management Console, select your distribution and click the "Edit" button in the top bar. In the CNAMEs field, add the subdomains you want to be serving assets from. These are the CNAMEs we're using for this blog:

assets0.beeblebroxapp.com
assets1.beeblebroxapp.com
assets2.beeblebroxapp.com
assets3.beeblebroxapp.com

Click "Yes, Edit" and see the changes applied.

These CNAMEs will make the CloudFront accept request for the domain names you will be using in your app.

Now, configure the subdomains as CNAMEs to your CloudFront distribution URL in your own DNS panel. Apply there and wait for the DNS changes to be propagated. This might take some time depending on the TTL for your domains.

When you are certain the DNS changes have been applied, change the asset_host line in your config/environments/production.rb the following, replacing beeblebroxapp.com with your own domain name.

config.action_controller.asset_host = "http://assets%d.beeblebroxapp.com"

The %d in the asset_host string will automatically make Rails pick random asset hosts from 0 - 3 when generating URLs for your assets.

The advantage of this is that your browser will be able to open a lot more concurrent connections to your assets which makes loading all the images, CSS and JavaScripts for your pages super fast since the threads will not have to wait for each other. Especially on image-heavy pages, this might be a great thing.

Concluding

The asset_sync gem is awesome for automatically synchronizing your precompiled assets to an S3 bucket, so you can serve them from there. When you are using Heroku, the assets will be automatically precompiled and uploaded to S3 on every deploy.

By setting up a CloudFront distribution you can speed up serving of your assets even more because the request will be routed to a geographically close edge location to the user.

By making use of CNAMEs in your CloudFront distribution and multiple subdomains in your Rails app for serving assets you can speed up loading of your assets in most browsers.

I hope you liked reading trough the guide and please let me know if you have any questions or comments. Also, please show me for which apps you implemented it, I'd love to know!

Most of this guide has been based on this guide on the Heroku Devcenter and a few hours of Googling.

Need help?

Need help implementing this stuff in your own app or are you receiving weird errors? Please let me know! In the comments, @michiels on Twitter or michiel [at] firmhouse [dot] com.

comments powered by Disqus