Amazon S3 for Your Ruby-on-Rails App

by Nikolai SharangovichMarch 22, 2013

Learn how to quickly configure your Ruby-on-Rails application to store assets and uploads on Amazon S3.

With the increased application penetration into the clouds, users may face the issue of keeping data in a cloud storage. One of the most popular solutions in this area is Amazon Simple Storage Service (Amazon S3).

Table of Contents

Getting started with Amazon S3

First, we need to include support for Amazon S3 in the Amazon Web Services (AWS) account.

A home page the Amazon S3 account

Create a bucket for your project, but remember that a name must be unique across all Amazon S3 accounts. Then, you are able to manage access permissions to your bucket in the Properties tab.

Configuring bucket permissions in Amazon S3

After this, we can go to our Ruby-on-Rails 3.0 application and start configuring it to interact with Amazon S3. Most of Ruby-on-Rails 3.0 gems that support cloud storages use the fog library.

Configuring CarrierWave uploader

One of the popular uploaders—CarrierWave—has a description of how to store uploads on Amazon S3. You can initialize a storage type in the Uploader class.

<span class="k">class</span> <span class="nc">AvatarUploader</span> <span class="o">&lt;</span> <span class="no">CarrierWave</span><span class="o">::</span><span class="no">Uploader</span><span class="o">::</span><span class="no">Base</span>
  <span class="n">storage</span> <span class="ss">:fog</span>
<span class="k">end</span>

class AvatarUploader < CarrierWave::Uploader::Base

storage :fog

end

You can do the same in initializer.

<span class="c1">#config/initializers/carrier_wave.rb</span>
<span class="no">CarrierWave</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
  <span class="n">config</span><span class="p">.</span><span class="nf">storage</span> <span class="o">=</span> <span class="ss">:fog</span>
<span class="k">end</span>

#config/initializers/carrier_wave.rb

CarrierWave.configure do |config|

config.storage = :fog

end

Now, we have an issue with the development and test environments. We don’t need to use Amazon S3 in these environments. I suggest that we do a simple trick: put a storage type into your config file and use initializer.

<span class="c1">#config/config.yaml</span>
<span class="ss">development: </span><span class="o">&amp;</span><span class="n">development</span>
  <span class="ss">carrier_wave:
    storage: </span><span class="n">file</span>

<span class="ss">production:
  amazon:
    provider: </span><span class="no">AWS</span>
    <span class="ss">bucket: </span><span class="n">altoros</span><span class="o">-</span><span class="n">blog</span>
    <span class="ss">aws_access_key_id: </span><span class="p">\</span><span class="no">YOUR</span> <span class="no">KEY</span><span class="p">\</span>
    <span class="ss">aws_secret_access_key: </span><span class="p">\</span><span class="no">YOUR</span> <span class="no">SECRET</span><span class="p">\</span>
  <span class="ss">carrier_wave:
    storage: </span><span class="n">fog</span>

<span class="ss">test:
  </span><span class="o">&lt;&lt;</span><span class="p">:</span> <span class="o">*</span><span class="n">development</span>

#config/config.yaml

development: &development

carrier_wave:

storage: file

production:

amazon:

provider: AWS

bucket: altoros-blog

aws_access_key_id: \YOUR KEY\

aws_secret_access_key: \YOUR SECRET\

carrier_wave:

storage: fog

test:

<<: *development

<span class="c1">#config/initializers/carrier_wave.rb</span>
<span class="no">CarrierWave</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
  <span class="k">if</span> <span class="p">(</span><span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:carrier_wave</span><span class="p">][</span><span class="ss">:storage</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'fog'</span><span class="p">)</span> <span class="o">&amp;&amp;</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">]</span>
	  <span class="n">config</span><span class="p">.</span><span class="nf">fog_credentials</span> <span class="o">=</span> <span class="p">{</span>
	    <span class="ss">provider:              </span><span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:provider</span><span class="p">],</span>
	    <span class="ss">aws_access_key_id:     </span><span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:aws_access_key_id</span><span class="p">],</span>
	    <span class="ss">aws_secret_access_key: </span><span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:aws_secret_access_key</span><span class="p">]</span>

	    <span class="c1"># :region                 =&gt; 'eu-west-1'</span>
	    <span class="c1"># :host                   =&gt; 's3.example.com'</span>
	    <span class="c1"># :endpoint               =&gt; 'https://s3.example.com:8080'</span>
	  <span class="p">}</span>
	  <span class="n">config</span><span class="p">.</span><span class="nf">fog_directory</span>  <span class="o">=</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:bucket</span><span class="p">]</span>
	  <span class="n">config</span><span class="p">.</span><span class="nf">fog_public</span>     <span class="o">=</span> <span class="kp">false</span>
	  <span class="n">config</span><span class="p">.</span><span class="nf">storage</span>        <span class="o">=</span> <span class="ss">:fog</span>
  <span class="k">else</span>
    <span class="n">config</span><span class="p">.</span><span class="nf">storage</span>        <span class="o">=</span> <span class="ss">:file</span>
  <span class="k">end</span>
<span class="k">end</span>

#config/initializers/carrier_wave.rb

CarrierWave.configure do |config|

if (APP_CONFIG[:carrier_wave][:storage] == 'fog') && APP_CONFIG[:amazon]

config.fog_credentials = {

provider: APP_CONFIG[:amazon][:provider],

aws_access_key_id: APP_CONFIG[:amazon][:aws_access_key_id],

aws_secret_access_key: APP_CONFIG[:amazon][:aws_secret_access_key]

# :region => 'eu-west-1'

# :host => 's3.example.com'

# :endpoint => 'https://s3.example.com:8080'

}

config.fog_directory = APP_CONFIG[:amazon][:bucket]

config.fog_public = false

config.storage = :fog

else

config.storage = :file

end

Configuring assets

Another useful gem that will help us to store assets on Amazon S3 is Asset Sync. To configure this gem, we should tell it a path where we want to store assets.

<span class="c1">#config/environments/production.rb</span>
<span class="n">config</span><span class="p">.</span><span class="nf">action_controller</span><span class="p">.</span><span class="nf">asset_host</span> <span class="o">=</span> <span class="s2">"//</span><span class="si">#{</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:bucket</span><span class="p">]</span> <span class="si">}</span><span class="s2">.s3.amazonaws.com"</span>

#config/environments/production.rb

config.action_controller.asset_host = "//#{ APP_CONFIG[:amazon][:bucket] }.s3.amazonaws.com"

The default matcher for compiling files includes application.js, application.css, and all non-JavaScript/CSS files (i.e., .coffee and .scss files are not automatically included, as they compile to JavaScript/CSS).

To include specific files, you should put them to a precompile config.

<span class="c1">#config/environments/production.rb</span>
<span class="n">config</span><span class="p">.</span><span class="nf">assets</span><span class="p">.</span><span class="nf">precompile</span> <span class="o">+=</span> <span class="p">[</span><span class="s1">'admin.js'</span><span class="p">,</span> <span class="s1">'admin.css'</span><span class="p">,</span> <span class="s1">'common.js'</span><span class="p">,</span> <span class="s1">'common.css'</span><span class="p">]</span>

#config/environments/production.rb

config.assets.precompile += ['admin.js', 'admin.css', 'common.js', 'common.css']

To include all your asset files, you should put the following regular expression to config.

<span class="c1">#config/environments/production.rb</span>
<span class="n">config</span><span class="p">.</span><span class="nf">assets</span><span class="p">.</span><span class="nf">precompile</span> <span class="o">&lt;&lt;</span> <span class="sr">/(^[^_\/]|\/[^_])[^\/]*$/</span>

1 2	<span class="c1">#config/environments/production.rb</span> <span class="n">config</span><span class="p">.</span><span class="nf">assets</span><span class="p">.</span><span class="nf">precompile</span> <span class="o"><<</span> <span class="sr">/(^[^_\/]\|\/[^_])[^\/]*$/</span>

We should also configure asset_sync. We can generate a default config file by executing the Rake task.

<span class="go">rails g asset_sync:install --provider=AWS</span>

1	<span class="go">rails g asset_sync:install --provider=AWS</span>

Then, we should put the Amazon S3 config information in it.

<span class="c1">#config/initializers/asset_sync.rb</span>
<span class="k">if</span> <span class="k">defined?</span><span class="p">(</span><span class="no">AssetSync</span><span class="p">)</span>
  <span class="no">AssetSync</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
    <span class="n">config</span><span class="p">.</span><span class="nf">fog_provider</span>          <span class="o">=</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:provider</span><span class="p">]</span>
    <span class="n">config</span><span class="p">.</span><span class="nf">aws_access_key_id</span>     <span class="o">=</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:aws_access_key_id</span><span class="p">]</span>
    <span class="n">config</span><span class="p">.</span><span class="nf">aws_secret_access_key</span> <span class="o">=</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:aws_secret_access_key</span><span class="p">]</span>
    <span class="n">config</span><span class="p">.</span><span class="nf">fog_directory</span>         <span class="o">=</span> <span class="no">APP_CONFIG</span><span class="p">[</span><span class="ss">:amazon</span><span class="p">][</span><span class="ss">:bucket</span><span class="p">]</span>
    <span class="c1"># Increase upload performance by configuring your region</span>
    <span class="c1"># config.fog_region = 'eu-west-1'</span>
    <span class="c1">#</span>
    <span class="c1"># Don't delete files from the store</span>
    <span class="c1"># config.existing_remote_files = "keep"</span>
    <span class="c1">#</span>
    <span class="c1"># Automatically replace files with their equivalent gzip compressed version</span>
    <span class="c1"># config.gzip_compression = true</span>
    <span class="c1">#</span>
    <span class="c1"># Use the Rails generated 'manifest.yml' file to produce the list of files to</span>
    <span class="c1"># upload instead of searching the assets directory.</span>
    <span class="c1"># config.manifest = true</span>
    <span class="c1">#</span>
    <span class="c1"># Fail silently.  Useful for environments such as Heroku</span>
    <span class="c1"># config.fail_silently = true</span>
  <span class="k">end</span>
<span class="k">end</span>

#config/initializers/asset_sync.rb

if defined?(AssetSync)

AssetSync.configure do |config|

config.fog_provider = APP_CONFIG[:amazon][:provider]

config.aws_access_key_id = APP_CONFIG[:amazon][:aws_access_key_id]

config.aws_secret_access_key = APP_CONFIG[:amazon][:aws_secret_access_key]

# Increase upload performance by configuring your region

# config.fog_region = 'eu-west-1'

#

# Don't delete files from the store

# config.existing_remote_files = "keep"

#

# Automatically replace files with their equivalent gzip compressed version

# config.gzip_compression = true

#

# Use the Rails generated 'manifest.yml' file to produce the list of files to

# upload instead of searching the assets directory.

# config.manifest = true

#

# Fail silently. Useful for environments such as Heroku

# config.fail_silently = true

end

In addition, we can see the region option in both gems. It is highly recomended to configure it correctly to increase upload performance. Now, when we call assets:precompile, assets will be uploaded to a specified Amazon S3 bucket.

So, these steps should be enough to basically configure your app to work with Amazon S3.

About the author

Nikolai Sharangovich is an experienced Ruby on Rails developer with a deep understanding of the object-oriented design and modern software principles. He likes to collaborate with product people to achieve a maximum impact. Find him on GitHub.

Posted In

Amazon S3 for Your Ruby-on-Rails App

Getting started with Amazon S3

Configuring CarrierWave uploader

Configuring assets

Further reading

About the author

Contact Us