Uploading Files to IBM Bluemix Object Storage: A Sample App

by Gastón RamosApril 8, 2016

The service from IBM can serve as an alternative to AWS S3.

Table of Contents

Saving objects on Bluemix

Imagine that you need to deploy an application to a PaaS and this application should allow for uploading files to the server. In such scenario, you cannot just use the local storage of the application container, because it is ephemeral.

This post shows you how to solve the problem. We will develop a simple application as an example and use IBM Bluemix Object Storage to host binary files.

The sample application is a form where a user can upload any binary file, a document, image, and so on under a selected category (a container). It also allows for listing files stored in a certain category: Music, Images, or Docs.

The source code for this post is available on GitHub.

An IBM Object Storage overview

IBM Object Storage is built upon Swift, an object/blob store from OpenStack. The architecture of the Swift API is very similar to Amazon S3: you have buckets and objects in S3, and in Swift, you have containers instead of buckets.

Here are some examples of the Swift API endpoints:

In fact, there is an endpoint to list all available endpoints: GET /v1/endpoints 🙂
Show container details and list objects: GET /v1/{account}/{container}
Get object content and metadata: GET /v1/{account}/{container}/{object}
Create or replace an object: PUT /v1/{account}/{container}/{object}
Delete an object: DELETE /v1/{account}/{container}/{object}

Creating the application

As I’ve mentioned before, the application will have two endpoints: one for uploading a file under a certain category and the other for listing files. Here are my mockups.

Uploading-files-to-IBM-Bluemix-Object-Storage-creating-the-app-v3

uploading-files-to-ibm-bluemix-storage-v3

Now, let’s start creating the basic files for the application. Similar to my previous post, I use Cuba. We need four files:

app.rb
config.ru
Gemfile
views/upload.mote

app.rb:

echo "require 'cuba'
require 'mote'
require 'mote/render'

Cuba.plugin Mote::Render

Cuba.define do

  on root do
    on get do
      render 'upload'
    end
  end
end" > app.rb

config.ru

echo "require './app'
run Cuba" > config.ru

Gemfile.rb

echo "source 'https://rubygems.org'
gem 'cuba'
gem 'mote'
gem 'mote-render'
gem 'pry'" > Gemfile.rb

views/upload.mote

mkdir views
echo '<h1>Upload Your File</h1>

<form method="post" enctype="multipart/form-data" action="/">
  <fieldset>
    <select>
      <option value="music">Music</option>
      <option value="images">Images</option>
      <option value="docs">Documents</option>
    </select>

    <input name="file" type="file">
    <br /><br />
    <button type="submit">Upload</button>
  </fieldset>
</form>' > views/upload.mote

Now, install the required gems:

bundle install

If you run the rackup command and point your browser at http://localhost:9292, you should see the upload form. It does nothing yet, so we will add a new endpoint to handle the POST request after a user clicks the upload button. Before doing this, add the Open Storage service through the Bluemix dashboard as described in Bluemix documentation.

Then, you can see your credentials in the Object Storage section on the Bluemix dashboard. With that data, test the service using curl:

curl -i \
-H "Content-Type: application/json" \
-d '
{
  "auth": {
    "identity": {
      "methods": ["password"],
        "password": {
          "user": {
            "name": "Admin_username",
            "domain": { "id": "domain" },
            "password": "password"
          }
        }
     },
     "scope": {
       "project": {
         "name": "project",
         "domain": { "id": "domainId" }
        }
      }
     }
}' \
https://identity.open.softlayer.com/v3/auth/tokens ; echo

You can find more examples in OpenStack Documentation.

Now, after we have tested the service and made sure it works, we are ready to interact with Object Storage. Let’s add the swift_client gem to Gemfile:

gem 'swift_client'

Then, create the containers in Bluemix from the dashboard by clicking Add container on the Action menu. Remember that the application will have three categories (containers): Music, Images, and Documents.

Adding the endpoint for uploading files

First, we have to connect to a Swift cluster. To do so, add the following module to your app.rb file:

module OStorage
  def self.client
    o_storage   = JSON.parse ENV['VCAP_SERVICES']
    credentials = o_storage['Object-Storage'].first['credentials']

    SwiftClient.new(
      :auth_url          => "#{ credentials['auth_url'] }/v3",
      :username          => credentials['username'],
      :password          => credentials['password'],
      :domain_id         => credentials['domainId'],
      :project_name      => credentials['project'],
      :project_domain_id =>  credentials['domainId'],
      :storage_url       => "https://dal.objectstorage.open.softlayer.com/v1/AUTH_#{ credentials['projectId'] }"
)
  end
end

Take into consideration two things that I found out when working on the module:

auth_url needs /v3 to be included.
storage_url has to be like the service documentation describes.

Second, to handle file uploading, we have to add a new endpoint to app.rb similar to the following:

on post do
    on root do
      OStorage.client.put_object(req['file'][:filename],
                                 req['file'][:tempfile], req['container'])
      res.redirect '/'
    end
  end
end

With this done, we are able to upload files into an Object Storage directory. We also want to list files under a certain directory. Let’s add a new route under get. The entire app.rb file is below:

require 'cuba'
require 'mote'
require 'mote/render'
require 'swift_client'
require 'json'

Cuba.plugin Mote::Render

module OStorage
  def self.client
    o_storage   = JSON.parse ENV['VCAP_SERVICES']
    credentials = o_storage['Object-Storage'].first['credentials']

    SwiftClient.new(
      :auth_url          => "#{ credentials['auth_url'] }/v3",
      :username          => credentials['username'],
      :password          => credentials['password'],
      :domain_id         => credentials['domainId'],
      :project_name      => credentials['project'],
      :project_domain_id =>  credentials['domainId'],
      :storage_url       => "https://dal.objectstorage.open.softlayer.com/v1/AUTH_#{ credentials['projectId'] }"
)
  end
end

Cuba.define do
  on get do
    on root do
      container = req.params['container'] || 'music'
      files = OStorage.client.get_objects(container).parsed_response

      render 'upload', container: container, files: files
    end
  end

  on post do
    on root do
      OStorage.client.put_object(req['file'][:filename],
                                 req['file'][:tempfile], req['container'])
      res.redirect '/'
    end

    on ':container/:filename' do |container, filename|
      OStorage.client.delete_object(filename, container)
      res.redirect '/'
    end

  end

end

The upload view:

<h1>Upload Your File</h1>
<form method="post" enctype="multipart/form-data"
      action="/" class="form-inline alert alert-info text-center">
  <label for="container">Container:</label>
  <select id="container" name="container">
    <option value="music">Music</option>
    <option value="images">Images</option>
    <option value="documents">Documents</option>
  </select>

  <label for="file">File:</label>
  <input name="file" type="file" class="form-control" id="file">
  <button type="submit" class="btn btn-primary">Upload</button>
</form>

<ul class="nav nav-pills nav-justified">
  <li role="presentation" class="active"><a href="/?container=music">Music</a></li>
  <li role="presentation"><a href="/?container=images">Images</a></li>
  <li role="presentation"><a href="/?container=documents">Documents</a></li>
</ul>

% files.each do |f|
  <a href="/file/{{ f['hash'] }}">
    {{ f['name'] }}
  </a>
  <form method="post" action="{{container}}/{{ f['name'] }}">
    <button type="submit">Delete</button>
  </form><br />
% end
<br/>

Now, deploy the application to Bluemix to see if everything is working properly:

cf push object-storage-example

If you have not pushed it before, you need to associate the Object Storage service with your application.

You can visit http://object-store-example.mybluemix.net/ to check that the application is working, right? 🙂

Conclusion

The Object Storage service worked well, and everything went as I expected. Creating a service for an application is fast because Bluemix provides credentials for the service and links the service to your application for you. My only concern is that the version of the Swift API is not specified in the Bluemix documentation. Doing so would let users know what version of the Swift API reference to read.

Anyway, you can figure this out by running the curl https://identity.open.softlayer.com/ command.