Cloud Foundry on OpenStack at IBM

by Volha KurylionakAugust 10, 2015

Animesh Singh speaks about IBM's experience with running and optimizing Cloud Foundry and OpenStack. He also provides a quick intro into the basic CF, OpenStack, and BOSH concepts.

Why read this?

FOR A CIO

Despite the fact that Cloud Foundry is a new technology, it has already been adopted by a number of giant companies. Watch the videos to learn why IBM uses Cloud Foundry, why they run it on top of OpenStack, and how they contribute to both projects.

FOR AN ARCHITECT

Without knowing the basics, it may be hard to grasp how Cloud Foundry and OpenStack work, let alone why they should be used together. Animesh Singh provides a concise intro into the architectures and main concepts of OpenStack, CF, and BOSH.

FOR A DEVOPS PRO

Deploying OpenStack and Cloud Foundry on top of it is a complex process that involves multiple steps and may take a lot of time. Learn how IBM’s engineers use Chef, Fog, and BOSH to go from bare metal to a full-blown CF deployment in just one hour.

FOR A DEVELOPER

Discover what developer experience looks like with Cloud Foundry running on top of OpenStack.

Table of Contents

How IBM has optimized Cloud Foundry and OpenStack

The OpenStack platform became the top open source cloud technology of 2014 (according to Crisp) for a reason. The numbers speak for themselves: OpenStack has grown from 52K to over 2M lines of code in just four years. Right now, it is supported by 461 companies, has over 18K of active members and 130K of commits.

Over the last 10 years, IBM’s strategy has been to move with the wave, gradually changing the corporate mindset and embracing the cloud. They have been investing heavily into cloud technologies. IBM is the #2 contributor to OpenStack. They are also using and contributing to Cloud Foundry, which is the basis for their own Bluemix PaaS.

ibm-contributions-to-openstack

A lot of work has been done to optimize and integrate Cloud Foundry and OpenStack. In particular, IBM has automated the complex process of deploying Cloud Foundry over OpenStack. Now, “you can go from bare metal to a running Cloud Foundry ready to push apps in an hour.” To make this possible, IBM used open source tools created by the Cloud Foundry and OpenStack communities.

Chef helps to deploy OpenStack in 15 minutes. Installing Cloud Foundry on top of it is a bit more challenging, because you have to gather and manually add a lot of information to the manifest. To automate this task, IBM uses Fog, a tool that not only can retrieve data necessary for the manifest, but also helps with setup. Finally, IBM uses BOSH and Ruby to automate Cloud Foundry deployment. As a result, it is possible to consistently replicate and recreate environments within an hour. Other optimizations include automated zero-downtime updates, improvements to the process of scaling OpenStack and Cloud Foundry/BOSH, etc.

The bottom line is that OpenStack and Cloud Foundry are a great fit for each other. Both these projects are 100% open source and are supported by vibrant communities. OpenStack can be configured/adapted to handle a PaaS, such as Cloud Foundry.

OpenStack + Cloud Foundry + BOSH

From an architect’s point of view, OpenStack can be described as a collection of well-integrated IaaS modules responsible for provisioning VMs, creating private/public, software-defined networks, storage, identity management, etc. Everything is tied together by the Horizon UI.

openstack-overview

Cloud Foundry is a platform-as-a-service designed to provide portability across cloud vendors. It’s the second 2014 OSS cloud technology on Crisp’s list with 1.3K of contributors and 800K lines of code. Any piece of software built for Cloud Foundry, including apps, tools, etc., can be easily migrated to another IaaS. CF also detects what runtimes are necessary for your software and can provision services to apps. The platform is supported by a large and active community of contributors and a governing body called the Cloud Foundry Foundation.

Developers can access Cloud Foundry via the CLI, an IDE, such as Eclipse, or a browser. Apps pushed to Cloud Foundry are handled by the Cloud Controller (CC) component. CC combines each app with an appropriate runtime, e.g., J2EE, if it’s a Java app, and deploys it in containers running on top of a dedicated pool of specialized VMs called DEAs. A deployed app can be accessed through a browser using the URL provided by the Router component.

cloud-foundry-architecture

Cloud Foundry ensures that the deployed apps stay available. The Health Manager component tracks app health, notifies the operator of any issues, and automatically deals with them, e.g. resurrects failed app instances. All the communication goes through the NATS messaging bus.

The BOSH tool chain provides the cloud provider interface (CPI) for deploying Cloud Foundry on OpenStack and other platforms, such as AWS, GCE, etc. Each new IaaS requires a separate CPI, in which certain methods are implemented.

Deploying Cloud Foundry with BOSH

Animesh Singh also describes the basics of deploying Cloud Foundry on OpenStack. He mentions some things that need to be in place before you can begin. These include:

support for static or floating IPs
persistent disks for storing blobs
security groups and segregated networks

Custom VM configuration sizes also need to be defined for different components before you attempt deploying CF to OpenStack.

bosh-deployment-overview

Animesh gives the general idea of how to install Cloud Foundry with BOSH. The main takeaway is that BOSH deploys software using:

releases (software packages)
stemcells (base OS images)
a manifest (a file where you determine what is going to be deployed and how)

A separate release needs to be created/obtained for each piece of software deployed by BOSH. Stemcells are base OS images that BOSH converts to different CF components. The deployment manifest combines all the information necessary for creating a Cloud Foundry cluster. Manifests may be very complex, exceeding 1K lines.

For more on OpenStack architecture, refer to this guide. For more details on Cloud Foundry and BOSH, see CF documentation and BOSH documentation.

Altoros Take

Many consider OpenStack to be one of the best infrastructure choices for Cloud Foundry, but there are some other options too. At the moment, there are officially supported BOSH Cloud Provider Interfaces (CPIs) for Amazon Web Services, vCloud, vSphere, and Warden. Community supported CPIs exist for Microsoft Azure, CloudStack, and Google Compute Engine (GCE).

When choosing an infrastructure for Cloud Foundry, the first thing to consider is the SLA, which can differ depending on the provider. Other considerations include compliance with management qualifications, server location, pricing, customer satisfaction studies, etc.

Finally, it is a good idea to perform independent benchmarking of the infrastructure before committing to it. You can find more advice on how to choose the right IaaS for Cloud Foundry in the Architect’s Guide to Implementing CF.

Want details? Watch the video

Table of contents

How IBM contributes to OpenStack and CF. (8:26)
How IBM has optimized OpenStack and CF deployment. (10:12)
How IBM has optimized scaling of OpenStack(14:33)
How IBM has optimized scaling of CF (15:48)
Tips on optimizing networking (16:33)
Why OpenStack and Cloud Foundry are a great match for each other. (17:21)

https://www.youtube.com/watch?v=CmcfXWSKlQA

About the speaker

Animesh Singh is Senior Lead Cloud Architect and Strategist for IMB Cloud Labs. Under his supervision, IBM is now creating, designing, and implementing its strategy for Cloud Foundry and OpenStack-based solutions. He is leading cutting edge projects of an enterprise level around cloud and virtualization technologies and in telecommunication, banking, and healthcare industries. Animesh is a frequent and renowned speaker at Cloud Foundry conferences and meetups of different scale.