How to Deploy Cloud Foundry v2 to AWS by Using Vagrant
Recently, we published an article on the Cloud Foundry blog in which we explained how to install Cloud Foundry with Vagrant. Although BOSH is suggested as the official method of setting up a system, the way described in the article is easier and faster. This blog post found on the ActiveState blog adds some more details to the subject. Don’t skip the comments made by our Argentinian team, in which we suggest the ways of automating some installation tasks.
Read the full article “How to Deploy Cloud Foundry v2 to AWS via Vagrant” to learn the details.
In this post, I’m going to quickly run through how I got up and running with Cloud Foundry v2. These notes are based on my colleague’s instructions, who is in the process of giving Cloud Foundry v2’s tires a good kicking.
The easiest way to deploy Cloud Foundry version 2 (a.k.a “ng” or “next generation”) seems to be via Vagrant. The official way is via BOSH, but we have created a method which makes it much easier to spin up a single instance of Cloud Foundry v2 on Amazon EC2. We found with BOSH we needed 14 instances to get up and running and it took much longer.
Install the installer
You start by git-cloning the cf-vagrant-installer
repository from GitHub.
$ git clone https://github.com/Altoros/cf-vagrant-installer $ cd cf-vagrant-installer $ cat README.md
As you will see in the README.md, there are a few vagrant dependencies, the first of which is Vagrant itself.
Install Vagrant
If you do not have Vagrant installed, you can install it from http://downloads.vagrantup.com/. I installed the .dmg for my Mac, which was pretty straightforward.
Install Vagrant plug-ins
The Vagrant plug-ins required (if they have not changed) were vagrant-berkshelf
, which adds Berkshelf integration to the Chef provisioners, vagrant-omnibus
, which ensures the desired version of Chef is installed via the platform-specific Omnibus packages, and vagrant-aws
, which adds an AWS provider to Vagrant, allowing Vagrant to control and provision machines in EC2.
Installation of these plug-ins could not be simpler:
$ vagrant plugin install vagrant-berkshelf $ vagrant plugin install vagrant-omnibus $ vagrant plugin install vagrant-aws
Run the bootstrap
Next, make sure we are in the cf-vagrant-installer
(which we cloned above) directory and run the rake command to download all the Cloud Foundry components.
$ rake host:bootstrap
The output of this rake command will look something like this:
(in /Users/phil/src/cfv2/cf-vagrant-installer) ==> Init Git submodules Submodule 'cloud_controller_ng' (https://github.com/cloudfoundry/cloud_controller_ng.git) registered for path 'cloud_controller_ng' Submodule 'dea_ng' (https://github.com/cloudfoundry-attic/dea_ng.git) registered for path 'dea_ng' Submodule 'gorouter' (https://github.com/cloudfoundry/gorouter.git) registered for path 'gorouter' Submodule 'health_manager' (https://github.com/cloudfoundry/health_manager.git) registered for path 'health_manager' Submodule 'uaa' (https://github.com/cloudfoundry/uaa.git) registered for path 'uaa' Submodule 'warden' (git://github.com/cloudfoundry/warden.git) registered for path 'warden' Cloning into 'cloud_controller_ng'... remote: Counting objects: 13057, done. remote: Compressing objects: 100% (7357/7357), done. remote: Total 13057 (delta 7851), reused 10513 (delta 5512) Receiving objects: 100% (13057/13057), 4.07 MiB | 1.34 MiB/s, done. Resolving deltas: 100% (7851/7851), done. Submodule path 'cloud_controller_ng': checked out '4b9208900c54181d539c9cc93519277d7c2702b5' Submodule 'vendor/errors' (https://github.com/cloudfoundry/errors.git) registered for path 'vendor/errors' Cloning into 'vendor/errors'... remote: Counting objects: 58, done. remote: Compressing objects: 100% (45/45), done. ... (truncated) ...
Set up AWS credentials
Next, you will need to edit the Vagrantfile
:
$ vim Vagrantfile
Add the following section directly above the config.vm.provider :vmware_fusion
line:
config.vm.provider :aws do |aws, override| override.vm.box_url = "http://files.vagrantup.com/precise64.box" aws.access_key_id = "YOUR AWS ACCESS KEY" aws.secret_access_key = "YOUR AWS SECRET KEY" aws.keypair_name = "YOUR AWS KEYPAIR NAME" aws.ami = "ami-23d9a94a" aws.instance_type = "m1.large" aws.region = "us-east-1" aws.security_groups = ["open"] aws.user_data = File.read('ec2-setup.sh') override.ssh.username = "vagrant" override.ssh.private_key_path = "THE LOCAL PATH TO YOUR AWS PRIVATE KEY" end
Then replace "YOUR AWS ACCESS KEY"
, "YOUR AWS SECRET KEY"
, and "YOUR AWS KEYPAIR NAME"
with your own AWS credentials.
An open security group
The AWS security group used in the above example is one called “open.” This is just one with all open ports. You will need to create it if you do not have it already. You can do this through the AWS console.
Create an EC2 set-up script
Next, you’ll need to create an ec2-setup.sh
file directly in the cf-vagrant-installer
directory. It should look exactly like the following:
#!/bin/bash -ex usermod -l vagrant ubuntu groupmod -n vagrant ubuntu usermod -d /home/vagrant -m vagrant mv /etc/sudoers.d/90-cloudimg-ubuntu /etc/sudoers.d/90-cloudimg-vagrant perl -pi -e "s/ubuntu/vagrant/g;" /etc/sudoers.d/90-cloudimg-vagrant
Build the EC2 instance running CFv2
Finally, run "vagrant up --provider=aws"
and your instance will be built:
$ vagrant up --provider=aws
My (truncated) output looked something like this:
Bringing machine 'cf-install' up with 'aws' provider... [Berkshelf] Updating Vagrant's berkshelf: '/Users/phil/.berkshelf/cf-install/vagrant/berkshelf-20130717-81754-5vjx63-cf-install' [Berkshelf] Using apt (1.10.0) [Berkshelf] Using git (2.5.2) [Berkshelf] Using sqlite (1.0.0) [Berkshelf] Using mysql (3.0.2) [Berkshelf] Using postgresql (3.0.2) [Berkshelf] Using chef-golang (1.0.1) [Berkshelf] Using java (1.12.0) [Berkshelf] Using ruby_build (0.8.0) [Berkshelf] Installing rbenv (0.7.3) from git: 'git://github.com/fnichol/chef-rbenv.git' with branch: 'master' at ref: 'e10f98d5fd07bdb8d212ebf42160b65c39036b90' [Berkshelf] Using rbenv-alias (0.0.0) at './chef/rbenv-alias' [Berkshelf] Using rbenv-sudo (0.0.1) at './chef/rbenv-sudo' [Berkshelf] Using cloudfoundry (0.0.0) at './chef/cloudfoundry' [Berkshelf] Using dmg (1.1.0) [Berkshelf] Using build-essential (1.4.0) [Berkshelf] Using yum (2.3.0) [Berkshelf] Using windows (1.10.0) [Berkshelf] Using chef_handler (1.1.4) [Berkshelf] Using runit (1.1.6) [Berkshelf] Using openssl (1.0.2) [cf-install] Warning! The AWS provider doesn't support any of the Vagrant high-level network configurations (`config.vm.network`). They will be silently ignored. [cf-install] Launching an instance with the following settings... [cf-install] -- Type: m1.large [cf-install] -- AMI: ami-23d9a94a [cf-install] -- Region: us-east-1 [cf-install] -- Security Groups: ["open"] [cf-install] Waiting for instance to become "ready"... [cf-install] Waiting for SSH to become available... [cf-install] Machine is booted and ready for use! [cf-install] Rsyncing folder: /Users/phil/src/cfv2/cf-vagrant-installer/ => /vagrant [cf-install] Rsyncing folder: /Users/phil/.berkshelf/cf-install/vagrant/berkshelf-20130717-81754-5vjx63-cf-install/ => /tmp/vagrant-chef-1/chef-solo-1/cookbooks [cf-install] Installing Chef 11.4.0 Omnibus package... [cf-install] Running provisioner: chef_solo... Generating chef JSON and uploading... Running chef-solo... stdin: is not a tty [2013-07-17T19:43:22+00:00] INFO: *** Chef 11.4.0 *** [2013-07-17T19:43:23+00:00] INFO: Setting the run_list to ["recipe[cloudfoundry::vagrant-provision-start]", "recipe[apt::default]", "recipe[git]", "recipe[chef-golang]", "recipe[ruby_build]", "recipe[rbenv::user]", "recipe", "recipe[sqlite]", "recipe[mysql::server]", "recipe[postgresql::server]", "recipe[rbenv-alias]", "recipe[rbenv-sudo]", "recipe[cloudfoundry::warden]", "recipe[cloudfoundry::dea]", "recipe[cloudfoundry::uaa]", "recipe[cloudfoundry::cf_bootstrap]", "recipe[cloudfoundry::vagrant-provision-end]"] from JSON [2013-07-17T19:43:23+00:00] INFO: Run List is [recipe[cloudfoundry::vagrant-provision-start], recipe[apt::default], recipe[git], recipe[chef-golang], recipe[ruby_build], recipe[rbenv::user], recipe, recipe[sqlite], recipe[mysql::server], recipe[postgresql::server], recipe[rbenv-alias], recipe[rbenv-sudo], recipe[cloudfoundry::warden], recipe[cloudfoundry::dea], recipe[cloudfoundry::uaa], recipe[cloudfoundry::cf_bootstrap], recipe[cloudfoundry::vagrant-provision-end]] [2013-07-17T19:43:23+00:00] INFO: Run List expands to [cloudfoundry::vagrant-provision-start, apt::default, git, chef-golang, ruby_build, rbenv::user, java::openjdk, sqlite, mysql::server, postgresql::server, rbenv-alias, rbenv-sudo, cloudfoundry::warden, cloudfoundry::dea, cloudfoundry::uaa, cloudfoundry::cf_bootstrap, cloudfoundry::vagrant-provision-end] [2013-07-17T19:43:23+00:00] INFO: Starting Chef Run for ip-10-77-71-207.ec2.internal [2013-07-17T19:43:23+00:00] INFO: Running start handlers [2013-07-17T19:43:23+00:00] INFO: Start handlers complete. [2013-07-17T19:43:24+00:00] INFO: AptPreference light-weight provider already initialized -- overriding! ... (truncated) ... [2013-07-17T19:58:50+00:00] INFO: Processing package[zip] action install (cloudfoundry::dea line 9) [2013-07-17T19:58:55+00:00] INFO: Processing package[unzip] action install (cloudfoundry::dea line 13) [2013-07-17T19:58:55+00:00] INFO: Processing package[maven] action install (cloudfoundry::uaa line 1) [2013-07-17T19:59:38+00:00] INFO: Processing execute[run rake cf:bootstrap] action run (cloudfoundry::cf_bootstrap line 3) [2013-07-17T20:05:35+00:00] INFO: execute[run rake cf:bootstrap] ran successfully [2013-07-17T20:05:35+00:00] INFO: Processing bash[emit provision complete] action run (cloudfoundry::vagrant-provision-end line 2) [2013-07-17T20:05:35+00:00] INFO: bash[emit provision complete] ran successfully [2013-07-17T20:05:35+00:00] INFO: Chef Run complete in 1332.027903781 seconds [2013-07-17T20:05:35+00:00] INFO: Running report handlers [2013-07-17T20:05:35+00:00] INFO: Report handlers complete
We can now log into our new EC2 instance, which is running Cloud Foundry v2:
$ vagrant ssh
Note: all commands that follow are intended to be run on the EC2 instance.
Push an app
First, we must initialize the Cloud Foundry v2 command-line interface with the following command:
$ cd /vagrant $ rake cf:init_cf_cli
Here is the output of that command:
==> Initializing cf CLI Setting target to http://127.0.0.1:8181... OK target: http://127.0.0.1:8181 Authenticating... OK There are no spaces. You may want to create one with create-space. Creating organization myorg... OK Switching to organization myorg... OK There are no spaces. You may want to create one with create-space. Creating space myspace... OK Adding you as a manager... OK Adding you as a developer... OK Space created! Use `cf switch-space myspace` to target it. Switching to space myspace... OK Target Information (where will apps be pushed): CF instance: http://127.0.0.1:8181 (API version: 2) user: admin target app space: myspace (org: myorg)
Now you can deploy one of the test apps. We will use a Node.js “Hello World” app:
$ cd test-apps/hello-node $ cf push
We see the output:
Warning: url is not a valid manifest attribute. Please remove this attribute from your manifest to get rid of this warning Using manifest file manifest.yml Creating hello-node... OK 1: hello-node 2: none Subdomain> hello-node 1: vcap.me 2: none Domain> 1 Creating route hello-node.vcap.me... OK Binding hello-node.vcap.me to hello-node... OK Uploading hello-node... OK Preparing to start hello-node... OK Checking status of app 'hello-node'........................... 0 of 1 instances running (1 starting) 0 of 1 instances running (1 starting) 1 of 1 instances running (1 running) Push successful! App 'hello-node' available at http://hello-node.vcap.me
Cloud Foundry v2 is running on localhost on our EC2 instance, so our app is not accessible from our web-browser, but we can test the app exists using curl from the EC2 instance:
$ curl http://hello-node.vcap.me/
Here is what is output by curl:
Hello from Cloud Foundry
Delete the app
To delete the app, you can use:
$ cf delete
The following output is seen:
Warning: url is not a valid manifest attribute. Please remove this attribute from your manifest to get rid of this warning Using manifest file manifest.yml Really delete hello-node?> y Deleting hello-node... OK
Inside out
From the notes I was given:
Now, to expose apps externally, it gets trickier. First, you’ll need to provision an elastic IP in the AWS console and attach it to the EC2 instance that’s running the cf v2 install. Then, you’ll need to set up a wildcard DNS record to point to that IP (*.domain and domain should point to that IP). xip.io might work here, but I’m not familiar enough with it to know for sure.
xip.io is actually perfect for this. All I need is my external IP, which was 50.19.50.63
, and I append ".xip.io"
, which gives me "50.19.50.63.xip.io"
as well as wildcard "*.50.19.50.63.xip.io"
for the Cloud Foundry API and any apps I deploy. This is a zero-configuration service. The IP that you want to resolve to is included in the hostname you create and the DNS service simply returns you the IP. This means you can have a valid globally resolvable DNS hostname instantly.
I can also get a simpler hostname by checking the DNS record of this hostname, which is actually just a CNAME.
$ host 50.19.50.63.xip.io
Which outputs:
50.19.50.63.xip.io is an alias for hj8raq.xip.io. hj8raq.xip.io has address 50.19.50.63 Host hj8raq.xip.io not found: 3(NXDOMAIN) Host hj8raq.xip.io not found: 3(NXDOMAIN)
So, I can use hj8raq.xip.io
instead, since it is shorter and I just want to use it temporarily.
Updating more config
Since we now have an external domain name, not just localhost, we need to update some configuration files within the custom_config_files
directory.
$ cd /vagrant/custom_config_files
Assuming you are running under the domain "yourdomain"
(or "hj8raq.xip.io"
in my case), you should edit the cloud_controller.yml
as follows:
$ (cd cloud_controller_ng; vim cloud_controller.yml)
- change
external_domain
toapi.yourdomain
- change
system_domain
toyourdomain
- change
app_domains
toyourdomain
- change
uaa:url
tohttp://yourdomain:8080/uaa
Next, edit the DEA configuration.
$ (cd dea_ng; vim dea.yml)
- change
domain
toyourdomain
And, finally, the configuration of the Health Manager:
$ (cd health_manager; vim health_manager.yml)
- change
bulk_api:host
tohttp://api.yourdomain:8181
Router-registry bug
There was a small bug on my AWS deployment that may have been fixed. This was related to a incompatibility with the JSON between the Cloud Controller and the Router when registering the API endpoint with the router. Here’s the fix:
$ cd /vagrant/cloud_controller_ng/lib/cloud_controller $ vim message_bus.rb
Then, change the line:
:uris => config[:external_domain],
To this:
:uris => [config[:external_domain]],
This will make :uris
an array, not a string. Probably, better to fix this in the gorouter, but this is quicker for now.
Restart CC DB
Now we need to reset the Cloud Controller database.
$ cd /vagrant/ $ rake cf:bootstrap
Finally, reboot the machine.
$ sudo reboot
When the machine comes back up, we can ssh
back into it:
$ vagrant ssh
And run the ./start.sh
command to start Cloud Foundry components.
$ cd /vagrant $ ./start.sh
Now, Cloud Foundry v2 should be running with your externally accessible endpoint.
Related video
In this meetup session, Gastón Ramos and Alan Morán of Altoros Argentina deliver an overview of Cloud Foundry and present CF Vagrant Installer to the audience.