Cloud Foundry After Dark: Bridging Brooklyn

Create new services as part of your app’s manifest!

Interest in Cloud Foundry continues to grow and there is now heavy development work ongoing to integrate it with Apache Brooklyn.

On February 22, 2015, in Cloud Foundry After Dark’s podcast, Pivotal Software’s James Watters, Andrew Clay Shafer, and a few more of their colleagues discussed with Duncan Johnston-Watt and Alex Heneveld of Cloudsoft what they are doing to integrate Cloud Foundry and Apache Brooklyn and how this would help the developer community.

“Essentially, the way we wanted to approach this was to allow people through a single service broker to access the whole sort of cornucopia of blueprints that Apache Brooklyn can stand up,” said Duncan.

Giving devs the power

He explained how they came up with a plugin that would help developers create their own blueprints that Apache Brooklyn did not support by default.

“Rather than just consuming what Brooklyn is exposing us to, we wanted to give you guys a plugin that would enable you to then actually push out your own blueprints and actually create new services on the fly,” he pointed out.

After Duncan’s explanation, Alex explained the technical details of how to actually use their code.

“The idea is that there’s a service broker and a plugin and, with the service broker, it wires up [Apache] Brooklyn and Apache Brooklyn then serves as a kind of a blueprinting management framework, so you can write blueprints in YAML, you can add them to a catalog, you can write them in Java or something else in JVM,” he said.

Alex then explained how they’ve already worked on developing catalogs with MongoDB, Basho, and Couchbase. He detailed how using the Cloud Foundry plugin differed from simply using Apache Brooklyn.

“In Brooklyn, you can just pick something and then say where you want to deploy it and off you go, but with the Cloud Foundry plugin, [it can] actually draw up a blueprint straight into your manifest,” he said.

Here’s how it works

Citing an example, he said: “For instance, say I want five shards of three replica sets each deployed into local host or deployed into Google Compute. The plugin takes the job of moving the deployment and passing it to the service broker and then the service broker, of course, passes to [Apache] Brooklyn which will go and deploy it and then, once it’s done, it will take all that and do the usual VCAP services, environment variables so all the sensors wherever the Mongo nodes are running will get passed into your app the usual way.”

He continued to describe how their plugin would work in complex setups.

“So here we got the things in the catalog but within [Apache] Brooklyn, you can go into the catalog and if you’ve got some other blueprint, you can just drop [it] right in and then make it available. What we wanted to do was to make it tighter. If you’ve got some complex topology of Storm or what not, you can describe that directly in your application’s manifest and so the combination of the plugin and the service broker can give you nice small modular pieces that, working together, can add up to letting you do things in what we think is a pretty idiomatic way.”

James added, “I think what’s [good] is that we have this network effect between each other. Everyone who has [Apache] Brooklyn gets the power and now Cloud Foundry leverages [Apache] Brooklyn.”

DevOps challenges

Not everything was positive though as Duncan raised concerns regarding how the Cloud Foundry development would affect the relationship between developers and DevOps people.

“There’s a concern [about] separation,” he said. “You’ll want developers to have access to perhaps a set of predefined services but you’ll also want more people in the more DevOps kind of idiom to develop new services. I don’t necessarily see that in an enterprise, they’re one and the same person.”

James replied, “We somewhat separate [it]. Here’s the DevOps or operator layer where they create services that developers consume and we’re not trying to get completely into cowboy coding where you can just create any service you want with little manifest and hope for the best.”

With the issue explained, Duncan and James agreed that the challenge could escalate once more developers are added to Cloud Foundry.

“It starts to matter when we have thousands and thousands of developers working on projects,” Andrew pitched in. “That’s one of the things about Cloud Foundry’s work flows and origin spaces that we have [in Apache] Brooklyn built in to the framework. I’m curious as [to] how this evolves where people see those. What would you put in Brooklyn versus what would you put in Cloud Foundry and how does that shift over time or does it?”

“We don’t know the answer to that. There’s a whole bunch of services that we can provide today that are actually a bridge to, in some cases, a more legacy environment where there’s no actual desire to import everything into Cloud Foundry and at the same time give people the power of Cloud Foundry,” answered Duncan.

“That’s worth seeing to develop new apps but those apps will ultimately need some backend services. So if that’s what we end up providing—robust production support for them I think that’s a win for everybody. I think giving people the ability to author new services without creating blueprints would I think also be attractive to some DevOps.

“So there’s a lot of there’s a fair amount of education and thinking to be done before this is fully baked but right now we’re very keen to push it into the Cloud Foundry community as it is and have other people play with it, give us opinions and so on.”

You can learn more about the plugin from this blog series by Cloudsoft: Integrating Cloud Foundry with Apache Brooklyn.

Want details? Watch the video!