Cloud Foundry Advisory Board Meeting, Nov 2016: Diego, Containers, and Persistence

This month’s worries

“Cloud Foundry is alive, well, and kicking, and we need to get more people involved, so we can have a broader reach.” Thus stated IBM’s Michael Maximilien (aka Dr. Max), leader of the monthly Cloud Foundry Community Advisory Board (CAB) calls. Dr. Max’s remarks came at the end of the call. He also urged attendees to “tweet about what you heard today. It’s interesting and we can create a little more buzz.”

Indeed, there were three topics of broad interest featured during this month’s call on November 9:

• a Diego update
• a container networking update
• a long discussion about persistence

 

Diego is very close to 1.0

Eric Malm from Pivotal discussed Diego, which has now successfully run in a large environment with 200K Cloud Foundry apps and 250K instances. This is a significant follow-up announcement from last month’s call, during which Eric announced a successful 50K-instance run.

“We’re finishing the long trek to coming to Diego 1.0 being able to replace DEA (the current Cloud Foundry architecture) everywhere, and are hoping to cut the actual 1.0 version soon,” Eric noted.

 

Try to contain this

Usha Ramachandran, a project manager for container networking, also gave a significant update. She described a CNI-based pluggable container networking stack that allows containers to talk directly among themselves. (Note: CNI stands for Container Networking Interface and was originally put forth by CoreOS.)

“An ‘app A’ can now talk to an ‘app B’ on this protocol, with no port mapping required.” —Usha Ramachandran, Pivotal

When asked about the use of overlay networks, she noted that the CNI plug-in can be swapped out for something different, “maybe an SDN-based CNI plug-in, for example.” She said the path through the CF Go Router is still the same, and “application security groups continue to work as is, though they will be folded into policy enforcement at some time in the future.”

Dr. Max asked her about the extent of policy language, to which she replied that it was very simple. “Cloud Foundry gives access to protocols and ports, and that’s the extent of the configuration. A list command gives you all policies that are enforced, and a delete command. This is enabled through a CLI (command-line interface) plug-in. We also have APIs you can use to configure them,” said Usha.

“We wanted something powerful to express simple policies without a complicated set of rules.” —Usha Ramachandran, Pivotal

According to Usha, the basic idea was to be very specific, so that default is deny root and then one starts to specify and enable. The underlying implementation uses IP tables, so then one creates an IP table rule within a VN packet.

Containers are, of course, all the rage within cloud-computing discussions today, and Dr. Max said the recent developments “bring Cloud Foundry to a level of competition to any other cloud platform out there.”

 

Be persistent

The discussion then moved on to persistence, led by project manager Julian Hjortshoj of EMC. He described Diego persistence “as a set of plumbing changes to allow Docker volume plug-ins to mount external file systems to Diego cells to use Cloud Foundry apps.” He added that there is an additional plumbing for service broker and a bind to apps.

“Shared file systems act like services, just like any other data resource,” he said. “It’s been around in experimental form in Pivotal CF 1.8, Diego 1381, and cf 238. So, it’s probably available in the versions most people are using, at least in test environments.”

He added that “minor changes force us to have slight incompatibilities with service brokers in the current version, but service brokers we’ve built have backward compatibility, and Cloud Foundry itself has a compatibility path. So, it’s ready to consume today.”

GA for it was announced at the recent Cloud Foundry Summit in Frankfurt, and “there’s been a lot of refining since then,” Julian said. “We’re making sure our volume drivers are deployable as a BOSH add-on. With the latest BOSH release we can filter add-ons to specify that jobs in a runtime config will only get placed on virtual machines that are running a certain job for certain deployments. So, you can target Diego cells specifically.”

He also noted the persistence team was working with the cf-dev team to get a local driver in cf-dev, to bypass the current method of having to run BOSH Light on top of Cloud Foundry.

This is major. It puts Cloud Foundry in a position where you have just a file system, without changes.” —Dr. Max, IBM

Julian agreed, noting “this addresses a class of apps that couldn’t run previously in Cloud Foundry because they required a file server, and now they can.”

He continued, “Within legacy systems there will be some well-behaved 12-factor apps, and for them this is a good thing. Volume services are perfect for many old apps, and there’s no real reason something use a file system can’t be a 12-factor app. There should be a class of legacy apps that can move into Cloud Foundry without rewrites.”

 

To BOSH or not…

Another discussion during the call referenced a lively, ongoing discussion within the Cloud Foundry mailing lists about an initial announcement to require BOSH as part of 2017 Cloud Foundry certifications, which are administered by the Cloud Foundry Foundation. Companies wishing to have a certified distro must renew each year.

The crux of the argument is whether having a strict BOSH requirement is an overreach by the CF Foundation (given the traditionally free and open zeitgeist of open-source software movements) that could potentially lock newer vendors out.

In the end, it’s likely the BOSH requirement will be removed. Time will tell if that influences the number of distros submitted and approved for 2017 certification.

The next call is scheduled for Wednesday, December 14, at 8 a.m. Pacific Time. The call is open to anyone who would like to attend, and all attendees are free to contribute to the discussion. The best way to get in the loop is to join the Cloud Foundry Slack channel and follow the #CAB track.