Lattice, Cloud Foundry, and the Pursuit of Micro

Is Lattice left for good?

Colleagues with whom I spoke shared my surprise to hear of the discontinuation of Lattice, the announcement being made by of Pivotal during the December CAB call.

Initial tweets from Pivotal VP said Lattice had been renamed MicroPCF within Pivotal, to which Cloud Foundry Foundation CEO Sam Ramji tweeted:

Lattice or no, the Cloud Foundry community continues to await a definitive, smaller, open-source Cloud Foundry implementation that would allow developers an easy and quick on-boarding, then the ability to play around (i.e., work) with it on their laptops.

Subsequent to the initial tweets, the reality appears to be that the Pivotal Lattice team is being migrated to its MicroPCF team.

Lattice remains the property of the Cloud Foundry Foundation and is lying there awaiting someone to jump in and develop it further. Pivotal cited an absence of significant traction or take-off for the Lattice project. This seems to be correlated with Lattice having a different interface than full Cloud Foundry. Two interfaces mean two products, yes?

To clear things up, it seemed like a good idea to go to the sources. So, I contacted Pivotal and I contacted Sam Ramji. My approach was to ask them four questions:

• What is Lattice’s original mission?
• To what degree that mission was accomplished?
• Current details on its status?
• What, if anything, you’d expect/wish the community to do with it in the future?

 

Sam Ramji’s response

Here is response in full:

“Lattice was inspired by Dokku. They are both designed to be very simple developer on-boarding paths compared to their much more sophisticated production software counterparts (Lattice:Cloud Foundry::Dokku:Heroku).”

“The intent was simple: have a desktop footprint of Diego, which resulted in artifacts that could be pushed to production. Vagrant was the underlying tool for the desktop and Terraform was the tool to push to larger environments (like Digital Ocean).”

“The result of the engineering constraints and design choices led to Lattice having a different interface than Cloud Foundry and so wasn’t really a direct on boarding path. Some people used Lattice as a sort of “snap circuits” environment to snap in modules like UAA from Cloud Foundry. It’s a more obviously container-focused tool than Cloud Foundry, which is “containers all the way down,” but hides that fact from the developer.”

“Lattice was launched confusingly and not marketed clearly. Despite that fact we saw growing adoption—although not at the breakout pace that we’d hoped for. The core Lattice team decided in November to stop development after the next iteration. At >700 Github stars, Lattice is no slouch as open-source projects go.”

“When the announcement that Lattice development would stop was clearly announced to the community and to the leading members of the Cloud Foundry Foundation (including the Board of Directors), the response was that they’d found Lattice useful and interesting and there was worthwhile effort in continuing it.”

“At this time, the Foundation is calling for volunteers to take Lattice development forward and define how it could be most useful to the Cloud Foundry and Docker communities.”

“Like all other code in the Cloud Foundry and Cloud Foundry Incubator GitHub repositories, the Lattice code is held in trust for the entire community by the non-profit Cloud Foundry Foundation.”

“We think it’s a great opportunity for Diego to demonstrate its general purpose utility for scheduling Docker containers as well as providing a minimum viable environment for scaling containers (logging, monitoring, routing/discovery). We’re open to a range of modifications including alignment of the Lattice API and CLI to match Cloud Foundry, pluggable networking and storage, or other changes the community would like to see.”

 

Pivotal’s response

, Senior Director of Product Management at Pivotal, also responded. He copied Sam on it, as I had told both Sam and Pivotal that I was asking them the same questions.

Here is James’s response in full:

“Pivotal originally puts forward engineering, product and marketing efforts behind lattice. We believed that an alternative packaging of some of the Cloud Foundry components with a lower level API would be valuable. The Lattice docs also answer some of your questions.”

“Once Pivotal had what we thought was a compelling start, we asked the Cloud Foundry Foundation to incubate the project.”

Over the course of about a year of ongoing development with releases, we at Pivotal learned several things:

• Cloud Foundry’s new features like Diego, Docker image support and TCP routing were hardened with great feedback from Lattice users.
• It was possible and actually worked well—take BOSH packages and colocate many components in a small footprint VM.
• Users and Pivotal’s customers loved both the small footprint of Lattice and being able to use new features in Cloud Foundry sooner.
• Users asked Pivotal for the full Cloud Foundry API and the CF CLI on their developer workstation in a way that was fully compatible with PCF.
• Users were also interested in other Pivotal technologies related to microservices like Spring Boot and Spring Cloud.

“Pivotal took these learnings and decided to act on them by having the Pivotal employees, working on Lattice, do a final release of Lattice by the Pivotal team and then start that same team working on MicroPCF, which uses the knowledge acquired developing Lattice.”

“Lattice remains in incubation in the Cloud Foundry Foundation. Other members of the CF Foundation, and anyone else in the Cloud Foundry community, may extend and contribute to the existing open-source Lattice codebase.”

 

The pursuit of micro

Sam got back to me again, writing “James is precisely correct. You can depend on his accuracy and impartiality.” Sam also noted that the Cloud Foundry Foundation is “still learning how to ignite and organize collective action around new projects that are outside the scope of core Cloud Foundry.”

I also got a response from , a Director of Altoros Argentina (Santa Fe), who said that Lattice “was extremely great in regards to its requirements and speed. Sadly, it was not a minimalistic Cloud Foundry (as it) didn’t even have the same CLI.”

At this point, Manuel regards it as “a sandbox environment to test Docker in a way that you can hope CF can do it the same way (but) I want a minimalistic version of Cloud Foundry, not another (separate) product. I want a real, (open) Cloud Foundry-like experience with the same CLI.”

The technical challenge is clearly not trivial for someone to deliver such a thing, the number of people who would use it is undefined, as is the tangible benefit to an organization that would take the project on. The project could also be swamped by arguments over which features to include, if and when to deviate from core Cloud Foundry, how to manage community participation (if desired), etc. Would (or should) an open MicroCF in reality be more like a MiniCF, a NanoCF, a PicoCF? Would semantics rule the day? How many versions can dance on the head of a pin?