Cloud Foundry Advisory Board Meeting, Sep 2017: Service Fabrik and Task Scheduler
Demos in the spotlight
Dr. Max
This month’s CAB call featured two demos, that of Service Fabrik—a multi-cloud provisioning broker developed by SAP—and a platform task scheduler developed by Pivotal that will be available via an API and CLI.
Both of this month’s demos have grown out of the cf-extensions program that’s been championed by CAB call leader Dr. Max. He said the philosophy behind extensions is “let a thousand flowers bloom. We’re trying to attract many projects because we believe in innovation.”
As usual, the members of the Cloud Foundry project development teams submitted monthly progress, and Dr. Max updated the participants on the upcoming Cloud Foundry Summit Europe to be held in Basel, Switzerland.
Service Fabrik
Ashish Jain
The project’s mission is “to provide a highly scalable and pluggable framework to develop and manage enterprise-grade backing services on multi-cloud infrastructure,” according to Ashish Jain, a User Experience Specialist at SAP Labs. (Backing services are, actually, any services consumed by apps in a cloud deployment, for example, databases, messaging systems, mail, caching systems, etc.)
At the Cloud Foundry Summit 2017 in Silicon Valley, SAP proposed Service Fabrik for incubation as part of the ongoing cf-extensions effort.
The project has a number of high-level goals:
- Support container- and BOSH-based provisioning
- Support Day 2 Operations, such as data backup and restore, service update, service monitoring and logging, etc.
- Bring in capabilities to visualize service deployments
- Support multi-cloud deployment and management
- Pluggable architecture to integrate third-party monitoring, logging and backup solutions
- Build an ecosystem of production-grade backing services
Service Fabrik also aims to “provide well defined APIs for operations, such as backup and restore or updates in an unattended, automated way in a multi-cloud environment,” according to its incubation proposal.
The project takes its inspiration from the generic cf-containers broker, Ashish said. According to him, a primary goal is to extend generic functionalities of provisioning and binding services to the Day 2 Operations listed above.
Jagadish Basavaraju, a Development Architect at SAP, took participants through a high-level view, which shows Service Fabrik using Docker Swarm (for container clusters) and BOSH (for various service deployments).
Service Fabrik’s architecture (Source)Ashish noted there are ten full-time engineers currently engaged in the Service Fabrik project. Its road map over time includes pluggability to integrate different monitoring and logging solutions, an ops management user interface, BOSH 2.0-based service provisioning, and a Cloud Foundry CLI plug-in for capabilities that are not supported by CF Cloud Controller.
Task scheduler
Scott L’Hommedieu
Scott L’Hommedieu, a Software Development Leader from Pivotal, presented a new platform task scheduler.
As outlined in his demo, the scheduler would “allow app developers to schedule the execution of workloads on Cloud Foundry without using external services. You may want to schedule tasks, such as database migrations, e-mails and reports, and batch jobs. This service also enables developers to more easily bring their scheduled jobs into a CI/CD pipeline by using the same Cloud Foundry resources and APIs used by apps and services.”
“We decided to put together an ecosystem that would abstract jobs and tasks that could be executed on a schedule in a Cloud Foundry way,” he said. “So, primitives of Cloud Foundry would apply in scheduling workloads. We wanted to help enterprise users move off of legacy scheduling platforms that haven’t quite kept up to cloud-native and development life cycles. This scheduler can also support scheduled workloads as part of a microservices development life cycle.”
Scott mentioned service interactions, such as binding and re-binding, adding and removing routes, and blue-green deploys as examples of tasks that would come under the scheduler’s purview. Key features include:
- Define jobs as Diego tasks
- Define calls as HTTP requests
- Execute jobs and calls ad-hoc or scheduled
- Use cron expressions for schedules
- Retain history/status of job and call execution
- Support tasks based on all CF buildpacks
- Interface via CLI or API
Runtime PMC
Dieu Cao
Dieu Cao from Pivotal reported that the container networking team went from incubation to active. “There is also a new project services API team based in London that will be working with the CAPI team,” she said, “to focus on an open service broker API and helping with services.”
According to Dieu, one of the first things to address is instance sharing, though she admitted it to be a challenging prospect. “We also have a CF permissions team looking at how to identify provider-group map-ins for Cloud Controller roles. (The team will also examine) a custom role for authorization for Cloud Controller and potentially other components in the ecosystem.”
For detailed updates on the CF projects, read the notes from the Project Management Committee meeting held on September 20.
Diego
Eric Malm
Eric Malm from Pivotal said there is now a test environment that “has been decoupled from dependence on Consul, and all is good so far.” This has been an ongoing issue for some time (as noted, for example, in our December 2016 CAB call report).
Eric added that the BOSH process manager is designed to containerize BOSH jobs and the team did some opt-ins for this. At the moment, the team is also working on a better security posture for BOSH releases.
BOSH
Dmitriy Kalinin
As far as BOSH goes, Dmitriy Kalinin from Pivotal reported the team “is continuing to finish up DNS-related things pushing other teams to use DNS in their environments.” He said there are also ongoing CLI v2 clean-ups. Dmitriy urged community members to check out the new “vendor-package” command, noting that docs and further info about it are coming soon.
Dimitriy also discussed what is called BOSH Process Manager (BPM) in this community, a project with “a goal to make it easier to manage the life cycle of jobs and allow them to be containerized.” It also aims to “simplify process management by replacing CTL scrips with the YAML files,” Dimitriy said, adding that this is still in beta.
“For 99% of jobs, you end up without any CTL script. Just a couple of the YAML files. This will be great for BOSH adoption and for simplified releases.” —Dmitriy Kalinin, Pivotal
The upcoming summits
Dr. Max urged members of the community to attend the upcoming day-long hackathon at the Cloud Foundry Summit Europe. “It was really cool last year,” he said. “Not just because it was held, but what came out of it. One example was how an effort between Pivotal and IBM added serverlist to Cloud Foundry in a really great way.” As was the case last year, the hackathon will offer prizes. There are more than 650 people registered at the moment for the summit, with a goal of exceeding 1,000 attendees.
At the upcoming summit, Altoros is going to deliver a lightning talk on full-stack Cloud Foundry monitoring.
Today, the Cloud Foundry Foundation also announced the CF 2018 North America Summit to be hosted in Boston.
The next CAB call will be done live from the Cloud Foundry Summit Europe in Basel, Switzerland, on Thursday, October 12, beginning at 2:40pm CET (8:40am EDT). The next regularly scheduled call will then be held on Wednesday, November 15, starting at 8 a.m. Pacific Time.
The calls are open to anyone within the Cloud Foundry community who is interested in joining in on them. The best way to stay apprised of the call schedule and agenda is to join the #cab channel at the Cloud Foundry Slack workspace.
Want details? Watch the video!
