DigitalGlobe Builds Satellite Control System with Microservices and Cloud Foundry

(Note: This use case is based on a presentation at the Cloud Foundry Summit 2015 in Santa Clara, CA by CTO and others from DigitalGlobe.)

DigitalGlobe provides satellite imagery for Google Maps. It develops its own satellites as well as the software that enables them. The company recently undertook an initiative built on Cloud Foundry as part of building a new ground system that controls its latest satellites. A couple of slides describing the company’s satellites and its latest undertaking are shown below:

“We identified how we wanted it to work and PaaS was the way it should go,” according to company CTO Mark Carlson. “We evaluated the offerings out there and Cloud Foundry was a pretty obvious winner.”

The company downloaded both the open-source and Pivotal versions of Cloud Foundry and pressure-tested them “to make sure they lived up to how we believed they should function,” according to Carlson. An overview of the technical decision process is shown in the slide below:

The testing and evaluation process

As part of the initial testing and evaluation process, DigitalGlobe team members ran a series of applications and services on OpenStack, Amazon Web Services (AWS), and VMware. They also employed Java, Ruby, and Python buildpacks, running apps and services on all the combinations of IaaS and language.

In doing so, “we wanted to challenge the status quo, so we moved a lot of people’s cheese,” Carlson notes. Scrum teams were created and made autonomous from one another–and from IT operations–as much as possible. With top management buy-in, they then expanded the concept and use of end-to-end testing and demonstration.

Along the way, they saw the benefits of focusing on an alignment of Scrum teams by work product, not technology. A summary of goals and processes is provided in the following table:

Carlson points out that the aim with the burstable environment “is the ability to have code on- and off-premises and be able to move it back and forth.” Microservice enablement was of course a key goal, but DigitalGlobe also found it needed to be pragmatic with the overall architecture.

An overview of technical accomplishments so far is provided in the following slide:

Three patterns emerge

Along the way, the company defined three architectural patterns. Ideally, all apps and services would fit into what DigitalGlobe calls Pattern 1, a top layer that fits well into a PaaS and conforms to a microservices architecture. But there were cases where legacy apps needed to be put into VMs instead, and formed what DigitalGlobe calls Pattern 2 applications. A third layer of Pattern 3 apps are those “where we need every millisecond of compute power, and the latency that a virtual environment causes is unacceptable,” according to DigitalGlobe’s Steve Wall. “The company needs a high-performance, bare-metal compute cluster in this case.”

“It’s not an all or nothing proposition,” Wall says. “With all of the apps and services, it’s a question of what fits well where?”

An overview of the three patterns is shown in the following slide:

To go where none have gone before

Culturally, “our path to adoption is taking the organization to a place it has never been before,” Wall points out. The process involved in doing so encompasses three key points:

• Develop one service. The company started small, with an eventing service. The service publishes an event and distributes it out.
• Use a small team of developers. They are known as “the pioneers.” After the first service, they developed a few more, “we got a BPM layer in to orchestrate workflow,” Wall says, “ and we had an end-to-end demo.”
• Do the Initial buildout and then roll it out to the company at large.

Using an agile development approach “got us into understanding what a microservice was,” according to Wall. “We also learned in this case that we had three microservices so now we had a template. We got it into Git and into Jenkins, and got our deployment pipeline started.”

The company is now doing a blue/green deployment and executing some automated tests.

In fact, “automated testing is bedrock of everything we’re doing,” Wall says. “We can’t do this with manual testing because we won’t get velocity through the pipeline.”

A move to self-provisioning (see slide below) has made “every happier,” Wall says. “In the legacy environment you had to coordinate among a lot organizations.If you wanted an environment build out, you had to get a lot of people involved, and the coordination among organizations was time-consuming.”

By contrast, “in a continuous delivery environment there are now clear lines of delineation between the organizations,” he says. “We gave the application development team the power to do what they needed to do. They have access to a pool of resources, so are now able to test instances in an afternoon instead of a week or two, or conduct scale-handling tests that simply were not possible before.”

Related videos

Here is the original video presented at the Cloud Foundry Summit 2015 in Santa Clara, CA.

Related slides

An overview of DigitalGlobe can be found on SlideShare: