Orange Saves Months When Building Apps with Cloud Foundry

A multi–data center Cloud Foundry environment is used for aсcelerating the development, operations, and delivery of apps reaching millions of customers.
Why read this?
Use Case for Cloud Foundry:

A multi-national telco is migrating hundreds of apps to its "federated" private cloud platform, increasing the speed of development and operations across its business units.

Business or Technical Result:
  • The internal IT department of Orange Labs experiments with migrating thousands of apps and services to a cloud environment, decomposing monoliths to microservices.
  • In its turn, Orange Technocentre managed to save 100 days of development for an MVP, and development velocity was increased by 1.5x due to the use of Pivotal Cloud Foundry.
  • Another division, Orange France, is running 3 data centers hosting ~300 apps, 3 open-source Cloud Foundry production deployments, and thousands of servers.
Lessons learned:
  • Operations' buy-in is essential for the platform's adoption.
  • A test lab, "playground" environment, is the best way to proceed.
  • Unique aspects of a particular company must be addressed specifically, and the use of Cloud Foundry must conform to the company, not the other way around.
CF deployed to:

OpenStack, vCloud

What else is in the stack?

Pivotal Cloud Foundry, Spring Boot, Java, Splunk, MySQL, PostgreSQL, Riak CS, Redis, MongoDB, Neo4j, RabbitMQ, S3 Object Storage, NFS, Hive, HDFS, Prometheus, Logsearch / ELK, Concourse, Kubernetes, MariaDB, Terraform, Cassandra, Xymon, Mattermost

Company Description:

The company is a European telco with €41 billion in annual revenue (2018). It serves 263 million customers, including 202 million mobile users and 20 million fixed broadband subscribers (2018). Orange employs 150,000 specialists worldwide, including 91,000 employees in France (as of March 2018).

Cool fact about the company:

Orange Labs (known as "France Télécom R&D" before 2007) is the research and development division of Orange. It was derived from different entities, which existed previously, such as CNET ("Centre national d'études des télécommunications") founded in 1944, the CCETT created in 1972, as well as other organizations. Orange claims it owns 240,000 km of undersea cable and invested €700 million in research and innovation (as of 2018).

Experiments with cloud migration

The French telco Orange has revenues of around €41 billion (as of 2018), serving 263 million customers across Europe, Africa, and the Middle East (March 2018). It is a federated group, which consists of many subsidiaries handling different business niches and operating in 28 countries.

Guillaume Berche

In 2010, the company realized that simply migrating to IaaS as a cloud strategy would “not be a silver bullet,” according to Guillaume Berche of Orange Labs. The question then arose as to how PaaS could handle “massive cloud migration,” he says.

That year, the company created their first, custom PaaS prototype on top of vCloud, since no open-source PaaS was available yet. In the course of three years, the prototype was migrated to Cloud Foundry—for that purpose, the team explored Pivotal Web Services.

By 2015, Orange created “a playground for cloud-native architecture” based on Cloud Foundry, aiming to ease the adoption of the PaaS within the company’s internal IT and experiment with microservices. The company has transformed the platform into an environment called ElPaaso to pave a path for widespread adoption of Cloud Foundry within a “federated environment that provides individual businesses with lots of local autonomy,” according to Guillaume.

 

The ElPaaso app templating add-on

The team discovered that meeting its goals would require “more than an out-of-the-box Cloud Foundry instance,” Guillaume says. It resulted in a dual-hosting mode, with a “raw” platform and its forked implementation with the ElPaaso app templating add-on. The team was able to work with either Cloud Controller API or use ElPaaso’s visual interface.

The ElPaaso add-on integrated with Cloud Foundry (Image credit)

The ElPaaso team had several high-level goals:

  • Achieving decentralized autonomy, having transparency preserved
  • Integrating the PaaS into the company’s internal IT ecosystem and work processes
  • Gradually adopting the PaaS for the majority of users in development and operations, beyond the initial innovators
  • Capturing expertise and best practices along the way
  • Creating stable interfaces to isolate projects from changes

Working within an on-premises, private cloud instance, the team utilized the impact mapping process, which addressed fundamental questions, such as why the lab team is doing what it’s doing, how it can measure if/when its goal is reached, and who can help the team to reach its goal. “The (impact) map is quite large,” noted Guillaume at Cloud Foundry Summit 2015.

Deployed to vCloud and OpenStack, the test lab environment was using Spring Boot and Java, utilized Splunk for unified logging, and provided various tools/products “as a service” (such as DBaaS).

 

Challenges met on the way

A couple of issues emerged during adoption, such as integration of legacy apps and buy-in by the operations teams.

“We have many legacy apps that we are taking time to transform and adapt…We must always see if we can get existing apps to transform to the 12-factor apps.” —Guillaume Berche, Orange Labs

As noted by Guillaume, “there are gaps in culture between developers and operations, and the tools they use.” He said the team would be successful “only when operations accept it,” and only when things are proven to be able to run in a multi–data center, multi-region environment. Moving toward operational buy-in, Guillaume said the team was working to establish a common baseline in operations “so there is less specificity among apps.”

A high-level view of the system’s internal architecture (Image credit)

A lot of customization was done—the ElPaaso team extended Cloud Foundry through many “hooks,” as Guillaume refered to them.

“We are working to capture what apps support in terms of environment variables, thus making this information explicit, so that the intimacy between developers and operations gets introduced with this explicit configuration.” —Guillaume Berche, Orange Labs

 

Spreading the use of the platform

After the experimental ElPaaso project proved to be effective, Orange moved further with production-ready PaaS impementation. It was then revealed that the company adopted Pivotal Cloud Foundry (PCF) for creating a new self-service mobile application. The idea started in March 2016 and the MVP was delivered in October later that year.

In June 2017, Orange joined the Cloud Foundry Foundation as a Silver member, while spreading the use of PaaS across its subdivisions.

Cloud Foundry CF Summit Europe 2017 Fabien GuichardFabien Guichard, Orange

According to Fabien Guichard of Orange, the company is now running a multi–data center Cloud Foundry deployment, employing Prometheus, Terraform, and Concourse for managing operations. To support this large infrastructure, Orange is actively building its own tools, such as cf-ops-automation and CF WebUI (offering a web interface for Cloud Foundry based on AngularJS and Bootstrap), among others.

The Orange France subsidiary alone, as noted by Fabien at CF Summit Europe 2017, is running 3 data centers, with ~300 apps, 3 open-source Cloud Foundry production deployments, and thousands of servers. (Currently, some of Orange Group’s business units are using OSS Cloud Foundry, while others are hosted on Pivotal Cloud Foundry.)

Multi–data center Prometheus federation (Image credit)

In 2018, the company also presented how the Orange Lean Application Facilities (OLAF) team is using PCF to accelerate cloud-native journey and CI/CD. As a result, for instance, Orange Technocentre managed to save 100 days of development for an MVP, while one of internal PMs mentioned that the development velocity was increased by 1.5x.

Using Pivotal Cloud Foundry in OLAF (Image credit)

Today, hundreads of developers at Orange are already working with the PaaS, aiming to onboard thousands of engineers more, according to Olivier Brun.

 

Want details? Watch the videos!

In this video from Cloud Foundry Summit 2015, Guillaume Berche talks about the experimental ElPaaso project.

 
Here is Fabian Guichard describing how Orange manages a multi–data center Cloud Foundry deployment (2017).

 
In this 2018 presentation, Olivier Brun shows how Orange utilizes Pivotal Cloud Foundry for the OLAF team.

 

About the experts

Guillaume Berche is an active BOSH and Cloud Foundry user since 2012. He is working at Orange, one of the leading European telco operator, where he has been contributing to private PaaS efforts since late 2010. Guillaume’s activities range from product management to software development to operations. He has been contributing to the Cloud Foundry community through cf-dev, cf-docs, specs proposals, and product management to the terraform-provider-cf project. In addition, he is contributing to the OSS orange-cloudfoundry/cf-ops-automation-broker on-demand service framework.

 

Fabien Guichard is a PaaS technical leader and manager for PaaS operations at Orange. He led the production of the first on-premises multi-site platform for Orange Digital Factory (based on BOSH and Cloud Foundry) and hosted services, such as musique.orange.fr and cinema.orange.fr. In his previous position, Fabien was part of the Greenwich project as a technical operations leader.

 

Olivier Brun has spent several years working on cloud infrastructures at Orange by leading the technical streams of the global OLAF platform—a cloud-native transformation plan of Orange at scale. His 10+ years spent at Orange provided him with varied experience from Java development, VoIP validation and tests, DNS skill center and support, before embracing the cloud.

 


The post is written by Alex Khizhniak and Roger Strukhoff with assistance from Hanna Kautur.