Cloud Foundry for Pricing Optimization: GAP’s Experience

by Alesia Bulanok and Alex KhizhniakJanuary 16, 2017

GAP is a global fashion business with multiple brands and retail stores. Philip Glebow shares the company's experience in automating localized promotional pricing using the Gurobi linear optimizer and Pivotal Cloud Foundry.

Why read this?

Use Case for Cloud Foundry:

Achieve essential optimization of pricing, while saving time and computational resources to process large quantities of input data from multiple sources.

Business or Technical Result:

With a new tool set and the continuous integration and delivery (CI/CD) capability, the company is now able to change user interface overnight or push a feature into production in just five minutes.

Lessons learned:

A DBaaS platform (e.g., Tesora) and a data virtualization platform (e.g., Denodo) can simplify data management.
An operational intelligence solution, such as Splunk, will give you valuable insights into the inner workings of your applications.
Use application performance monitoring tools to get a comprehensive view of what's going on across all of your systems.
With the IPsec add-on, security of sensitive data within the Cloud Foundry is no longer an issue.

CF deployed to:

OpenStack, with plans to migrate to a public cloud—Azure or AWS

What else is in the stack?

Gurobi, Spring Boot, RabbitMQ, MongoDB, Java, Jenkins CI, Splunk, Tesora, Denodo, Vault, Consul

Cool fact about deployment:

Moving to Pivotal CF saves developers a lot of trouble with flowing new features into production. The company can now actually "deploy changes faster than people can consume them."

Company Description:

GAP Inc. is a global clothing and accessories retailer. The company has over 3,000 stores in operation across 52 countries and delivers its products to 214 countries. It is the 3rd largest international retailer with annual revenue of $16+ billion.

Cool fact about the company:

Donald and Doris Fisher, the company's founders, came up with the idea to open their first store as once they had difficult time finding jeans that would fit them.

Table of Contents

Price optimization issues at GAP

Philip Glebow, GAP

At the Cloud Foundry Summit 2016, Philip Glebow mentioned localized promotional pricing as one of the GAP‘s initiatives in action. This marketing model relies on the principle of price adjustments, based on the local factors of demand and customer responses to changes in prices. The model has proven to be efficient enough to keep the inventory moving; however, this entails a bunch of issues for the company:

Multiple parameters (including even weather conditions) need to be taken into account for the company to be able “to adjust the price and still move through the inventory.”
Pretty sensitive deadlines—6,000 optimization instances to be completed in about 4 hours.
Requires extensive computational resources (e.g., GAP uses about 90 Amazon’s c4.4xlarge instances).

Why cloud and Cloud Foundry?

With the above issues in mind, GAP started to look for a solution that would satisfy their requirements in a most efficient way. Eventually, the company opted for cloud services since those, as Philip Glebow explained, enable to:

Vary costs—resources are scaled up or down to match the demand, and the company pays for just as much resources as it actually uses at a given moment
Vary performance—make the process faster or slower as needed
Eliminate the need for maintenance—the company does not have “to build a bunch of servers that sit around a data center.”

“The ability to build an elastic pricing solution that can scale on demand is very powerful.”
—Philip Glebow, GAP

Explaining the company’s choice of Pivotal Cloud Foundry, Philip Glebow pointed out the following advantages:

Fast and easy integration and delivery of cloud services on a continuous basis with maximum flexibility

“If we need to change the algorithm—fine, we can run it through a unit test, it flows automatically to production.” —Philip Glebow, GAP

Reduced processing time

“Something that would normally, perhaps, take days or weeks—we can compress that into a cycle of few minutes.” —Philip Glebow, GAP

Very quick to scale
Simplifies the infrastructure
Imposes a set of useful patterns (e.g., microservices, 12-factor apps, etc.)

“Constrains somewhat what people can do, but in a way that I think is very empowering.”
—Philip Glebow, GAP

The adopted concept of microservices has brought a major change to the company’s development workflow, speeding up and automating deployment via a federated continuous integration pipeline.

Deployment overview

Detailing the technical part of the Cloud Foundry deployment project, Philip Glebow described the system they actually built at the company. The system comprises:

Pivotal Cloud Foundry as a PaaS
a Java-based application that talks to MongoDB and RabbitMQ
the Gurobi linear solver
Spring Boot to work with applications
a Spring actuator

Client access to the system is via a web-browser or the REST services.

The GAP and Cloud Foundry story actually started with Pivotal CF that was installed internally. Then, the company felt it has to have access to some other essential process components:

the existing MongoDB database and RabbitMQ (both running on OpenStack)
the Gurobi linear solver (running on Amazon)

That’s how they came to use Cloud Foundry and Spring config to manage their processes, as well as acquire and provision data.

Among major tasks of the existing system, the speaker mentioned managing large data quantities from multiples sources. To solve the task, the company employs the data virtualization technology that functions as an interface layer, enabling to access a variety of databases in use: Oracle, MySQL, MongoDB, etc.

“It allows us to expose an interface layer that can look very much the same across brands and, for us, that has been very powerful.” —Philip Glebow, GAP

Optimization workflow

At GAP, the workflow used to solve the optimization tasks is as follows:

Creating cloud instances of the compute servers, scaling those up or down to the needs.

“The process takes a good two minutes.” —Philip Glebow, GAP

The RabbitMQ component builds a central queue of optimization jobs, and then a dedicated group of workers deals with the queue, thus enabling to solve independent optimization problems in parallel.

Processing data and reports.

What’s the price?

Just like any other project, building a Cloud Foundry-based system at GAP involved certain challenges:

Experts in data science and optimization would use Python, MATLAB, or SAS—rather than Java chosen for the project.
Essentially waterfall environment is unwilling to adopt the Agile development approach.
Though Spring Cloud Config helped the company to accelerate scaling, simplify adding new instances, and cope with some other complex problems, its use revealed a number of inherent deficiencies:

Hand encryption is inconvenient on an enterprise level.
Git becomes a single soft spot, as its failure can cause an overall stop.
One needs to mind issues with web hooks during Blue-Green deployment.
The audit trail in Git missing information about the effective date of a committed change.

There are more than 60 TB of logging data from numerous applications, which needs to be managed.
Due to specifics of the operational process at GAP and the wide variety of databases used at the company, setting up data services for applications takes up much time and efforts.
The company has strict security requirements concerning protection of personal data (e.g., names, addresses, credit card details, etc.).

Lessons learned

Any project of as big a scale is sure a source of invaluable experience, both for the implementing company and anyone who follows its footsteps:

Adopting a technology or a development approach that is different from what is widely accepted in the industry requires a major cultural change. The main thing here is try to be flexible, investigating into options and opportunities, and be prepared to take reasonable risks.

“We’re happy to make mistakes as long as the blast radius isn’t too large if it fails.”
—Philip Glebow, GAP

Collaborating with a broader community can save time and resources in finding proper solutions to eliminate major deficiencies.
Splunk can do the job of managing logging data just all right, while providing additional capabilities, such as alerting.

Setting up data services could be made much simpler with a DBaaS platform, e.g., Tesora. Data virtualization technologies (e.g., Denodo data virtualization platform) can come in handy, too, especially if one has to deal with a lot of legacy systems and data centers.
Additional use of specialized tools for application monitoring can help to quickly identify and solve any performance problems across the entire infrastructure—both in Cloud Foundry and legacy systems.
Installing the IPsec add-on is enough to protect any sensitive information within Pivotal Cloud Foundry.

What’s next?

The company’s vision of its future is dominated by the idea of migrating to a public cloud—most probably, Microsoft Azure. Philip Glebow shared these plans at the SpringOne Platform conference in December 2017, with the following reasons mentioned as underlying the move:

Better speed and a larger scale
Geographic proximity of services
New technologies to adopt, including machine learning

However, despite the advantages, there’s the financial side of the question that can’t but be taken into consideration.

“Some of these services are very cheap at the low end but, if you start to really scale them up and if you look at the cost modeling, I think it is really expensive, really fast,” —Philip Glebow, GAP

Therefore, the company will still have to “figure out the right way” when migrating to a public cloud. It’s interesting that another company, WMG, did a similar move with their Cloud Foundry deployment, migrating from OpenStack to AWS last year.

Want details? Watch the video!

Table of contents

What are the price optimization models employed at GAP? (1:44)
What were the company’s cloud requirements? (2:59)
Technology stack: integrated components (4:53)
How did the company organize the deployment process? (6:22)
What were major tasks to achieve price optimization? (7:21)
What challenges did GAP face when adopting Pivotal Cloud Foundry? (8:40)

About the speaker

Philip Glebow is now the product architect for pricing in the planning domain at GAP Inc. He can boast of over 15 years of experience in software system design and cooperation with companies such as KPMG Consulting Inc., Blackrock Inc., and Barclays. Mr. Glebow holds a Bachelor of Science in Computer Engineering (Cal Poly San Luis Obispo) and a Master of Science in Bioinformatics (John Hopkins University)

The post is written by Alesia Bulanok and Alex Khizhniak.

Posted In