CSAA Insurance Group Deploys Three Times a Day with Cloud Foundry

by Carlo GutierrezNovember 22, 2017
One of the world's oldest insurance companies reduces the development-to-production cycle from eight days to 20 minutes.
Why read this?
Use Case for Cloud Foundry:

The insurance provider for the American Automobile Association (AAA) uses Pivotal Cloud Foundry to speed up deployment frequency by over 1,000%.

Business or Technical Result:
  • New applications are being deployed at a rate of three times a day.
  • Ideas are being brought into production in just 2.5 days.
  •  

    Lessons learned:

    Break transformation disbelief by building something that works and is better than the current system.

    CF deployed to:

    VMware vSphere

    What else is in the stack?

    Code Climate, BrowserStack, Concourse, Cucumber, TravisTorrent, Sonar, Runscope, Datadog, New Relic, Statuspage, Rollbar, PagerDuty, BIME, Amazon Redshift, Google Analytics, Segment, Mixpanel, Databox

    Cool fact about deployment:

    All flagship applications now run on Pivotal Cloud Foundry.

    Company Description:

    CSAA Insurance Group is one of the largest insurance providers in the United States serving over 17 million AAA members. In 2015, the company wrote $3.3 billion in premiums.

    Cool fact about the company:

    CSAA Insurance Group can trace its origin all the way back to 1900, where it all started with a meeting between automobile enthusiasts at the Cliff House in San Francisco. The organization began offering automobile insurance in 1913 and homeowner insurance in 1974.

    Detecting the pain points

    Several organizations shared their success stories during the Cloud Foundry Summit Europe 2017. Among these companies is CSAA Insurance Group, an insurance provider for the American Automobile Association, Inc.

    Kyle Campos, a Technology Operations Manager, took to the stage and explained how the company began their digital transformation journey back in 2012, and how they are now rolling out new applications within three days using Pivotal Cloud Foundry.

    CF Summit Europe 2017 Kyle Campos CSAA InsuranceKyle Campos at Cloud Foundry Summit Europe 2017 (Source)

    “If you thought getting people to believe transformation is possible was tough, wait until you try and convince people that you actually accomplished successful transformation.”
    —Kyle Campos, CSAA Insurance

    The early days of the CSAA Insurance’s transformation were devoted to looking at the current pain points within the organization. According to Kyle, CSAA Insurance had the following issues:

    • No automation
    • Too much tech debt
    • Overreliance on silos
    • No insights and analysis

    On top of this list, there was an even bigger issue with the culture at work. “The worst part of it was that almost no one believed it could be any different,” said Kyle. “The ones that did believe it could be different were either powerless or didn’t have the insight or ability to help in any meaningful way.”

    In this scenario, deployment of new applications would take at least a week if everything ran smoothly. Patches were rolled on a quarterly basis. All in all, it could be better.

     

    Enabling continuous integration with Cloud Foundry

    The DevOps team at CSAA Insurance began with improvements related to enabling monitoring and analytics, automated testing, and continuous integration with static analysis engines.

    Next, the most significant change for the team began with collaboration with Pivotal Labs, as Kyle noted. Through these engagements, the organization was introduced to Agile development and Cloud Foundry.

    CSAA Insurance’s first foray into Cloud Foundry was with Pivotal CF on-premises. This was quickly shut down before even touching production.

    “After the deal was signed, we were told ‘you guys are actually thinking of using this thing that we spent a bunch of money on in production? That’s not going to happen’,” explained Kyle. “Needless to say, we were bummed out about that. This was not just a dev project for us to prove that we could deploy to our dev environments quickly.”

    CF Summit Europe 2017 CSAA Insurance Pivotal Cloud Foundry Digital Services PlatformAll flagships apps now run on Pivotal CF (Source)

    In the next attempt, the team made use of Pivotal CF in a public cloud, which was set up within a week. In another few weeks, three foundations were up and running with automated CI/CD pipelines.

    “We had monitoring across infrastructure and APM. We had centralized logging,” said Kyle. “Most shocking was that engineering had access to it all.”

    “In our most successful enterprise version of this, it would take at least three months to even get development VMs. That was obviously world shifting.” —Kyle Campos, CSAA Insurance

    Though this configuration found success, there were concerns about putting information on a public cloud. This was resolved by setting up Pivotal CF on a private cloud.

     

    Faster development cycles

    The shift from legacy to Pivotal CF had an impact throughout CSAA Insurance. In development, this manifested as:

    • 205% increase to developer productivity
    • 1,400% increase to a deployment frequency
    • 1,614% increase to a patch frequency
    • 99.8% decrease to development to a production cycle
    • 96.5% decrease to the initial environment cost

    In real numbers, this translated to:

    • The deployment frequency from once a week to three times a day
    • The development-to-production cycle from eight days to 20 minutes
    • The path frequency from once a quarter to once a week
    • People required to maintain the development-to-production cycle from four to none

    CF Summit Europe 2017 CSAA Insurance Pivotal Cloud Foundry Time to MarketFrom a concept to production in just 2.5 days (Source)

    “One of the challenges for us around value stream mapping is some of the numbers are so high that you just want to arbitrarily bring them down, so executives believe it.”
    —Kyle Campos, CSAA Insurance

     

    Lessons learned

    Though the DevOps team needed three attempts to finally nail down Pivotal CF for CSAA Insurance, Kyle noted that it wasn’t all a waste of time. “At the end of the day, our action in take two forced the conversation,” he explained. “Without doing that, Pivotal CF would never have happened for us.

    CF Summit Europe 2017 CSAA Insurance Pivotal Cloud Foundry infrastructure comparisonResource utilization is much better on Pivotal CF (Source)

    Throughout the journey, the team learned valuable lessons, which anyone looking to adopt Cloud Foundry should be mindful of:

    • You have to build the real thing before others are convinced that it’s the right thing.
    • Sometimes you have to put your credibility on the line.
    • Build a new team that can learn new technologies and then teach the old team.
    • Be prepared for rapidly increased development time.
    • Not everyone will be happy with change and transformation.
    • Cloud Foundry will be blamed for whatever goes wrong with the new applications.

    Now that CSAA Insurance is over the Cloud Foundry hump, Kyle and his team are looking at other improvements down the line, including high availability, multi-cloud, and Pivotal Cloud Cache.

     

    Want details? Watch the video!

    Table of contents
    1. When did CSAA Insurance’s digital transformation start? (1:40)
    2. What were the problems? (3:40)
    3. What were the first steps for improvement? (5:00)
    4. Enabling Pivotal CF on-premises (8:30)
    5. Moving to Pivotal CF on a public cloud (9:50)
    6. Moving to Pivotal CF on a private cloud (12:30)
    7. What ended up in Pivotal CF? (16:00)
    8. How much of an impact did Pivotal CF have? (17:10)
    9. How much faster is the development with Pivotal CF? (20:00)
    10. What were the lessons learned? (21:20)

     

    Further reading

     

    About the expert

    Kyle Campos, CSAA Insurance bio
    Kyle Campos is a digital transformation and cloud operations leader, who is highly motivated by turning non-scalable, resource-intensive, and painful systems into automated, scalable, and efficient ones. He is a full-stack engineer, who loves attacking challenging operational and infrastructure issues through both organizational leadership and software architecture, specializing in legacy Ops to DevOps transformations in either startup or enterprise contexts. Kyle is currently working hard to deliver to application teams all the promises of a highly available, highly performant PaaS through Pivotal Cloud Foundry.