How to Push Private Docker Images and Enable Caching on Cloud Foundry Diego
In this post, we will go over two solutions for those who cannot have their data in the open: private Docker registries and Diego Docker Cache. Below, you will find installation steps for both and a demo of how to deploy Diego Docker Cache.
Prerequisites
These features were tested against the Cloud Foundry release v234 and the corresponding Diego release v0.1463.0 on an OpenStack deployment. (If you are not familiar with Diego, check out this GitHub repo to learn how to run Docker images on Cloud Foundry.)
This post assumes that you are fairly comfortable with BOSH, Cloud Foundry’s native deployment tool, since both solutions come as BOSH releases. You do not have to use BOSH to install a registry for Docker, but I strongly recommend that you do, because:
- BOSH is good at working with image storage.
- BOSH can easily scale your registry when you run out of space.
How Docker containers work with Diego
First of all, let’s define what Docker containers and images are. An image is a read-only set of root file system changes and corresponding startup parameters. A container is a runtime instance of an image.
To support containers, Diego uses the Garden Linux back end, which can create Linux containers using namespaces and cgroups—core features of the Linux kernel. With Garden, you can extract all Docker image layers and then use Docker libraries to unite and mount them as a root file system. For more information about Docker support in Diego, see this documentation.
Options for storing Docker images privately
Docker images are usually stored in a registry. By default, Docker uses the public Docker Hub registry and interacts with it through the Docker Registry HTTP API. Those who would like to hide their images from the rest of the world have several options, the most obvious ones being:
- Setting up their own Docker registry in a private cloud.
- Getting a private registry on Docker Hub and using Diego Docker Cache to push images to Diego. As the name suggests, Diego Docke Cache also adds features for caching Docker images. This helps you to stay independent from Docker Hub availability and speed up scaling of apps.
Let’s take a closer look at each of the solutions.
Option #1: A private Docker registry
A stand-alone Docker registry is the most straightforward way to make your Docker images private and still be able to push them to Diego. Getting one is easy:
- Download and deploy the Docker Registry BOSH release (see the README file for installation steps).
- Provide Cloud Foundry with the address of the private registry by passing the
insecure_docker_registry_listproperty into yourproperty-overridesstub file for the Diego deployment. See this example from the Diego manifest generation stubs.
After completing the steps, you should be able to push images using the Cloud Foundry CLI:
cf push myapp –o docker-registry.example.com:5000/myapp
As a short summary, I would recommend a self-hosted Docker registry under the following circumstances:
- You want to have an unlimited number of private Docker repositories.
- You need to stay independent from third-party services.
- You have to keep all images inside your private cloud.
- You don’t want to be affected by network bandwidth.
If you decide to roll out your own registry, keep in mind that:
- Your will be solely responsible for supporting the self-hosted Docker registry. This also means you will need to be prepared for and deal with any disasters.
- For small teams, a private Docker registry (e.g., on S3) may turn out to be much more expensive than using Docker Hub.
- No UI is available, so it will probably be hard to search and maintain the image storage.
Option #2: Diego Docker Cache
The second option is to use a private Docker Hub repository and Diego Docker Cache—a BOSH-manageable release that consists of two parts: a Docker registry and caching support for Diego.
To successfully push a private Docker image to Cloud Foundry, you will need to provide Docker Hub credentials during the application staging process. This can be done with the cf curl command. I have prepared a small script that helps to simplify this procedure. Below, you can find a demo that demonstrates the process. As a result, we will get a running application, which was pushed from a private Docker Hub repository.
By adding the caching capabilities, we can solve the following problems:
- Speed up the application scaling process
- Become independent of Docker Hub availability
- Get a way to push private Docker Hub images to Diego
The main issue with Diego Docker Cache is that the whole caching process runs inside a Garden container where the OverlayFS file system is not available. So, it may take a lot of time and resources for initial application staging.
In order to avoid failures during staging, I suggest that you change disk size for the staging process. To do that, update these properties in the Cloud Foundry deployment manifest and configure it in accordance with your environment.
Can I have a private Docker registry and caching, too?
When choosing storage for your Docker images, start by considering your actual needs and possibilities. Even if you end up using a self-managed Docker registry, I would still recommend that you install Docker Cache. It will help you to stay independent of the image storage and ensure that your apps can be scaled up regardless of registry availability. So, the answer is “Yes, you can.”
Further reading
