Cloud Foundry Monitoring with Admin UI: Technical Overview
Technology basics (Ruby, the Cuba framework, Redis)
Admin UI is a web service that allows to gather metrics from the Cloud Foundry components. The metrics of Admin UI are based on data taken from the UAA, Cloud Controller databases, and NATS. Admin UI checks the /varz and /healthz endpoints of the Cloud Foundry components. You can find more information about these endpoints in the official documentation. You can read about the Admin UI dependencies in this GitHub repo.
Admin UI is a Sinatra-based application that uses SQLite3 to cache data and store stats. To support multiple possible UAA and CC databases (currently, Cloud Foundry supports PostgreSQL and MySQL), Admin UI uses the Sequel gem. That’s why it requires some additional packets to be installed, including the PostgreSQL and MySQL client libraries.
With Admin UI BOSH release, you can expose Admin UI through the CF Router, using cf-registrar. Still, it is not the best way to do it.
Components and layers
Admin UI is a JavaScript-based Web application: user’s browser gets a static file (application.html) that makes multiple AJAX requests to the server side (a single AJAX request for each tab). These AJAX requests are sent only once, so to update information, you need to refresh a browser page. Data is mostly transferred in the JSON format.
UAA is used as an authentication tool in the latest version of Admin UI. When a user opens the application for the first time, he/she is redirected to the UAA endpoint with a login form. Then, a user is redirected back to Admin UI with username and access scope data in parameters. A username and a scope are stored to a session and are further utilized to identify if a user has access to specific tabs.
The authentication process requires a special registration of Admin UI within UAA. In order to add Admin UI as the UAA client, you can use a script from Admin UI that provides access to UAA via the cf-uaac gem. If you install Admin UI with the BOSH v4 release, you can run the errand jobs that will do it for you.
The Admin UI server has a special thread that updates the Cloud Foundry components data in the background. It has a low priority and doesn’t hamper the server’s work. This background thread gathers data from the Cloud Foundry components and stores it into SQLite3. Time period required for the thread to gather data is set in configurations. After data is saved to SQLite3, it is accessible by the main thread and is used in responses of a client. The server allows for performing some management tasks for the Cloud Foundry deployment, too. Since the Admin UI client access scope is used to get data or perform any management task, you will not be able to perform actions allowed to your user, but restricted to Admin UI.
Functionality
Admin UI provides access to the CF components data and helps to manage Cloud Foundry deployments. Deeply integrated with Cloud Foundry, Admin UI brings forth loads of relevant information: a list of DEA components, a list of each DEA app containers, the UAA organizations, users and groups, statistics of resource consumption, etc.
Integration with Cloud Foundry APIs
To enjoy a full scope of possibilities offered by Admin UI, it should be connected to the following components:
- NATS: the NATS client gem is used.
- The Cloud Controller REST API: no special library, all work with an API is implemented within Admin UI.
- the Cloud Controller DB URI: the sequel gem is used to connect the PostgreSQL or MySQL database.
- the UAA REST API: no special library, all work with an API is implemented within Admin UI.
- the UAA DB URI: the sequel gem is used to connect the PostgreSQL or MySQL database.
Performance / scaling
Admin UI is designed to work within the Ruby process, which implies strict constraints on scaling. While you can scale vertically without any limitations, horizontal scaling calls for using a load balancer with a sticky session. As a rule, Admin UI is used by a limited number of operators, so there is no data about request workload it can handle.
Error handling, logging, and debugging
If you get the “This page requires data from services that are currently unavailable” message, it means that Admin UI doesn’t have access to the UAA or CC databases or is not able to get data from NATS.
To check if databases are available, you should SSH to the instance, where Admin UI is installed, and try to get access to the databases with the client tools. Make sure Admin UI has correct settings for the databases, too. Learn more about the bug in this GitHub issue.
To find out if NATS is available, you need to get its client and connect to NATS from within the Admin UI virtual machine. To do it, check out the NATS version in the Admin UI Gemfile and install a correspondent version of a gem. Then, you need to connect to NATS to prove it is possible.
gem install nats -v <nats-version> nats-sub ‘>’ nats://nats-user@nats-password@nats-host:nats-port
Another issue occurs, when you have multiple instances of Admin UI and the load balancer that works in the “round robin” mode (you can find details here). In this case, only the Admin UI header will be shown.
If you get the “Wrong Scope” message after authentication to Admin UI, it means you haven’t configured the Admin UI UAA client properly.
Localization (Chinese and Russian)
Support for the Russian and Chinese languages is implemented in the Altoros’s fork of Admin UI in the i18n branch.
The pull request with the changes to the official repository mentioned is on approval to be merged.
You can deploy a version with the Chinese and Russian support, using our special BOSH release. Discover more about the Admin UI BOSH release with i18n in this blog post.
Issues / bugs
While using Admin UI, you may face the following issues:
- The Admin UI server uses the SQLite3 file system database for caching information about Cloud Foundry. It may cause problems during horizontal scaling. See the “Performance / scaling” section of this blog post to learn how to solve this issue.
- The authentication process doesn’t recognize the changes made to the user access scope. It means if a user is logged in Admin UI, the result is the same, even after he/she is removed from DEA or his/her access scope is changed.
Admin UI proved to be a great tool with a strong CF bond, using not only the Cloud Foundry REST API, but databases of such core components as the Cloud Controller and UAA. Therefore, Admin UI is able to monitor the Cloud Foundry vital data in real time without network overloading.
Further reading
