Backgroud
Tanzu Application Platform
Tanzu Application Platform (TAP) is an amazing tool and can run on ANY kubernetes cluster.
One of the great parts of TAP is how easy it is to deploy and get started with.
TAP is a platform that allows us to easily go from source code to a running app, in a secure, efficient, and really easy way while baking into our processes throughout the entire supply chain best practices.
Tanzu Community Edition
Tanzu Community Edition (TCE) is the open source kubernetes distribution from VMware. beyond being an amazing kubernetes distribution, TCE includes within it much of the opensource tooling that backs the entire Tanzu commercial portfolio. this includes things like kpack which is the base technology in Tanzu build service, Cartographer which is a core component in TAP, knative which is the core technology in Cloud native runtimes and much more.
Tanzu Community edition is very similar from a UX perspective to TKGm and actually is based on the same underlying framework which has been open sourced by VMware called Tanzu Framework.
TCE Unmanaged Clusters
Within TCE, beyond having the ClusterAPI based cluster lifecycle that is similar to the commercial Tanzu offerings, we have the ability to provision local development clusters using a feature called unmanaged clusters.
Unmanaged Clusters, utilize either KinD or minikube as the backend technology for deploying a local kubernetes cluster and then “Tanzuify” them by installing all of the bits and pieces that all Tanzu clusters include within them such as a CNI (Calico or Antrea), CSI, Kapp Controller, Metrics server and more!
Why run TAP locally on your desktop
As i have been playing with TAP for a long time, and always want to test the new versions and spin up ephemeral environments to try out different things, I decided to work on building a solution to easily install TAP on a local machine.
This can also be helpful if your company uses TAP. This can allow your developers to have a local development cluster, where they will have the same experience and tools that you have in your organizations clusters, locally on their desktops for quick iterative development.
TAP + TCE = Magic!
TCE 0.12.0 added in some amazing new capabilities around unmanaged clusters, this includes for example the ability to add addition carvel package repositories and auto install packages onto a cluster as part of its bootstrapping process.
Because TCE is truly giving us a Tanzu like cluster on our local machine with the unmanaged cluster functionality, this becomes a great choice for how to deploy TAP locally on our desktop.
All of the prerequisites that a cluster must have in order to install TAP, are already installed and managed via TCE, which makes the installation process very simple.
TCE 0.12.0 also added the ability to stop and start unmanaged clusters via a simple command. this allows us to save resources on our local machines when we dont need our cluster up and running without needing to recreate it from scratch every time.
How I automated the setup
As part of my testing of TAP, i decided to put together a simple bash script that can manage the entire lifecycle of deploying TAP in an easy way on a TCE unmanaged cluster, with as few required inputs as possible.
Prerequisites
In order to install TAP and TCE we need to have a few things on our machines:
- Docker – this can be Docker Desktop on Windows or MAC, or Docker itself on Linux.
- Tanzu CLI – this is the main CLI for both TAP and TCE
- Kubectl – this is needed for interaction with our kubernetes cluster
Because of the many different ways of installing Docker, and the significant differences in the process based on the OS you are using, I decided for the meantime to not automate the installation of Docker.
For Tanzu CLI and Kubectl however, I have included the functionality in the script to automatically install / upgrade both of these tools on both MAC and Linux. if you are using Windows, this has been tested and works on WSL, however native windows support is currently not available, as the script itself is written in bash and in general, managing kubernetes natively in windows is not the best experience in my opinion.
We also need to relocate the TAP package repository to our own container registry so as to not be reliant on the VMware registry for our platform to work on a day to day basis.
Relocating the Package repository is documented well in the TAP documentation, however i decided to also write a small script to make the process even easier and more streamlined.
This script does require you to have docker as well as the imgpkg cli installed on your machine.
What does the script create
Beyond simply creating a local cluster and installing TAP on to the cluster, I have also included a few additional things to make the experience even more streamlined and user friendly.
The first addition i made is installing a local docker registry alongside the kubernetes cluster. this allows for really quick image pulling and pushing, as well as removes additional dependencies on an external container registry for hosting our images. while i am a huge fan of Harbor, which is included in TCE, I decided to go with a local registry in order to save resources as harbor requires a lot more in terms of resources than a simple docker registry container does.
Because we are using a local registry without TLS configurations to simplify the setup, I have also automated the required configuration within our TCE cluster so that Containerd within the cluster can trust our registry. I have also added in the functionality to update the local hosts file on your machine so that the docker registry can be accessed locally using the docker CLI from your machine for any needs you may have.
Another thing that has been added is Kyverno, along with a custom ClusterPolicy.
Kyverno is an amazing tool which is currently a CNCF sandbox project and hopefully soon will graduate to incubating.
Kyverno is a policy engine designed for Kubernetes. With Kyverno, policies are managed as Kubernetes resources and no new language is required to write policies. This allows using familiar tools such as kubectl, git, and kustomize to manage policies. Kyverno policies can validate, mutate, and generate Kubernetes resources plus ensure OCI image supply chain security. The Kyverno CLI can be used to test policies and validate resources as part of a CI/CD pipeline.
Kyverno in this setup is utilized in order to automate the steps needed when creating new namespaces to “prepare them” for TAP. this includes steps such as, adding the needed secrets, roles and rolebindings, image and source code scanning templates, tekton tasks, service accounts and more.
By adding in kyverno and the custom policy, its as easy as creating a namespace and within seconds the namespace is ready for you to install workloads into.
I’ve also used the great port mapping capabilities of TCE unmanaged clusters, in order to expose all of our needed UIs and workloads locally on our machine on both ports 443 and 80. this allows us to have access to the wealth of TAP GUI, and test our web applications from our browsers without needing any DNS management, port forwarding, and other steps otherwise required.
Additional Capabilities
The script also includes functions to allow you to easily, stop, start, and delete the environment. It also includes a handy function to check on the status of the environment.
How to get started
Both of the scripts have been uploaded to a git repo which you can easily clone down to your local machine.
git clone https://github.com/vrabbi/local-tap-setup cd local-tap-setup
Relocating The TAP Repo
Once you have the repo cloned locally, and have at minimum Docker and imgpkg installed, you can now relocate the TAP Package repository to a Registry of your choosing.
In my case i chose to relocate the images to my local harbor instance in my lab.
chmod +x ./relocate-tap-repo.sh ./relocate-tap-repo.sh
This script will run a validation check to make sure you have the needed prerequisites installed and then will ask you for some information:
- Your username for Tanzu Network
- Your password for Tanzu Network
- Your destination registry FQDN
- Your destination registry repo / project / namespace
- Your destination registry username
- Your destination registry password
- TAP Version to relocate
Once the relocation is complete, you can now utilize this repo from anywhere that has access to this repository which is really awesome!
Installing Prerequisites for Environment creation
Now that we have our repo relocated, we next need to make sure we have all the needed CLI tools installed.
chmod +x ./local-tap.sh ./local-tap.sh --action prepare
This will make sure that your installed version of Tanzu CLI and Kubectl are at the correct versions and will install / upgrade them as needed. it will also validate that you have docker installed however it will not automate the installation if it is not present.
Creating an environment
Now that we have all of our prerequisites installed and configured we can install our local environment. I have added a bunch of customization options via flags to the scripts which are all documented in the help command
./local-tap.sh --help
For a simple installation using the default configuration the installation command would be:
export TANZUNET_USER="YOUR USERNAME" export TANZUNET_PASSWORD="YOUR PASSWORD" export TAP_REPO="THE RELOCATED REPO URL" ./local-tap.sh --action create \ --tanzunet-user $TANZUNET_USER \ --tanzunet-password $TANZUNET_PASSWORD \ --tap-package-repo-url $TAP_REPO
Example of configurations you could also provide are, which TAP profile to install (by default its the full profile but the iterate profile is also supported), which TAP Version to install (by default its 1.1.1) and more!
The script will take typically between 8-12 minutes to run if you are using a registry within your environment and not pulling from the internet and a bit longer when using a registry like Dockerhub or GHCR due to latency.
The script will tell you what step it is on and will at the end, also print exactly how long it took to run.
Accessing the TAP GUI and also any application you deploy on the platform is very easy, as the script configures the ingress suffix to be *.127.0.0.1.nip.io.
This means that you can simply access the TAP GUI for example at:
http://tap-gui.127.0.0.1.nip.io
Checking the status of the environment
If you ever want to check the status of your local TAP environment, you can run a simple command:
./local-tap.sh --action status
Shutting down the environment
When you don’t need the environment up and running, but want the workloads and configurations to not be lost, you can simply shutdown the environment using the command:
./local-tap.sh --action stop
Starting back up the environment
When you want to bring back up the environment, you can simply run:
./local-tap.sh --action start
This can take around 5 minutes after completion until all of the system is back up and running. to check on the status you can use the status command mentioned above.
Deleting the environment
When you are done with the environment and want to delete it, its as simple as:
./local-tap.sh --action delete
Summary
Its truly awesome to see how the different parts of the Tanzu portfolio can work together and offer a really great user experience for both platform operators as well as developers.
TAP and TCE are both amazing technologies and using them together is a really great experience!
If you try out the scripts and have any issues or feature requests, please open up GitHub issues on the repo and I will try and get the feedback incorporated into the automation.