--- title: "kr8 – Configuration Management for Kubernetes Clusters" source: "https://wordpress-1075849-4005834.cloudwaysapps.com/blog/kr8-configuration-management-for-kubernetes-cluster/" --- [Technical Blog](https://wordpress-1075849-4005834.cloudwaysapps.com/category/technical-blog/) # kr8 – Configuration Management for Kubernetes Clusters By [Lee Briggs](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/author/lee-briggs/) on Jun 27, 2019 • 5 min read This article originated from [http://leebriggs.co.uk///blog/2018/11/07/kr8-kubernetes-config-mgmt.html](http://leebriggs.co.uk///blog/2018/11/07/kr8-kubernetes-config-mgmt.html) Previous visitors to this blog will remember [I wrote about configuration management for Kubernetes clusters](http://leebriggs.co.uk/blog/2018/05/08/kubernetes-config-mgmt.html), and how the space was lacking. For those not familiar, the problem statement is this: it’s really hard to maintain and manage configuration for components of multiple Kubernetes clusters. As the number of clusters you have starts to scale, keeping the things you need to run in them (such as ingress controllers) configured and in sync, as well as managed the subtle differences that need to be managed across accounts and regions. With that in mind, it’s my pleasure to announce that at my employer, [Apptio](http://www.apptio.com) we have tried to solve this problem with [kr8](http://github.com/apptio/kr8). kr8 is an opinionated Kubernetes cluster configuration management tool, designed to be simple, flexible and use off the shelf tools where possible. This blog post details some of the design goals of kr8, as well as some of the benefits and a few examples. ### Design Goals The intention when making kr8 was to allow us to generate manifests for a variety of Kubernetes clusters, and give us the ability to template and override yaml parameters where possible. We took inspiration from a variety of different tools such as [Kustomize](http://github.com/kubernetes-sigs/kustomize), [Kasane](http://github.com/google/kasane), [Ksonnet](http://github.com/ksonnet/ksonnet) and many others on our journey to creating a configuration management framework that is relatively simple to use, and follows some of the practices we’re used to as Puppet administrators. Other design goals included: - No templating engine - Flexibility - Compatibility with existing deployment tools, like [Helm](http://www.helm.sh/) - Small binaries, with off the shelf tools used where needed The end goal was to be able to take existing helm charts, or yaml installation manifests, then manipulate them to our needs. We chose to use [jsonnet](http://jsonnet.org/) as the language of kr8 due to its data templating capabilities and ability to work with both JSON and YAML. ### Terminology & Tools #### kr8 kr8 itself is the only component of the kr8 framework that we wrote at Apptio. Its purposes are: - Discover clusters in a [hierarchical directory tree](http://github.com/apptio/kr8-configs/tree/master/clusters) - Discover components in a [components directory](http://github.com/apptio/kr8-configs/tree/master/components) - Map components to clusters, using a [cluster.jsonnet](http://github.com/apptio/kr8-configs/blob/master/clusters/gke/cluster.jsonnet) file You can see the result of these purposes using a few of the tools in the kr8 binary, for example, listing clusters: ![kr8 cluster list showing cluster names (do, gcloud, etc.) and their config file paths, illustrating Kubernetes cluster](/app/uploads/2019/06/Screenshot-45.png) However, using the kr8 binary alone is probably not what you want to do. We bundle and use a variety of other tools with kr8 to achieve the ability to generate manifests for multiple clusters and deploy them. ### Task [Task](http://github.com/go-task/task) does a lot of the heavy lifting for kr8. It is a task runner, much like [Make](http://www.gnu.org/software/make/) but with a more flexible DSL (yep, it’s yaml/json!) and the ability to run tasks in parallel. We use [Taskfiles](http://taskfile.org/#/usage?id=getting-started) for each component to allow us to build the component config. This gives us the flexibility to use rendering options for each component that make sense, whether it be pulling in a Helm chart or plain yaml. We can then input that yaml with kr8, and manipulate it with jsonnet code to add, modify the resulting kubernetes manifest. Alongside this, we use a taskfile to generate deployment tasks and to generate *all* components for a Task. This gives us the ability to execute lots of generate manifest jobs in relatively short periods of time. ### Kubecfg We use [Kubecfg](http://github.com/ksonnet/kubecfg) to do the actual deployment of these manifests. Kubecfg gives us the ability to validate, diff and iteratively deploy Kubernetes manifests which `kubectl` does not. The kubecfg jobs are generally inside Taskfiles at the cluster level. ### Components Components are very similar to helm charts. They are installable resource collections for something you’d like to deploy to your Kubernetes clusters. A component has 3 minimal requirements: - `params.jsonnet`: contains configurable params for the component - `Taskfile.yml`: Instructions to render the component - An installation source: This can be anything from a pure jsonnet file to a helm input values file. Ultimately, this needs to be able to generate some yaml I intend to write many more kr8 blog posts and docs, detailing how kr8 can work, examples of different components and tutorials. Until then, take a look at these resources: - [kr8](http://github.com/apptio/kr8) binary - [kr8-configs](http://github.com/apptio/kr8-configs) example repo - [cluster_config](http://github.com/jaxxstorm/cluster_config) my cluster configuration repo ### Acknowledgements I want to thank Colin Spargo, who came up with the original concept of kr8 and how it might work, as well as contributing large amounts of the kr8 code. I also want to thank Sanyu Melwani, who had valuable input into the concept, as well as writing many kr8 components. Finally, a thank you to our employer, Apptio, who has allowed us to spend time creating this tool to ease our Kubernetes deployment frustrations. If you’re interested in working on fun projects like this, we are [hiring for remote team members](http://www.apptio.com/about/careers/job-openings) [Lee Briggs](http://www.linkedin.com/in/briggsl/) is a Tigera guest blogger. He is an experienced DevOps Engineer Specialties: Redhat, Unix, Linux, Monitoring, Kubernetes, AWS,Containers, Hashicorp, DevOps, SRE ————————————————- [**Free Online Training**](https://wordpress-1075849-4005834.cloudwaysapps.com/events/) Access Live and On-Demand Kubernetes Tutorials [**Calico Enterprise – Free Trial**](https://www.calicocloud.io/home) Solve Common Kubernetes Roadblocks and Advance Your Enterprise Adoption [Open Source](https://wordpress-1075849-4005834.cloudwaysapps.com/tags/open-source/) Add Tigera as a preferred source on Google to see more of our content in your search results. [![Add as a preferred source on Google](https://wordpress-1075849-4005834.cloudwaysapps.com/app/themes/tigera2021/dist/img/google-preferred-source-light.png)](https://www.google.com/preferences/source?q=tigera.io) ## Related posts [![Join Calico at KubeCon Europe 2026: AI Agents, Silent Discos, and Dutch Delights!](https://wordpress-1075849-4005834.cloudwaysapps.com/app/uploads/2026/02/Blog-Kubecon-Euro-2026-.png)](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/calico-at-kubecon-cloudnativecon-europe2026/) [Company Blog](https://wordpress-1075849-4005834.cloudwaysapps.com/category/company-blog/) #### [Join Calico at KubeCon Europe 2026: AI Agents, Silent Discos, and Dutch Delights!](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/calico-at-kubecon-cloudnativecon-europe2026/) By [Tigera Team](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/author/marketingtigera-io/) on Feb 25, 2026 The cloud-native community is heading to the historic canals and vibrant tech scene of Amsterdam for KubeCon + CloudNativeCon Europe 2026! From March 23–26, Amsterdam will be buzzing with the latest in Kubernetes, platform engineering,... [Read more](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/calico-at-kubecon-cloudnativecon-europe2026/) [![Project Calico 3.30+ Hackathon: Show Us What You Can Build!](https://wordpress-1075849-4005834.cloudwaysapps.com/app/uploads/2026/02/Blog-Hackathon-.png)](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/project-calico-3-30-hackathon-show-us-what-you-can-build/) #### [Project Calico 3.30+ Hackathon: Show Us What You Can Build!](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/project-calico-3-30-hackathon-show-us-what-you-can-build/) By [Tigera Team](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/author/marketingtigera-io/) on Feb 17, 2026 Build the Future of Cloud-Native Networking! 🚀 The Calico community moves fast. With the releases of Calico 3.30 and 3.31, brings improvements in scalability, network security, and visibility. Now, we want to see what YOU... [Read more](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/project-calico-3-30-hackathon-show-us-what-you-can-build/) [![Why Kubernetes Flat Networks Fail at Scale—and Why Your Cluster Needs a Security Hierarchy](https://wordpress-1075849-4005834.cloudwaysapps.com/app/uploads/2026/01/Why-Kubernetes-Flat-Networks-Fail-at-Scale.png)](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/why-kubernetes-flat-networks-fail-at-scale-and-why-your-cluster-needs-a-security-hierarchy/) #### [Why Kubernetes Flat Networks Fail at Scale—and Why Your Cluster Needs a Security Hierarchy](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/why-kubernetes-flat-networks-fail-at-scale-and-why-your-cluster-needs-a-security-hierarchy/) By [Reza Ramezanpour](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/author/rezar/) on Jan 28, 2026 Kubernetes networking offers incredible power, but scaling that power often transforms a clean architecture into a tangled web of complexity. Managing traffic flow between hundreds of microservices across dozens of namespaces presents a challenge that... [Read more](https://wordpress-1075849-4005834.cloudwaysapps.com/blog/why-kubernetes-flat-networks-fail-at-scale-and-why-your-cluster-needs-a-security-hierarchy/)