At Intuit, developers use Argo CD to deploy their services to a specific cluster. However, we also need to install and configure cluster addons over our fleet of 200+ clusters. Our earlier solution for addon installation/management was a proprietary installer that managed all addons in a monolithic fashion. We couldn’t install or upgrade a single addon without changing the code of the central installer, leading to bottle-necks and deployment delays. We wanted to decouple the addon management from the primary cluster upgrade process and enable additional teams (beyond the central platform team) to deploy and manage cluster addons. Additionally, some addons are needed in all clusters, while others are only required in a few clusters. This presentation will describe how we accomplished this using Argo CD and Argo ApplicationSets with the following capabilities:

• Deploy cluster addons on a set of clusters (either 20 of 200 clusters or all 200 clusters) to iterate faster on addon changes
• We'll show you how to reduce risk and seamlessly roll out Addons by using a phased approach (per cluster, environment, region, etc.)
• We decoupled namespaced scoped resources (managed by the addon development team) from cluster resources (managed by the cluster platform team) to allow a secure operational environment
• The least privileged security model using Argo CD project cluster/namespace allow/deny lists
• Variable configuration of addons per cluster (defaults vs. overrides)

You will learn to utilize Argo CD & Argo ApplicationSets to deploy and manage cluster Addons over a large fleet of Kubernetes clusters and to maintain a centrally governed security model by decoupling RBAC from addons.