Use Gitops delivery with Flux CD

This topic explains how you can deliver Carvel Packages, created by the Carvel Package Supply Chains, from a GitOps repository to one or more run clusters using Flux CD and Supply Chain Choreographer.

Prerequisites

To use Gitops Delivery with Flux CD, you must complete the following prerequisites:

  • You must create a Workload that uses a Carvel Package Supply Chain. You must have at least one Carvel Package generated by this Workload stored in your GitOps repository. See Carvel Package Supply Chains.
  • You must have at least one run cluster. Run clusters are your deployment environments. They can either be Tanzu Application Platform clusters, or regular Kubernetes clusters, but they must have kapp-controller and Contour installed. See the Carvel documentation and the Contour documentation.
  • To use a build cluster to control the deployment on all the run clusters, you must create a build cluster that has network access to your run clusters. You must also install Flux CD Kustomize Controller. See Install Kustomize Controller Prerequisite. If you intend to deploy directly on the run cluster without a build cluster, a build cluster is only necessary for building the package.

Install Kustomize Controller Prerequisite

As mentioned earlier, to use a build cluster to control the deployment on run clusters, you must install Flux CD kustomize-controller. You can do the installation using the flux cli. Use release v2.1.2.

Caution

Newer releases of Kustomize Controller are not compatible with Tanzu Application Platform v1.7.x and later.

To install Flux CD:

# https://fluxcd.io/flux/get-started/
brew install fluxcd/tap/flux@0.41  (for example, when on macOS)
flux install -n flux-system --components kustomize-controller --version v2.1.2

Set up run cluster namespaces

Each run cluster must have a namespace and ServiceAccount with the correct permissions to deploy the Carvel Packages, PackageInstalls and Kubernetes Secrets. Create a namespace and ServiceAccount with the following permissions:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: <run-cluster-ns>
  name: app-package-and-pkgi-install-role
rules:
  - apiGroups: ["data.packaging.carvel.dev"]
    resources: ["packages"]
    verbs: ["get", "list", "create", "update", "delete", "patch"]
  - apiGroups: ["packaging.carvel.dev"]
    resources: ["packageinstalls"]
    verbs: ["get", "list", "create", "update", "delete", "patch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "create", "update", "delete", "patch"]

If your run cluster is a Tanzu Application Platform cluster, see Set up developer namespaces to use your installed packages.

If your run cluster is not a Tanzu Application Platform cluster, the ServiceAccount must also have the following permissions:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: <run-cluster-ns>
  name: app-cr-role
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "create", "update", "delete"]
- apiGroups: [""]
  resources: ["configmaps", "services"]
  verbs: ["get", "list", "create", "update", "delete"]
- apiGroups: ["networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["get", "list", "create", "update", "delete"]

Create Carvel PackageInstalls and secrets

For each Carvel Package and each run cluster, you must create a Carvel PackageInstall and a Secret. The Carvel PackageInstall and the Secret are stored in your GitOps repository and deployed to run clusters by Flux CD.

The following example shows GitOps repository structure after completing this section:

app.default.tap/
  packages/
    20230321004057.0.0.yaml  # Package
  staging/
    packageinstall.yaml      # PackageInstall
    params.yaml              # Secret
  prod/
    packageinstall.yaml      # PackageInstall
    params.yaml              # Secret

  1. For each run cluster, create a Secret that has the values for each Package parameter. You can see the configurable properties of the Package by inspecting the Package CR’s valuesSchema, or in the Carvel Package Supply Chains documentation. Store the Secret in your GitOps repository at <package_name>/<run_cluster>/params.yaml.

    ---
apiVersion: v1
kind: Secret
metadata:
  name: app-values
stringData:
  values.yaml: |
    ---
    workload_name: app
    replicas: 2
    hostname: app.mycompany.com
    Note

    You must set a value for the workload_name parameter. You can skip setting other fields to use the default parameter values.

  2. For each run cluster, create a PackageInstall. Reference the Secret you created earlier. Store the PackageInstall in your GitOps repository at <package_name>/<run_cluster>/packageinstall.yaml.

    ---
apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageInstall
metadata:
  name: app
spec:
  serviceAccountName: <run-cluster-ns-sa> # ServiceAccount on run cluster with permissions to deploy Package, see "Set up run Cluster Namespaces"
  packageRef:
    refName: app.default.tap # name of the Package
    versionSelection:
      constraints: 20230321004057.0.0 # version of the Package
  values:
  - secretRef:
      name: app-values # Secret created in previous step

To continuously deploy the latest version of your Package, set versionSelection.constraints: >=0.0.0. To revert to a previous version, update the versionSelection.constraints: field and annotate the PackageInstall:

packaging.carvel.dev/downgradable: ""

See the Carvel documentation.

  1. Push the newly created PackageInstalls and Secrets to your GitOps repository.

Create Flux CD GitRepository and Flux CD Kustomizations on the build cluster

Configure Flux CD on the Build cluster to deploy your Packages, PackageInstalls, and Secrets to each of your run clusters.

  1. Give your Build cluster access to your run clusters. On the Build cluster, for each run cluster, create a Secret containing the run cluster’s kubeconfig. Create the Kubernetes Secret in the same namespace as the Kustomization resource:

    kubectl create secret generic <run-cluster>-kubeconfig \
   -n <build-cluster-ns> \
   --from-file=value.yaml=<path-to-run-cluster-kubeconfig>
    Note

    The kubeconfig must be self-contained and not rely on binaries, or environment and credential files from the kustomize-controller pod. Kubeconfigs that specify a command to provide client credentials such as gke-gcloud-auth-plugin won’t work without a custom per-provider installation of kustomize-controller.

  2. Configure your Build cluster to clone the GitOps repository. On the Build cluster, create the following Flux CD GitRepository:

    ---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: <package-name>-gitops-repo
  namespace: <build-cluster-ns>
spec:
  url: # GitOps repo URL
  ignore: |
    !.git
  interval: 30s
  ref:
    branch: # GitOps repo branch
  timeout: 60s

  # only required if GitOps repo is private (recommended). The secret below should be present in the same namespace as the GitRepository.
  secretRef:
    name: <package-name>-gitops-auth

# only required if GitOps repo is private (recommended)
---
apiVersion: v1
kind: Secret
metadata:
  name: <package-name>-gitops-auth
  namespace: <build-cluster-ns>
type: Opaque
data:
  username: # base64 encoded GitHub (or other git remote) username
  password: # base64 encoded GitHub (or other git remote) personal access token

  3. Configure your Build cluster to deploy your Package to the run clusters. For each run cluster, on the Build cluster, create the following Flux CD Kustomization:

    ---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: <package-name>-<run-cluster>-packages
  namespace: <build-cluster-ns>
spec:
  sourceRef:
    kind: GitRepository
    name: <package-name>-gitops-repo
    namespace: <build-cluster-ns>
  path: "./<package-name>/packages"
  interval: 5m
  timeout: 5m
  prune: true
  wait: true

  # where to deploy
  kubeConfig:
    secretRef:
      name: <run-cluster>-kubeconfig
  targetNamespace: <run-cluster-ns>
  serviceAccountName: <run-cluster-ns-sa>
    Note

    The Kustomization resource does not accept a metadata.name field longer than 63 characters.

  4. Configure your Build cluster to deploy your PackageInstalls and Secrets to the run clusters. For each run cluster, on the Build cluster, create the following Flux CD Kustomization:

    ---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  # for the second run cluster, for example hello-app-prod2-packages
  name: <package-name>-<run-cluster>-packageinstalls
  namespace: <build-cluster-ns>
spec:
  sourceRef:
    kind: GitRepository
    name: <package-name>-gitops-repo
    namespace: <build-cluster-ns>
  path: "./<package-name>/<run-cluster>"
  interval: 5m
  timeout: 5m
  prune: true
  wait: true

  # where to deploy
  kubeConfig:
    secretRef:
      name: <run-cluster>-kubeconfig
  targetNamespace: <run-cluster-ns>
  serviceAccountName: <run-cluster-ns-sa>
    Note

    The Kustomization resource does not accept a metadata.name field longer than 63 characters.

Verify installation

To verify your installation:

  1. On your Build cluster, confirm that your Flux CD GitRepository and Kustomizations are reconciling:

    kubectl get gitrepositories,kustomizations -A

  2. Target a run cluster. Confirm that all Packages from the GitOps repository are deployed:

    kubectl get packages -A

  3. Target a run cluster. Confirm that all PackageInstalls are reconciled:

    kubectl get packageinstalls -A

Now you can access your application on each run cluster.

check-circle-line exclamation-circle-line close-line
Scroll to top icon