Configuration Files and Object Specs

This topic explains how to configure Tanzu Kubernetes Grid (TKG) workload clusters using a flat configuration file or a Kubernetes-style object spec. For more specific instructions, see vSphere with Standalone Management Cluster Configuration Files.

Important

This section describes configuration files and object specs for workload clusters that you create by using TKG with a standalone management cluster. For information about configuration files and object specs to use with a vSphere IaaS control plane (formerly vSphere with Tanzu) Supervisor, see the vSphere IaaS control plane docs.

Cluster Types

How you configure a workload cluster depends on the cluster type, as described in the sections below.

• (Default type) Class-based clusters use a Kubernetes-style object spec for the Cluster object. To create a class-based cluster, you pass either this object spec or a cluster configuration file with a flat structure that sets uppercase-underscore variables like CLUSTER_NAME to tanzu cluster create.

  • More advanced customizations may also require a specification for ClusterBootstrap and objects that it references.

• (Legacy type) Plan-based clusters use a cluster configuration file with a flat structure that sets uppercase-underscore variables like CLUSTER_NAME. To create a legacy cluster, you pass this configuration file to tanzu cluster create.

  • More advanced customizations may require ytt Overlays to customize workload clusters or cluster plans. For information about how to download and install ytt, see Install the Carvel Tools.

For more information about the above cluster types and the Cluster API providers that they use, see:

• Workload Cluster Types
• Cluster Types and Cluster API
• Ways of Creating Workload Clusters, which includes the Tanzu CLI, Tanzu Mission Control, and kubectl.

For information about which configuration file to choose for your workload cluster, see the table below.

Configuration Files

Configuring a Workload Cluster with a Cluster Configuration File

You can use a cluster configuration file to configure class-based clusters and (legacy) plan-based clusters on vSphere 7 and 8 with a standalone management cluster.

About the Cluster Configuration File

Before deploying a workload cluster to vSphere, you create a configuration file for the cluster. When you pass this file to the --file option of tanzu cluster create, the Tanzu CLI uses the configuration information defined in the file to connect to your target platform and create the resources that the cluster will use.

To create a cluster configuration file, you can copy an existing configuration file for a previous deployment and update it. Alternatively, you can create a file from scratch by using an empty template:

• To set the configuration information relevant to your infrastructure, see vSphere with a Standalone Management Cluster.

• For the full list of options that you can specify in the cluster configuration file, see Configuration File Variable Reference.

Creating Your First Cluster Configuration File

When you deploy a workload cluster, most of the configuration for the cluster is the same as the configuration of the standalone management cluster that you use to deploy it. Because of this, the easiest way to create a configuration file for a workload cluster is to start with a copy of the standalone management cluster configuration file.

1. Locate the YAML configuration file for the management cluster:

   • If you deployed the management cluster from the installer interface and you did not specify the --file option when you ran tanzu mc create --ui, the configuration file is saved in ~/.config/tanzu/tkg/clusterconfigs/. The file has a randomly generated name, for example, bm8xk9bv1v.yaml.
   • If you deployed the management cluster from the installer interface and you did specify the --file option, the management cluster configuration is taken from the file that you specified.
   • If you deployed the management cluster from the Tanzu CLI without using the installer interface, the management cluster configuration is taken from either a file that you specified in the --file option, or from the default location, ~/.config/tanzu/tkg/cluster-config.yaml.

2. Make a copy of the management cluster configuration file and save it with a new name. For example, save the file as my-vsphere-cluster.yaml.

3. Update the settings in the cluster configuration file. If you want to configure a workload cluster to use an OS other than the default Ubuntu 22.04 for Kubernetes v1.28 or Ubuntu 20.04 for prior versions, you must set the OS_NAME and OS_VERSION values. The installer interface does not include node VM OS values in the management cluster configuration files that it saves to ~/.config/tanzu/tkg/clusterconfigs.

4. Save the file.

Configuring a Workload Cluster With an Object Spec

You can use a Kubernetes-style object spec file to deploy a class-based workload cluster to vSphere 7 and 8.

About the Object Spec File

An object spec file serves the same purpose as a cluster configuration file. When you pass an object spec to the --file option of tanzu cluster create, the Tanzu CLI creates the cluster using the configuration information defined in the object spec.

Creating Your First Object Spec File

As with other Kubernetes objects, you can configure a class-based workload cluster by creating and editing object specs. The Cluster object configures most cluster options, such as cluster topology. You can change and re-apply this object spec in order to change the configuration of a running cluster.

To create your first object spec file, you can use the Tanzu CLI to convert a cluster configuration file into an object spec file without deploying the cluster:

• (Global configuration) Use the auto-apply-generated-clusterclass-based-configuration feature of the Tanzu CLI. This feature is set to false by default. When auto-apply-generated-clusterclass-based-configuration is set to false and you run tanzu cluster create with the --file flag, the command converts your cluster configuration file into an object spec file and exits without creating the cluster. After reviewing the configuration, you re-run tanzu cluster create with the object spec file generated by the Tanzu CLI, as described in Create a Class-Based Cluster from the Object Spec. If you have updated the default configuration, to set it back to false, run:

  tanzu config set features.cluster.auto-apply-generated-clusterclass-based-configuration false
  

• (Single cluster) Pass the --dry-run option to tanzu cluster create and save the output to a file. Use the same options and configuration --file that you would use if you were creating the cluster, for example:

  tanzu cluster create my-cluster --file my-cluster-config.yaml --dry-run > my-cluster-spec.yaml
  

You can then use this object spec to deploy a cluster as described in Create a Class-Based Cluster from the Object Spec.

Workload Clusters, kubectl, and kubeconfig

Tanzu Kubernetes Grid does not automatically set the kubectl context to a workload cluster when you create it. You must set the kubectl context to a workload cluster manually by using the kubectl config use-context command.

By default, unless you specify the KUBECONFIG option to save the kubeconfig for a cluster to a specific file, all workload clusters that you deploy are added to a shared .kube/config file. If you delete the shared .kube/config file and you still have the .kube-tkg/config file for the management cluster, you can recover the .kube/config of the workload clusters with the tanzu cluster kubeconfig get CLUSTER-NAME command.

Do not change context or edit the .kube-tkg/config or .kube/config files while Tanzu Kubernetes Grid operations are running.

What to Do Next

Proceed to vSphere with Standalone Management Cluster Configuration Files.