Manage Kubernetes custom resources

19min
|
Terraform

Custom Resource Definitions (CRDs) extend Kubernetes to allow you to manage resources controlled by in-cluster applications with the same tools and workflow as built-in Kubernetes resources, such as pods and nodes. You can manage CRDs with the kubernetes_manifest Terraform resource type. Managing the resources running inside your Kubernetes clusters with Terraform allows you to use a single workflow to manage your infrastructure and take advantage of Terraform's workflow and features.

OpenFaaS, an open source Functions as a Service (FaaS) platform that you can run inside your Kubernetes cluster, includes a Kubernetes operator that defines a CRD you can use to manage OpenFaaS functions.

In this tutorial, you will create a Kubernetes cluster on your local machine with Docker and kind. Then, you will deploy OpenFaaS to your cluster with Terraform. Next, you will use Terraform to deploy a function to your Kubernetes cluster using the OpenFaaS functions CRD. Finally, you will configure CPU and memory limits for your function and scale it to run on multiple pods.

Prerequisites

This tutorial assumes that you are familiar with the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first.

For this tutorial, you will need:

The Terraform CLI (1.0.1+).
Docker Desktop installed and running.
kubectl.
kind.

Clone the example configuration

Clone the Manage Kubernetes Custom Resources with Terraform GitHub repository for this tutorial.

$ git clone https://github.com/hashicorp-education/learn-terraform-k8s-crd-openfaas

Change to the repository directory.

$ cd learn-terraform-k8s-crd-openfaas

Start Kubernetes cluster

Create a Kubernetes cluster running in Docker on your local machine.

$ kind create cluster --name=openfaasCreating cluster "openfaas" ... ✓ Ensuring node image (kindest/node:v1.21.1) 🖼  ✓ Preparing nodes 📦   ✓ Writing configuration 📜  ✓ Starting control-plane 🕹️  ✓ Installing CNI 🔌  ✓ Installing StorageClass 💾 Set kubectl context to "kind-openfaas"You can now use your cluster with:kubectl cluster-info --context kind-openfaasThanks for using kind! 😊

Verify that your cluster exists by listing your kind clusters.

$ kind get clustersopenfaas

Then, use kubectl to print out information about your cluster. The context is kind- followed by the name of your cluster.

$ kubectl cluster-info --context=kind-openfaasKubernetes master is running at https://127.0.0.1:56300CoreDNS is running at https://127.0.0.1:56300/api/v1/namespaces/kube-system/services/kube-dns:dns/proxyTo further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Review example configuration

Open main.tf. This configuration uses the Kubernetes provider to create namespaces for OpenFaaS and the Helm provider to deploy OpenFaaS on your Kubernetes cluster.

First, the configuration sets up the Kubernetes provider with client certificate authentication using four variables. You will configure these variables in the next section. The configuration creates two namespaces: one for the OpenFaaS application, named openfaas, and another for your functions, named openfaas-fn.

provider "kubernetes" {  host = var.k8s_host  client_certificate     = base64decode(var.k8s_client_certificate)  client_key             = base64decode(var.k8s_client_key)  cluster_ca_certificate = base64decode(var.k8s_cluster_ca_certificate)}resource "kubernetes_namespace" "openfaas" {  lifecycle {    ignore_changes = [metadata]  }  metadata {    name = "openfaas"    labels = {      role            = "openfaas-system"      access          = "openfaas-system"      istio-injection = "enabled"    }  }}resource "kubernetes_namespace" "openfaas-fn" {  lifecycle {    ignore_changes = [metadata]  }  metadata {    name = "openfaas-fn"    labels = {      role            = "openfaas-fn"      istio-injection = "enabled"    }  }}

Then, the helm provider block configures Kubernetes authentication using the same values as the Kubernetes provider. The time_sleep resource avoids an error from Helm when you destroy your OpenFaaS cluster.

provider "helm" {  kubernetes {    host = var.k8s_host    client_certificate     = base64decode(var.k8s_client_certificate)    client_key             = base64decode(var.k8s_client_key)    cluster_ca_certificate = base64decode(var.k8s_cluster_ca_certificate)  }}# Need to wait a few seconds when removing the openfaas resource to give helm# time to finish cleaning up.## Otherwise, after `terraform destroy`:# │ Error: uninstallation completed with 1 error(s): uninstall: Failed to purge#   the release: release: not foundresource "time_sleep" "wait_30_seconds" {  depends_on = [kubernetes_namespace.openfaas]  destroy_duration = "30s"}

Finally, the helm_release.openfaas resource defines your OpenFaaS chart. This resource configures your OpenFaaS deployment, sets the namespace for your functions, enables basic authentication, and creates the operator that you will use to deploy functions using a CRD.

resource "helm_release" "openfaas" {  repository = "https://openfaas.github.io/faas-netes"  chart      = "openfaas"  name       = "openfaas"  namespace  = "openfaas"  depends_on = [time_sleep.wait_30_seconds]  set {    name  = "functionNamepsace"    value = "openfaas-fn"  }  set {    name  = "generateBasicAuth"    value = "true"  }  set {    name  = "operator.create"    value = "true"  }}

Deploy OpenFaaS

Terraform must authenticate with Kubernetes to deploy resources to your cluster. Use the kubectl config view command to generate a terraform.tfvars file from the template in the example repository. This file sets values for variables in the example configuration, which the Kubernetes provider will use to authenticate with your Kubernetes cluster.

$ kubectl config view --context=kind-openfaas --raw --output="go-template-file=cluster.tfvars.gotemplate" > terraform.tfvars

The terraform.tfvars file now contains values the Kubernetes provider will use to authenticate with your cluster. These values will be different from the example below.

k8s_host                          = "https://127.0.0.1:50390"k8s_cluster_ca_certificate        = "LS0tLS1CR...EXAMPLE...0FURS0tLS0tCg=="k8s_client_certificate            = "LS0tLS1CR...EXAMPLE...0FURS0tLS0tCg=="k8s_client_key                    = "LS0tLS1CR...EXAMPLE...0FURS0tLS0tCg=="

Now initialize Terraform.

$ terraform initInitializing the backend...Initializing provider plugins...- Reusing previous version of hashicorp/helm from the dependency lock file- Reusing previous version of hashicorp/time from the dependency lock file- Reusing previous version of hashicorp/kubernetes from the dependency lock file- Installing hashicorp/helm v2.2.0...- Installed hashicorp/helm v2.2.0 (signed by HashiCorp)- Installing hashicorp/time v0.7.2...- Installed hashicorp/time v0.7.2 (signed by HashiCorp)- Installing hashicorp/kubernetes v2.6.1...- Installed hashicorp/kubernetes v2.6.1 (signed by HashiCorp)Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or backend configuration for Terraform,rerun this command to reinitialize your working directory. If you forget, othercommands will detect it and remind you to do so if necessary.

Apply the configuration to deploy OpenFaaS to your Kubernetes cluster. Respond yes to the prompt to confirm the apply.

$ terraform apply Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with thefollowing symbols:  + createTerraform will perform the following actions:  # helm_release.openfaas will be created  + resource "helm_release" "openfaas" {      + atomic                     = false      + chart                      = "openfaas"      + cleanup_on_fail            = false      + create_namespace           = false##...Plan: 4 to add, 0 to change, 0 to destroy.Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yeskubernetes_namespace.openfaas-fn: Creating...kubernetes_namespace.openfaas: Creating...kubernetes_namespace.openfaas-fn: Creation complete after 1s [id=openfaas-fn]kubernetes_namespace.openfaas: Creation complete after 1s [id=openfaas]time_sleep.wait_30_seconds: Creating...time_sleep.wait_30_seconds: Creation complete after 0s [id=2021-08-02T21:47:33Z]helm_release.openfaas: Creating...helm_release.openfaas: Still creating... [10s elapsed]helm_release.openfaas: Still creating... [20s elapsed]helm_release.openfaas: Still creating... [30s elapsed]helm_release.openfaas: Still creating... [40s elapsed]helm_release.openfaas: Still creating... [50s elapsed]helm_release.openfaas: Creation complete after 56s [id=openfaas]Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Verify OpenFaaS deployment

List the deployments in your OpenFaaS namespace.

$ kubectl get deployments --context=kind-openfaas --namespace=openfaasNAME                READY   UP-TO-DATE   AVAILABLE   AGEalertmanager        1/1     1            1           74sbasic-auth-plugin   1/1     1            1           74sgateway             1/1     1            1           74snats                1/1     1            1           74sprometheus          1/1     1            1           74squeue-worker        1/1     1            1           74s

List the two CRDs installed with the OpenFaaS operator.

$ kubectl get crds --context=kind-openfaasNAME                     CREATED ATfunctions.openfaas.com   2021-08-02T21:47:40Zprofiles.openfaas.com    2021-08-02T21:47:40Z

Inspect the functions.openfaas.com CRD.

$ kubectl describe crds/functions.openfaas.com --context=kind-openfaasName:         functions.openfaas.comNamespace:    Labels:       app.kubernetes.io/managed-by=HelmAnnotations:  controller-gen.kubebuilder.io/version: v0.4.0              meta.helm.sh/release-name: openfaas              meta.helm.sh/release-namespace: openfaasAPI Version:  apiextensions.k8s.io/v1Kind:         CustomResourceDefinitionMetadata:## ...Status:  Accepted Names:    Kind:       Function    List Kind:  FunctionList    Plural:     functions    Singular:   function  Conditions:    Last Transition Time:  2021-08-02T21:47:40Z    Message:               no conflicts found    Reason:                NoConflicts    Status:                True    Type:                  NamesAccepted    Last Transition Time:  2021-08-02T21:47:40Z    Message:               the initial names have been accepted    Reason:                InitialNamesAccepted    Status:                True    Type:                  Established  Stored Versions:    v1Events:  <none>

You will use this CRD with the Kubernetes Terraform provider to manage your OpenFaaS functions.

Forward OpenFaaS API gateway

The OpenFaaS gateway allows you to call OpenFaaS functions over HTTP. Before you deploy a function to OpenFaaS, forward the port so you can connect to the gateway running inside your Kubernetes cluster.

$ kubectl port-forward svc/gateway 8080:8080 --context=kind-openfaas --namespace=openfaas

This command will continue running in your current terminal until you cancel it. Open a new terminal window or tab to proceed with this tutorial.

In that new terminal, navigate to the top-level learn-terraform-k8s-faas-crd directory you cloned from GitHub earlier in this tutorial. Verify your current directory with pwd.

$ pwd/Users/<YOU>/learn-terraform-k8s-crd-openfaas

Deploy an OpenFaaS function

The kubernetes_manifest resource type allows you to use Terraform to manage resources defined by a Kubernetes manifest. In this section, you will use a manifest to deploy a function to OpenFaaS with the functions.openfaas.com CRD.

First, change into the function's directory. This function returns a randomly selected ASCII art featuring cows.

$ cd functions/cows

Review main.tf.

provider "kubernetes" {  host = var.k8s_host  client_certificate     = base64decode(var.k8s_client_certificate)  client_key             = base64decode(var.k8s_client_key)  cluster_ca_certificate = base64decode(var.k8s_cluster_ca_certificate)}

The configuration in main.tf does not define any resources. You will add a kubernetes_manifest resource to this configuration to manage your OpenFaaS function.

First, reconfigure kubectl to point to your openfaas Kubernetes cluster.

$ kind export kubeconfig --name=openfaasSet kubectl context to "kind-openfaas"

Generate a terraform.tfvars file to store your Kubernetes authentication secrets as you did earlier.

$ kubectl config view  --context=kind-openfaas --raw --output="go-template-file=../../cluster.tfvars.gotemplate" > terraform.tfvars

Initialize this configuration to install the provider and set up Terraform.

$ terraform initInitializing the backend...Initializing provider plugins...- Finding hashicorp/kubernetes versions matching "~> 2.6.1"...- Installing hashicorp/kubernetes v2.6.1...- Installed hashicorp/kubernetes v2.6.1 (signed by HashiCorp)Terraform has made some changes to the provider dependency selections recordedin the .terraform.lock.hcl file. Review those changes and commit them to yourversion control system if they represent changes you intended to make.Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or backend configuration for Terraform,rerun this command to reinitialize your working directory. If you forget, othercommands will detect it and remind you to do so if necessary.

Review cows.yaml. This file is a Kubernetes manifest that defines the function you will deploy with the OpenFaaS functions CRD.

apiVersion: openfaas.com/v1kind: Functionmetadata:  name: showcow  namespace: openfaas-fnspec:  name: showcow  handler: node show_cow.js  image: alexellis2/ascii-cows-openfaas:0.1

You must convert this manifest from YAML to HCL to use it with the Terraform Kubernetes provider.

Convert cows.yaml to HCL with the yamldecode() Terraform function and the terraform console command.

$ echo 'yamldecode(file("cows.yaml"))' | terraform console{  "apiVersion" = "openfaas.com/v1"  "kind" = "Function"  "metadata" = {    "name" = "showcow"    "namespace" = "openfaas-fn"  }  "spec" = {    "handler" = "node show_cow.js"    "image" = "alexellis2/ascii-cows-openfaas:0.1"    "name" = "showcow"  }}

Add a kubernetest_manifest resource for the showcow function. Assign the value of the manifest attribute to the HCL generated by the previous command.

resource "kubernetes_manifest" "openfaas_fn_showcow" {  manifest = {  "apiVersion" = "openfaas.com/v1"  "kind" = "Function"  "metadata" = {    "name" = "showcow"    "namespace" = "openfaas-fn"  }  "spec" = {    "handler" = "node show_cow.js"    "image" = "alexellis2/ascii-cows-openfaas:0.1"    "name" = "showcow"  }}}

The HCL manifest you pasted into your configuration includes additional whitespace and does not line up with the rest of your configuration. Save main.tf and use terraform fmt to format your configuration.

$ terraform fmtmain.tf

Apply your configuration to deploy the function to your OpenFaaS cluster. Respond yes to the prompt to confirm the apply.

$ terraform applyTerraform used the selected providers to generate the following execution plan. Resource actions are indicated with thefollowing symbols:  + createTerraform will perform the following actions:  # kubernetes_manifest.openfaas_fn_showcow will be created  + resource "kubernetes_manifest" "openfaas_fn_showcow" {      + manifest = {          + apiVersion = "openfaas.com/v1"          + kind       = "Function"          + metadata   = {              + name      = "showcow"              + namespace = "openfaas-fn"            }          + spec       = {              + handler = "node show_cow.js"              + image   = "alexellis2/ascii-cows-openfaas:0.1"              + name    = "showcow"            }        }      + object   = {          + apiVersion = "openfaas.com/v1"          + kind       = "Function"          + metadata   = {              + name      = "showcow"              + namespace = "openfaas-fn"            }          + spec       = {              + annotations            = (known after apply)              + constraints            = (known after apply)              + environment            = (known after apply)              + handler                = "node show_cow.js"              + image                  = "alexellis2/ascii-cows-openfaas:0.1"              + labels                 = (known after apply)              + limits                 = {                  + cpu    = (known after apply)                  + memory = (known after apply)                }              + name                   = "showcow"              + readOnlyRootFilesystem = (known after apply)              + requests               = {                  + cpu    = (known after apply)                  + memory = (known after apply)                }              + secrets                = (known after apply)            }        }    }Plan: 1 to add, 0 to change, 0 to destroy.Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yeskubernetes_manifest.openfaas_fn_showcow: Creating...kubernetes_manifest.openfaas_fn_showcow: Creation complete after 0sApply complete! Resources: 1 added, 0 changed, 0 destroyed.

Now, call your function by using the curl command to send a request to the gateway you set up earlier. The function will return a randomly selected piece of cow-related ASCII art.

$ curl http://127.0.0.1:8080/function/showcow  (__) |  (__) |  (__) |  (__) |  (__) |  (__) |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |-------------------------------------------------  (__) |  (__) |  (__) |  (__) |  (__) |  (__) |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |-------------------------------------------------  (__) |  (__) |  (__) |  (__) |  (__) |  (__) |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |-------------------------------------------------  (__) |  (__) |  (__) |  (__) |  (__) |  (__) |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  ( oo |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |  /\_| |-------------------------------------------------              Andy Warhol Cow

List OpenFaaS function pods

Your OpenFaaS functions run on pods in the openfaas-fn namespace in your Kubernetes cluster.

List the pods in your openfaas-fn namespace.

$ kubectl get pods --context=kind-openfaas --namespace=openfaas-fnNAME                       READY   STATUS    RESTARTS   AGEshowcow-845ff7bdcc-2rvgq   1/1     Running   0          43s

Right now, there is only one instance of the showcow function running on your Kubernetes cluster.

Scale the function

Add the following configuration to the "spec" section of your openfaas_fn_showcow resource in main.tf to manage scaling and resource limits for the function. This configuration ensures that there are between four and six instances of your function running at any given time. It also sets CPU and memory usage limits for each instance.

      "labels" = {        "com.openfaas.scale.max" = "6"        "com.openfaas.scale.min" = "4"      }      "limits" = {        "cpu"    = "100m"        "memory" = "64Mi"      }

Your configuration should now match the following.

resource "kubernetes_manifest" "openfaas_fn_showcow" {  manifest = {    "apiVersion" = "openfaas.com/v1"    "kind"       = "Function"    "metadata" = {      "name"      = "showcow"      "namespace" = "openfaas-fn"    }    "spec" = {      "handler" = "node show_cow.js"      "image"   = "alexellis2/ascii-cows-openfaas:0.1"      "name"    = "showcow"      "labels" = {        "com.openfaas.scale.max" = "6"        "com.openfaas.scale.min" = "4"      }      "limits" = {        "cpu" = "100m"        "memory" = "64Mi"      }    }  }}

Apply the new configuration. Respond yes to the prompt to confirm the apply and update your function.

$ terraform applykubernetes_manifest.openfaas_fn_showcow: Refreshing state...Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with thefollowing symbols:  ~ update in-placeTerraform will perform the following actions:  # kubernetes_manifest.openfaas_fn_showcow will be updated in-place  ~ resource "kubernetes_manifest" "openfaas_fn_showcow" {      ~ manifest = {          ~ spec       = {              + labels  = {                  + com.openfaas.scale.max = "6"                  + com.openfaas.scale.min = "4"                }              + limits  = {                  + cpu    = "100m"                  + memory = "64Mi"                }                # (3 unchanged elements hidden)            }            # (3 unchanged elements hidden)        }      ~ object   = {          ~ spec       = {              ~ labels                 = null -> {                  + "com.openfaas.scale.max" = "6"                  + "com.openfaas.scale.min" = "4"                }              ~ limits                 = {                  ~ cpu    = null -> "100m"                  ~ memory = null -> "64Mi"                }                # (9 unchanged elements hidden)            }            # (3 unchanged elements hidden)        }    }Plan: 0 to add, 1 to change, 0 to destroy.Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yeskubernetes_manifest.openfaas_fn_showcow: Modifying...kubernetes_manifest.openfaas_fn_showcow: Modifications complete after 0sApply complete! Resources: 0 added, 1 changed, 0 destroyed.

Now, list your OpenFaas function pods again to confirm that Terraform scaled your function.

$ kubectl get pods --context=kind-openfaas --namespace=openfaas-fnNAME                       READY   STATUS        RESTARTS   AGEshowcow-6cdbc9cc48-4dhv7   1/1     Running       0          12sshowcow-6cdbc9cc48-92x8p   1/1     Running       0          24sshowcow-6cdbc9cc48-lhvlx   1/1     Running       0          16sshowcow-6cdbc9cc48-rkt66   1/1     Running       0          20sshowcow-845ff7bdcc-2rvgq   0/1     Terminating   0          2m31sshowcow-845ff7bdcc-dnkfr   0/1     Terminating   0          24sshowcow-845ff7bdcc-k9mnv   0/1     Terminating   0          24s

It may take a few minutes for Kubernetes to terminate the old pod and make the new ones available. When this process is complete, your cluster will have four running showcow-* pods.

$ kubectl get pods --context=kind-openfaas --namespace=openfaas-fnNAME                       READY   STATUS    RESTARTS   AGEshowcow-6cdbc9cc48-4dhv7   1/1     Running   0          54sshowcow-6cdbc9cc48-92x8p   1/1     Running   0          66sshowcow-6cdbc9cc48-lhvlx   1/1     Running   0          58sshowcow-6cdbc9cc48-rkt66   1/1     Running   0          62s

Retrieve more cow-related ASCII art from your function.

$ curl http://127.0.0.1:8080/function/showcow    ___                 __  __   (( /\   (__)        ( /\/ \(__)    \\ /\  (oo)         \ /\\/(oo) ,----\ /\--\/      ,----\ /\--\/( ) ) ) // ||      ( ) ) ) // || `-----//--||       `-----//--||      ^^   ^^            ^^   ^^              beecows

Clean up resources

In this section, you will delete your OpenFaaS function, remove OpenFaaS, and destroy your Kubernetes cluster.

Working in your learn-terraform-k8s-faas-crd/functions/cows directory, use the terraform destroy command to remove your function from OpenFaaS. Respond yes to the prompt to confirm the destroy.

$ terraform destroykubernetes_manifest.openfaas_fn_showcow: Refreshing state...Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with thefollowing symbols:  - destroyTerraform will perform the following actions:  # kubernetes_manifest.openfaas_fn_showcow will be destroyed  - resource "kubernetes_manifest" "openfaas_fn_showcow" {      - manifest = {          - apiVersion = "openfaas.com/v1"          - kind       = "Function"          - metadata   = {              - name      = "showcow"              - namespace = "openfaas-fn"            }          - spec       = {              - handler = "node show_cow.js"              - image   = "alexellis2/ascii-cows-openfaas:0.1"              - labels  = {                  - com.openfaas.scale.max = "6"                  - com.openfaas.scale.min = "4"                }              - limits  = {                  - cpu    = "100m"                  - memory = "64Mi"                }              - name    = "showcow"            }        } -> null      - object   = {          - apiVersion = "openfaas.com/v1"          - kind       = "Function"          - metadata   = {              - name      = "showcow"              - namespace = "openfaas-fn"            }          - spec       = {              - annotations            = null              - constraints            = null              - environment            = null              - handler                = "node show_cow.js"              - image                  = "alexellis2/ascii-cows-openfaas:0.1"              - labels                 = {                  - "com.openfaas.scale.max" = "6"                  - "com.openfaas.scale.min" = "4"                }              - limits                 = {                  - cpu    = "100m"                  - memory = "64Mi"                }              - name                   = "showcow"              - readOnlyRootFilesystem = null              - requests               = {                  - cpu    = null                  - memory = null                }              - secrets                = null            }        } -> null    }Plan: 0 to add, 0 to change, 1 to destroy.Do you really want to destroy all resources?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.  Enter a value: yeskubernetes_manifest.openfaas_fn_showcow: Destroying...kubernetes_manifest.openfaas_fn_showcow: Destruction complete after 0sDestroy complete! Resources: 1 destroyed.

Switch to the terminal window running the kubectl port-forward command, and press <ctrl-c> to cancel it.

Ensure that you are in the root directory of the example repository.

$ pwd/Users/<YOU>/learn-terraform-k8s-crd-openfaas

Run terraform destroy again to remove OpenFaaS and the associated resources from your cluster. Respond yes to the prompt to confirm the destroy.

$ terraform destroy##...Plan: 0 to add, 0 to change, 4 to destroy.Do you really want to destroy all resources?  Terraform will destroy all your managed infrastructure, as shown above.  There is no undo. Only 'yes' will be accepted to confirm.  Enter a value: yeshelm_release.openfaas: Destroying... [id=openfaas]kubernetes_namespace.openfaas-fn: Destroying... [id=openfaas-fn]helm_release.openfaas: Destruction complete after 1stime_sleep.wait_30_seconds: Destroying... [id=2021-08-02T21:47:33Z]kubernetes_namespace.openfaas-fn: Destruction complete after 6stime_sleep.wait_30_seconds: Still destroying... [id=2021-08-02T21:47:33Z, 10s elapsed]time_sleep.wait_30_seconds: Still destroying... [id=2021-08-02T21:47:33Z, 20s elapsed]time_sleep.wait_30_seconds: Destruction complete after 30skubernetes_namespace.openfaas: Destroying... [id=openfaas]kubernetes_namespace.openfaas: Destruction complete after 6sDestroy complete! Resources: 4 destroyed.

Finally, delete your cluster.

$ kind delete cluster --name=openfaasDeleting cluster "openfaas" ...

Next steps

In this tutorial, you used Terraform to deploy OpenFaaS to a Kubernetes cluster. You then used the Kubernetes Terraform provider to deploy a function defined as a CRD to OpenFaaS. You also modified the CRD to scale your function.

Learn more about how to use Terraform to manage Kubernetes resources with CRDs and the OpenFaaS project by reviewing the following resources:

The Kubernetes provider documentation includes other examples of managing resources with kubernetes_manifest resource.
The OpenFaaS Project website includes links to documentation and other resources.

Helm

Multi-cloud Kubernetes