Don't fear YAML files!

If you are working with modern applications, you cannot avoid working with YAML files.

When you deploy an application with kubectl or Helm, you will need to add or edit entries in the YAML file, for example to activate/deactivate a feature, change a network port, a path to certificates.

All commands have been tested on Ubuntu 22.04 but should work on any Linux or MacOS systems.

In this blog, you will see that many tools can assist in your day-to-day job, but first let’s define YAML.

YAML files fundamentals

Depending on whom you ask, YAML stands for yet another markup language or YAML ain’t markup language (a recursive acronym), which emphasizes that YAML is for data, not documents.

To some experts, it can seem more human readable/editable than markup languages like XML or JSON because it is only based on indentation using spaces (tab characters are not allowed).

Example of YAML file (from https://www.cloudbees.com/blog/yaml-tutorial-everything-you-need-get-started):

---
ray: "a drop of golden sun"
pi: 3.14159
xmas: true
french-hens: 3
calling-birds:
 - huey
 - dewey
 - louie
 - fred
xmas-fifth-day:
 calling-birds: four
 french-hens: 3
 golden-rings: 5
partridges:
  count: 1
  location: "a pear tree"
turtle-doves: two

The file starts with three dashes, and examples of lists, strings and Booleans. Please note, that the three dashes are not mandatory with kubectl.

Configure VIM for YAML

A tip if you are using VIM editor:

In order to get the nice 2-space YAML as the default when you hit carriage return after the colon, add this to your .vimrc configuration file:

autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtab

Create YAML file with kubectl

Whether you want to create a new pod, deployment or service, it can be difficult to remember the apiVersion for each component, the syntax, the indentation… kubectl can create the YAML for you.

Below you will find two examples of pod and deployment file creation. If you want the result in a file, you must redirect the output with >file.yaml at the end of the command line.

Using “—dry-run=client”, no modification will be made to your Kubernetes cluster.

And with “-o yaml”, the output is the desired YAML file.

Creation of a YAML file to describe a pod

For example, if you want to create a new nginx pod, run the following command:

$ kubectl run nginx --image=nginx --dry-run=client -o yaml
apiVersion: v1 
kind: Pod 
metadata:
   creationTimestamp: null
   labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
  name: nginx
  resources: {}
 dnsPolicy: ClusterFirst
 restartPolicy: Always
status: {}

Creation of a YAML file to describe a deployment

To create a nginx deployment, where you can easily modify replicas, run this command:

$ kubectl create deployment --image=nginx nginx --dry-run=client -o yaml 
apiVersion: apps/v1 
kind: Deployment 
metadata: 
   creationTimestamp: null 
   labels: 
     app: nginx 
  name: nginx 
spec:
replicas: 1 
selector: 
   matchLabels:
     app: nginx
   strategy: {} 
   template:
     metadata:
       creationTimestamp: null
       labels:
         app: nginx
       spec:
         containers:
         - image: nginx
         name: nginx
         resources: {}
status: {}

Get YAML files from Helm charts

Helm chart is the package manager for Kubernetes. Before installing a package, you can configure it via a YAML file: values.yaml

Let’s take the example of the popular application “Apache Airflow”. This application can be highly configured via the values.yaml configuration file:

helm repo add apache-airflow 
helm show values apache-airflow/airflow > values.yaml
…

EDIT values.yaml 
…

helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f values.yaml

For all applications packaged with Helm, you don’t need to create the YAML file; you just need to fine tune some parameters.
Let’s see how to automate YAML modification with yq.

Edit YAML file with yq

The application yq is a portable command-line YAML processor that you can install easily on Linux and MacOS. For JSON files, we have the application jq, for YAML files, we have yq. The letters are a helpful hint to remember which application to use.

To install yq on Ubuntu 22.04, it is as easy as: snap install yq

To install yq on any Linux:

sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
sudo chmod a+x /usr/local/bin/yq

yq is available on a large variety of other OS on GitHub.

Now let’s look at an example. If you have a YAML file and you want to change the configuration of the second webserver:

• http port from 80 to 8080
• https port from 443 to 8443

You cannot use sed to change 80 to 8080, because it would change the configuration of both servers.

Nor can you use the line number, because if someone changes a comment in the file, the line number will change.

To modify this example.yaml file, the solution is to use yq.

--- 
# First web server
webserver1:
  http:
    port: 80 
  https: 
    port: 443
# Second web server
webserver2: 
  http:
    port: 80 
  https:
    port: 443

Using yq, you can select a specific field and modify the value:

$ yq e -i '.webserver2.http.port = 8080' example.yaml
$ yq e -i '.webserver2.https.port = 8443' example.yaml 
$ cat example.yaml
---
webserver1:
   http:
    port: 80
  https:
    port: 443
webserver2:
  http:
    port: 8080
  https:
    port: 8443

Only webserver2 ports have been modified, and we can automate this change easily.

Check the syntax of YAML files

Online, you can find a lot of YAML linter, viewer and converter, but you must not use them.

For security reason, do not use online tools for this, YAML files can contain passwords, API IDs, secrets, and certificates.

YAML files can be pushed on GitHub, or somewhere in the cloud, so from a security perspective, be mindful of what is contained within any YAML file. It is an error so common that startup GitGuardian was created to scan source code.

You can install yamllint to validate your YAML file on premise:

$ sudo apt install -y yamllint 
$ yamllint good.yaml  
$ yamllint bad.yaml  
bad.yaml 
   6:2 error  syntax error: expected <block end>, but found '<block mapping start>' (syntax)

Errors may be difficult to understand, but this will give you the line number and row number to find the error location.

Try before deploying

Don’t forget you can always use the “—dry-run=client” parameter to check your yaml file before deployment.

$ kubectl create deployment --image=nginx nginx --dry-run=client -o yaml >nginx.yaml 
…
EDIT nginx.yaml
… 
$ kubectl apply -f nginx.yaml --dry-run=client 
deployment.apps/nginx created (dry run)
 
$ kubectl apply -f nginx.yaml 
deployment.apps/nginx created

Conclusion

You don’t fear YAML anymore!

Use yamllint and yq to manipulate your YAML files on premise.

I hope you enjoyed this blog, and don’t hesitate to send feedbacks and ideas!