If you are working with modern applications, you cannot avoid working with YAML files.
When you deploy an application with kubectl or Helm, you will need to add or edit entries in the YAML file, for example to activate/deactivate a feature, change a network port, a path to certificates.
All commands have been tested on Ubuntu 22.04 but should work on any Linux or MacOS systems.
In this blog, you will see that many tools can assist in your day-to-day job, but first let’s define YAML.
YAML files fundamentals
Depending on whom you ask, YAML stands for yet another markup language or YAML ain’t markup language (a recursive acronym), which emphasizes that YAML is for data, not documents.
To some experts, it can seem more human readable/editable than markup languages like XML or JSON because it is only based on indentation using spaces (tab characters are not allowed).
Example of YAML file (from https://www.cloudbees.com/blog/yaml-tutorial-everything-you-need-get-started):
---
ray: "a drop of golden sun"
pi: 3.14159
xmas: true
french-hens: 3
calling-birds:
- huey
- dewey
- louie
- fred
xmas-fifth-day:
calling-birds: four
french-hens: 3
golden-rings: 5
partridges:
count: 1
location: "a pear tree"
turtle-doves: twoThe file starts with three dashes, and examples of lists, strings and Booleans. Please note, that the three dashes are not mandatory with kubectl.
Configure VIM for YAML
A tip if you are using VIM editor:
In order to get the nice 2-space YAML as the default when you hit carriage return after the colon, add this to your .vimrc configuration file:
autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtabCreate YAML file with kubectl
Whether you want to create a new pod, deployment or service, it can be difficult to remember the apiVersion for each component, the syntax, the indentation… kubectl can create the YAML for you.
Below you will find two examples of pod and deployment file creation. If you want the result in a file, you must redirect the output with >file.yaml at the end of the command line.
Using “—dry-run=client”, no modification will be made to your Kubernetes cluster.
And with “-o yaml”, the output is the desired YAML file.
Creation of a YAML file to describe a pod
For example, if you want to create a new nginx pod, run the following command:
$ kubectl run nginx --image=nginx --dry-run=client -o yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: nginx
name: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}Creation of a YAML file to describe a deployment
To create a nginx deployment, where you can easily modify replicas, run this command:
$ kubectl create deployment --image=nginx nginx --dry-run=client -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}Get YAML files from Helm charts
Helm chart is the package manager for Kubernetes. Before installing a package, you can configure it via a YAML file: values.yaml
Let’s take the example of the popular application “Apache Airflow”. This application can be highly configured via the values.yaml configuration file:
helm repo add apache-airflow
helm show values apache-airflow/airflow > values.yaml
…
EDIT values.yaml
…
helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f values.yaml
For all applications packaged with Helm, you don’t need to create the YAML file; you just need to fine tune some parameters.
Let’s see how to automate YAML modification with yq.
Edit YAML file with yq
The application yq is a portable command-line YAML processor that you can install easily on Linux and MacOS. For JSON files, we have the application jq, for YAML files, we have yq. The letters are a helpful hint to remember which application to use.
To install yq on Ubuntu 22.04, it is as easy as: snap install yq
To install yq on any Linux:
sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
sudo chmod a+x /usr/local/bin/yq
yq is available on a large variety of other OS on GitHub.
Now let’s look at an example. If you have a YAML file and you want to change the configuration of the second webserver:
http port from 80 to 8080https port from 443 to 8443
You cannot use sed to change 80 to 8080, because it would change the configuration of both servers.
Nor can you use the line number, because if someone changes a comment in the file, the line number will change.
To modify this example.yaml file, the solution is to use yq.
---
# First web server
webserver1:
http:
port: 80
https:
port: 443
# Second web server
webserver2:
http:
port: 80
https:
port: 443
Using yq, you can select a specific field and modify the value:
$ yq e -i '.webserver2.http.port = 8080' example.yaml
$ yq e -i '.webserver2.https.port = 8443' example.yaml
$ cat example.yaml
---
webserver1:
http:
port: 80
https:
port: 443
webserver2:
http:
port: 8080
https:
port: 8443
Only webserver2 ports have been modified, and we can automate this change easily.
Check the syntax of YAML files
Online, you can find a lot of YAML linter, viewer and converter, but you must not use them.
For security reason, do not use online tools for this, YAML files can contain passwords, API IDs, secrets, and certificates.
YAML files can be pushed on GitHub, or somewhere in the cloud, so from a security perspective, be mindful of what is contained within any YAML file. It is an error so common that startup GitGuardian was created to scan source code.
You can install yamllint to validate your YAML file on premise:
$ sudo apt install -y yamllint
$ yamllint good.yaml
$ yamllint bad.yaml
bad.yaml
6:2 error syntax error: expected <block end>, but found '<block mapping start>' (syntax)
Errors may be difficult to understand, but this will give you the line number and row number to find the error location.
Try before deploying
Don’t forget you can always use the “—dry-run=client” parameter to check your yaml file before deployment.
$ kubectl create deployment --image=nginx nginx --dry-run=client -o yaml >nginx.yaml
…
EDIT nginx.yaml
…
$ kubectl apply -f nginx.yaml --dry-run=client
deployment.apps/nginx created (dry run)
$ kubectl apply -f nginx.yaml
deployment.apps/nginx created
Conclusion
You don’t fear YAML anymore!
Use yamllint and yq to manipulate your YAML files on premise.
I hope you enjoyed this blog, and don’t hesitate to send feedbacks and ideas!