Kubernetes toleration for FIO

[mmotejlek]

Kubestr currently does not allow scheduling FIO benchmark pods on nodes with taints.

I've come up with several ways that could be achieved:

• Ignore all taints (changes current behaviour)
• Have a flag to ignore all taints
• Have a flag for toleration expressions (but only implement an expression for universal toleration for now)

For the last option, I suggest a syntax based off kubectl's taint expression:

key[=value][:effect]|:effect

However, it needs a way to specify a universal toleration. I came up with these possibilities:

• : – An empty effect in the syntax above would mean "any". This is in line with the fact that "" specifies any effect in the definition of Toleration.
• :Any – This mimics an actual effect name (effect names use an uppercase starting letter and are case sensitive), but this also means it could clash with an actual "Any" effect (I find this unlikely to be added though).
• :any – Using a lowercase starting letter to denote a special symbol
• ::Any – Extra : to denote a special symbol

Or maybe these is an established solution that I'm not aware of.

I'm not sure which way would be preferred. To me, toleration :/toleration=: seems to be the cleanest and most future-proof solution.

@ihcsim @smasset-orange

[ihcsim]

I prefer to not over-complicate the CLI UX. It's better to not have to deal with the different permutations of taint/toleration behaviour using a CLI.

One other way to tackle this is to update kubestr fio to accept a pod override template (-T) flag. E.g.,

cat<<EOF | kubestr fio -T -
tolerations:
- key: "example-key"
  operator: "Exists"
  effect: "NoSchedule"
EOF

This offers the most flexibility to configure the pod. But I am not sure if it's desirable to let user modify container images, and security context.