Tekton Operator SCC / UID CreateContainerError

[ArthurVardevanyan]

Expected Behavior

Deployments:

• openshift-pipelines-operator
• tekton-operator-webhook

Actual Behavior

Error: container has runAsNonRoot and image will run as root (pod: "openshift-pipelines-operator-ccc7779d6-556lb_openshift-pipelines-operator(7aae737c-ba63-487e-821a-c9b03880c6ef)", container: openshift-pipelines-operator-cluster-operations)

Steps to Reproduce the Problem

1. Install: https://github.com/tektoncd/operator/releases/download/v0.68.0/openshift-release.yaml

Additional Info

Client Version: 4.14.0-rc.5
Server Version: 4.14.0-0.okd-2023-10-12-150331
Kubernetes Version: v1.27.1-3342+98158f96f30cce-dirty

Tekton Pipeline version:

• Operator: v0.68.0

[pratap0007]

/assign

[vdemeester]

I think we need to update the base image from .ko.yaml to use an image that defines a user (ubi-minimal probably doesn't).

[ArthurVardevanyan]

I don't seem to have an issue with ubi9-minimal /w KO /w OpenShift:

KO: https://github.com/ArthurVardevanyan/openshift-monitoring-cr-controller/blob/production/Makefile#L8
Deployment: https://github.com/ArthurVardevanyan/HomeLab/blob/production/okd/openshift-monitoring/base/openshift-monitoring-cr-controller/deployment.yaml#L51-L59

[ArthurVardevanyan]

The issue appears to be due to the elevated SCCs assigned to the operator pods, it is picking up the AnyUID SCC when it doesn't need it, and OpenShift is no longer injecting the random ID.

Forcing the Deployment to use restriced-v2 takes care of the issue:

• ArthurVardevanyan/HomeLab@f6a0c23
• ArthurVardevanyan/HomeLab@3824c11

[Elyytscha]

same or similar over here when installing the latest tekton operator from operatorhub on okd 4.15

[vdemeester]

@ArthurVardevanyan from ArthurVardevanyan/HomeLab@f6a0c23, the previous state was pod-security.kubernetes.io/enforce: privileged which is not something the operator sets, anywhere.

Just tried on a brand new cluster, oc apply -f https://github.com/tektoncd/operator/releases/download/v0.68.0/openshift-release.yaml.

NAME                                            READY   STATUS    RESTARTS   AGE
openshift-pipelines-operator-7989dfd96b-fjkl4   2/2     Running   0          9s
tekton-operator-webhook-76c89b7bb7-5rmcb        1/1     Running   0          9s

All just work.

Looking at openshift-operators, it does have it:

  labels:
    kubernetes.io/metadata.name: openshift-operators
    openshift.io/cluster-monitoring: "true"
    openshift.io/scc: ""
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/enforce-version: v1.24

But this is not set by openshift-pipelines, it's probably the default label and is not required for OpenShift Pipelines operator. Also, from your repo, you are deploying it in openshift-pipelines-operator namespace, so you own that namespace, and it's up to you to create it with restricted-v2, it's not something the operator controls.

@Elyytscha which operator from the hub did you install ? If it is https://operatorhub.io/operator/tektoncd-operator it is "tailored" for Kubernetes (e.g. it doesn't remove the specified user if the operand payload such as pipeline, …) and will not work properly on OpenShift (or OKD).

[ArthurVardevanyan]

@vdemeester, Generally I avoid installing anything into the openshift-operators, I have ran into issues in the past, when installing many operators into that single namespace.

[Elyytscha]

@Elyytscha which operator from the hub did you install ? If it is https://operatorhub.io/operator/tektoncd-operator it is "tailored" for Kubernetes (e.g. it doesn't remove the specified user if the operand payload such as pipeline, …) and will not work properly on OpenShift (or OKD).

why its like this? the operator should check if he runs on openshift or not and do the appropriate stuff

the tekton operator is only available via the link you posted, there is no tekton operator in the openshift repos so its the only option to install tekton
https://github.com/redhat-openshift-ecosystem/community-operators-prod here is no tekton operator
(i think we all agree that OLM is superior and we should avoid helm, kustomize, plain yamls when we can use olm for operator install)

[vdemeester]

why its like this? the operator should check if he runs on openshift or not and do the appropriate stuff

this is not the case today at least. Today there is one build for vanilla k8s and one for openshift (and the latter is supported by red hat). But yeah ideally, we could have one build that adapts, but it would/will require a lot of work/refactoring 😇.

the tekton operator is only available via the link you posted, there is no tekton operator in the openshift repos so its the only option to install tekton

For openshift it is called openshift-pipelines, but it is only available from red hat catalog. I think it is available for OKD but I am not 100% sure of that 😅

[titou10titou10]

For openshift it is called openshift-pipelines, but it is only available from red hat catalog. I think it is available for OKD but I am not 100% sure of that 😅

For OKD, it is indeed available if you installed OKD with a "RH pull secret". This will setup the "redhat operators" CatalogSources and so the "Red Hat OpenShift Pipelines" operator can be installed

BUT... OKD is going through a major revamping (the next GA version of OKD will use SCOS instead of FCOS as base images)
It will not be possible/allowed to install OKD with a RH pull secret, and so the "redhat operators" CatalogSources is not available and so it will not be possible to install the "Red Hat OpenShift Pipelines" operator

So the only way to use tekton operator in OKD will be to use this operator...

And there is another problem with the upcoming OKD v4.16/v4.17. All the projects now include such annotations that limit the UID that can be used in a namespace and so is in conflict with the SCC:

kind: Project
apiVersion: project.openshift.io/v1
metadata:
  {...}
  annotations:
    openshift.io/description: ''
    openshift.io/display-name: ''
    openshift.io/requester: 'system:admin'
    openshift.io/sa.scc.mcs: 's0:c26,c10'
    openshift.io/sa.scc.supplemental-groups: 1000670000/10000
    openshift.io/sa.scc.uid-range: 1000670000/10000