[Bug] Chainsaw tries to create namespace which is already present

[sandert-k8s]

chainsaw version Version

v0.2.8

Description

When we start tests with the parameter --namespace chainsaw-tests, we get the error that the namespace already exist. This is correct, the namespace does already exist, but it should just run inside this namespace.

We have this behaviour when the tests are started in the official container, not when launched from a client where chainsaw is installed through brew.

Also; the pod doesn't exit with a error code; so Kubernetes says the pod is completed instead of failed (normally, when a test fails, the pod fails).

Steps to reproduce

1. Put the test files in the container (for example with a Dockerfile)

FROM ghcr.io/kyverno/chainsaw:v0.2.8
`COPY chainsaw-tests/ /chainsaw-tests/`

2. Create manually a namespace chainsaw-tests
3. Launch the container/pod with the following commands:

    - args:
        - test
        - '--test-dir=/chainsaw-tests'
        - '--namespace=chainsaw-tests'
      command:
        - chainsaw

When trying with <=v.0.2.7, it does work. Also tried with 0.2.9 and x.10, doesn't solve it.

Expected behavior

Chainsaw tests would run in the provided namespace.

Screenshots

No response

Logs

Version: 0.2.8
Loading default configuration...
- Using test file: chainsaw-test
- TestDirs [/chainsaw-tests]
- SkipDelete false
- FailFast false
- Namespace 'chainsaw-tests'
- FullName false
- IncludeTestRegex ''
- ExcludeTestRegex ''
- ApplyTimeout 5s
- AssertTimeout 30s
- CleanupTimeout 30s
- DeleteTimeout 15s
- ErrorTimeout 30s
- ExecTimeout 5s
- DeletionPropagationPolicy Background
- Template true
- NoCluster false
- PauseOnFailure false
Loading tests...
- e2etest-argo-instance-and-apps (/chainsaw-tests/deploy-argo)
Loading values...
Running tests...
=== RUN   chainsaw
=== PAUSE chainsaw
=== CONT  chainsaw    l.go:52:| 12:10:16 | chainsaw | @chainsaw | CREATE    | ERROR | v1/Namespace @ chainsaw-tests       
=== ERROR       
 namespaces "chainsaw-tests" already exists    
l.go:52:| 12:10:16 | chainsaw | @chainsaw | INTERNAL  | ERROR |        
=== ERROR        
namespaces "chainsaw-tests" already exists
--- FAIL: chainsaw (0.05s)
FAIL
Tests Summary...
- Passed  tests 0
- Failed  tests 0
- Skipped tests 0
Done.

Slack discussion

No response

Troubleshooting

• I have searched other issues in this repository and mine is not recorded.

[eddycharly]

Thanks for reporting, i will fix it asap!

[eddycharly]

@sandert-k8s I couldn't reproduce the issue with v0.2.10.

I created the chainsaw-tests namespace in my cluster and ran:

docker run \
	-v ./testdata/e2e/:/chainsaw/ \
	-v ${HOME}/.kube/:/etc/kubeconfig/ \
	-e KUBECONFIG=/etc/kubeconfig/config \
	--network=host \
	--user $(id -u):$(id -g) \
	--name chainsaw \
	--rm \
	ghcr.io/kyverno/chainsaw:v0.2.10 test /chainsaw/examples/quick-start --namespace chainsaw-tests

It seems to work fine:

Version: 0.2.10
Loading default configuration...
- Using test file: chainsaw-test
- TestDirs [/chainsaw/examples/quick-start]
- SkipDelete false
- FailFast false
- Namespace 'chainsaw-tests'
- FullName false
- IncludeTestRegex ''
- ExcludeTestRegex ''
- ApplyTimeout 5s
- AssertTimeout 30s
- CleanupTimeout 30s
- DeleteTimeout 15s
- ErrorTimeout 30s
- ExecTimeout 5s
- DeletionPropagationPolicy Background
- Template true
- NoCluster false
- PauseOnFailure false
Loading tests...
- quick-start (/chainsaw/examples/quick-start)
Loading values...
Running tests...
=== RUN   chainsaw
=== PAUSE chainsaw
=== CONT  chainsaw
=== RUN   chainsaw/quick-start
=== PAUSE chainsaw/quick-start
=== CONT  chainsaw/quick-start
    | 09:22:09 | quick-start | step-1    | TRY       | BEGIN |
    | 09:22:09 | quick-start | step-1    | APPLY     | RUN   | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | CREATE    | OK    | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | APPLY     | DONE  | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | ASSERT    | RUN   | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | ASSERT    | DONE  | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | TRY       | END   |
    | 09:22:09 | quick-start | step-1    | CLEANUP   | BEGIN |
    | 09:22:09 | quick-start | step-1    | DELETE    | OK    | v1/ConfigMap @ chainsaw-tests/chainsaw-quick-start
    | 09:22:09 | quick-start | step-1    | CLEANUP   | END   |
--- PASS: chainsaw (0.01s)
    --- PASS: chainsaw/quick-start (0.12s)
PASS
Tests Summary...
- Passed  tests 1
- Failed  tests 0
- Skipped tests 0
Done.

Can you provide more details on how to reproduce it ?