Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable compliance tests to use plugins for cluster provisioning #753

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Enable compliance tests to use plugins for cluster provisioning #753

wants to merge 3 commits into from

Conversation

tonifinger
Copy link
Contributor

This PR introduces an interface that enables the provision of kubeconfig files for the sonobuoy test framework.
Each Kubernetes provider must derive its own specific plugin from this interface class in order to provide a cluster for testing

@tonifinger tonifinger self-assigned this Sep 18, 2024
@tonifinger tonifinger linked an issue Sep 18, 2024 that may be closed by this pull request
[Feature Request] enable compliance tests to use plugins for cluster provisioning #710
Open
@tonifinger tonifinger force-pushed the 710-feature-request-enable-compliance-tests-to-use-plugins-for-cluster-provisioning branch 2 times, most recently from d4b2b66 to bfd1a0f Compare September 18, 2024 13:57
@mbuechse
Copy link
Contributor

@tonifinger Can you please post what a run would look like in the shell? (Just paste something from your terminal.)

mbuechse
mbuechse reviewed Sep 19, 2024
View reviewed changes
Copy link
Contributor

@mbuechse mbuechse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to use the test infrastructure that we already have, i.e., the scripts

and the spec files

The spec file for KaaS already includes the CNCF conformance test. What I think we could do:

  • We make a list of clusters that we need for testing purposes, and for each cluster, we say where we expect the Kubeconfig to sit,
  • then your tool can use the plugin to create all these clusters and the corresponding Kubeconfig files at the specified locations,
  • it could also use the plugin and this specification to remove the clusters.
  • We add calls to your tool to our Zuul job BEFORE and AFTER it runs the actual tests to create and remove clusters, respectively.

@tonifinger
Copy link
Contributor Author

@tonifinger Can you please post what a run would look like in the shell? (Just paste something from your terminal.)

This is the output to the shell using the plugin of kind and the logging lvl set to info:

No kind clusters found.
INFO:root:Creating cluster scs-cluster..
Creating cluster "scs-cluster" ...
 ✓ Ensuring node image (kindest/node:v1.25.3) 🖼
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-scs-cluster"
You can now use your cluster with:

kubectl cluster-info --context kind-scs-cluster --kubeconfig .pytest-kind/scs-cluster/kubeconfig

Have a nice day! 👋
INFO:interface:check kubeconfig
INFO:interface:kubeconfigfile loaded successfully
Sonobuoy Version: v0.56.16
MinimumKubeVersion: 1.17.0
MaximumKubeVersion: 1.99.99
GitSHA: c7712478228e3b50a225783119fee1286b5104af
GoVersion: go1.19.10
Platform: linux/amd64
API Version:  v1.25.3
INFO:interface: invoke cncf conformance test
INFO[0000] create request issued                         name=sonobuoy namespace= resource=namespaces
INFO[0000] create request issued                         name=sonobuoy-serviceaccount namespace=sonobuoy resource=serviceaccounts
INFO[0000] create request issued                         name=sonobuoy-serviceaccount-sonobuoy namespace= resource=clusterrolebindings
INFO[0000] create request issued                         name=sonobuoy-serviceaccount-sonobuoy namespace= resource=clusterroles
INFO[0000] create request issued                         name=sonobuoy-config-cm namespace=sonobuoy resource=configmaps
INFO[0000] create request issued                         name=sonobuoy-plugins-cm namespace=sonobuoy resource=configmaps
INFO[0000] create request issued                         name=sonobuoy namespace=sonobuoy resource=pods
INFO[0000] create request issued                         name=sonobuoy-aggregator namespace=sonobuoy resource=services
14:08:41          PLUGIN                        NODE    STATUS   RESULT   PROGRESS
14:08:41    systemd-logs   scs-cluster-control-plane   running                    
14:08:41             e2e                      global   running                    
14:08:41 
14:08:41 Sonobuoy is still running. Runs can take 60 minutes or more depending on cluster and plugin configuration.
...
...
14:09:41    systemd-logs   scs-cluster-control-plane   complete                    
...
14:10:21    systemd-logs   scs-cluster-control-plane   complete   passed                         
14:10:21             e2e                      global   complete   passed   Passed:960, Failed:  0
14:10:21 Sonobuoy has completed. Use `sonobuoy retrieve` to get results.
INFO:interface: 1094 passed, 5976 failed of which 5976 were skipped
INFO:interface:removing sonobuoy tests from cluster
INFO[0000] delete request issued                         kind=namespace namespace=sonobuoy
INFO[0000] delete request issued                         kind=clusterrolebindings
INFO[0000] delete request issued                         kind=clusterroles

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[]}

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 2 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 1 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-09-19 14:10:26 +0200 CEST Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}
...
...
...

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.
INFO:interface:removing sonobuoy tests from cluster
INFO[0000] already deleted                               kind=namespace namespace=sonobuoy
INFO[0000] delete request issued                         kind=clusterrolebindings
INFO[0000] delete request issued                         kind=clusterroles

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.
INFO:root:Deleting cluster scs-cluster..
Deleting cluster "scs-cluster" ...

@tonifinger
Enable compliance tests to use plugins for cluster provisioning
a326e99 
Signed-off-by: Toni Finger <[email protected]>
@tonifinger
Fix python style
ab0803b 
And apply static plugin

Signed-off-by: Toni Finger <[email protected]>
@tonifinger
Fix path to junit file
199fdf2 
Signed-off-by: Toni Finger <[email protected]>
@tonifinger tonifinger force-pushed the 710-feature-request-enable-compliance-tests-to-use-plugins-for-cluster-provisioning branch from a42542d to 199fdf2 Compare September 20, 2024 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mbuechse mbuechse mbuechse left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@tonifinger tonifinger

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Feature Request] enable compliance tests to use plugins for cluster provisioning
2 participants
@tonifinger @mbuechse