Implement test script for Openinfra Interop Guidelines

[mbuechse]

The certification level "SCS compatible" includes the interoperability guidelines from Openinfra.

Openinfra publishes these guidelines as json files that can be tested via Tempest using, e.g., the refstack client also provided by Openinfra. See the repository for both the guidelines and the client: https://opendev.org/openinfra/interop/

The aim is to fetch recent test results from an SCS-specific refstack server (to be made available within lot 6d) and convert them into pass/fail.

---

Obsolete old text (June 6th) following:

Would it be possible to either

• download and install the refstack client as part of our test script
• or "vendor" the client, i.e., include a copy (maybe as a git submodule) with the test script

To this end,

• check license -- Apache license, seems very permissive
• check time taken for the client to be installed during test (compared to the actual test)

[mbuechse]

Apparently, the refstack client installs half the internet: https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt

So maybe we can prepare a fully equipped venv with the Docker container that is used to run the tests? (see https://github.com/SovereignCloudStack/standards/blob/main/.github/workflows/build-docker.yml)

[berendt]

For refstack we've osism.validations.refstack: https://github.com/osism/ansible-collection-validations/tree/main/roles/refstack. We use this as part of the osism/testbed CI and it works as expected. Switching to container image can be done and makes sense to me. But I would prefer if we don't have 2 different ways afterwards to use refstack.

Refstack container image can be added to https://github.com/osism/container-images. This way it will be published at quay.io/osism/refstack.

[mbuechse]

@berendt Thanks for weighing in. Your input is exactly what I sought after when writing this issue, because I had the impression that it shouldn't be too hard to do, but I didn't know what had been done already. Of course I agree that redundant solutions aren't desirable.

[berendt]

@mbuechse Let's discuss on Wednesday the next steps. I think it makes sense to add a refstack container image and then to rewrite the scripts in osism.validations.refstack. The installation of Refstack is really something weird.

Sample usage for osism.validations.refstack: https://github.com/osism/testbed/blob/main/scripts/check/301-openstack-refstack.sh#L11-L12.

[fkr]

I'd suggest to include @master-caster and @o-otte since as part of the tender they're working on CI is a big part and I can image that they can assist with work that needs to be done here.

[fkr]

I'll ask @mbuechse to join Team OPS call on Thursday (2023-06-15) so that we can have a discussion with @master-caster

[mbuechse]

As per https://input.scs.community/2023-scs-team-ops?both#OpenInfra-Compliance-Checks: the OpenInfra compliance check will be run independently of the SCS-compatible certification check and the results stored on an SCS-specific refstack server. The certification check will then merely have to fetch the results from this server and convert into pass/fail. So this issue has to be postponed until the corresponding infrastructure is availabe.

[mbuechse]

After a chat with @gtema I have the impression that this topic is dead. Maybe we should close this for good? Or better yet, rededicate this issue for removing the Interop guidelines from the certificate scope. Let's hope our standard on required APIs will come along quickly. Addendum: this is the issue regarding said standard: SovereignCloudStack/issues#528

[gtema]

Guideline updates stopped in 2022 (https://opendev.org/openinfra/interop/src/branch/master/guidelines), so yes, there is no sense in continuing watching this