-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create standard for mandatory and supported IaaS services #587
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some suggestions for minor rephrasing of some things inline.
Also one thing I only addressed in the review in a shallow manner so far: I believe we discussed in the community calls previously that we need to make a clearer distinction between OpenStack APIs and the services and be careful about the wording in standards. The OpenStack services are essentially reference implementations of the APIs. Wherever possible, the SCS should mandate the APIs and their functionalities, not the (backend) services. CSPs should be free to use whatever implementation they like as long as it provides the same API and behavior1. Especially when mandating components, this can be a make or break deal for CSPs. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two cosmetic issues and one question, which requires clarification.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the service lists themselves mostly LGTM, but I think some other parts could need some clarification, see the comments.
Thanks for working on this.
I think we should discuss the supported services list in the IaaS call, as this topic is very important and should not be merged without having overall agreement also for the supported service list. |
We discussed this PR today in the IaaS Call. Especially what "supported service" means.
In conclusion the PR should focus mor on APIs than OpenStack services. Which has the downside, that we would also need to clarify for each API endpoint, whether this is mandatory, recommended or optional / not needed. This is a direction which is worth going for, but this will add a load of complexity and might need specific documents for each service API. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I found mostly some typos and some missing replacements of /services/APIs/.
Signed-off-by: josephineSei <[email protected]>
Co-authored-by: Markus Hentsch <[email protected]> Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Co-authored-by: anjastrunk <[email protected]> Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Co-authored-by: Sven <[email protected]> Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Co-authored-by: Markus Hentsch <[email protected]> Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
## Introduction | ||
|
||
To be SCS-compliant a Cloud Service Provider (CSP) has to fulfill all SCS standards. | ||
Some of those standards are broad and consider all APIs of all services on the IaaS-Layer. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please be specific and name at least one example?
…t mandatory and supported services. Signed-off-by: josephineSei <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
S3 with/vs Swift requires a bit more precision in my eyes
| **image** | Glance | Image service | | ||
| **load-balancer** | Octavia | Load-balancer service | | ||
| **network** | Neutron | Networking service | | ||
| **s3** or **object-store** | S3 API object storage | No formal standard exists, many implementations: Swift, RadosGW, minio... | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally dislike "or" here. There are user loads supporting only one of both so having here an OR does not bring much benefit. S3 support in it's own is not standardized enough to have multiple compatibility issues, especially when it comes to the OpenStack Identity related stuff.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point. It was a requirement from the beginning to have an object store API as mandatory in each scs-cloud.
CSPs right now use different kinds of object-stores. So we have two options: either make one specific implementation mandatory or we state that CSPs can and must choose an implementation and users have to live with these different options.
Another option would be to make this optionally, but we would have to discuss this again with the IaaS team and Kurt.
What is your opinion on this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We discussed this in the standardization SIG and following items were raised:
- S3 was always intended to be mandatory (also for KaaS layer)
- Swift should be at least "recommended"
Even though S3 and Swift both implement the same service (object-store) it can be seen as: HDMI output is a must and DisplayPort is recommended
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And it should be two separate rows
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will change the standard accordingly.
BUT, if S3 is also listed as "object-store" equal to Swift, we have no chance to distinguish between those two services in the tests.
This is a little bit unfortunate, because we will never know for certain, whether the endpoints are from an S3 service or from Swift. So I can change the lines here, but we will not be able to test them!
An option may be to use gaia-x credentials. Which raises the question: Do we have a point, were we state which credentials we require? @garloff ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, in this context, we need a standard for object storage too, similar like for block storage. CSPs need to know the requirements for object store and users need to know, which capabilities an object store has. See, #725
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree to @josephineSei: We can not distinguish between two object store APIs in tests. We should define just one as mandatory. Furthermore, setting S3 as mandatory is problematic. As far as I known, S3 API is not in control of OpenStack. There is no official documentation on how S3 for a specific OpenStack release looks like. The latest documentation is from Mitaka. There is no documentation for current release, e.g. Hence, we standardize a moving target IMO.
logger = logging.getLogger(__name__) | ||
mandatory_services = ["compute", "identity", "image", "block-storage", | ||
"network", "load-balancer", "placement"] | ||
object_store_service = ["s3", "object-store"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have never seen a single cloud having S3 service/endpoint in the catalog. Typically object-store
is also supporting S3 (either it is swift with S3 enabled or it is rgw catalog pointing to swift endpoint).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can remove it, if there really is no cloud with "s3" as service type. But can we certainly know, that this will never be the case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no tooling existing in the world (known to me) that expects S3 and consumes OpenStack service catalog searching for "s3" service type
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
An other question came to my mind: How does a user know, which API version a certain SCS cloud supports? There is a document on certificates listing all effective standards for a certain scope and version. Standard on mandatory services will be part of a scopes/version, soon. How does the user/reader figure out, which API of which OpenStack service belongs to a given certificate scope and version. |
As discussed in the IaaS meeting, we want to standardize, that the s3 should be mandatory and swift only supported. The one open question to me (which needs to be tested) is how do we get an endpoint for s3 when we do not have swift? or do we require that swift should be always there - even just as a stump to provide that endpoint in the service catalog? |
I was able to test my script and Kurts script against a deployment from Cloud&Heat, which provides a Swift endpoint and also offers S3 behind this endpoint.
But it also came up, that there was another description for "block-storage" that was just volume. I need to adjust the test, to allow both options - or do we want to only allow one specific endpoint name:
I would like to answer the first question with yes, because in this way we will always have access to the object storage endpoint, regardless, what service is used. The second question is more difficult: we may need to adjust the tests to allow more than one name per service - each time a new name comes up. On the other hand: that will have a huge impact on CSPs, so I don't think, we should strictly only allow one name. |
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
I am installing a minIO server on the same machine my devstack runs on to create a setup, which has an OpenStack and an s3 on the same level but next to it and no Swift enabled. I am looking into minIO to create a user and credentials, which i can use for the test. |
I installed minio in a test environment and started it temporarily:
In another console tab I installed the minio client, logged in as admin, created a new user and gave that users rights to read and write:
After this I was able to test the script and made adjustments to it until I reached the point, where it succeeded:
When not giving the parameters it will fail:
|
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
Signed-off-by: josephineSei <[email protected]>
closes SovereignCloudStack/issues#528