Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate gx conformant credentials for server flavors #81

Merged

Conversation

anjastrunk
Copy link
Contributor

Duplicate to #78 to sign off commits.

closes #77
closes #63

Signed-off-by: Anja Strunk <[email protected]>
Copy link
Contributor

@mbuechse mbuechse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks a bit unfinished with the formatting issues (missing spaces, shuffled imports), but most importantly, I'm of the opinion that the flavor-name parsing routines should be used, which live in another repo unfortunately. We could try a git submodule, or we publish them on PyPI, or we might even just copy the file here (but then we have to update the copy every now and then). @martinmo what do you think?

generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/vm_images_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/server_flavor_discovery.py Outdated Show resolved Hide resolved
Copy link
Member

@martinmo martinmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm of the opinion that the flavor-name parsing routines should be used.

@mbuechse I agree.

We could try a git submodule, or we publish them on PyPI, or we might even just copy the file here (but then we have to update the copy every now and then). @martinmo what do you think?

I think copying is not a nice option but probably the best short term fix.

The submodule approach will lead to very "funny" import statements, because the flavor_name module is hidden in a deep hierarchy and most of it doesn't conform to python package requiremets (e.g., dashes in the name).

I like the idea of a PyPI module with the reusable/shareable parts, but this would involve a lot of effort.

generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
generator/discovery/openstack/openstack_discovery.py Outdated Show resolved Hide resolved
@martinmo
Copy link
Member

martinmo commented Apr 5, 2024

Duplicate to #78 to sign off commits.

FTR, to retroactively sign-off commits in an existing PR, you can use

git rebase --signoff $(git merge-base HEAD main)

(in the PR branch) and

git push --force-with-lease

to (safely) force push the rewritten commits. The git merge-base HEAD main part just finds the "starting point" of your branch (common ancestor).

@mbuechse
Copy link
Contributor

mbuechse commented Apr 5, 2024

I think copying is not a nice option but probably the best short term fix.

Who knows? Maybe soon(TM) we won't need to parse the names any more because we get better discoverability at last...

From this vantage point, publishing a flavor-name parser on PyPI might send a very wrong signal.

anjastrunk and others added 4 commits April 8, 2024 14:52
@anjastrunk
Copy link
Contributor Author

I think copying is not a nice option but probably the best short term fix.

Who knows? Maybe soon(TM) we won't need to parse the names any more because we get better discoverability at last...

From this vantage point, publishing a flavor-name parser on PyPI might send a very wrong signal.

I agree, we should use a copy until we get rid of these artificial flavor naming...

Signed-off-by: Anja Strunk <[email protected]>
Signed-off-by: Anja Strunk <[email protected]>
Signed-off-by: Anja Strunk <[email protected]>
@anjastrunk anjastrunk self-assigned this Apr 8, 2024
@anjastrunk anjastrunk added the SCS-VP10 Related to tender lot SCS-VP10 label Apr 10, 2024
@anjastrunk
Copy link
Contributor Author

@martinmo @mbuechse Unit tests are fixed and pipeline is green. Please do a final review. Thanks

Copy link
Contributor

@mbuechse mbuechse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is still the issue of mistaken renaming of discovery -> openstack_discovery -- I caught a few instances, but maybe you can do a search.

Also, here is the (relevant part of) the output of flake8:

./generator/discovery/openstack/openstack_discovery.py:7:1: F401 'abc.ABCMeta' imported but unused
./generator/discovery/openstack/openstack_discovery.py:7:1: F401 'abc.abstractmethod' imported but unused
./generator/discovery/openstack/openstack_discovery.py:12:1: F401 'generator.common.const' imported but unused
./generator/discovery/openstack/openstack_discovery.py:37:13: W503 line break before binary operator
./generator/discovery/openstack/openstack_discovery.py:38:10: W292 no newline at end of file
./generator/discovery/openstack/server_flavor_discovery.py:95:9: F841 local variable 'e' is assigned to but never used
./generator/discovery/openstack/server_flavor_discovery.py:228:1: W293 blank line contains whitespace
./generator/discovery/openstack/server_flavor_discovery.py:228:5: W292 no newline at end of file
./generator/discovery/openstack/vm_images_discovery.py:208:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:521:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:522:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:628:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:645:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:650:21: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:651:21: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:664:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:689:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:693:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:697:13: W503 line break before binary operator
./tests/common.py:30:1: E122 continuation line missing indentation or outdented
./tests/common.py:259:1: E302 expected 2 blank lines, found 1
./tests/test_json_ld.py:3:1: F401 'generator.common.gx_schema.InstantiationRequirement' imported but unused
./tests/test_server_flavor_discovery.py:13:1: F401 'generator.vendor.flavor_names.Flavorname' imported but unused

generator/discovery/openstack/server_flavor_discovery.py Outdated Show resolved Hide resolved
k8s-discovery.py Outdated Show resolved Hide resolved
openstack-discovery.py Outdated Show resolved Hide resolved
openstack-discovery.py Outdated Show resolved Hide resolved
tests/common.py Outdated Show resolved Hide resolved
tests/common.py Outdated Show resolved Hide resolved
tests/common.py Outdated Show resolved Hide resolved
tests/common.py Outdated Show resolved Hide resolved
tests/common.py Outdated Show resolved Hide resolved
@anjastrunk
Copy link
Contributor Author

There is still the issue of mistaken renaming of discovery -> openstack_discovery -- I caught a few instances, but maybe you can do a search.

This rename was intended. I re-stored the original setting. See https://github.com/SovereignCloudStack/gx-credential-generator/tree/main/generator/discovery/openstack.

@anjastrunk
Copy link
Contributor Author

Also, here is the (relevant part of) the output of flake8:

./generator/discovery/openstack/openstack_discovery.py:7:1: F401 'abc.ABCMeta' imported but unused
./generator/discovery/openstack/openstack_discovery.py:7:1: F401 'abc.abstractmethod' imported but unused
./generator/discovery/openstack/openstack_discovery.py:12:1: F401 'generator.common.const' imported but unused
./generator/discovery/openstack/openstack_discovery.py:37:13: W503 line break before binary operator
./generator/discovery/openstack/openstack_discovery.py:38:10: W292 no newline at end of file
./generator/discovery/openstack/server_flavor_discovery.py:95:9: F841 local variable 'e' is assigned to but never used
./generator/discovery/openstack/server_flavor_discovery.py:228:1: W293 blank line contains whitespace
./generator/discovery/openstack/server_flavor_discovery.py:228:5: W292 no newline at end of file
./generator/discovery/openstack/vm_images_discovery.py:208:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:521:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:522:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:628:17: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:645:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:650:21: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:651:21: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:664:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:689:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:693:13: W503 line break before binary operator
./generator/discovery/openstack/vm_images_discovery.py:697:13: W503 line break before binary operator
./tests/common.py:30:1: E122 continuation line missing indentation or outdented
./tests/common.py:259:1: E302 expected 2 blank lines, found 1
./tests/test_json_ld.py:3:1: F401 'generator.common.gx_schema.InstantiationRequirement' imported but unused
./tests/test_server_flavor_discovery.py:13:1: F401 'generator.vendor.flavor_names.Flavorname' imported but unused

flake8 errors were fixed. See f974b97

anjastrunk and others added 4 commits April 11, 2024 10:32
Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>
Signed-off-by: Anja Strunk <[email protected]>
…s_new' of github.com:SovereignCloudStack/gx-self-description-generator into 77-generate-gx-conformant-credentials-for-server-flavors_new
Copy link
Contributor

@mbuechse mbuechse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still a few admonitions by flake8, but they are really minor. I approve.

./generator/discovery/openstack/server_flavor_discovery.py:130:13: E265 block comment should start with '# '
./generator/discovery/openstack/server_flavor_discovery.py:132:13: E265 block comment should start with '# '
./tests/common.py:266:22: E222 multiple spaces after operator
./tests/common.py:274:1: W293 blank line contains whitespace

@anjastrunk anjastrunk merged commit c96fb45 into main Apr 11, 2024
2 checks passed
@anjastrunk anjastrunk deleted the 77-generate-gx-conformant-credentials-for-server-flavors_new branch April 11, 2024 10:03
anjastrunk added a commit that referenced this pull request Apr 15, 2024
* Update unit tests

Signed-off-by: Anja Strunk <[email protected]>

* Update generator/discovery/openstack/openstack_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/openstack_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/vm_images_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/server_flavor_discovery.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Refactor code

Signed-off-by: Anja Strunk <[email protected]>

* Fix typo

Signed-off-by: Anja Strunk <[email protected]>

* Refactor OsCloud

Signed-off-by: Anja Strunk <[email protected]>

* incorporate scs module for flavor-name parsing

Signed-off-by: Matthias Büchse <[email protected]>

* Minor improvements on previous commit

Signed-off-by: Matthias Büchse <[email protected]>

* remove FIXME regarding vmware hypervisor type

Signed-off-by: Matthias Büchse <[email protected]>

* fix a few unit tests

Signed-off-by: Matthias Büchse <[email protected]>

* Remove obsolete assigment of ram size and number of cpu from flavor name

Signed-off-by: Anja Strunk <[email protected]>

* Fix unit tests

Signed-off-by: Anja Strunk <[email protected]>

* Increase code coverage of server_flavor_discovery.py to 100%

Signed-off-by: Anja Strunk <[email protected]>

* Reformt files to solve flake8 errors

Signed-off-by: Anja Strunk <[email protected]>

* Revert unintended renaming of discovery --> openstack_discovery

Signed-off-by: Anja Strunk <[email protected]>

* Update tests/common.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Simplify tests/common.py

Signed-off-by: Anja Strunk <[email protected]>

* Fix unit tests and flake8 errors

Signed-off-by: Anja Strunk <[email protected]>

---------

Signed-off-by: Anja Strunk <[email protected]>
Signed-off-by: anjastrunk <[email protected]>
Signed-off-by: Matthias Büchse <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Co-authored-by: Martin Morgenstern <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: Anja Strunk <[email protected]>
anjastrunk added a commit that referenced this pull request Apr 25, 2024
* Add python lint

Signed-off-by: Anja Strunk <[email protected]>

* Test trgger  python lint workflow

Signed-off-by: Anja Strunk <[email protected]>

* Reformat code to apply with flake8

Signed-off-by: Anja Strunk <[email protected]>

* Trigger lint workflow

Signed-off-by: Anja Strunk <[email protected]>

* Enable isort

Signed-off-by: Anja Strunk <[email protected]>

* Sort imports

Signed-off-by: Anja Strunk <[email protected]>

* Enable code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Add coverage

Signed-off-by: Anja Strunk <[email protected]>

* Update requirements

Signed-off-by: Anja Strunk <[email protected]>

* Fix unit tests

Signed-off-by: Anja Strunk <[email protected]>

* Generate code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add pytest-cov

Signed-off-by: Anja Strunk <[email protected]>

* Fix pytest

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Remove obsolete files

Signed-off-by: Anja Strunk <[email protected]>

* Fix code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Fix code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Include code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Debug errno 2

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Add coverage report

Signed-off-by: Anja Strunk <[email protected]>

* Ad code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Generate gx conformant credentials for server flavors (#81)

* Update unit tests

Signed-off-by: Anja Strunk <[email protected]>

* Update generator/discovery/openstack/openstack_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/openstack_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/vm_images_discovery.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update generator/discovery/openstack/server_flavor_discovery.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Refactor code

Signed-off-by: Anja Strunk <[email protected]>

* Fix typo

Signed-off-by: Anja Strunk <[email protected]>

* Refactor OsCloud

Signed-off-by: Anja Strunk <[email protected]>

* incorporate scs module for flavor-name parsing

Signed-off-by: Matthias Büchse <[email protected]>

* Minor improvements on previous commit

Signed-off-by: Matthias Büchse <[email protected]>

* remove FIXME regarding vmware hypervisor type

Signed-off-by: Matthias Büchse <[email protected]>

* fix a few unit tests

Signed-off-by: Matthias Büchse <[email protected]>

* Remove obsolete assigment of ram size and number of cpu from flavor name

Signed-off-by: Anja Strunk <[email protected]>

* Fix unit tests

Signed-off-by: Anja Strunk <[email protected]>

* Increase code coverage of server_flavor_discovery.py to 100%

Signed-off-by: Anja Strunk <[email protected]>

* Reformt files to solve flake8 errors

Signed-off-by: Anja Strunk <[email protected]>

* Revert unintended renaming of discovery --> openstack_discovery

Signed-off-by: Anja Strunk <[email protected]>

* Update tests/common.py

Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Simplify tests/common.py

Signed-off-by: Anja Strunk <[email protected]>

* Fix unit tests and flake8 errors

Signed-off-by: Anja Strunk <[email protected]>

---------

Signed-off-by: Anja Strunk <[email protected]>
Signed-off-by: anjastrunk <[email protected]>
Signed-off-by: Matthias Büchse <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Co-authored-by: Martin Morgenstern <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Signed-off-by: Anja Strunk <[email protected]>

* Fix errors

Signed-off-by: Anja Strunk <[email protected]>

* Sort imports

Signed-off-by: Anja Strunk <[email protected]>

* Add coverage to job summary

Signed-off-by: Anja Strunk <[email protected]>

* Add code coverage report to pull request

Signed-off-by: Anja Strunk <[email protected]>

* Add permission to write to pull requrest

Signed-off-by: Anja Strunk <[email protected]>

* Change Code coverage thresholds

Signed-off-by: Anja Strunk <[email protected]>

* Change Code coverage thresholds

Signed-off-by: Anja Strunk <[email protected]>

* Change oder of job steps

Signed-off-by: Anja Strunk <[email protected]>

* Increase code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Sort imports

Signed-off-by: Anja Strunk <[email protected]>

* Reformat files

Signed-off-by: Anja Strunk <[email protected]>

* Sort inputs and reformat files

Signed-off-by: Anja Strunk <[email protected]>

* Increase code coverage

Signed-off-by: Anja Strunk <[email protected]>

* Reformat files

Signed-off-by: Anja Strunk <[email protected]>

* Reformat files

Signed-off-by: Anja Strunk <[email protected]>

* Add markdown validation

Signed-off-by: Anja Strunk <[email protected]>

* Move pr-markdowlint.yml to coorect folder

Signed-off-by: Anja Strunk <[email protected]>

* Add markdown linter and link validator

Signed-off-by: Anja Strunk <[email protected]>

* Add markdown linter and link validator

Signed-off-by: Anja Strunk <[email protected]>

* Add package.json

Signed-off-by: Anja Strunk <[email protected]>

* Fix MD013/line-length Line length error

Signed-off-by: Anja Strunk <[email protected]>

* Fix MD013/line-length Line length error

Signed-off-by: Anja Strunk <[email protected]>

* Reformat markdown files

Signed-off-by: Anja Strunk <[email protected]>

* Add missing files for markdown lint

Signed-off-by: Anja Strunk <[email protected]>

* Exclude README_old.md from markdown lint check

Signed-off-by: Anja Strunk <[email protected]>

* Fix markdown lint errors

Signed-off-by: Anja Strunk <[email protected]>

* Fix markdown lint errors

Signed-off-by: Anja Strunk <[email protected]>

* Sort imports and reformat files

Signed-off-by: Anja Strunk <[email protected]>

* Reformt files according to flake8

Signed-off-by: Anja Strunk <[email protected]>

* Sort imports

Signed-off-by: Anja Strunk <[email protected]>

* Reformat markdown

Signed-off-by: Anja Strunk <[email protected]>

* Reformat files to remove flake8 errors

Signed-off-by: Anja Strunk <[email protected]>

* Remove outdated parts of READMe.md

Signed-off-by: Anja Strunk <[email protected]>

* Fix test cases and markdown link errors

Signed-off-by: Anja Strunk <[email protected]>

* Remove package-lock.json

Signed-off-by: Anja Strunk <[email protected]>

* Revert delete of package-lock.json

Signed-off-by: Anja Strunk <[email protected]>

* Update gx-cred-generator.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update k8s-discovery.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update tests/test_cli.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update gx-cred-generator.py

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Use pytest-cov instead of coverage

Signed-off-by: Anja Strunk <[email protected]>

* Replace ctions/checkout@v3 with ctions/checkout@v4

Signed-off-by: Anja Strunk <[email protected]>

* Remove coverage.xml

Signed-off-by: Anja Strunk <[email protected]>

* Update .coveragerc

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update .github/workflows/lint-python-syntax.yml

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Update .github/workflows/lint-python-syntax.yml

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

* Remove duplicate flake8 file

Signed-off-by: Anja Strunk <[email protected]>

* Ignore line length in flake8

Signed-off-by: Anja Strunk <[email protected]>

* Update .flake8

Co-authored-by: Martin Morgenstern <[email protected]>
Signed-off-by: anjastrunk <[email protected]>

---------

Signed-off-by: Anja Strunk <[email protected]>
Signed-off-by: anjastrunk <[email protected]>
Signed-off-by: Matthias Büchse <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Co-authored-by: Martin Morgenstern <[email protected]>
Co-authored-by: Matthias Büchse <[email protected]>
Co-authored-by: Martin Morgenstern <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SCS-VP10 Related to tender lot SCS-VP10
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Generate GX conformant credentials for Server Flavors Align IaaS Flavor description between SCS and Gaia-X
3 participants