Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for multiple git resolver configurations #8263

Merged
merged 1 commit into from
Sep 30, 2024

Conversation

piyush-garg
Copy link
Contributor

@piyush-garg piyush-garg commented Sep 15, 2024

This will add support for providing multiple configurations
in git resolver configmap
Configurations can be provided using unique key and that key
can be passed as a param in the taskrun/pipelinerun to resolver

Old format is supported to provide backward compatibility and
docs added in details about how to use the feature.

Unit and e2e test added

Fixes #5487

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
  • Has Tests included if any functionality added or changed
  • pre-commit Passed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Add support for multiple git resolver configurations

@tekton-robot tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 15, 2024
@piyush-garg piyush-garg force-pushed the add_multi_git_res_auth_ref branch 2 times, most recently from 52b64a2 to 16592ca Compare September 15, 2024 20:01
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-integration-tests

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-go-coverage-df

@tekton-robot
Copy link
Collaborator

@piyush-garg: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-tekton-pipeline-alpha-integration-tests
  • /test pull-tekton-pipeline-beta-integration-tests
  • /test pull-tekton-pipeline-build-tests
  • /test pull-tekton-pipeline-integration-tests
  • /test pull-tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

  • /test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test pull-tekton-pipeline-go-coverage-df

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-alpha-integration-tests

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-go-coverage-df

@tekton-robot
Copy link
Collaborator

@piyush-garg: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-tekton-pipeline-alpha-integration-tests
  • /test pull-tekton-pipeline-beta-integration-tests
  • /test pull-tekton-pipeline-build-tests
  • /test pull-tekton-pipeline-integration-tests
  • /test pull-tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

  • /test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test pull-tekton-pipeline-go-coverage-df

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-integration-tests

@piyush-garg
Copy link
Contributor Author

/test pull-tekton-pipeline-go-coverage-df

@tekton-robot
Copy link
Collaborator

@piyush-garg: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-tekton-pipeline-alpha-integration-tests
  • /test pull-tekton-pipeline-beta-integration-tests
  • /test pull-tekton-pipeline-build-tests
  • /test pull-tekton-pipeline-integration-tests
  • /test pull-tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

  • /test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test pull-tekton-pipeline-go-coverage-df

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 19, 2024
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL is not happy 😅.

So I am ok to get this merge but it doesn't feel like something that scale that much. It will work fine for small teams, but I do feel like bigger team that would require tens or hundreds of identifier would instead use the parameter. Maybe it's just a documentation issue.

/cc @afrittoli

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chitrangpatel, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [chitrangpatel,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 88.2% 1.6
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 86.5% 1.3

@piyush-garg
Copy link
Contributor Author

/retest

Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR, it's a great functionality to add.

I left a new comments, the most important are about handling error cases and adding some code coverage to config.go; plus I would really like to see the redundant identifier go away from the configuration.

I think changing the GitResolverConfig from a struct to a map would simplify the code in a few places.

I know that comments about names sound picky, so feel free to ignore if you prefer. I think it would help with code readability though as I struggled a bit to associate variables with the value they carried as I could not associated the name to the value easily.

There are a few functions that take the param map as input now, only to extract the parameter that identifies the config name, so maybe they could take a string only. On the plus side, passing the params is more future-proof, so it's probably fine to keep them as they are.

docs/git-resolver.md Outdated Show resolved Hide resolved
docs/git-resolver.md Outdated Show resolved Hide resolved
docs/git-resolver.md Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/resolver.go Outdated Show resolved Hide resolved
@afrittoli
Copy link
Member

/hold

@tekton-robot tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 24, 2024
@afrittoli
Copy link
Member

Sorry to hold the PR, but since the structure of the configuration and name of the parameters are user facing, we should fix them before this is merge. @vdemeester @chitrangpatel WDYT

@afrittoli afrittoli added this to the Pipeline v0.64 milestone Sep 24, 2024
@piyush-garg
Copy link
Contributor Author

piyush-garg commented Sep 24, 2024

Sorry to hold the PR, but since the structure of the configuration and name of the parameters are user facing, we should fix them before this is merge. @vdemeester @chitrangpatel WDYT

No worries regarding this. Lets all approve and agree with user facing things, then we can proceed with merge

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 84.4% -2.3
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 96.3%
pkg/resolution/resolver/git/resolver.go 85.3% 83.7% -1.6

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 84.4% -2.3
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 96.3%
pkg/resolution/resolver/git/resolver.go 85.3% 83.7% -1.6

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 84.4% -2.3
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 84.6% -0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.7% 1.0
pkg/remoteresolution/resolver/git/resolver.go 86.7% 84.4% -2.3
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.3% 1.2
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 84.6% -0.7

@piyush-garg
Copy link
Contributor Author

Sorry to hold the PR, but since the structure of the configuration and name of the parameters are user facing, we should fix them before this is merge. @vdemeester @chitrangpatel WDYT

Hey @afrittoli @chitrangpatel @vdemeester

I have done all the changes based on comments, please take a look now. Thanks

Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for all the updates.
I couple more comments, mostly cosmetic, some about error handling which I think needs to be addressed.

@@ -195,6 +191,114 @@ spec:
value: Ranni
```

### Specifying Configuration for Multiple Git Providers

You can specify configurations for multiple providers and even multiple configurations for same provider to use in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I would recommend using impersonal sentences, i.e. instead of "You can specify...", "It is possible to specify configurations..." etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

docs/git-resolver.md Show resolved Hide resolved
docs/git-resolver.md Outdated Show resolved Hide resolved
docs/git-resolver.md Outdated Show resolved Hide resolved
docs/git-resolver.md Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/config.go Show resolved Hide resolved
pkg/resolution/resolver/git/config_test.go Show resolved Hide resolved
pkg/resolution/resolver/git/config_test.go Outdated Show resolved Hide resolved
pkg/resolution/resolver/git/params.go Show resolved Hide resolved
pkg/resolution/resolver/git/resolver.go Outdated Show resolved Hide resolved
This will add support for providing multiple configurations
in git resolver configmap
Configurations can be provided using unique key and that key
can be passed as a param in the taskrun/pipelinerun to resolver

Old format is supported to provide backward compatibility and
docs added in details about how to use the feature.

Unit and e2e test added

Fixes tektoncd#5487
@piyush-garg
Copy link
Contributor Author

Thank you for all the updates. I couple more comments, mostly cosmetic, some about error handling which I think needs to be addressed.

@afrittoli I have addressed all the comments

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.3% 0.7
pkg/remoteresolution/resolver/git/resolver.go 86.7% 81.8% -4.8
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.0% 0.8
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 84.2% -1.0

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/remoteresolution/resolver/framework/reconciler.go 74.6% 75.3% 0.7
pkg/remoteresolution/resolver/git/resolver.go 86.7% 81.8% -4.8
pkg/resolution/resolver/framework/reconciler.go 73.1% 74.0% 0.8
pkg/resolution/resolver/git/config.go Do not exist 100.0%
pkg/resolution/resolver/git/resolver.go 85.3% 84.2% -1.0

Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for all the updates!
Docs could use a couple of minor updates, but that can be done as a follow-up PR, good candidate for Hacktoberfest :)
/lgtm


### Example Configmap

You can add multiple configuration to `git-resolver-configmap` like this. All keys mentioned above are supported.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: "you can..."

It is possible to specify configurations for multiple providers and even multiple configurations for same provider to use in
different tekton resources. You need to first add details in configmap with your unique identifier key prefix.
To use them in tekton resources, pass the unique key mentioned in configmap as an extra param to resolver with key
`configKey` and value your unique key. If no `configKey` param is passed, `default` will be used. You can specify
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: you can...

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 30, 2024
@afrittoli
Copy link
Member

/hold cancel

@tekton-robot tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 30, 2024
@tekton-robot tekton-robot merged commit 006c9ac into tektoncd:main Sep 30, 2024
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for using multiple remote resolution Git providers in the same cluster
6 participants