Issue with Allstar Branch Protection Enforcement (404 errors upon action: fix)

[ArisBee]

Hello,

Over the past couple of months, I've noticed a recurring issue where Allstar returns 404 error messages when attempting to enforce branch protection, as shown in the logs below, causing the application to crash. The issue is still there with Allstar latest version (4.2)

allstar {"severity":"INFO","org":"Org","repo":"test0","area":"bot","enabled":true,"time":"2024-08-20T08:29:17Z","message":"Check repo enabled"}                                                                                                  
allstar {"severity":"INFO","org":"Org","repo":"test0","area":"Binary Artifacts","enabled":false,"time":"2024-08-20T08:29:17Z","message":"Check repo enabled"}                                                                                    
allstar {"severity":"INFO","org":"Org","repo":"test0","area":"Binary Artifacts","result":true,"enabled":false,"notify":"","details":{"Artifacts":null},"time":"2024-08-20T08:29:17Z","message":"Policy run result."}                             
allstar {"severity":"INFO","org":"Org","repo":"test0","area":"Branch Protection","enabled":true,"time":"2024-08-20T08:29:17Z","message":"Check repo enabled"}                                                                                    
allstar {"severity":"INFO","org":"Org","repo":"test0","area":"Branch Protection","result":false,"enabled":true,"notify":"No protection found for branch master\n","details":{"master":{"PRReviews":false,"NumReviews":0,"DismissStale":false,"Bl 
allstar {"severity":"ERROR","error":"PUT [https://api.github.com/repos/Org/test0/branches/master/protection](https://api.github.com/repos/Org/test0/branches/master/protection): 404 Branch protection has been disabled on this repository. []","time":"2024-08-20T08:29:18Z","message":"Unexpected error running policies."}
allstar {"severity":"INFO","area":"bot","count":7,"results":{"CODEOWNERS":{"totalFailed":1},"OpenSSF Scorecard":{"totalFailed":1},"Repository Administrators":{"totalFailed":1}},"time":"2024-08-20T08:29:18Z","message":"EnforceAll complete."}           

After investigating, I found that this issue occurs when branch protection rules are disabled on a repository. I also noticed in the GitHub API documentation that while there is an API to update branch protection, there doesn’t seem to be one to set it initially. Could this be a new limitation from GitHub?

Currently, I’ve implemented a custom alerting system to manually re-enable branch protection rules when Allstar fails, but this is not an ideal solution. Is there a better workaround that you would recommend?

For reference, here are the current permissions for my Allstar GitHub App.

[jeffmendoza]

Thanks for bringing this up! I didn't know about this "disable branch protection" option. It looks like new repos have branch protection enabled by default, but just with empty rules. So that the "fix" option will just add the correct protection and everything works as long as a user didn't use this disable option.

If there is a way to re-enable via API, then we definitely want Allstar to just do that. I did a quick look through the REST docs as you did, and also the GraphQL docs, and didn't see any way to query or enable this either. It might be impossible, or might just take more digging.

Shorter term. We can update the code to expect the 404 on some repos, and just log a Warning and continue without exiting the enforcement loop.

[ArisBee]

Hi Jeff,

It would be great if you could take a look at this. If not, I can try submitting a PR to handle the 404 error by setting an exception. I believe it would go somewhere in this function: branch.go#L294-L313.

Thanks for your help!

[ArisBee]

@jeffmendoza I opened a support ticket at GitHub following the missing API and got the below answer:

Your ticket was transferred to the Ecosystem team since it's more in line with our scope of support. We agree that we should have an API endpoint for this; however, our engineering team have suggested you whatever tooling you are using should migrate to rulesets. We announced rulesets last year in this blog post, and while we don't have fixed plans to sunset branch protection rules yet, that's something we're considering.

Regardless, we'll still raise this as customer feedback since the limitation is causing you significant friction, but we can't say if or when such functionality may be available. We could only recommend that you keep an eye on the GitHub Blog and the Changelog for related updates. We understand this may be frustrating and wish we had better news to share regarding a workaround, but we're afraid this just simply isn't possible at the moment with our available APIs.

That raises the priority of this Feature Request: #475