Automatically set milestones on merged PRs

[jasongrout]

One issue we have a lot is that merged PRs are assigned to the wrong milestones. For example, a PR was originally assigned to a milestone, then merged earlier or later than that milestone, or a PR was merged with no milestone.

Just brainstorming here: it would great if there was some way to automatically set the appropriate milestone when a PR is merged. For example, if a PR was merged into a backport branch, it could be assigned to the appropriate previous release milestone, or if a PR was merged into master, even if it had a different milestone already set, it could be reassigned to an appropriate milestone.

It seems that the tricky thing here is deciding what the appropriate milestone is. Perhaps the easiest way to do it is to check what branch it was merged into, and somewhere have a table of branches to milestones. Or maybe even more appropriate is to have a revision spec (like master --not 0.34) for each milestone, i.e., take the merge commit from the PR, and check to see if it is each revision spec, and the first revision spec to win is the milestone to assign. Of course, you'd want to be able to update these tables, so perhaps they could be stored in a file in the repo? Does meeseeks have access to files in the repo? (And if so, which version of the file should be used? Probably just the one at the tip of master?)

I'm not quite sure how it could work out, but it would be wonderful if this could be automated. We're very often making errors, which makes the release process full of tedious work to check the PRs in the milestone against the actual merged commits.

[jasongrout]

I have a short, very specific script that compares the PRs in a milestone with the commits in a specific revision range at https://github.com/jupyterlab/jupyterlab/blob/b4dbcca12c3304d333c66f81104245304d98bb04/scripts/milestone_check.py, if it's helpful.

[Carreau]

How long does your script takes to run ?

That's definitively implementable, Mr Meeseeks is "just" a framework that triggers based on action so it could run something on every PR merged and similar and handle credentials for you.

I would suggest trying first to have MrMeeseeks set a "Check Milestone" label and comment what it thinks the correct milestone is.

I'll see if I can write the necessary hooks/placeholder so that you can complete the rest with the actual code you need.

[stefanv]

Related: https://github.com/scientific-python/attach-next-milestone-action