Beta Release - Requiring workflows with Repository Rulesets!

[tonyjcamp]

Requiring workflows with Repository Rulesets is now available in beta!

Available in public beta today, GitHub organization administrators on Enterprise Cloud plans can set up and require workflows to run and pass before a pull request can be merged into an organization’s repositories. This can be enabled with Repository Rulesets!

This idea may sound familiar - it was originally introduced in beta as Actions Required Workflows. After receiving feedback from early users we saw an opportunity to improve the experience by moving this functionality to be a part of Repository Rulesets. Not only do Repository Rulesets enhance the experience of setting up and managing these workflows but it also provides organization administrators with several other benefits like branch targeting, bypass rules and dry running rules in an evaluation mode.

Getting Started

Step 1 - If you haven’t already, check out the documentation for Repository Rulesets

Step 2 - Learn more about requiring workflows with Repository Rulesets

Step 3 - After you’ve set up a few required workflows, leave us some feedback.

Leaving Feedback

It’s so important to us that we know things are working as expected and we’re solving the right use cases as we get closer to moving this feature from Public Beta to releasing it to the general public.

Did you find a strange bug? Use this discussion to tell us!
Did you notice an area of improvement? Use this discussion to tell us!
Do you have a general question about the feature? Use this discussion to… ask us!

With that said, your feedback means the world to us! We hope you’ll take the time to share your thoughts with the team.

[patrick-knight]

For our visual friends out there here a quick overview of setting up a workflow rule.

[stasostrovskyi]

Any timeline when this will be added to Rules API?

[elbrenn]

Good question! Rest and GraphQL support will go live in a couple weeks with our GA release

[stasostrovskyi]

Great! Thank you for the quick response!

[GuillermoWS]

Hello, we are using this funtionality, do you now, When do you plan to deploy the GA that adds this functionality to the API?

[elbrenn]

@GuillermoWS API functionalist will be added this week 🙂

[felipesu19]

API functionality is now live

[jon-ruckwood]

Do you have any tips on debugging the error:

Invalid rules: 'Workflows'

(The workflow I'm configuring already works as a "Required Workflow", so its visibility and permissions are configured according to the docs).

[jon-ruckwood]

Yes, the file exists at the given path for the selected ref.

Interestingly, I have got it working when selecting a branch, e.g. main.

It seems to error but only when I select a tag (the tag definitely exists).

[felipesu19]

Seems like an unexpected behavior, I've filed an internal issue to check it out, and see if its a bug on our end 👍

[elbrenn]

@jon-ruckwood the team pushed a fix here and selecting a tag should be unblocked. Please let us know if you still have problems with this setup

[jon-ruckwood]

I can now select a tag and add the workflow successfully to the ruleset.

However, when I click "Save changes" I get a "Failed to Update" error message. So I'm still unable to use a tag or pin to that tag's commit sha.

[elbrenn]

@jon-ruckwood I'm not able to reproduce this problem. Can you verify that the file exists at that location on that tag? Is it still a generic error message?

[SrinivasKashi]

Does this feature work with workflow files that are in other organization

[elbrenn]

It does not. For now, the scope of this feature is within a single organization

[SrinivasKashi]

Thank you. Any roadmap to enable it outside of the organization?

[dstrzelec]

This feature only appears to work if the workflows are stored in a public repo? We only use private repositories. Is there a reason why this feature wasn't designed to work with private repos?

[tonyjcamp]

Hi @dstrzelec - There are some repository visibility details that are important and will hopefully help:

The workflow file needs to be in a repository that matches the visibility of the repositories you want to run it in. Specifically, a public workflow can run on any repository in your organization, an internal workflow can only run on internal and private repositories, and a private workflow can only run on private repositories. For more information, see "About workflows."

Please check to be sure the repos you're working with match the visibility rules listed above. Let us know if this solves the issue for you.

[cgundy]

We are in the process of migrating from required workflows to repository rulesets and discovered one limitation. We need our workflow to run on forked repositories and access a secret, which is why we used the trigger pull_request_target. But according to the docs ruleset workflows can only be triggered on pull_request or merge_group. The workflow will be triggered on pull_request but will fail because it can't access repository secrets. Is there a workaround for this issue?

[cgundy]

Thanks a lot for the response! Do you know how quickly these additional triggers would be introduced? Specifically, if these will be available before required workflows are deprecated on the October 18 deadline?

[tonyjcamp]

@cgundy NP! To be clear, you will continue to have access after the 18th. We are continuing to work on migrating users but aren't quite ready yet. We will not remove access to the old feature until we have a majority of users migrated and have given remaining users advanced notice of the final deprecation.

We strongly believe that pull_request_target will be available with repository rules before we deprecate the old feature. Once we finalize the date I will share it out. Until then, you don't need to worry about your existing Required Workflows.

[cgundy]

Perfect, thanks for clarifying! Will keep my eye out for any updates. 👍

[tonyjcamp]

Hi @cgundy - we added support for pull_request_target today! Let us know if you have any questions. Here's the changelog announcement

[cgundy]

Thanks so much! This is a huge help and unblocks our team!

[bewuethr]

Previously, I would store the required workflows in a directory other than .github/workflows. When I try this as part of a repository ruleset, I get this error:

When moving this into .github/workflows, I have to add pull_request as a trigger, but I might not want to run that workflow for the repository that hosts it.

I can work around this with a repository name check in a job-level if-condition, but I'd prefer being able to store my workflow somewhere else, where it won't be triggered for the repo that hosts it.

[felipesu19]

This is a deliberate choice, and we don't intend to support workflows outside of the the .github/workflows directory, right now an if condition is the best way of handling things I think. We may revisit this if it turns out to be a major pain point, so we are listening to this feedback 👍

[bewuethr]

In that case, the UX should be more clear about this; why even let me input a path, if I can only choose from .github/workflows anyway?

This also means that the workflow will run on PRs in the repo that hosts it, but it might be irrelevant to that repo? It's not a major pain point, though.

[felipesu19]

I do think that's a great potential UI enhancement, I'll surface it in the right places 👍

[beeequeue]

It seems that currently an extra check is added to each PR/commit(?) that this is configured to run on. This adds a lot of visual clutter when you have 2+ ruleset workflows.

i.e. if we have a Change Motivation with one job in it it will add two checks:

• ✅ change-motivation - Required workflow passed
• ✅ change-motivation / check (pull_request)

Is there any chance it be changed to only show one of these?

[elbrenn]

Thanks @beeequeue we are tracking this and working on an improved experience

[heavymachinery]

Hi @beeequeue, we recently rolled out a change to improve this experience so that only one status check will be added per ruleset workflow 😄 Thanks for the feedback!

[dexterorion]

hi there. is there any workaround to handle this while we don't have option to use pull_request_target or any other trigger?

we use [skip ci] commit to skip some workflow executions, which end up skipping our required workflows as well (which we don't want to)

We are in the process of migrating from required workflows to repository rulesets and discovered one limitation. We need our workflow to run on forked repositories and access a secret, which is why we used the trigger pull_request_target. But according to the docs ruleset workflows can only be triggered on pull_request or merge_group. The workflow will be triggered on pull_request but will fail because it can't access repository secrets. Is there a workaround for this issue?

[dexterorion]

thank you for the response!

[tonyjcamp]

Hi @dexterorion - we just announced support for pull_request_target. Let us know if you have any questions!

[dexterorion]

hi @tonyjcamp

you the best! I will try and let you know

thank you

[dexterorion]

hi there

I am testing the use of pull_request_target and I am probably missing something, but I thought it worth asking

My required workflow triggers when running a Pull Request, using pull_request_target

Lets say the PR is from branch my-feature to xyz

What are the expected values for:

• github.base_ref
• github.head_ref

I suppose github.head_ref should be my-feature and github.base_ref should be xyz... right?

In my case, head looks fine, but base points to main every time.
Am I doing something wrong? Or my understanding regarding which information I should see is incorrect?

Also, I thought I misunderstood the use of github.base_ref and tried to get this info from a more specific place:

• github.event.pull_request.base.ref

still showing main

Thanks in advace

[heavymachinery]

Hi @dexterorion, thanks for trying this out!

In the scenario you described, I would expect github.head_ref to be my-feature and github.base_ref to be xyz. The base_ref should only be main if the PR is targeting main branch.

I tested this out on my end and it worked as expected. If you'd like, you can reach out to [email protected] and we can look further into your workflow to figure out what's going on 🙂

[jfrost-mo]

An issue I've encountered is that PRs created by Actions workflows don't trigger the required checks to run. This has long been a problem, and before this I've been able to work around it by running those workflows manually. With these required workflows however the status doesn't seem to be tied to the commit status, and is instead some kind of PR status that can't be changed by manually running the workflow outside of the PR.

For example, the checks in MetOffice/CSET#203 were manually run in MetOffice/CSET/actions/runs/6455607589 but that hasn't marked the status as suceeded.

I guess the two issues here are that it doesn't pick up when the commits do pass the checks, and that automatically generated PRs can't be merged as the workflow doesn't run.

[grahame-student]

Apologies for resurrecting this discussion, but my team are finding this scenario a blocker when implementing automation that pushes back to the repo.

Here's a simple example:

• set up a required workflow to carry out linting
• implement some automation to automatically apply safe linting fixes and push the changes back to the branch

Changes applied via automation can't trigger other workflows, a perfectly sane design choice by GitHub, however we now have to make a small change or do an empty commit to get the required workflow to run.
Running the workflow manually doesn't cause the necessary status to get set, if it did that would be an acceptable compromise that we could live with.

We were inspired by a couple of the GitHub articles around IssueOps and implemented a few proof of concepts allowing on demand formatting fixes, however the inability to tie up with the related workflows is making it difficult to roll out on a larger scale.

[jfrost-mo]

The solution (workaround?) I've found for this is to use a GitHub App token to make the PR (or the commit). This can be done from inside Actions via actions/create-github-app-token.

For an example, see https://github.com/MetOffice/CSET/blob/ed9674b24f0f60e58c6fc62b2d729f601a0e4d3c/.github/workflows/pre-commit-update.yml#L37-L62

[igwejk]

@tonyjcamp will push events be supported in required workflows?

[tonyjcamp]

I'm sorry but push is not supported at this time.

[reiniertimmer]

Would it be possible (or is it discussed) to add more types of triggers?

Example: in our company, we have a shared set of workflows for building applications. These are now distributed (ie. pushed) to individual repositories. These workflows handle both the PR validation (on: pull_request) as well as building/releasing the application after merge (e.g. on: push to main or on: pull_request closed/merged)

It would be great if we could capture both of these cases in such a system. Then the entire CI/CD process could effectively be offloaded to rulesets without the need to distribute workflows to individual repositories.

[dvaske]

@tonyjcamp is there plans to support any other events, I'm looking at pull_request_review and pull_request_review_comment and status

[tonyjcamp]

Hi @dvaske 👋 - we don't have any immediate plans to add triggers at this time but we do see the feedback in this area growing. I'd love to learn more about your use cases if you don't mind sharing!

[dvaske]

Sure, @tonyjcamp here is the quick use case.

In our setup we have multiple git repositories with different components, but all the different repositories are required for a build, and often/sometimes changes or refacs can span multiple components or often multiple repositories.
We also have an requirement which states that a review must be made by a person who is in an "approver group" this group is just an AD group where we look up members by means of an action.

We therefore have a lot of custom actions to combine pull-requests across repositories.
To merge a PR builds of cause needs to pass, and an the PR bust be approves as descibed above. We therefore depend on the status event and the pull_request_review to trigger merges across all repositories once all pull requests is reviewed and approved and the build status is ✅

For now this has been configured for every single repository, but with the rulesets it would be possible to just have it in one place.

[tonyjcamp]

Thank you!

[devinburnette]

i also have a use case for this.. please keep us updated.

[leszekbulawa]

Does this feature work for the pull_request edited event? Editing a pull request does not trigger the workflow.

[lrotschy]

We do not currently support non-default activity types. For pull_request, default activity types are opened, synchronize, and reopened. We are planning to add other activity types, as we have received feedback from other customers as well that they want to use other activity types.

[leszekbulawa]

Thanks for the reply! Please add +1 from me to the other activity types feature request :)

[lrotschy]

Will do! Thank you for the feedback!

[cmpsoares91]

Does the ruleset modify the workflow type configuration. For instance I've created a workflow that should only be triggered on pull_request.review_requested But it gets triggered with all other types. And I'm not sure on how to stop this?

Any ideas to setup rulesets like this? Not required but run everywhere on a specific event type?

[leszekbulawa]

I guess this explains:
https://github.com/orgs/community/discussions/67754#discussioncomment-7844416

[lrotschy]

Yes, we are planning to add other activity types. Thank you for your feedback!

[amoralesc]

@tonyjcamp This seems like a good place to ask if this feature will ever support compatibility with the paths filter on workflows (similar problem presented in this StackOverflow question).

Let's say we have a workflow that should run on PRs to main, only when Python files have changed:

name: CI
on:
  pull_request:
    branches:
      - main
    paths:
      - "**/*.py"

jobs:
  lint:
    name: Lint
    # ...

And let's also say we set a status check that targets main and sets Require status checks to pass with the Lint job from this workflow.

In a situation where we have a PR to main with no changes to Python files, the PR will be blocked by this status check, but no job is going to run to mark it as successful or failed (the message shows Expected — Waiting for status to be reported). Example images of the current behavior are shown below:

 

Current workarounds (like the one in the StackOverflow question) include not setting the paths filter, but instead, running an action to check if the files changed and then using the output of that action to determine if the rest of the steps need to run. The problem with this workaround is that: a) it ignores the already native solution of skipping workflows when the filter doesn't match and b) it requires actually running the workflow; it seems like a waste to run an action to check for this, specially when running the whole job will last more or less the same (plus will be billed the same on sub 1-min jobs).

It would be amazing if skipped checks could be simply marked as successful, or perhaps even extend the workflows API to choose what to report as an skipped status check (success or failure).

[tonyjcamp]

Hi @amoralesc - thank you for sharing this feedback. It's not something we have plans to solve in the near term (next few quarters) but we have plans to discuss this in the future and decide if we will support it.

[Bzaid94]

Hello @tonyjcamp !

When a developer makes a development and uploads his changes to the repository, it is necessary to make a PR to the main branch, and for there to be a code reviewer. Here we have a ruleset in the organization so that all the repositories maintain the same rules.

We created a workflow so that there is an approval by the QA team, and they have read-only permissions within the repositories. The workflow handles these conditions:

• It expects that there is a comment and a keyword in the PR, example: Approved
• That comment and the keyword must be from a member of the QA team

And that triggers the workflow within the repository, but it acts as a:

on:
issue_comment:
types: [created, edited]

permissions:
contents: write
statuses: write
pull-requests: read

This guarantees that it has passed the necessary tests. But it is not functional for us within the rulesets, because the event is not started with a , with pull_request, pull_request_target or merge_group, but it is started with an issue_comment

Have you considered adding more events? This would help us a lot to not add that workflow to each of the repositories that we have within the organization, and thus the maintenance would be less.