Unable to create annotations

[xt0rted]

I have two actions that create annotations after running eslint & stylelint but I have yet to move them to v2. They run fine in v2 workflows though and annotations are created as you'd expect.

I'm trying to setup a new action that creates annotations just like my other actions but using the toolkit packages. I'm able to create a checks run using GITHUB_TOKEN and the github client, but I'm not able to call the update checks function with the annotations. I keep getting a 422 response back. Are there differences between v1 and v2 actions that would prevent this from working?

This call works:

const githubClient = new github.GitHub(core.getInput("repo-token", { required: true }));

const { data } = await githubClient.checks.create({
  ...github.context.repo,
  name: github.context.action,
  head_sha: github.context.sha,
  started_at: new Date().toISOString(),
});

But this call doesn't:

const githubClient = new github.GitHub(core.getInput("repo-token", { required: true }));

const { data} = await githubClient.checks.update({
  ...github.context.repo,
  check_run_id: id, // data.id from the first call
  completed_at: new Date().toISOString(),
  conclusion: conclusion,
  output,
  status: "completed",
});

[thboop]

Hi @xt0rted,

I'm having trouble reproducing this issue, I've outlined my steps below in case they are helpful!
Could you also provide an example output variable?
If possible, could you also provide a link to a run using your action, or a link to your action.

I tried the following below:

Here are the packages I am using:

    "@actions/core": "^1.1.0",
    "@actions/github": "^1.1.0"

Here is my action main.ts

import * as core from '@actions/core';
import * as github from '@actions/github';

async function run() {
  try {
    const githubClient = new github.GitHub(core.getInput("repo-token", { required: true }));
    const { data } = await githubClient.checks.create({
      ...github.context.repo,
      name: github.context.action,
      head_sha: github.context.sha,
      started_at: new Date().toISOString(),
    });
    console.log(JSON.stringify(data));
    const update = await githubClient.checks.update({
      ...github.context.repo,
      check_run_id: data.id, // data.id from the first call
      completed_at: new Date().toISOString(),
      conclusion: "success",
      output: 
      { 
        summary: "summary",
        title: "title",
        annotations: [
          {
            path: "./dummyfile.txt",
            end_line: 1,
            start_line: 1,
            annotation_level: "failure",
            message: "missing value"
          },
        ]
      },
      status: "completed",
    });
    console.log(JSON.stringify(update));
  } catch (error) {
    core.setFailed(error.message);
  }
}

run();

Here is how I invoke the action

name: test
on: push
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Test Action
      uses: bbq-beets/thboop-action@master
      with:
        repo-token: ${{secrets.GITHUB_TOKEN}}

[xt0rted]

@thboop I'll have to take a look at this again later, but the main difference I see between the two setups is yours isn't creating annotations in the update call. I'm able to create the checks run just fine, but when it comes to the annotations that's when the request fails.

[thboop]

@xt0rted
Sounds great, thank you!
I can confirm I can push annotations as well, I've updated the above code snippets.

[xt0rted]

@thboop this is embarrassing, the issue was on my end. I was setting the start_column for the annotation, but not the end_column. That's why the requests kept failing. The docs don't really mention this but after some trial and error that's what I narrowed it down to.

The issue I'm facing now is how to get the annotations to be associated with the job they ran in. In v1 I did this by using GITHUB_ACTION as the check name, but with v2 this gets set to self (or maybe still {user}{repo} if using one outside the current repo) and then another job shows in the left, and that's where the annotation results show.

[xt0rted]

After some more testing I'm seeing that if the workflow runs on push the annotations are created and shown, but if the workflow runs on pull_request the annotations are created but they don't get shown anywhere.

[thboop]

Hey @xt0rted
The checks API requires the head_sha, the sha provided by github.context.sha for pull_request builds is a merge sha, so these do not end up aligning.

I'm going to take this feedback to the team and see how we can improve this experience! Thanks for your feedback and quick responses on this item we really appreciate it!

For now, you can use access the head_sha for pull request builds via github.context.payload.pull_request.head.sha. I've confirmed that I can see the annotations for those builds.

I will also file an item for the 422 response code you were seeing and see how we can improve that messaging and error handling.

[bryanmacfarlane]

@thboop @xt0rted - OK to close this issue from a toolkit perspective? It seems like we figure it out but I want to make sure.

[xt0rted]

@bryanmacfarlane the annotations are working now, but I'm not sure how to associate them with the job that created them. This worked in v1 by using GITHUB_ACTION as the check name but not in v2 so far. This doesn't really feel like a toolkit issue, just lack of docs or server side functionality.

An example run can be seen here. The job self was created from the annotations in the ESLint step. When the action is in the repo it's running from the job is called self, if it's loaded from an external repo it's {user}{repo} or in this case xt0rtedeslint-action.

[thboop]

Hey @xt0rted ,
We are currently working on smoothly this experience out for you!

Our recommended flow for this is going to be using problem matchers, as these will automatically create issues on the run. We have a feature in development which will create annotations for issues on your run, providing the end to end functionality I believe you are trying to accomplish. This should allow you to more easily create annotations without having to manually use the sdk to create another check!

We have a ticket to update our documentation #56 which will cover how to do this.
I also want to cover in the docs on the toolkit more information about annotations and how users can create them from the toolkit, I've updated that issue to indicate that.

[xt0rted]

@thboop I'll give this a try and see how it works compared to the api approach.

[xt0rted]

@thboop I forgot to ask, when creating annotations with the api you can add action buttons to them. The UI shows them and reacts to clicking them, but I wasn't able to get that to trigger a workflow on check_run:requested_action. Is this something that's possible with actions right now?

[thboop]

@thboop I forgot to ask, when creating annotations with the api you can add action buttons to them. The UI shows them and reacts to clicking them, but I wasn't able to get that to trigger a workflow on check_run:requested_action. Is this something that's possible with actions right now?

@xt0rted , that doesn't seem intended. Other users recently reported similar issues with triggering workflows from apps. I'll pass along that you are seeing this issue as well to our engineering team. You can also reply in the community forums.

Just for clarity's sake, I do not think that issue is related to the toolkit.