Swap workflow sometimes generates incorrect release notes (OSOE-1112)

[BenedekFarkas]

The Swap workflow also creates a release in the code repository, but the release notes are not always correct, i.e. different than what it should be according to the diff between the previous release's commit and the current one.

Noticed in DoNest that one the Swap workflow instance's step responsible for moving the production/latest tag is one that doesn't seem to exist (in the code repo, neither in the swap repo): 23f505af

Deleted tag 'production/latest' (was 23f505af)
To https://github.com/Lombiq/DotNest-Tenants-Core
 - [deleted]           production/latest
Deleted tag 'staging/latest' (was 96837ea7)
To https://github.com/Lombiq/DotNest-Tenants-Core
 - [deleted]           staging/latest
To https://github.com/Lombiq/DotNest-Tenants-Core
 * [new tag]           staging/latest -> staging/latest
To https://github.com/Lombiq/DotNest-Tenants-Core
 * [new tag]           production/latest -> production/latest
To https://github.com/Lombiq/DotNest-Tenants-Core
 * [new tag]           production/2025-05-15T13-55-[53](https://github.com/Lombiq/DotNest-Tenants-Core-Swap/actions/runs/15046359395/job/42290772611#step:9:54) -> production/2025-05-15T13-55-53
Notice: The commit from the app code repository with the SHA 96837ea743efc0dbf32002993af0b31c646c9c77 was swapped out.

This could be a clue as to why it's generating the release notes based on the incorrect diff.

Jira issue

[Piedone]

Just adding some context (also for Copilot):

This is coming from the "Move Latest Tag (Swap) and Add Timestamp Tag" step. Attached the full workflow log to the Jira issue in case it gets cleaned up in the meantime.

[Piedone]

After some investigation:

It seems that usually, the previous tag determined by GitHub for the release (i.e. the one it computes the diff from to the current production tag) is the one two or more tags before. E.g., in DotNest-Tenants-Core (https://github.com/Lombiq/DotNest-Tenants-Core/releases) there were releases with the following tags, i.e. at the following times:

• production/2025-06-14T01-06-06
• production/2025-05-15T14-43-13
• production/2025-04-13T16-58-00
• production/2025-03-21T00-21-43
• production/2025-03-19T08-58-47
• production/2025-03-17T23-57-54
• production/2025-03-17T17-12-57
• production/2025-03-14T16-37-23
• production/2025-03-07T01-12-31
• production/2025-03-06T20-51-01
• production/2025-03-06T14-25-28

Then, the GitHub releases use the following diffs, which are too wide, encompassing not just the previous production tag, but at least the second back (the ending tags are correct):

• production/2025-03-21T00-21-43...production/2025-06-14T01-06-06 (this is the 3rd tag back)
• production/2025-03-21T00-21-43...production/2025-05-15T14-43-13
• production/2025-03-19T08-58-47...production/2025-04-13T16-58-00 (this should actually start with production/2025-03-21T00-21-43, what was used in the previous two, but jumped a tag)
• production/2025-03-17T17-12-57...production/2025-03-21T00-21-43
• production/2025-03-17T17-12-57...production/2025-03-19T08-58-47
• production/2025-03-14T16-37-23...production/2025-03-17T23-57-54
• production/2025-03-06T14-25-28...production/2025-03-17T17-12-57 (this is the 4th tag back)
• production/2025-03-06T14-25-28...production/2025-03-14T16-37-23 (this is the 3rd tag back)

There's no simple pattern. Maybe the algorithm that GitHub uses to determine the previous tag can't order these properly, and that's why there's a mix-up (although based on this discussion, it should work with alphabetic ordering). We could try using the version number that specify for the assemblies during the Build and Static Code Analysis step, since that's a proper version number that GitHub might understand and order correctly.

Or, we can explicitly specify the previous tag in the Create Release step, since we already know the previous production/latest commit. However, https://github.com/ncipollo/release-action that we use, doesn't support specifying the previous tag.

Note that the https://github.com/Lombiq/DotNest-Tenants-Core/releases/tag/production%2F2025-05-15T13-55-53 release was draft, with its tag not existing; I fixed it by hand, so this is expectedly out of the pattern.

[BenedekFarkas]

Yep, according to GHCP (the context is release-action's code):

When you generate release notes via the GitHub API (for example, using the generateReleaseNotes endpoint) and do not specify a previous tag, GitHub will attempt to automatically determine the previous tag.

How GitHub selects the previous tag:

GitHub will look at the list of tags in your repository.
It typically selects the tag associated with the most recent previous release (by semantic versioning or date, depending on your tags).
This previous tag is used as the starting point for the comparison when generating release notes, showing changes between the previous and the new tag.

It indeed looks like that GH is selecting the wrong release to find which tag to use as a base for the one being created.

BTW:

• 2025-01-28T01-57-46...production/2025-01-28T02-47-37 is the last one that is correct.
• 2025-01-28T01-57-46...production/2025-03-06T14-25-28 is already wrong as it's based on the same tag as the previous release is based on.
• 2025-01-28T02-47-37...production/2025-03-06T20-51-01 is based on the last correct one (second back).

Note that the https://github.com/Lombiq/DotNest-Tenants-Core/releases/tag/production%2F2025-05-15T13-55-53 release was draft, with its tag not existing; I fixed it by hand, so this is expectedly out of the pattern.

That's weird. I haven't found anything in the workflow log that would explain why it was created as a draft.

I fixed the base tag of each release, but unfortunately 2025-05-15T13:55:53 is out of order (since it was published most recently), but set 2025-06-14T01:06:06 as the Latest release instead.

Or, we can explicitly specify the previous tag in the Create Release step, since we already know the previous production/latest commit. However, https://github.com/ncipollo/release-action that we use, doesn't support specifying the previous tag.

What I don't understand is why the /releases/generate-notes API endpoint allows specifying the base tag, but /releases (to create a release) doesn't.

It's super to simple to determine that tag though and we can use the GH CLI instead to create the release. There are other places where we use release-action, so that's less than ideal, but at least it would work exactly as expected here.

EDIT: Since release-action allows overriding the release description (body), we can generate the release notes from the GH API (with the previous release tag specified) and pass that on.

[Piedone]

OK, I think I got it now. This looks like rather an issue in how we use ncipollo/release-action. We should set makeLatest to true. That way, the release will be marked as "latest" (which is not the same as being the most recent), and thus, regardless of version number ordering, the next release will be compared against it.

[BenedekFarkas]

I see we're not overriding that and the default value is legacy, but (as far as I can tell) it results in the same behavior in our case as true. At least what I'm seeing in our repositories using this action is that the most recently published release is marked as Latest.

Anyway, if setting makeLatest to true doesn't fix it, then I've got the scripts locally to grab the correct tag and generate the release description based on that, so we can adapt our workflow with those.