Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gomod): preserve go/toolchain directives #34779

Open
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

trim21
Copy link
Contributor

@trim21 trim21 commented Mar 13, 2025

Changes

If go.mod has go>=1.21, add toolchain@{toolchain directive} go@${go directive} to preserve the directives. use toolchain@none as a special case to not add toolchain directive.

Only add -d flag if go.mod has old go version < 1.17.

close #34097

Context

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@trim21 trim21 marked this pull request as draft March 13, 2025 09:42
@trim21 trim21 marked this pull request as ready for review March 13, 2025 09:47
@viceice viceice changed the title fix: preserve go mod toolchain directive fix(gomod): preserve go mod toolchain directive Mar 13, 2025
@trim21

This comment was marked as off-topic.

@trim21 trim21 requested a review from viceice March 13, 2025 10:17
@trim21 trim21 marked this pull request as draft March 13, 2025 10:22
@trim21 trim21 marked this pull request as ready for review March 13, 2025 10:27
@trim21 trim21 changed the title fix(gomod): preserve go mod toolchain directive fix(gomod): preserve go/toolchain directives Mar 13, 2025
@trim21
Copy link
Contributor Author

trim21 commented Mar 13, 2025

@viceice I changed the implementation due to I didn't fully understand this problem, please check the PR description for currently implementation .

@trim21
Copy link
Contributor Author

trim21 commented Mar 13, 2025

@rarkins can you take a look? I believe this issue extends beyond the scope of just gomod.

Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs extensive testing on existing repos

@viceice
Copy link
Member

viceice commented Mar 13, 2025

I think we should revert #34564 and start new here.

@rarkins wdyt?

@rarkins
Copy link
Collaborator

rarkins commented Mar 14, 2025

I think we should revert #34564 and start new here.

@rarkins wdyt?

I thought that #34564 was an improvement over what we had before though. i.e. using the toolchain version instead of the latest?

@viceice
Copy link
Member

viceice commented Mar 14, 2025

I think we should revert #34564 and start new here.

@rarkins wdyt?

I thought that #34564 was an improvement over what we had before though. i.e. using the toolchain version instead of the latest?

sure, but the hosted app was reverted because of issues with that fix. see slack discussion

@rarkins
Copy link
Collaborator

rarkins commented Mar 14, 2025

Definitely revert then

@trim21
Copy link
Contributor Author

trim21 commented Apr 3, 2025

How would artifactUpdate error behave? Would it NOT create the PR? But the err would appear in the dashboard?

oapi-codegen/oapi-codegen#1641 (comment)

@trim21
Copy link
Contributor Author

trim21 commented Apr 3, 2025

I suggest adding updated package to go get command, go command provide a clear enough error message and we write less code.

#34779 (comment)

@trim21
Copy link
Contributor Author

trim21 commented Apr 3, 2025

I get my other PRs merged, I can go back work on this if you don't want to go this direction.

@casaqori
Copy link

casaqori commented Apr 3, 2025

$ go get [email protected] toolchain@none golang.org/x/[email protected] ./...
go: golang.org/x/[email protected] requires [email protected], not [email protected]

Also, without explicitly adding package name to command we see that error message.
Just currently I do not see an example of capturing the std output. And further, it's multiple commands stacked up that are being called.
So, maybe you can commit your ideas and we will see further.
What still causes headaches is that the PR is left in a state where it has 'golang.org/x/[email protected]' in the go.mod
When I "cleaned" go.mod to the state it was before the PR (since no upgrade was done) then I get warnings in console about having detected an empty commit which will not be pushed.

@trim21 trim21 requested a review from viceice April 5, 2025 17:46
@trim21
Copy link
Contributor Author

trim21 commented Apr 5, 2025

we get a expected artifact update error now:

trim21-bot/oapi-codegen#6

image

@trim21
Copy link
Contributor Author

trim21 commented Apr 5, 2025

added some more code to handle the major upgrade case, because we need to also update package name

config.constraints?.go ?? getGoConstraints(newGoModContent);

const goMod = getGoConfig(newGoModContent);
const goConstraints = config.constraints?.go ?? `^${goMod.minimalGoVersion}`;
Copy link

@casaqori casaqori Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code does enforce the version currently declared by the go directive.
Which in turn will prevent any dependency upgrade to succeed that requires even a higher minor version of go.

go 1.23.0

config.constraints.go 1.23 (or GOTOOLCHAIN="local")
installed toolchain go1.23.1

required by dependency: go1.23.7

enforced by constraint: go1.23.0

Wasn't there agreement that this is undesired?

Copy link
Contributor Author

@trim21 trim21 Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/renovatebot/renovate/pull/34779/files#diff-4c37b8edc62aa49295d264c221717bd0361ccff0a791d4327f4421a0ea23521fR2214

Consider x/crypto would specify go1.23.7 in it's go.mod

[email protected] would not allow that ...
And users can not control that behavior in renovate. They would explicitly need to bump go before that. Why not allow the bump in same PR?

Copy link
Contributor Author

@trim21 trim21 Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the language version should have more priority than library version. Users can always enable updating go directive, but they can't suppress updating go directive caused by library if they want, therefore is this PR

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is what a sane default is:

  1. allow automated go version bumping that is supported by local toolchain when dependency requires that (have PRs that can be merged, toolchain always latest, current state of affairs)
  2. avoid that (have PRs that break), enforce users to maintain their go directive separately
  3. allow users to choose from 1) and 2) - e.g. if config.constraints.go is set, use strategy 1) for allowing updates within the constrained minor only

Who would be able to take such decision @viceice ?

Copy link

@casaqori casaqori Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..., but they can't suppress updating go directive caused by library if they want

That is only the case for auto-merges of PRs. Otherwise users can transparently review and close unwanted PRs.

Please also consider that the changes currently WILL ALLOW go directive bumping. Here is the observation:

go.mod is at 1.22.0

go get -t ./... toolchain@none [email protected] github.com/someorg/[email protected]
go: github.com/someorg/[email protected] requires go >= 1.22.2; switching to go1.23.8
go: github.com/someorg/[email protected] requires [email protected], not [email protected]
go.mod is at 1.22

go get -t ./... toolchain@none [email protected] github.com/someorg/[email protected]
go: github.com/someorg/[email protected] requires go >= 1.22.2; switching to go1.23.8
go: upgraded go 1.22.0 => 1.22.12

go.mod is now at 1.22.12

And that is if GOTOOLCHAIN is "auto". On "local" with a 1.22.0 toolchain, we get no switching:

go get -t ./... toolchain@none [email protected] github.com/someorg/[email protected]
go: github.com/someorg/[email protected] requires [email protected], not [email protected]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I don't know this case.

A workaround would be using go 1.23 instead of go 1.23.0 in go mod, but that requires user to change their code ...

A better solution would be disabling go version switching and pick 1.23.0 when it has go 1.23 directive 😅

Copy link
Contributor Author

@trim21 trim21 Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is what a sane default is:

  1. allow automated go version bumping that is supported by local toolchain when dependency requires that (have PRs that can be merged, toolchain always latest, current state of affairs)
  2. avoid that (have PRs that break), enforce users to maintain their go directive separately
  3. allow users to choose from 1) and 2) - e.g. if config.constraints.go is set, use strategy 1) for allowing updates within the constrained minor only

Who would be able to take such decision @viceice ?

IMO, if users want renovate to touch go version, they can manually enable it.

So renovate will update the go directive to expected version based on user config.

If they group go directive with packages, they get one working PR.

And if they have seprated PRs for go direcive and package, they can still get a working PR for package after they merge the PR for go directive.

But I'm not opposed to a new config that let go command updating go directive, it would very easy to add such feature. we just don't push anything into extraGetArguments.

For example

const preserveGoDirective = config.preserveGoDirectives ?? 'enabled';
if (
  preserveGoDirective !== 'enabled' &&
  semver.satisfies(goMod.minimalGoVersion, '>=1.21.0')
) {
  extraGetArguments.push(`toolchain@${goMod.toolchainDirective ?? 'none'}`);
  extraGetArguments.push(`go@${goMod.goDirective}`);

  ...
}

But it should be done in a seprated PR.

@trim21
Copy link
Contributor Author

trim21 commented Apr 11, 2025

to get what we want, we need:

  1. add go@${go directive} , toolchain@${toolchain directive} and all updated package to go get commands, for example go get -t ./... [email protected] [email protected] golang.org/x/[email protected]
  2. add env GOTOOLCHAIN=local to disable to toolchain switching.
  3. pick the minimal supported go version defined in go.mod.

So in the race case that go.mod has go major.minor, and a package requires go major.minor.patch where patch != 0, go still doesn't touch go directive.

Copy link
Collaborator

@rarkins rarkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes far beyond the #34097 that it closes. It's also a pretty fundamental change to how Renovate handles go compatibility, and needs its own discussion, which I don't want to do here in a PR review comment

@trim21
Copy link
Contributor Author

trim21 commented Apr 12, 2025

it close #34097 by the way, but it's not the main target of this pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Skip -d flag for "go get" if go 1.18 or later is run
5 participants