Skip to content

cmd/go: incorrect pseudo-versions for merge commits #29798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jefferai opened this issue Jan 17, 2019 · 6 comments
Closed

cmd/go: incorrect pseudo-versions for merge commits #29798

jefferai opened this issue Jan 17, 2019 · 6 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@jefferai
Copy link

jefferai commented Jan 17, 2019

What version of Go are you using (go version)?

$ go version
go version devel +4b3f04c63b Thu Jan 10 18:15:48 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes. Tested on 1.11.4 and tried with 1.12-beta2.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jeff/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jeff/go"
GOPROXY=""
GORACE=""
GOROOT="/home/jeff/src/go"
GOTMPDIR=""
GOTOOLDIR="/home/jeff/src/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jeff/hc/vault/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build737217089=/tmp/go-build -gno-record-gcc-switches"

What did you do?

When trying to convert a large program from govendor to go modules, I'm seeing behavior where many of the dependencies are being overwritten after I set them; a version using a git commit will be overwritten by the latest tag, even if that latest tag is extremely old.

Invoking any go command causes this overwriting.

This only seems to affect specified revisions that are merge commits (as in, the product of a PR merge). When the merge commit is used the version is v0.0.0-<timestamp>-<revision>. When I use the non-merge commit, it shows up with v0.9.2-<timestamp>-<revision>.

What did you expect to see?

Once I set a version, it is left alone.

What did you see instead?

(You can follow along in the gomodtry2 branch in github.com/hashicorp/vault if you like)

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@a0450ddff675

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
# github.com/SermoDigital/jose
github.com/hashicorp/vault/audit
github.com/SermoDigital/jose/jws
github.com/SermoDigital/jose

 $ grep SermoDigital go.mod
        github.com/SermoDigital/jose v0.9.1
@jefferai
Copy link
Author

The inconsistent part of the issue subject is that I created a dead simple test program:

package main

import "github.com/SermoDigital/jose/jwt"

func main() {
        var _ jwt.JWT
}

and I do not see the behavior here:

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.9.1

$ go get github.com/SermoDigital/jose@803625baeddc

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go get github.com/SermoDigital/jose@a0450ddff675

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.9.2-0.20180104202408-a0450ddff675

$ go get github.com/SermoDigital/jose@803625baeddc

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

$ go mod why github.com/SermoDigital/jose
# github.com/SermoDigital/jose
github.com/jefferai/modbug
github.com/SermoDigital/jose/jwt
github.com/SermoDigital/jose

$ grep SermoDigital go.mod
require github.com/SermoDigital/jose v0.0.0-20180104203859-803625baeddc

Other than "the other go.mod file has many more deps and is more complicated" I can't figure out why I would see different behavior given the same set of commands and revisions for the same dep.

@ChrsMark
Copy link

+1

Having exactly the same issue. This can be really painful in my case since CIs are dependent on the fact that dependencies are located in versioned directories, so extracting them to another format break things.

@odeke-em odeke-em changed the title go mod inconsistently changes versions without user interaction cmd/go: mod inconsistently changes versions without user interaction Jan 22, 2019
@odeke-em odeke-em added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 22, 2019
@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

You are running into two separate issues here.

One is #27171: for some reason, your module cache maps commit 803625baeddc to version v0.0.0-20180104203859-803625baeddc. With a clean module cache, I'm seeing that map to version v0.9.2-0.20180104203859-803625baeddc instead.

The other issue is that you have a dependency on github.com/influxdata/platform. The latest version of github.com/influxdata/platform has a go.mod file that requires github.com/SermoDigital/jose v0.9.1. When you try to go get a version older than that, the go command attempts to downgrade all other modules to compatible versions, and because github.com/influxdata/platform does not have version tags, the only downgrade that go get can perform is to remove it entirely.

Once the dependency on github.com/influxdata/platform has been removed, you keep the dependency on the older-versioned github.com/SermoDigital/jose until the next time something tries to resolve the import from github.com/influxdata/platform. At that point, you pick up the latest version of github.com/influxdata/platform again, and that upgrades github.com/SermoDigital/jose back to v0.9.1.

vault$ rm go.sum

vault$ go mod tidy
[…]

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.1

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/[email protected]
github.com/influxdata/[email protected] github.com/SermoDigital/[email protected]

vault$ go get github.com/SermoDigital/jose@803625baeddc
go: finding github.com/SermoDigital/jose 803625baeddc
go: downloading github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc
go: extracting github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/[email protected]
github.com/influxdata/[email protected] github.com/SermoDigital/[email protected]

vault$ go mod tidy

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc

vault$ go get github.com/SermoDigital/jose@6686625
go: finding github.com/SermoDigital/jose 6686625
go: downloading github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55
go: extracting github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.0.0-20151001034355-66866257aa55

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/[email protected]

vault$ go list -m github.com/influxdata/...
github.com/influxdata/flux v0.7.2
github.com/influxdata/influxdb v1.7.1
github.com/influxdata/influxql v0.0.0-20180925231337-1cbfca8e56b6
github.com/influxdata/line-protocol v0.0.0-20180522152040-32c6aa80de5e
github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9
github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368

vault$ go mod tidy
go: finding github.com/influxdata/platform/models latest
go: finding github.com/influxdata/platform latest
go: downloading github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589
go: extracting github.com/influxdata/platform v0.0.0-20190117200541-d500d3cf5589
go: finding github.com/circonus-labs/circonusllhist v0.1.3
go: finding github.com/influxdata/flux v0.13.0
go: finding github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: finding github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82
go: finding github.com/benbjohnson/tmpl v1.0.0
go: finding github.com/yudai/pp v2.0.1+incompatible
go: finding github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88
go: finding github.com/golang/gddo v0.0.0-20181116215533-9bd4a3295021
go: finding github.com/apache/arrow/go/arrow v0.0.0-20181217213538-e9ed591db9cb
go: finding github.com/goreleaser/goreleaser v0.94.0
go: finding github.com/yudai/gojsondiff v1.0.0
go: finding github.com/dustin/go-humanize v1.0.0
go: finding github.com/miekg/dns v1.1.1
go: finding gopkg.in/src-d/go-git.v4 v4.8.1
go: finding github.com/aokoli/goutils v1.0.1
go: finding github.com/Masterminds/sprig v2.16.0+incompatible
go: finding github.com/aws/aws-sdk-go v1.15.64
go: finding github.com/apache/arrow/go/arrow v0.0.0-20181031164735-a56c009257a7
go: finding github.com/huandu/xstrings v1.0.0
go: finding github.com/stretchr/testify v1.2.0
go: finding github.com/xanzy/ssh-agent v0.2.0
go: finding github.com/emirpasic/gods v1.9.0
go: finding github.com/gliderlabs/ssh v0.1.1
go: finding golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: finding golang.org/x/sys v0.0.0-20181030150119-7e31e0c00fa0
go: finding github.com/kevinburke/ssh_config v0.0.0-20180830205328-81db2a75821e
go: finding gopkg.in/src-d/go-billy.v4 v4.2.1
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.1.1
go: finding golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: finding golang.org/x/crypto v0.0.0-20180505025534-4ec37c66abab
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding github.com/src-d/gcfg v1.4.0
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding golang.org/x/sys v0.0.0-20180903190138-2b024373dcd9
go: finding golang.org/x/tools v0.0.0-20181221154417-3ad2d988d5e2
go: downloading github.com/aws/aws-sdk-go v1.15.64
go: downloading golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: downloading golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: extracting golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: extracting golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: extracting github.com/aws/aws-sdk-go v1.15.64
go: downloading github.com/miekg/dns v1.1.1
go: downloading github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: extracting github.com/circonus-labs/circonus-gometrics v2.2.5+incompatible
go: downloading github.com/circonus-labs/circonusllhist v0.1.3
go: extracting github.com/circonus-labs/circonusllhist v0.1.3
go: extracting github.com/miekg/dns v1.1.1

vault$ go list -m github.com/SermoDigital/jose
github.com/SermoDigital/jose v0.9.1

vault$ go mod graph | grep github.com/SermoDigital/jose
github.com/hashicorp/vault github.com/SermoDigital/[email protected]
github.com/influxdata/[email protected] github.com/SermoDigital/[email protected]

@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

Are you able to reproduce this issue using a clean module cache (go clean -modcache)?

If so, could you post the output of git --version? There may be a bad interaction between go and git that is interfering with our ability to infer the correct pseudo-version of merge commits.

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. modules labels Jan 23, 2019
@bcmills bcmills added this to the Go1.13 milestone Jan 23, 2019
@bcmills bcmills changed the title cmd/go: mod inconsistently changes versions without user interaction cmd/go: incorrect pseudo-versions for merge commits Jan 23, 2019
@jefferai
Copy link
Author

I'll give it a try with a wiped module cache. However, one problem with your analysis:

The latest version of github.com/influxdata/platform has a go.mod file that requires github.com/SermoDigital/jose v0.9.1. When you try to go get a version older than that, the go command attempts to downgrade all other modules to compatible versions ....... At that point, you pick up the latest version of github.com/influxdata/platform again, and that upgrades github.com/SermoDigital/jose back to v0.9.1

The issue here is that go mod is assuming that 0.9.1 is in fact newer than v0.0.0-20180104203859-803625baeddc -- but in fact it's way older. I would think looking at the commit date for the given tag and comparing it against the existing commit date would cause it to realize that -- but maybe it's due to the bug you mentioned.

@jefferai
Copy link
Author

I can confirm that with a cleaned modcache this works. I have no idea how it got unclean, but...there you go. I'll close the bug -- thanks for the response!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

5 participants