Omnicomplete deletes text before cursor when completeopt=longest,menuone and shows erroneous matches

[shabbyrobe]

What did you do? (required: The issue will be closed when not provided)

I created an empty folder, ran go mod init example.com/yep, then made sure completeopt was to longest,menuone, created two variables with a common prefix, started typing fo, then invoked <C-X>, <C-O>:

package main
func main() {
        var fooo = 1
        var food = 2
        fo_ <--- here at the underscore, I press <C-X>, <C-O>
}

What did you expect to happen?

I expect to see the fo prefix I typed in-tact and the two completion options based on that prefix (fooo and food) in the popup:

package main
func main() {
        var fooo = 1
        var food = 2

        fo
}       food    v int                       
~       fooo    v int                                                                                     

What happened instead?

The prefix is partially deleted and incorrect completion options appear (float32, float64). Sometimes the prefix is deleted in its entirety, but I still haven't got a reliable repro for the full deletion scenario yet, which is unfortunate because it happens frequently in practice:

package main
func main() {
        var fooo = 1
        var food = 2

        f
}       food    v int                       
~       fooo    v int                                                                                     
~       for     0                                                                                         
~       float32 t                                                                                         
~       float64 t                                                                                         
~       format  0 "go/format"                                                                             
~       font    0 "golang.org/x/image/font"  
                                                              

Configuration (MUST fill this out):

Unfortunately, I couldn't work out how to get vim to start with a fully separate vim and vimrc while still loading plugins (--clean ignores them), so I just built from source after using ruplacer to replace every single instance of ~/.vim in the source with /tmp/vimconf/vim (to force it to use /tmp/vimconf/vimrc and /tmp/vimconf/vim and ignore my home dir):

wget https://github.com/vim/vim/archive/v8.2.0268.tar.gz
tar xf v8.2.0268.tar.gz && cd vim-8.2.0268
ruplacer '~/.vim' `/tmp/vimconf/vim` --go
./configure --prefix=/tmp/viminst
make -j8 && make install

I then set up the following hierarchy in /tmp/vimconf, where vim-go is a clone of this repo at master, and vimrc contains the lines pasted below (very minimal):

.
├── vim
│   └── pack
│       └── bundle
│           └── start
│               └── vim-go
└── vimrc

I could then invoke vim from within my Go module's root like so: /tmp/viminst/bin/vim -u /tmp/vimconf/vimrc main.go. Even after my dirty hack on the source, I still needed the -u argument, but at least that got me the clean result I was after in the end.

(As an aside, if there's an easier way to do this kind of repro with a clean Vim that doesn't involve the silly things I've done here hacking the source up, I'd be grateful for any pointers.)

vim-go version:

master, but this has been going on for months.

vimrc you used to reproduce (use a minimal vimrc with other plugins disabled; do not link to a 2,000 line vimrc):

vimrc

filetype plugin on
syntax on
set nocompatible
set completeopt=longest,menuone

Vim version (first three lines from :version):

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Feb 18 2020 01:11:14)
Included patches: 1-268
Compiled by bl@bx1

I tried to set this repro up in 8.2.0268, but the issue has also been present in all versions of 8.2 I have used. May have also been present in 8.1, but I don't believe the problem is in vim, I think it's in vim-go, as my TypeScript, Rust and Python setups all continue to work exactly as expected.

Go version (go version):

Tried with 1.13 and 1.14rc1

Go environment

go env Output:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bl/.cache/go-build"
GOENV="/home/bl/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/bl/code/gopath"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/home/bl/apps/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/bl/apps/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build710199065=/tmp/go-build -gno-record-gcc-switches"

gopls version

gopls version Output:

golang.org/x/tools/gopls v0.3.2
    golang.org/x/tools/[email protected] h1:eP1aj1AvT6ynElQH6KP0mmOT2gnWa1gYclHL4wGUbMo=

Problem was also present with pinned version.

[bhcleek]

Does adding let g:go_gopls_fuzzy_matching=0 to your vimrc help?

[shabbyrobe]

I gave it a try, same result. I verified that it was set correctly using :echo g:go_gopls_fuzzy_matching, which outputs 0.

[bhcleek]

The completion candidates are provided by gopls. The only options that vim-go has to control how gopls behaves are documented in vim-go's help (e.g. :help g:go_gopls_fuzzy_matching); if it's not behaving as expected, then there may be an issue with gopls.

@stamblerre would you expect the candidates that @shabbyrobe mentioned to be provided by gopls?

@shabbyrobe if you can reproduce the prefix deletions you're seeing, I'll see if I can find an explanation. I'll say up front, though, that I use completion daily and haven't noticed the problem you're describing. However, there are some sometimes surprising behaviors by Vim itself. :help ins-completion-menu fully describes Vim's behavior; take a look at the description of the three states there and the related :help popupmenu-keys and its descriptions (including how <Enter> is handled differently in each of the three states). Does any of that apply for what you're trying to duplicate?

[shabbyrobe]

Thanks for the detailed response, @bhcleek!

I think the details I've provided here alone are sufficient to reproduce the issue - I can see it reliably when I follow the steps on my own machine. I mentioned earlier that I haven't worked out an exact repro for one aspect of it (where it deletes the entire word before the cursor, rather than just part of it), but the part I have reported appears to be reproducible with the details I've provided here.

If it'd save you the trouble of risking some time on it, I can try to confirm inside a VM with an absolutely fresh install of Vim, Go, vim-go and gopls just to make extra-double-sure something from my environment is not confounding the result, but I'm 95% confident the recompiled Vim did the trick for excluding my normal .vim setup. Is there a set of versions you'd prefer I used for that? Would probably just use ubuntu 19.10 with the "jonathonf" PPA as a starting point.

[shabbyrobe]

For ins-completion-menu, I've given it a read and it seems to describe what happens when I do something after I've pressed <C-x>,<C-o>, i.e. it describes the next steps. The issue I'm having happens in response to the <C-x>,<C-o> itself. Nothing I can interpret in ins-completion-menu describes what I'm seeing here. I'll re-read it a couple of times now to make sure I've read it correctly.

Also the help for completeopt describes insertion, but never deletion/removal/backspacing/unwinding. I don't experience this with any of my other language integrations - longestone only ever inserts as much as it can, it never deletes.

Is there something I can do with gopls to help surface more info for you too? I can install it from source and bang in some debug info or something if that'd help.

[bhcleek]

With regard to gopls, the adding let g:go_debug=['lsp'] to your vimrc will cause vim-go to populate a window with all the interactions with gopls.

If you can find a way to duplicate deletion you're seeing, it's probably quick to resolve if it's due to a bug in vim-go. The only thing I can think of that could potentially be an issue is if there are multi-byte characters on the line where you're trying to do completion. While that should be handled correctly, I can imagine scenarios where a bug around the multi-byte character handling could cause what you're describing.

[shabbyrobe]

The only thing I can think of that could potentially be an issue is if there are multi-byte characters on the line where you're trying to do completion.

Ah yep, makes sense. I reckon that's not what's going on here, there are no multibyte chars in the repro I posted, and they occur very, very rarely in the code I work with.

I grabbed the gopls log from the repro, here are the lines that appeared as I pressed <C-x>, <C-o>:

sent: Content-Length: 155

{"method":"textDocument/completion","jsonrpc":"2.0","id":3,"params":{"textDocument":{"uri":"file:///tmp/wat/main.go"},"position":{"character":3,"line":5}}}
sent: Content-Length: 226

{"method":"textDocument/didChange","jsonrpc":"2.0","params":{"contentChanges":[{"text":"package main\n\nfunc main() {\n\tvar fooo = 1\n\tvar food = 2\n\tfo\n}\n"}],"textDocument":{"uri":"file:///tmp/wat/main.go","version":3}}}
received: Content-Length: 2651

{"jsonrpc":"2.0","result":{"isIncomplete":true,"items":[{"label":"food","kind":6,"detail":"int","preselect":true,"sortText":"00000","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"food"}},{"label":"fooo","kind":6,"detail":"int","sortText":"00001","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"fooo"}},{"label":"for","kind":14,"sortText":"00002","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"for"}},{"label":"float32","kind":7,"sortText":"00003","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"float32"}},{"label":"float64","kind":7,"sortText":"00004","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"float64"}},{"label":"format","kind":9,"detail":"\"go/format\"","sortText":"00005","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"format"},"additionalTextEdits":[{"range":{"start":{"line":1,"character":0},"end":{"line":1,"character":0}},"newText":"\nimport \"go/format\"\n"}]},{"label":"font","kind":9,"detail":"\"golang.org/x/image/font\"","sortText":"00006","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"font"},"additionalTextEdits":[{"range":{"start":{"line":1,"character":0},"end":{"line":1,"character":0}},"newText":"\nimport \"golang.org/x/image/font\"\n"}]},{"label":"format","kind":9,"detail":"\"honnef.co/go/tools/lint/lintutil/format\"","sortText":"00007","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"format"},"additionalTextEdits":[{"range":{"start":{"line":1,"character":0},"end":{"line":1,"character":0}},"newText":"\nimport \"honnef.co/go/tools/lint/lintutil/format\"\n"}]},{"label":"format","kind":9,"detail":"\"github.com/klauspost/asmfmt/cmd/gofmt/format\"","sortText":"00008","filterText":"food","insertTextFormat":1,"textEdit":{"range":{"start":{"line":5,"character":1},"end":{"line":5,"character":3}},"newText":"format"},"additionalTextEdits":[{"range":{"start":{"line":1,"character":0},"end":{"line":1,"character":0}},"newText":"\nimport \"github.com/klauspost/asmfmt/cmd/gofmt/format\"\n"}]}]},"id":3}

[bhcleek]

Those logs are what I would expect.

I'll wait for Rebecca to weigh-in regarding whether the candidates are expected (my suspicion is that they are).

Let me know if you find a way to replicate the too-aggressive deletion.

[shabbyrobe]

Let me know if you find a way to replicate the too-aggressive deletion

I just had a bit more of a play around and I think I have it!

package main

type Stuff struct{}
type ThingoStuff struct{}

func main() {
        var v Stuff_ # <--- here at the "cursor", I press <C-X>, <C-O>
}

The menu shows Stuff and ThingoStuff, but deletes Stuff altogether. This happens regardless of whether g:go_gopls_fuzzy_matching is 0 or 1.

gopls log

sent: Content-Length: 156
{"method":"textDocument/completion","jsonrpc":"2.0","id":7,"params":{"textDocument":{"uri":"file:///tmp/wat/main.go"},"position":{"character":12,"line":6}}}

sent: Content-Length: 253
{"method":"textDocument/didChange","jsonrpc":"2.0","params":{"contentChanges":[{"text":"package main\n\ntype Stuff struct{}\ntype ThingoStuff struct{}\n\nfunc main() {\n\tvar v Stuff\n}\n"}],"textDocument":{"uri":"file:///tmp/wat/main.go","version":7}}}

received: Content-Length: 530
{"jsonrpc":"2.0","result":{"isIncomplete":true,"items":[{"label":"Stuff","kind":22,"detail":"struct{...}","preselect":true,"sortText":"00000","filterText":"Stuff","insertTextFormat":1,"textEdit":{"range":{"start":{"line":6,"character":7},"end":{"line":6,"character":12}},"newText":"Stuff"}},{"label":"ThingoStuff","kind":22,"detail":"struct{...}","sortText":"00001","filterText":"Stuff","insertTextFormat":1,"textEdit":{"range":{"start":{"line":6,"character":7},"end":{"line":6,"character":12}},"newText":"ThingoStuff"}}]},"id":7}

[bhcleek]

Thank you! That's really helpful. I can duplicate what you're seeing.

Strangely, it only occurs when completeopt includes longest. 🤔 And it doesn't seem to delete the full word in all cases. From what I can tell this is a Vim issue that only manifests itself in some cases, but I'm fully open to being wrong. It's worth noting that vim-go's behavior does not change based on the value of completeopt.

Here's what I know based off of experimenting with the repro path you provided:

1. This only happens when longest is in completeopt
2. I can't seem to get deletion to happen except for on type name completion
3. deletion doesn't happen in all cases.

• trying to complete S doesn't delete.
• trying to complete flo doesn't delete.
• trying to complete f doesn't delete.
• trying to complete fl deletes the l.
• trying to complete ThingoStuff doesn't delete
• I defined two new types, too,type ThingoStuff2 struct{} and type Stack struct{}, and got some more results:
  • trying to complete T deletes
  • trying to complete Th doesn't delete
  • trying to complete Thing doesn't delete

I'm not 100% sure, but I think this might be a Vim issue that occurs in some cases when completeopt uses longest.

[stamblerre]

@muirdm would know best, but I don't believe gopls will ever suggest a prefix deletion. It may only delete a suffix.

[muirdm]

gopls candidates contain text edits to overwrite the prefix with the entire candidate. For example completing "fo" to "food" will give a text edit that inserts "food" over "fo", as opposed to just appending "od" to "fo".

[bhcleek]

That's fine for vim-go's purposes; it only means that vim-go returns that information from the completion function. I suspect that @shabbyrobe is encountering has to do with how Vim deals with longest matches when:

1. there are multiple matches
2. the start of the match is before the the cursor position
3. some of the matches don't have the same prefix as the text being matched

[shabbyrobe]

I pulled my repro process together into a script to try to bisect when this issue appeared in vim, pulled a version way before this started to appear as an issue for me (it definitely did not used to do this) as the start point, and found it happens there too. Shouldn't've bothered with the script 😂

I think that eliminates this as an issue introduced into vim with the longest setting in the period I've been using gopls/vim-go, but it doesn't eliminate vim altogether; the issue could've always been there but just getting tripped by some combo of factors which I'll see if I can narrow down.

I'll try to find some time to do the same for gopls and vim-go versions next. If I can find the earliest version that doesn't reproduce the problem I'll report back. It's also possible that my memory is murky: the Go Modules transition period made a lot of things about my previously-working setup a bit creaky (that have been steadily getting resolved) so for all I remember, it has just always done this since I switched from gocode to gopls. I apologise, this is getting a bit spongy, I'll go and try to get some more hard data.

[bhcleek]

Given the differences in what gocode provided vs what gopls provides, it would not surprise me if the behavior your seeing correlates to gopls and gopls, specifically around #2291. However, what vim-go and gopls are providing is well within the documented requirements for Vim's completion.

I've looked for an issue in github.com/vim/vim/issues about this, but haven't been able to find anything.

[shabbyrobe]

Ah cool, so you reckon I should look at gathering material to submit a bug to Vim instead?

[bhcleek]

Yes. I don't see anything here that makes me think this is anything other than a Vim issue.

ref:

• Omnicomplete deletes text before cursor when completeopt=longest,menuone and shows erroneous matches #2730 (comment)
• Omnicomplete deletes text before cursor when completeopt=longest,menuone and shows erroneous matches #2730 (comment)

[ekotlikoff]

Hey just checking, was a bug raised to Vim for this?