[Bazel CI] Bazel build failed with "no such package '@@io_bazel_rules_go//go/toolchain': The repository '@@io_bazel_rules_go' could not be resolved"

[sgowroji]

CI: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3975#019095c0-e9a7-4ee3-8676-383d4ffee65f

Platform: Windows

Logs:

bazel --output_user_root=C:/b --nosystem_rc --nohome_rc version
WARNING: Option 'experimental_enable_bzlmod' is deprecated: Use --enable_bzlmod instead
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Jan 01 00:00:00 1970 (0)
Build timestamp: Thu Jan 01 00:00:00 1970 (0)
Build timestamp as int: 0
bazel --output_user_root=C:/b --nosystem_rc --nohome_rc info
WARNING: Option 'experimental_enable_bzlmod' is deprecated: Use --enable_bzlmod instead
ERROR: com.google.devtools.build.lib.packages.BuildFileNotFoundException: no such package '@@io_bazel_rules_go//go/toolchain': The repository '@@io_bazel_rules_go' could not be resolved: Repository '@@io_bazel_rules_go' is not defined
Traceback (most recent call last):
  File "c:\b\bk-windows-dmg8\bazel\bazel-at-head-plus-downstream\bazelci.py", line 4520, in <module>
    sys.exit(main())
  File "c:\b\bk-windows-dmg8\bazel\bazel-at-head-plus-downstream\bazelci.py", line 4488, in main
    execute_commands(
  File "c:\b\bk-windows-dmg8\bazel\bazel-at-head-plus-downstream\bazelci.py", line 1354, in execute_commands
    bazel_version = print_bazel_version_info(bazel_binary, platform)
  File "c:\b\bk-windows-dmg8\bazel\bazel-at-head-plus-downstream\bazelci.py", line 1651, in print_bazel_version_info
    execute_command(
  File "c:\b\bk-windows-dmg8\bazel\bazel-at-head-plus-downstream\bazelci.py", line 2757, in execute_command
    return subprocess.run(
  File "C:\python3\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bazel', '--output_user_root=C:/b', '--nosystem_rc', '--nohome_rc', 'info']' returned non-zero exit status 2.
🚨 Error: The command exited with status 1

CC Greenteam @meteorcloudy

[sgowroji]

CC @tyler-french

[fmeum]

@jin Friendly ping due to bazelbuild/continuous-integration#1986

[fmeum]

@sgowroji I don't think that this is a rules_go issue, it looks more likely to be a regression in Bazel.

[jin]

@fmeum sorry, I missed your comment. I'll need to get access to a Windows VM to repro this, but I can't repro this on macOS and Linux with HEAD Bazel, so it's probably a Windows specific issue.