Need advice/to contribute support for google artifact registry

[cognitiaclaeves]

The company where I work uses_ bazel (just upgraded to 8, I think) for a very significant project, which they use for a framework for Google Cloud.

I need to be able to use Google Artifact Registry to resolve java dependencies in Bazel. I'm 90% certain that this capability currently doesn't exist.

Though I'm fine with adding it, java is not a language that I'm very familiar with, so I'd like some guidance on how I'd implement something that can take a "artifactregistry://" url and access it using google's oauth token. Since we're not yet using bzlmod, I think this will need to be configured through maven_install(), which is how we're setting up our other repos.

I appreciate any pointers.

[cognitiaclaeves]

Update: I discovered that the project is already in progress for switching to bzlmod. So I'm looking to contribute a way to use Google Artifact Registry to resolve dependencies with Bazel, the same way that Maven can. And looking for pointers on where I can do that. I have this PR as to use as a guide as to what will need to be changed to support GAR: #779 ; I just need some pointers on where to make the kinds of changes that are in that PR for a module that resolves maven dependencies.

[shs96c]

As long as the GAR can be used by maven, you just need to add the URL to the repositories attribute of your maven_install or install tag

[shs96c]

Looking at the source of the gradle plugin for the Google Artifact Registry, it's enough to replace artifactregistry: with https: and to ensure your authentication credentials are in your .netrc

[cognitiaclaeves]

It's taken me a little time to wrap my head around this. The primary issue is probably that the corporate environment that I'm in does not allow service accounts. So, no username + password authentication. I need access to the http_file object to pass the auth_patterns argument to allow for the Google auth token.

[shs96c]

I don't know enough about the Google Artifact Registry and how it handles authentication. Are there any docs that succinctly describe what you need? And how do other build systems cope without the username and password?

[cognitiaclaeves]

One detail that I have omitted explicitly is that the GAR repo that I'm trying to access is a Maven repo. Google has an authentication helper for maven that makes the process pretty seamless. (with this, it's not necessary to use a service acct).

... however, such an authentication helper does not exist for bazel. ( This is not to be confused with the bazel credential helper/spec that seems to be starting to gain traction or with this generic credential helper -- the credential helper by itself does not change the call to the artifact registry; I think, rules_jvm_external would need to implement something to get the credentials from the credential helper; that might be what this ticket is for. )

I don't really know how the maven authentication helper (wagon) works, but I tracked it down to here. . ... going through that code was nontrivial for me. During breaks from the task, I was also trying to find a curl example. It was really hard to find/figure out. But this works:

Given:

group = com.mycompany.localc
artifact-id = artifact-id
version = ve.rsi.on

file = file-name.jar

curl -v -L -o testcurl.jar -H "Authorization: Bearer $(gcloud auth print-access-token)" 'https://us-maven.pkg.dev/$gcloud-project/$gcloud-repo/com/mycompany/localc/artifact-id/ve.rsi.on/file-name.jar'

key points here are that the group is com/mycompany/localc instead of com.mycompany.localc, whereas the version and the filename seem to be fine with "." separators

( this can also be seen with gcloud artifacts files list ... ) -- but is not really shown in Google Artifact Registry's UI.

It is also possible to d/l artifacts through https://artifactregistry.googleapis.com, with something like this:

curl -v -L -o testcurl.jar -H "Authorization: Bearer $(gcloud auth print-access-token)" 'https://artifactregistry.googleapis.com/download/v1/projects/$gcloud-project/locations/us/repositories/$gcloud-repo/files/com%2Fmycompany%2Flocalc%2Fartifact-id%2Fve.rsi.on%2Ffile-name.jar?alt=media

In addition to the group using slashes instead of dots, What's weird about that one, aside from piping artifacts through something that looks like googleapis.com, is that the FILE_STRING is URL-encoded.

I don't think that maven wagon uses the artifactregistry.googleapis.com URL, as "artifactregistry://us-maven.pkg.dev://" URLs are what are returned by google to add to the POM.xml, but I'm including the info anyway. ( The protocol "artifactregistry" appears to be parsed as "https" )

.. That's what I was able to determine so far.

[cognitiaclaeves]

By the way, I think I've traced where a change will need to be made to rules_jvm_external:
https://github.com/bazel-contrib/rules_jvm_external/blob/master/private/extensions/download_pinned_deps.bzl#L37

It looks to me like rules_jvm_external ultimately invokes http_file(). I think I need to be able to pass an auth_patterns argument here. I currently have a local copy of rules_jvm_external where I'm doing this, but I'm getting a 401, and I'm having trouble figuring out why. It's really hard to debug. I'd like to be able to write print statements in http_file, maybe through invoking an http_file that is pulled from a local git repo copy of bazel.

I think if I had access to that, I could write some print statements that I could use to determine why I'm getting 401s. But there doesn't seem to be a way to debug at that level. I can't really even get the project to sync with bazel on the account of some android dependency (we aren't developing android).

.. but I think I'm still missing something. It is breaking when I try to bazel run @gar_maven//:pin ... I'm wondering if there is another place where http_file is being invoked when pinning new versions.

echo -e "machine us-maven.pkg.dev\n\tpassword `gcloud auth print-access-token`" > ~/.netrc
bazel run @gar_maven//:pin