Release notes of git-tags dependencies using git@ scheme on gitlab.com and self-hosted Gitlab not retrieved

[pdecat]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Self-hosted Gitlab with Renovate 39.222.3

Please tell us more about your question or problem

Hi,

release notes from dependencies hosted on gitlab.com and self-hosted Gitlab instances are not appearing in merge requests opened by renovate in our self-hosted setup:

When examining the debug logs, we can see that sourceUrl is missing for dependencies not hosted on github.com, and that property is later used by renovate to retrieve the release notes.

Note: I'm opening this discussion as suggested in the PR I've submitted earlier. While it may not be perfect, as it is more of a fallback, these changes do make the changelog retrieval work:

FWIW, we are using these changes ourselves since last week.

Logs (if relevant)

Logs

Logs with unmodified Renovate 39.222.3:

DEBUG: packageFiles with updates (repository=ourgroup/ourprojects/renovatebot/renovate-test-changelog, baseBranch=main)
       "config": {
         "ansible-galaxy": [
           {
             "deps": [
               {
                 "depType": "role",
                 "currentValue": "11.7.2",
                 "datasource": "git-tags",
                 "depName": "base",
                 "packageName": "[email protected]:ourgroup/ourprojects/ansible/base.git",
                 "updates": [
                   {
                     "bucket": "major",
                     "newVersion": "12.3.0",
                     "newValue": "12.3.0",
                     "newMajor": 12,
                     "newMinor": 3,
                     "newPatch": 0,
                     "updateType": "major",
                     "branchName": "renovate/base-12.x"
                   }
                 ],
                 "versioning": "semver-coerced",
                 "warnings": [],
                 "currentVersion": "11.7.2",
                 "isSingleVersion": true,
                 "fixedVersion": "11.7.2"
               },
               {
                 "depType": "role",
                 "currentValue": "0.0.1",
                 "datasource": "git-tags",
                 "depName": "docker_management",
                 "packageName": "[email protected]:nofusscomputing/projects/ansible/docker_management.git",
                 "updates": [
                   {
                     "bucket": "non-major",
                     "newVersion": "0.2.0",
                     "newValue": "0.2.0",
                     "newMajor": 0,
                     "newMinor": 2,
                     "newPatch": 0,
                     "updateType": "minor",
                     "branchName": "renovate/docker_management-0.x"
                   }
                 ],
                 "versioning": "semver-coerced",
                 "warnings": [],
                 "currentVersion": "0.0.1",
                 "isSingleVersion": true,
                 "fixedVersion": "0.0.1"
               },
               {
                 "depType": "role",
                 "currentValue": "3.2.6",
                 "datasource": "git-tags",
                 "depName": "golang",
                 "packageName": "[email protected]:gantsign/ansible-role-golang.git",
                 "updates": [
                   {
                     "bucket": "non-major",
                     "newVersion": "3.4.1",
                     "newValue": "3.4.1",
                     "newMajor": 3,
                     "newMinor": 4,
                     "newPatch": 1,
                     "updateType": "minor",
                     "branchName": "renovate/golang-3.x"
                   }
                 ],
                 "versioning": "semver-coerced",
                 "warnings": [],
                 "sourceUrl": "https://github.com/gantsign/ansible-role-golang",
                 "currentVersion": "3.2.6",
                 "isSingleVersion": true,
                 "fixedVersion": "3.2.6"
               }
             ],
             "packageFile": "roles/requirements.yml"
           }
         ]
       }

[github-actions[bot]]

Hi there,

Please help this Discussion progress by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team

[rarkins]

Please provide an example of gitlab.com release notes not working. Based on a quick glance at the code, I thought they were meant to work.

[pdecat]

In our self-hosted setup, we also have gitlab.com hosted dependencies with the same behavior:

               {
                 "depType": "role",
                 "currentValue": "0.0.1",
                 "datasource": "git-tags",
                 "depName": "docker_management",
                 "packageName": "[email protected]:nofusscomputing/projects/ansible/docker_management.git",
                 "updates": [
                   {
                     "bucket": "non-major",
                     "newVersion": "0.2.0",
                     "newValue": "0.2.0",
                     "newMajor": 0,
                     "newMinor": 2,
                     "newPatch": 0,
                     "updateType": "minor",
                     "branchName": "renovate/docker_management-0.x"
                   }
                 ],
                 "versioning": "semver-coerced",
                 "warnings": [],
                 "currentVersion": "0.0.1",
                 "isSingleVersion": true,
                 "fixedVersion": "0.0.1"
               },

I've tried to setup a reproduction case with a gitlab.com hosted dependency, both on gitlab.com and github.com but:

• for gitlab.com (https://gitlab.com/pdecat/renovate-repro-35123), I did not find a way to run Cloud Renovate against it.
• for github.com (https://github.com/pdecat/renovatebot-27740), Cloud Renovate fails to look it up because it apparently does not trust gitlab.com (see Dependency Dashboard pdecat/renovatebot-27740#2):

Renovate failed to look up the following dependencies: Failed to look up git-tags package [email protected]:nofusscomputing/projects/ansible/docker_management.git.

Logs from https://developer.mend.io/github/pdecat/renovatebot-27740/-/job/0195e475-ab37-7c6b-9e5f-daa15958f334:

DEBUG: Datasource unknown error
{
  "datasource": "git-tags"
  "packageName": "[email protected]:nofusscomputing/projects/ansible/docker_management.git"
  "err": {
    "task": {
      "commands": [
        "ls-remote",
        "[email protected]:nofusscomputing/projects/ansible/docker_management.git"
      ],
      "format": "utf-8",
      "parser": "[function]"
    },
    "message": "Host key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n",
    "stack": "Error: Host key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n\n    at Object.action (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/src/lib/plugins/error-detection.plugin.ts:42:29)\n    at PluginStore.exec (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/src/lib/plugins/plugin-store.ts:54:29)\n    at /usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:124:42\n    at new Promise (<anonymous>)\n    at GitExecutorChain.handleTaskData (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:121:14)\n    at GitExecutorChain.<anonymous> (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:97:40)\n    at Generator.next (<anonymous>)\n    at fulfilled (/usr/local/renovate/node_modules/.pnpm/[email protected]/node_modules/simple-git/dist/cjs/index.js:52:24)\n    at processTicksAndRejections (node:internal/process/task_queues:105:5)"
  }
}

[rarkins]

Can you change the dependency to be https instead of git and retry?

[pdecat]

It does work with https:// instead of git@ because then the sourceUrl is restored here and preserved here.

[pdecat]

The job did work correctly: pdecat/renovatebot-27740#3

DEBUG: packageFiles with updates
{
  "baseBranch": "main"
  "config": {
    "ansible-galaxy": [
      {
        "deps": [
          {
            "depType": "role",
            "currentValue": "0.0.1",
            "datasource": "git-tags",
            "depName": "docker_management",
            "packageName": "https://gitlab.com/nofusscomputing/projects/ansible/docker_management.git",
            "updates": [
              {
                "bucket": "non-major",
                "newVersion": "0.2.0",
                "newValue": "0.2.0",
                "newMajor": 0,
                "newMinor": 2,
                "newPatch": 0,
                "updateType": "minor",
                "branchName": "renovate/docker_management-0.x"
              }
            ],
            "versioning": "semver-coerced",
            "warnings": [],
            "sourceUrl": "https://gitlab.com/nofusscomputing/projects/ansible/docker_management",
            "currentVersion": "0.0.1",
            "isSingleVersion": true,
            "fixedVersion": "0.0.1"
          },
          {
            "depType": "role",
            "currentValue": "3.2.6",
            "datasource": "git-tags",
            "depName": "golang",
            "packageName": "https://github.com/gantsign/ansible-role-golang.git",
            "updates": [
              {
                "bucket": "non-major",
                "newVersion": "3.4.1",
                "newValue": "3.4.1",
                "newMajor": 3,
                "newMinor": 4,
                "newPatch": 1,
                "updateType": "minor",
                "branchName": "renovate/golang-3.x"
              }
            ],
            "versioning": "semver-coerced",
            "warnings": [],
            "sourceUrl": "https://github.com/gantsign/ansible-role-golang",
            "currentVersion": "3.2.6",
            "isSingleVersion": true,
            "fixedVersion": "3.2.6"
          }
        ],
        "packageFile": "roles/requirements.yml"
      }
    ]
  }
}

https://developer.mend.io/github/pdecat/renovatebot-27740/-/job/0195e997-4f1d-7439-8850-413eddf609f5

[rarkins]

For self-hosted GitLab, is this the same server/endpoint as you run Renovate on? e.g. it matches with RENOVATE_ENDPOINT and you're not trying to source changelogs from random/third party public self-hosted gitlab on the internet?

[pdecat]

Thanks, I've added the call to detectPlatform() to only fallback on massaged URL if the platform is gitlab in my branch.
Should I re-open my PR now to continue discussing those changes there, or should I wait for further discussion to happen here?

[rarkins]

I'm happy to review the changes again

[pdecat]

Turns out I cannot re-open the PR myself:

[pdecat]

Should I open a new PR or can you re-open #35110 ?

[pdecat]

Hi, so I went ahead a created a new PR: #35258