Add gitea

[gabriel-samfira]

This branch adds initial Gitea support. What works:

• Adding gitea endpoint
• adding gitea credentials
• adding repositories
• adding orgs
• Webhooks properly record jobs
• Runners are managed properly
• Installing webhooks
• PAT auth

What doesn't work:

• Orgs (API endpoints and CLI need to be added, the rest is there)
• automatica webhook install. Gitea requires webhook type. We need to use the official SDK.
• System and user level entities. Gitea doesn't have "enterprises".
• Application auth.

Right now, we use the go-github client to interact with Gitea, but that has limitations. Before merging, we'll have to switch to the official SDK. For the initial support, we might stick with go-github.

TODO: fix existing tests, write new ones

Steps for testing this feature out:

Create a gitea endpoint

Gitea being self hosted, you will need to add a new endpoint, the same way you would do for GHES. This only needs to be done once for a Gitea instance.

For the purpose of this example, we'll assume that the endpoint is reachable at http://10.0.9.5/ (I'm just too lazy to adapt my bash history).

garm-cli gitea endpoint create \
    --api-base-url http://10.0.9.5/ \
    --base-url http://10.0.9.5/ \
    --description "My first Gitea endpoint" \
    --name local-gitea

Add gitea credentials

This branch supports PATs with Gitea. Apps may be added in the future. You will need to generate a PAT with read/write access to repositories and orgs. Then we can add the credentials to GARM:

garm-cli gitea credentials add \
    --endpoint local-gitea \
    --auth-type pat \
    --pat-oauth-token yourSuperSecretPAT \
    --name gsamfira-token \
    --description "Gitea token"

You now have a Gitea endpoint and credentials tied to it.

Create a repository

Repositories and Orgs are common for both GitHub and Gitea. We use the same database model for both with slight changes. Future entities (System and User) will be added as distinct models, available only for gitea, the same way enterprises are exclusive to GitHub.

But I digress. Adding the repo:

garm-cli repo add \
    --credentials gsamfira-token \
    --name testrepo \
    --owner gsamfira \
    --random-webhook-secret \
    --install-webhook \
    --forge-type gitea

Make a note of the repo UUID. You will need it when defining the pool.

Important note: At this stage, GARM can't yet install webhooks in Gitea. It can remove it (which happens by default when removing a repo from GARM), but creating it diverges from how github works. We will use the official SDK before merging this change, so we can expect webhook management to work. But for now, instead of --random-webhook-secret, you will have to explicitly set a --webhook-secret and make sure you use the same secret when creating the webhook in the repo or org.

Installing webhooks now works.

Adding a pool

For testing I used the LXD provider, but you can use any of the existing supported providers, which offer a way to override the runner install template.

For Gitea to work, the provider needs to know how to install the act runner. This is done by an installation script which is generated from a template. We can override the default template in most providers using the runner_install_template field in extra_specs. You can download a sample extra specs from here. Feel free to customize.

Download the extra specs file:

curl -s -L https://gist.githubusercontent.com/gabriel-samfira/d132169ec41d990bbe17e6097af94c4c/raw/67d226d9115eca5e10b69eac5ecb04be91a48991/gitea-extra-specs.json -o ~/garm-extra-specs.json

Create the pool:

garm-cli pool add \
    --repo THE_UUID_OF_THE_REPO_YOU_CREATED_ABOBVE \
    --provider-name arrakis \
    --image ubuntu:24.04 \
    --tags ubuntu-latest \
    --flavor default \
    --extra-specs-file $HOME/garm-extra-specs.json \
    --enabled=true \
    --min-idle-runners=1

You should soon see an idle runner being spun up in the provider and associated to your gitea instance. You should now be able to create a new workflow and target the runner with the ubuntu-latest label. You can add multiple labels, set a different image or flavor.

GARM will automatically spin up runners within the constraints set by the pool (min/max runners).

Fixes: #323

[ChristopherHX]

Right now, we use the go-github client to interact with Gitea, but that has limitations

This client is very very extensible,

for example here I replaced PUT with POST. The gitea/go-sdk currently tests against gitea 1.23.x, so I have delayed my initiative a bit to add the much more missing endpoints that require 1.24 to be able to provide required tests.

func createWorkflow(ctx context.Context, t *testing.T, commitMessage string, workflowContent string, client *github.Client, owner string, repoName string, workflowPath string) *github.RepositoryContentResponse {
	fileOpts := &github.RepositoryContentFileOptions{
		Message: &commitMessage,
		Content: []byte(workflowContent),
		Branch:  github.String("main"),
	}
	req, err := client.NewRequest(http.MethodPost, fmt.Sprintf("repos/%s/%s/contents/%s", owner, repoName, workflowPath), fileOpts)
	assert.NoError(t, err)
	createResponse := new(github.RepositoryContentResponse)
	_, err = client.Do(ctx, req, createResponse)
	assert.NoError(t, err)
	return createResponse
}

Now just derive the struct to add a single new member e.g. for webhook.

Your existing e2e test assumes an existing forge instance e.g. github.

Do you have any preference for an gitea + garm e2e test? I think I can create one.

• use a gitea.com user
  • problem makes it harder to run it locally
  • no control over deployed gitea version
• download and cache native gitea binary and use it for tests?
  • setup gitea via an action (https://github.com/ChristopherHX/gitea-actions-runner/blob/test-ephemeral-runner-0/.github/actions/setup-gitea/action.yml / https://github.com/ChristopherHX/gitea-actions-runner/blob/test-ephemeral-runner-0/.github/workflows/test.yml)
  • setup gitea via go code
• use docker api client and use a dockerd / podman container runtime
• use docker/podman cli from the golang test
• use an kubernetes api client and local port forwarding (<= did this in my keda experiment, example https://github.com/kedacore/keda/pull/6765/files)

[gabriel-samfira]

This client is very very extensible,

I tend to prefer to use official clients if available. It saves us from having to maintain that code as well.

Your existing e2e test assumes an existing forge instance e.g. github.

Yep. It's too early to think about e2e for gitea. But the easiest way would be to just set up gitea on the runner that picks up the job. Gitea even has a snap from what I see so it should be really easy to set up on any VM. Even creating a custom image with gitea pre-installed and just spinning that up via GARM itself would do.

So we will see. First, I need to add orgs and perhaps system as an entity, then unit tests, then integration tests.

[gabriel-samfira]

@ChristopherHX

Interesting. Apparently gitea doesn't properly validate the webhook config. I can create a new webhook via the API, without an actual "target URL". The UI validates correctly. The API does not. Notice the missing URL bellow:

And here you can see I'm editing hook with ID 14 (visible in the URL bar), but the target URL is empty:

Using this body:

createOpts := &createGiteaHookOptions{
		Type:         "gitea",
		Events:       hook.Events,
		Active:       hook.GetActive(),
		BranchFilter: "*",
		Config: map[string]string{
			"content_type": hook.GetConfig().GetContentType(),
			"url":          "",
			"http_method":  "post",
		},
	}

The struct is defined as:

type createGiteaHookOptions struct {
	Type                string            `json:"type"`
	Config              map[string]string `json:"config"`
	Events              []string          `json:"events"`
	BranchFilter        string            `json:"branch_filter"`
	Active              bool              `json:"active"`
	AuthorizationHeader string            `json:"authorization_header"`
}

[ChristopherHX]

I have not yet tried calling this webhook create api (while my integration test on Gitea side might implicitly call it), maybe open a new gitea issue about this.

Could be an older bug.

[gabriel-samfira]

Yup. Will do.

[gabriel-samfira]

added a bug report here:

• Webhooks - API does not validate target URL go-gitea/gitea#34491

[lunny]

• Webhooks - API does not validate target URL Webhooks - API does not validate target URL go-gitea/gitea#34491

go-gitea/gitea#34492

[gabriel-samfira]

Added orgs. I think that's a sufficient target for initial support. Most people will probably need repo and org level runners.

Will focus on tests from now on. While this branch may work, for anyone that wants to test it, expect bugs.

[gabriel-samfira]

Merging this as is. New PRs will be made with integration tests and more unit tests. The pool manager will also need to be migrated to its own worker similar to scalesets and the rest before we can make a new release.