Memory leak when using Pure SSH for LFS

[Danstiv]

Description

Hello!
I tried to upload 1GB file through pure SSH connection, during the upload gitea occupied more and more memory.
After 920 megabytes of uploaded data, memory on the server was exhausted, and the upload was interrupted.
I also did not notice that during this process something was written to the disk.
Thus, probably gitea downloads the entire file to RAM if pure ssh is used.
When uploading over https, there is no such problem.

Gitea Version

1.23.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.47.1

Operating System

Ubuntu 24.04

How are you running Gitea?

docker

Database

PostgreSQL

[Danstiv]

@ConcurrentCrab It seems you recently implemented this feature, good work!
But perhaps there is a certain flaw, due to which the leak occurs, can you pay attention to this?
Thanks.

[Danstiv]

Can you reproduce the bug on the Gitea demo site?

As far as I know, LFS is turned off there, and in any case it will not be a very good idea to upload such a large file there.

[wxiaoguang]

Could you provide a diagnosis report from admin panel -> monitor -> trace? "Download diagnosis report", when the memory leak occurs.

---

If it isn't a real memory leak (the memory will be recycled eventually after the requests) but just "writes the entire file to RAM" and causes out-of-memory, then it could be another problem. Maybe it could be reproduced by the SSH LFS test code.

[Danstiv]

No problem!
Should i wipe some private data there?

[Danstiv]

If it isn't a real memory leak (the memory will be recycled eventually after the requests

Sorry, it seems I used the wrong terminology.
The memory is really released after the file upload is completed, probably this cannot be considered a memory leak.

[wxiaoguang]

The generated "heap" file doesn't contain sensitive data. It looks like this:

---

And thank you for the update, I think we do not need the diagnosis report now ...... just need to reproduce it by the SSH LFS tests.

[ConcurrentCrab]

Hi @Danstiv,

yes, the LFS SSH proxy reads the entire file into memory before uploads/downloads. More specifically, at these points:

GiteaBackend.Download():

gitea/modules/lfstransfer/backend/backend.go

Line 199 in 4af6417

respBytes, err := io.ReadAll(resp.Body)

GiteaBackend.Upload():

gitea/modules/lfstransfer/backend/backend.go

Line 237 in 4af6417

reqBytes, err := io.ReadAll(r)

(There are other calls to io.ReadAll(), but they're for comparatively tiny JSON files, and necessary because json.Unmarshal() only accepts bytes.)

Those calls to io.ReadAll() can be removed and just the reader can be passed around to potentially reduce memory usage, but you will need to verify that somewhere down the stack those readers are closed.

[wxiaoguang]

We need to read/write the stream on the fly, but not fill into memory.

[wxiaoguang]

-> Fix SSH LFS memory usage #33455