Error when accessing a commit with lots of file changes

[Lifeismana]

Description

When accessing a commit with around 6000 files changes in it, it results in a 404 error after around 10min and any subsequent commit can't be accessed due to the same error but commit before that big one are fine
Repo is around 15GB (14GB of pngs) so i didn't try on the gitea demo website because i assume i won't be able to put it there but i can provide the repo if that could help to repro the problem

looking at the resource usage, when the request is running, VM has around 95% memory usage but cpu wise it seems to be only using 1 core ( git seems to be using around 12GB of ram)
(i could try to throw a bit more ram at it but i'm kinda running out of free ram)

Gitea Version

1.21.0+dev-364-gebb98d724

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/Lifeismana/39b4063594375c2d3e0e78a2a1109838#file-error-log

Screenshots

No response

Git Version

2.40.1

Operating System

Debian

How are you running Gitea?

Docker using the official image with a postgresql db

Database

PostgreSQL

[cassiozareck]

Let me try to help you! I think your issue lives here:

The reason is because gitdiff.go (where the error is occurring first) uses Git Default timeout that is set to around 6 minutes (360 seconds). If you're building gitea yourself try to increase this timeout. About the resource usage it seems Git is who determine resource usage and parallelization while loading commits. But Im not so sure so I will search more deeply if there may be another reason for your issue.

Please comeback again to say if it worked or not.

[lunny]

Those are default timeout. You can change them via app.ini

;[git.timeout]
;DEFAULT = 360
;MIGRATE = 600
;MIRROR = 300
;CLONE = 300
;PULL = 300
;GC = 60

[Lifeismana]

Thank you both for your suggestion, totally missed that setting, after bumping it to 40min, it finally loads after 23mins

i guess there's no easy way to improve that git diff speed other than giving it more ram, more processing power or changing the diff algorithm ?

[yp05327]

So it is a timeout request, we need to return 408 (Request Timeout) instead of 404.