Upload fails on Termux (Linux arm64) while working from Windows - unexpected ping API response

[helios210]

Platform:

Termux (Android)

Linux ARM64 binary

immich-go versions tested (same error on each):

v0.25.0

v0.25.2

v0.25.3

Command issued:

immich-go upload from-folder --server=<myserver> --api-key=<mysecretkey> --log-level=DEBUG --folder-as-album=FOLDER "/sdcard/DCIM/Albums"

Behavior:

Works fine over HTTPS from Windows (x86-64 binary).

Fails from Termux Linux (arm64 binary) over HTTPS.

Local HTTP upload (http://192.168.0.x) works fine through Termux.

Remote HTTPS upload (https://mydomain) fails immediately with error:

"unexpected response to the immich's ping API at this address: https://mydomain/api/server/ping"

immich-go outputs the CLI command list after error (as if parsing error happened).

Other Tests Run:

curl to /api/server/ping works fine in Termux.

HTTPS CA certs are valid/trusted (curl shows "pong" as expected).

--skip-verify-ssl flag does not solve the issue.

Fresh binaries directly from GitHub releases tested.

Notes:

Could this be an issue in immich-go ARM64 binary’s internal HTTP client behavior on Termux?

All Termux packages and CA certs are fully up-to-date

This used to work just fine, I thought it was a server-side issue, but given this works fine from Windows I now think it's client-side

API key and server address redacted for privacy, but these are entered correctly, http://my-server:2283 works, while https://immich.my-domain doesn't work in Termux, but works fine in browsers. Additionally https://immich.my-domain works with curl, just not immich-go.

Why don't I just use the app?

The app doesn't allow me the same flexibility as immich-go does, with immich-go I can seamlessly tie together my Tasker workflows with custom upload commands, timings and checks, that is why I would like to use immich-go instead of the app.

Logs

I have checked the log after the failure, it simply re-states my command, and has the single error as written above, no additional detail is given, even with --log-level=DEBUG added into the command

[helios210]

Okay, so an interesting development here is:

• the windows binary uses http/1.1 regardless of whether connecting over http or https and works fine.
• when forced to use http1.1 using curl --http1.1 -v /api/server/ping on termux I get a "pong" response AND http 200 OK
• when using curl -v /api/server/ping on termux it uses http/2 by default and I get "pong" and http/2 200 response but no "OK" as this isn't part of http/2
• using curl -v /api/server/ping on termux uses http/1.1 and I get "pong" and http 200 OK response
• attempting to force http1.1 through SWAG doesn't seem to work for Termux

Is there a way to force http1.1 in immich-go as a test? This would 100% confirm that this is the source of the issue...

[simulot]

I don't think this is linked to the version of the HTTP protocol.
I'd rather think this is related to your network configuration.

You can turn on the --api-trace flag and check the trace log with all the dialogs between immich-go and the server.

[helios210]

Hi, thanks for the reply. you're probably right, I'm grasping a little as I seem to have tried everything and wondered if this might be some strange behaviour with immich-go expecting to parse an "OK" response with http/1.1 and then not receiving that with http/2.

I tried what you suggested, unfortunately it doesn't add any flavour to the output, here is the full log output with that option enabled:

2025-04-28 13:40:04 INF immich-go version:0.25.3, commit:20fb6882da9e6650b5385a76449254a8dcc3b382, date:2025-04-12T06:49:21Z
2025-04-28 13:40:04 INF Running environment: architecture=arm64 os=linux
2025-04-28 13:40:04 INF Command: immich-go upload from-folder
2025-04-28 13:40:04 INF Flags:
2025-04-28 13:40:04 INF --album-path-joiner= /
2025-04-28 13:40:04 INF --album-picasa=false
2025-04-28 13:40:04 INF --api-key=**************************************i8rY
2025-04-28 13:40:04 INF --api-trace=true
2025-04-28 13:40:04 INF --ban-file='@eaDir/', '@_thumb/', 'SYNOFILE_THUMB.', 'Lightroom Catalog/', 'thumbnails/', '.DS_Store/', '/._*', '.photostructure/'
2025-04-28 13:40:04 INF --client-timeout=5m0s
2025-04-28 13:40:04 INF --date-from-name=true
2025-04-28 13:40:04 INF --date-range=unset
2025-04-28 13:40:04 INF --device-uuid=localhost
2025-04-28 13:40:04 INF --dry-run=false
2025-04-28 13:40:04 INF --exclude-extensions=
2025-04-28 13:40:04 INF --folder-as-album=FOLDER
2025-04-28 13:40:04 INF --folder-as-tags=false
2025-04-28 13:40:04 INF --help=false
2025-04-28 13:40:04 INF --ignore-sidecar-files=false
2025-04-28 13:40:04 INF --include-extensions=
2025-04-28 13:40:04 INF --include-type=
2025-04-28 13:40:04 INF --into-album=
2025-04-28 13:40:04 INF --log-file=/data/data/com.termux/files/home/.cache/immich-go/immich-go_2025-04-28_13-40-04.log
2025-04-28 13:40:04 INF --log-level=DEBUG
2025-04-28 13:40:04 INF --log-type=text
2025-04-28 13:40:04 INF --manage-burst=NoStack
2025-04-28 13:40:04 INF --manage-epson-fastfoto=false
2025-04-28 13:40:04 INF --manage-heic-jpeg=NoStack
2025-04-28 13:40:04 INF --manage-raw-jpeg=NoStack
2025-04-28 13:40:04 INF --no-ui=false
2025-04-28 13:40:04 INF --on-server-errors=stop
2025-04-28 13:40:04 INF --recursive=true
2025-04-28 13:40:04 INF --server=https://immich.myserver/
2025-04-28 13:40:04 INF --session-tag=false
2025-04-28 13:40:04 INF --skip-verify-ssl=false
2025-04-28 13:40:04 INF --tag=[]
2025-04-28 13:40:04 INF --time-zone=
2025-04-28 13:40:04 INF Arguments:
2025-04-28 13:40:04 INF "/sdcard/DCIM/Albums/"
2025-04-28 13:40:04 INF Connection to the server https://immich.myserver
2025-04-28 13:40:04 ERR unexpected response to the immich's ping API at this address: https://immich.myserver/api/server/ping:

What I'm noticing now is that trailing ":" which is odd - if I curl to that address I get an error.

However I don't see anything wrong with the command I'm running, only with the address that it is trying to ping...

[simulot]

Could share the file /data/data/com.termux/files/home/.cache/immich-go/immich-go_2025-04-28_13-40-04.trace.log?

[helios210]

Hi again, it's quite short, here it is:

2025-04-28T13:40:04Z QUERY 1 PingServer GET https://immich.myserver/api/server/ping
Accept [application/json]
X-Api-Key redacted
-- binary body not dumped --

Looks like it is pinging the correct server, I just don't understand why only immich-go seems to be struggling.

• Browser access works local and via domain (on desktop + Android)
• immich-go works locally (over http)
• The official immich app works via the domain

I can successfully hit the URL above with curl and get a nice response, but immich-go just borks, but only on Termux. If I run the exact same immich-go command as in my original post, remotely, using https, on Windows, it works just fine...

[simulot]

Is your server reachable from internet? I could try to debug the connection. Share the address on a private message on discord @simulot

[helios210]

Yes it's fully reachable from the internet, Discord says you're only accepting messages from friends so I've sent you a request, should be from Helios210

[simulot]

I'll try to reproduce your particular setup

[simulot]

I have improved the error management, and here is the problem:

error while calling the immich's ping API at this address: https://immich.me/api/server/ping:
PingServer, GET, https://immich.me/api/server/ping
Get "https://immich.me/api/server/ping": tls: failed to verify certificate: x509: certificate signed by unknown authority

Possible actions:

• update termux's root certificates.
• add --skip-verify-ssl option to bypass the SSL verification.
• use the new android's terminal enabled with devlopper's optiopn

The error handling correction will be delivered with the next release of immich-go.

[helios210]

Thanks for this, did the --skip-verify-ssl work for you in testing? It still returns the same error for me.

I've done additional testing and found a few interesting things, though I'm not well versed in SSL/TLS so forgive any naivety please!

curl -Iv from Windows succeeds and returns three messages (among others) that caught my eye:

1. schannel: disabled automatic use of client certificate
2. schannel: remote party requests renegotiation
3. schannel: renegotiating SSL/TLS connection

However curl -Iv from Termux uses TLS 1.3 which, according to the log, forbids re-negotiation.

Therefore I think you're mostly right, that this is a network issue, however I'm still a bit stumped as to why immich-go is ignoring my use of the --skip-verify-ssl flag which would at least give me a temporary solution while I try to find something more permanent.

[helios210]

Okay, another update, the issue seems to be that CloudFlare is dropping the connection because Termux forces TLS1.3, if I do:

curl --tls-max 1.2 --tlsv1.2 -Iv mydomain < this works

curl -Iv mydomain < this doesn't

Can we force TLS1.2 in a test build of immich-go to see if this resolves the issue completely? If we can confirm that is the issue then I can focus on why CloudFlare is dropping Termux

[helios210]

Last update for today:

✅ What I did:

• I copied the working certificate bundle from a Windows system (where curl and immich-go succeed) to Termux.
• I then configured curl on Termux to explicitly use that certificate with --cacert.
• Despite using the exact same trusted root certs, the TLS handshake still fails on Termux, while it succeeds on Windows.

🚫 What this rules out:

• The issue would appear not to be due to outdated or missing root certificates.
• The server's certificate is valid and properly trusted.

✅ What I think this shows:

• The failure occurs during the TLS handshake phase, before certificate validation happens.
• The problem lies in the TLS ClientHello fingerprint generated by Termux's curl (using OpenSSL 3.x) or Go's crypto/tls stack.
• Cloudflare is rejecting the TLS 1.3 handshake due to this fingerprint — this is apparently a known behavior when dealing with minimal or “non-browser” TLS 1.3 clients.

If we can force TLS1.2 in Immich-go as a test, I think this will confirm the hypothesis.

[simulot]

Thank you for the investigation.

The immich-go's option --skip-verify-ssl enables the connection. The option is transmitted to go round tripper configuration roundTripper.TLSClientConfig.InsecureSkipVerify = false

The GO documentation says:

field InsecureSkipVerify bool
InsecureSkipVerify controls whether a client verifies the server's certificate chain and host name. If InsecureSkipVerify is true, crypto/tls accepts any certificate presented by the server and any host name in that certificate. In this mode, TLS is susceptible to machine-in-the-middle attacks unless custom verification is used. This should be used only for testing or in combination with VerifyConnection or VerifyPeerCertificate.

The same immich-go binary runs on the terminal app provided by google. Same phone, same server using the same Cloudflare service, and probably the same TLS version. My guess is that Termux environment is causing the error.

[simulot]

Thanks for this, did the --skip-verify-ssl work for you in testing? It still returns the same error for me.

Use the binary sent on your discord