Structured logging mode

[raymyers]

• This change is worth documenting at https://docs.all-hands.dev/
• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description of the problem this fixes or functionality that this introduces.

Add the option of structured logging in JSON for better observability. You can turn this on by setting the LOG_JSON environment variable.

---

Give a summary of what the PR does, explaining any non-trivial design decisions.

Add an option to use LOG_JSON env var to switch the main logger to output JSON lines instead of plaintext. This will allow observability pipelines to cleanly read long errors as one event and extract extra={...} fields in a structured way.

This is fairly vendor agnostic. Adding the LOG_JSON_LEVEL_KEY env var to rename level should be enough to work in AWS, Google Cloud, Splunk and Elastic Stack.

This uses the python library json-logger to allow existing log statements to be unchanged.

Example

LOG_JSON=1 LOG_JSON_LEVEL_KEY=severity make start-backend 

{"message": "Building image: ghcr.io/all-hands-ai/runtime:oh_v0.27.0_cm02idw7zgobbjks_onvc5j4pkl9nt07u", "severity": "INFO", "timestamp": "2025-03-01T02:04:45.521173+00:00"}

Alternatives

this doc shows how to do this in the plain Python SDK, but it requires changing all the log statements to something non-standard - not desirable.

Another multi-vendor o11y option would be OpenTelemetry, which might be a nice option at some point but seems like more of a commitment, whereas making existing logs structured is high-value for low-investment. 

Recently we added the MonitoringListener class, it's possible these structured logs could remove the need for that more complicated solution.

---

Link of any specific issues this addresses.