Error in logs duplicate key value violates unique constraint "argo_workflows_pkey"
#14344
Open
3 of 4 tasks
Assignees
Labels
Comments
|
Thanks for the report. Some of the lines in those logs appear to be truncated, which makes it hard to tell what's happening here. Can you provide the untruncated logs? Specifically, I'd like to see the rest of this line: |
MasonM
added
the
problem/more information needed
|
Oh, right, good point. Here is a full log (the inline json is removed to just |
|
These typically occur very close together as well
|
MasonM
removed
the
problem/more information needed
MasonM
added a commit
to MasonM/argo-workflows
that referenced
this issue
Apr 4, 2025
When offloading a workflow, it's possible for multiple workers to concurrently call `Save()` with the same workflow, which leads to `ERROR: duplicate key value violates unique constraint "argo_workflows_pkey"`` messages in the logs, along with a stack trace. These messages are cluttering the logs and confusing users, but duplicate key errors are harmless. When it detects a duplicate key error, `Save()` will return the `version` hash, and `version` is part of the primary key of the `argo_workflows` table, which means it's guaranteed to be identical to the previously-inserted row. This decreases the log level of that message to `DEBUG` so that it doesn't clutter the logs. I thought about removing it entirely, but I figured it's worth keeping just in case. I wasn't able to reproduce the error locally, but I verified workflow offloading works locally using `make PROFILE=postgres UI=true ALWAYS_OFFLOAD_NODE_STATUS=true` Signed-off-by: Mason Malone <[email protected]>
|
Thanks for the details @hanneskaeufler! I think these messages are harmless: the code already detects and handles duplicate key errors properly. But the messages are misleading (since they appear to be a real problem), and the fact they're cluttering the logs is definitely a problem. I entered a PR to reduce the log level to "debug" so they won't show up by default: #14357 |
Thanks! Downgrading the log works for me if we know this is harmless 💯 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Pre-requisites
:latestimage tag (i.e.quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on:latest. If not, I have explained why, in detail, in my description below.What happened? What did you expect to happen?
No DB errors
Version(s)
v3.6.4
Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.
UnclearLogs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: