Skip to content

"supervisorctl start xxx"command stuck #942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Flyalay opened this issue Jun 4, 2017 · 4 comments
Closed

"supervisorctl start xxx"command stuck #942

Flyalay opened this issue Jun 4, 2017 · 4 comments

Comments

@Flyalay
Copy link

Flyalay commented Jun 4, 2017

Hello everybody
I had a problem with supversior 3.2.0 on ubuntu:
When running "supervisorctl start xxx"command will be stuck.
The strace supervisorctl and supervisord processes found that supervisorctl stuck in recvform calls
Analysis supervisord log, it is because there is no send a message, so supervisorctl has been blocked.
Correct behavior seems to be supervisord will reply message "HTTP / 1.1 200 OK", supervisorctl exit after receiving.
I don't know if anyone have this problem, thank you very much for any help.

Good log (excerpt) :
Supervisord:
Recvfrom (5, "POST/RPC2 HTTP / 1.1 \ r \ nHost: local"..., 4096, 0, NULL) = 362
Sendto (5, "HTTP / 1.1 200 OK \ r \ nDate: Sat, 03 J"..., 251, 0, NULL, 0) = 251
Recvfrom (5, ", 4096, 0, NULL, NULL) = 0
Close (5)

Supervisorctl:
The socket (PF_LOCAL, SOCK_STREAM, 0) = 3
Connect (3, {sa_family = AF_LOCAL, sun_path = "/ var/run/supervisor.") = 0
Sendto (3, "POST/RPC2 HTTP / 1.1 \ r \ nHost: local"..., 362, 0, NULL, 0) = 362
Recvfrom (3, "H", 1, 0, NULL) = 1
Recvfrom (3, "T", 1, 0, NULL) = 1
Recvfrom (3, "T", 1, 0, NULL) = 1
Recvfrom (3, P, 1, 0, NULL, NULL) = 1
...

Stuck log (excerpt) :
Supervisord:
Recvfrom (5, "POST/RPC2 HTTP / 1.1 \ r \ nHost: local"..., 4096, 0, NULL) = 362
Recvfrom (5, ", 4096, 0, NULL, NULL) = 0
Close (5) = 0

Supervisorctl:
The socket (PF_LOCAL, SOCK_STREAM, 0) = 3
Connect (3, {sa_family = AF_LOCAL, sun_path = "/ var/run/supervisor.") = 0
Sendto (3, "POST/RPC2 HTTP / 1.1 \ r \ nHost: local"..., 362, 0, NULL, 0) = 362
Recvfrom (3,

@mnaberez
Copy link
Member

mnaberez commented Jun 4, 2017

When you start a process with supervisorctl start foo, the time it takes for the command to finish depends on the value of startsecs in the section [program:foo]. supervisord will block until the process stays up for as long as startsecs. If startsecs is set too high, supervisord can block for a long time and supervisorctl will appear stuck.

@Flyalay
Copy link
Author

Flyalay commented Jun 5, 2017

[program: foo] the startsecs value in the configuration is used by default (1s).

Starting foo, supervisord log has a tip:
foo entered RUNNING state, and process has stayed up for > than 1 seconds (startsecs)

When supervisorctl start foo stuck, press ctrl + c, then supervisorctl status foo show the foo is RUNNING status
It doesn't seem to matter to startsecs value.

Not clear supervisord reply "200 OK" in normal, when stuck just close the socket without sending anything.

@Flyalay
Copy link
Author

Flyalay commented Jun 8, 2017

I found a way to reproduce the problem.

For example, the host current time is "2017/6/8 17:00"

When the supervisorctl start foo is executing, the host time is changed to "2017/6/8 16:00"

The supervisorctl start foo command is stuck at this time.

@mnaberez
Copy link
Member

Closing this issue in favor of #1043 which has an explanation of the cause and a pull request to fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants