-
Notifications
You must be signed in to change notification settings - Fork 1.3k
EADDRINUSE during a reload #1596
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
There is a comment in ServerOptions.close_httpservers stating that:
It might be related somehow. |
mnaberez
added a commit
that referenced
this issue
Aug 9, 2023
mnaberez
added a commit
that referenced
this issue
Mar 2, 2024
``` error: uncaptured python exception, closing channel <supervisor.http.deferring_http_channel connected 127.0.0.1:58792 at 0x102d3a310 channel#: 0 requests:4> (<class 'OSError'>:[Errno 57] Socket is not connected [/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py|handle_read|89] [/Users/username/git/supervisor/supervisor/medusa/http_server.py|recv|528] [/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py|recv|354] [/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py|handle_close|156] [/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py|close|361]) error: uncaptured python exception, closing channel <supervisor.http.deferring_http_channel connected 127.0.0.1:58792 at 0x102d3a310 channel#: 0 requests:4> (<class 'OSError'>:[Errno 57] Socket is not connected [/Users/username/git/supervisor/supervisor/supervisord.py|runforever|218] [/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py|handle_read_event|392] [/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py|handle_read|91] [/Users/username/git/supervisor/supervisor/medusa/http_server.py|handle_error|546] [/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py|handle_error|422] [/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py|close|361]) Traceback (most recent call last): File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 347, in recv self.handle_close() File "/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py", line 156, in handle_close self.close() File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 361, in close self.socket.shutdown(socket.SHUT_RDWR) OSError: [Errno 57] Socket is not connected During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py", line 89, in handle_read data = self.recv (self.ac_in_buffer_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/git/supervisor/supervisor/medusa/http_server.py", line 528, in recv result = asynchat.async_chat.recv (self, buffer_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 354, in recv self.handle_close() File "/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py", line 156, in handle_close self.close() File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 361, in close self.socket.shutdown(socket.SHUT_RDWR) OSError: [Errno 57] Socket is not connected During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/username/git/supervisor/supervisor/supervisord.py", line 218, in runforever dispatcher.handle_read_event() File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 392, in handle_read_event self.handle_read() File "/Users/username/git/supervisor/supervisor/medusa/asynchat_25.py", line 91, in handle_read self.handle_error() File "/Users/username/git/supervisor/supervisor/medusa/http_server.py", line 546, in handle_error asynchat.async_chat.handle_error (self) File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 422, in handle_error self.close() File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 361, in close self.socket.shutdown(socket.SHUT_RDWR) OSError: [Errno 57] Socket is not connected During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/homebrew/bin/supervisord", line 33, in <module> sys.exit(load_entry_point('supervisor', 'console_scripts', 'supervisord')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/git/supervisor/supervisor/supervisord.py", line 373, in main go(options) File "/Users/username/git/supervisor/supervisor/supervisord.py", line 383, in go d.main() File "/Users/username/git/supervisor/supervisor/supervisord.py", line 78, in main self.run() File "/Users/username/git/supervisor/supervisor/supervisord.py", line 94, in run self.runforever() File "/Users/username/git/supervisor/supervisor/supervisord.py", line 224, in runforever combined_map[fd].handle_error() File "/Users/username/git/supervisor/supervisor/medusa/http_server.py", line 546, in handle_error asynchat.async_chat.handle_error (self) File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 422, in handle_error self.close() File "/Users/username/git/supervisor/supervisor/medusa/asyncore_25.py", line 361, in close self.socket.shutdown(socket.SHUT_RDWR) OSError: [Errno 57] Socket is not connected ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm using Supervisor 4.2.4 and I have a use case where I can repeat the following failure almost 100% of the time when restarting Supervisor:
I suspect it happens when the HTTP socket is still under use when the
close()
is called on the socket.As long as there is a handle on the socket, the operating system will not actually deallocate the "closed" socket.
Indeed, I am using a lot of
REMOTE_COMMUNICATION_EVENT
that make the HTTP socket a bit busy.When the Supervisor restart is called, the socket is most of the time not deallocated, although it has been closed.
The restart being swift, the socket bind cannot reuse the port (still in
TIME_WAIT
) and fails.The socket has to be shutdown before it's closed, so that the
FIN
event is sent to the peers.Then the socket close triggers the deallocation quite immediately.
Once applied, I never had the issue again and it's good practice anyway.
The text was updated successfully, but these errors were encountered: