-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Labels
bugSomething isn't workingSomething isn't workingneeds triageCause of bug still unknown, needs investigation.Cause of bug still unknown, needs investigation.
Description
It seems under certain conditions there is a memory leak in the wiregrid encoder agent. We ran into this on satp1 last night:
Right around that time the logs go from filled with "Received IRIG timeout packet" to "while calling from thread":
2024-10-24 15:39:08.363 2024-10-24T15:39:08+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:18.819 2024-10-24T15:39:18+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:28.838 2024-10-24T15:39:28+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:38.862 2024-10-24T15:39:38+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:48.894 2024-10-24T15:39:48+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:58.904 2024-10-24T15:39:58+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:08.922 2024-10-24T15:40:08+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:21.938 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.939 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.940 2024-10-24T15:40:21+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:21.940 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.941 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.942 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.944 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.944 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.946 2024-10-24T15:40:21+0000 while calling from thread
I'm not sure what's generating that "while calling from thread" message. I don't see anything with that string in it in the agent or Encoder beaglebone code, but maybe I'm missing something.
My immediate suspicion is in the nested while True:
loops in grab_and_parse_data()
. That's a good place to start for a deeper investigation. (And this should probably be improved, even if it doesn't end up being the problem.)
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingneeds triageCause of bug still unknown, needs investigation.Cause of bug still unknown, needs investigation.