Skip to content

Memory leak in wiregrid encoder agent #784

@BrianJKoopman

Description

@BrianJKoopman

It seems under certain conditions there is a memory leak in the wiregrid encoder agent. We ran into this on satp1 last night:
image

Right around that time the logs go from filled with "Received IRIG timeout packet" to "while calling from thread":

2024-10-24 15:39:08.363 2024-10-24T15:39:08+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:18.819 2024-10-24T15:39:18+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:28.838 2024-10-24T15:39:28+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:38.862 2024-10-24T15:39:38+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:48.894 2024-10-24T15:39:48+0000 Recieved IRIG timeout packet.
2024-10-24 15:39:58.904 2024-10-24T15:39:58+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:08.922 2024-10-24T15:40:08+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:21.938 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.939 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.940 2024-10-24T15:40:21+0000 Recieved IRIG timeout packet.
2024-10-24 15:40:21.940 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.941 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.942 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.944 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.944 2024-10-24T15:40:21+0000 while calling from thread
2024-10-24 15:40:21.946 2024-10-24T15:40:21+0000 while calling from thread

I'm not sure what's generating that "while calling from thread" message. I don't see anything with that string in it in the agent or Encoder beaglebone code, but maybe I'm missing something.

My immediate suspicion is in the nested while True: loops in grab_and_parse_data(). That's a good place to start for a deeper investigation. (And this should probably be improved, even if it doesn't end up being the problem.)

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingneeds triageCause of bug still unknown, needs investigation.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions