11Labs websocket connection

[EdgarMMProsper]

Pipecat version: 0.0.60

I know there has been a lot of working around websocket connections. I am sharing this in case someone else has seen this and we can deepdive together. At certain points my elevenlabs connection falls and is unable to reconnect:

2025-03-27 14:09:50.774 | ERROR | pipecat.services.websocket_service:_receive_task_handler:77 - CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent
2025-03-27 14:09:50.774 | WARNING | pipecat.services.websocket_service:_receive_task_handler:85 - CustomElevenLabsTTSService#0 connection error, will retry: no close frame received or sent
2025-03-27 14:09:50.775 | WARNING | pipecat.pipeline.task:_process_up_queue:479 - Something went wrong: ErrorFrame#0(error: CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent, fatal: False)
2025-03-27 14:09:50.776 | DEBUG | pipecat.services.elevenlabs:_disconnect_websocket:374 - Disconnecting from ElevenLabs
2025-03-27 14:09:50.776 | ERROR | pipecat.services.elevenlabs:_disconnect_websocket:381 - CustomElevenLabsTTSService#0 error closing websocket: no close frame received or sent
2025-03-27 14:09:50.776 | WARNING | pipecat.services.websocket_service:_reconnect_websocket:51 - CustomElevenLabsTTSService#0 reconnecting (attempt: 1)
2025-03-27 14:09:50.777 | ERROR | pipecat.services.websocket_service:_verify_connection:39 - CustomElevenLabsTTSService#0 connection verification failed: no close frame received or sent
2025-03-27 14:09:50.813 | WARNING | pipecat.services.elevenlabs:_keepalive_task_handler:409 - CustomElevenLabsTTSService#0 keepalive error: no close frame received or sent
2025-03-27 14:09:54.778 | ERROR | pipecat.services.websocket_service:_receive_task_handler:77 - CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent
2025-03-27 14:09:54.778 | WARNING | pipecat.services.websocket_service:_receive_task_handler:85 - CustomElevenLabsTTSService#0 connection error, will retry: no close frame received or sent
2025-03-27 14:09:54.779 | DEBUG | pipecat.services.elevenlabs:_disconnect_websocket:374 - Disconnecting from ElevenLabs
2025-03-27 14:09:54.779 | WARNING | pipecat.pipeline.task:_process_up_queue:479 - Something went wrong: ErrorFrame#1(error: CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent, fatal: False)
2025-03-27 14:09:54.779 | WARNING | pipecat.services.websocket_service:_reconnect_websocket:51 - CustomElevenLabsTTSService#0 reconnecting (attempt: 2)
2025-03-27 14:09:54.781 | ERROR | pipecat.services.elevenlabs:_disconnect_websocket:381 - CustomElevenLabsTTSService#0 error closing websocket: no close frame received or sent
2025-03-27 14:09:54.781 | ERROR | pipecat.services.websocket_service:_verify_connection:39 - CustomElevenLabsTTSService#0 connection verification failed: no close frame received or sent
2025-03-27 14:09:58.782 | ERROR | pipecat.pipeline.task:_process_up_queue:473 - A fatal error occurred: ErrorFrame#2(error: CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent, fatal: True)
2025-03-27 14:09:58.782 | ERROR | pipecat.services.websocket_service:_receive_task_handler:77 - CustomElevenLabsTTSService#0 error receiving messages: no close frame received or sent
2025-03-27 14:09:58.784 | DEBUG | pipecat.services.elevenlabs:_disconnect_websocket:374 - Disconnecting from ElevenLabs
2025-03-27 14:09:58.784 | ERROR | pipecat.services.elevenlabs:_disconnect_websocket:381 - CustomElevenLabsTTSService#0 error closing websocket: no close frame received or sent

If I understand it right no close frame received or sent means that we can not connect succesfully from our side the 11labs server. I might try to make MAX_RETRIES to a higher number but the wait time is already 4 seconds so I guess that it is odd that the connection is off for that long time. Anyone has seen this / any thoughts?

[EdgarMMProsper]

Okay I think I found something. I think that ElevenLabsTTSService._disconnect_websocket should be

  async def _disconnect_websocket(self):
       try:
           await self.stop_all_metrics()

           if self._websocket:
               logger.debug("Disconnecting from ElevenLabs")
               await self._websocket.send(json.dumps({"text": ""}))
               await self._websocket.close()
               self._websocket = None

           self._started = False
       except Exception as e:
           logger.error(f"{self} error closing websocket: {e}")
           # CUSTOM CODE: if the websocket can not disconnect, force it to be None
           self._websocket = None

This is because when we get an error so bad that we can not even disconnect, we need to force the disconnection

[aweitz]

We just started seeing this behavior with the ElevenLabsTTSService, but only for one voice out of 5 tested. Pipecat version 0.0.61.

Edit: The above suggestion seems to work (avoids the fatal error).

[YongeBai]

I've ran into this issue before. I believe it's due to using a voice Id that is not added to your 11labs library