Running larger LLMs on multiple local computers/servers/appliances via Exo Cluster (directly or via Fabric?)?

[Hedda]

Has anyone tested experimenting with running a large LLM on on Exo Cluster which consists of multiple local computers/servers/appliance or similar computers as Exo cluster nodes?

• https://github.com/exo-explore/exo

Could you add Exo API to this AI Automation Suggester integration?

YouTuber NetworkChuck posted a very entertaining video showcasing an Exo cluster:

https://youtube.com/watch?v=Ju0ndy2kwlw
and Zen van Riel posted a more technical video:

https://youtube.com/watch?v=25QhgZoPXPM
Both show that LAN (Local Area Network) speed between nodes are a bottleneck for speed, but if speed not a showstopper then could be an awesome option.

FYI, NetworkChuck said that Fabric framework will also get Exo support soon:

https://github.com/danielmiessler/fabric

PS: Off-topic but wondering if anyone is working on a "Fabric" AI conversion integration for Home Assistant?

fabric (from danielmiessler) is an open-source framework for augmenting humans using AI. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere.

fabric intro video:

• Network Chuck
• David Bombal
• My Own Intro to the Tool
• More Fabric YouTube Videos