Questions people ask first
Is it free?
Yes. MirbadLLM is open source under the MIT licence. There is no paid tier and no hosted version.
What hardware does it support?
A PC with an NVIDIA GPU on Windows 11 (llama.cpp) or Linux (vLLM in Docker), and an Apple Silicon Mac (llama.cpp on Metal, or MLX). AMD GPUs and machines without a GPU are not supported yet.
The master needs a machine that stays on. Any Linux PC with an NVIDIA GPU works, from an old gaming tower to a DGX Spark.
Can several people use it at the same time?
Yes, and that is what it is built for. Give each person and each tool its own API key. Models are spread across your machines, and models on vLLM serve many requests at once, so parallel coding agents, a small team or a family do not wait behind each other.
Which apps and tools work with it?
Anything that speaks the OpenAI API: coding agents such as Cline, Roo Code and Continue, chat apps, and the OpenAI SDKs. The Connect page gives ready-to-paste settings, including a VS Code configuration export.
Where do my prompts go?
From your apps to the master and on to your own machines. MirbadLLM is self-hosted; it is not a cloud service. Models are downloaded from Hugging Face when you add them.
Can I reach it from the internet?
It is built for your local network and listens on plain HTTP, port 8080, by default. To reach it from outside, put a TLS reverse proxy in front and require an API key on every request.
What does "memory checked before a start" mean?
Every model has a memory budget. Before a start, the agent compares it with the machine's free memory minus a reserve, per GPU on machines with several cards. If it does not fit, the start is refused and you are told why, instead of the machine running out of memory halfway through.
What happens after a restart?
Models that were running come back on their own, one at a time per machine. A model you stopped stays stopped.
How are the node agents updated?
The master pushes signed agent updates to every machine. Agents on Windows, Linux and macOS update in place.
What does a backup hold?
Model definitions, running models, node settings, managed host services, history, API keys and saved tokens. Keys and tokens are encrypted with your passphrase. Backups can run on a schedule to a local folder or an SMB or NFS share.
Why the name?
It comes from the Arabic word مِرْبَد (mirbad), the enclosure where camels are gathered and kept. MirbadLLM does the same for models scattered across your machines.