Will it run?
What can your old machine run?
The memory on the GPU, or the unified memory on a Mac, decides how large a model a machine can serve. Pick yours.
Up to about 14B
- Memory for the model
- 12 to 16 GB on the GPU
- Runtime
- llama.cpp, as a native process
- Note
- Rules of thumb for 4-bit GGUF with a moderate context. Context length, quantization and parallel requests move them. The agent reads each GGUF header and checks the real fit before every start.
The whole table
| Machine you may already have | Memory for the model | Comfortable model size |
|---|---|---|
| Gaming PC or gaming laptop with an 8 GB NVIDIA card (RTX 2070, 3060 Ti, 4060) | 8 GB | 7B to 8B |
| Gaming PC with a 12 to 16 GB card (RTX 3060 12 GB, 4070, 4080) | 12 to 16 GB | up to about 14B |
| Gaming PC with a 24 GB card (RTX 3090, 4090) | 24 GB | up to about 32B |
| Apple Silicon MacBook or Mac with 16 GB | shared with macOS | 7B to 8B |
| Apple Silicon Mac with 32 GB or more | shared with macOS | 14B to 32B |
| A DGX Spark or a large GPU workstation | 128 GB unified, or several cards | large models on vLLM |
Why you do not have to do the arithmetic
MirbadLLM gives every model a memory budget. Before a start, the agent compares it with the machine's free memory minus a reserve, per GPU on machines with several cards, and refuses a start that would not fit, with the reason. Other programs on the machine change what is free; the check happens every time.