What can your hardware run?

How the estimate works

1. Each size is one real model. We rate each size on a model people actually run, at 4-bit with an 8K-token context: the published file plus the memory that context needs.

   8B

   Llama 3.1 8B Instruct, Q4_K_M, about 6 GB

   32B

   Qwen3 32B, Q4_K_M, about 22 GB

   70B

   Llama 3.3 70B Instruct, Q4_K_M, about 45 GB

   ~120B MoE

   gpt-oss-120b, MXFP4, about 64 GB

   400B+ MoE

   DeepSeek-V3 671B, Q4_K_M, about 405 GB

2. Usable memory. We hold back 1 GB on each graphics card for the driver, and on a unified-memory computer the larger of 8 GB or 15% for the system. On a Mac the largest sizes need macOS's GPU memory limit raised (a one-line setting).
3. Runs, or runs with offload. A model runs when it fits in that memory. A mixture-of-experts model can also run with its experts in the PC's system RAM, since each token reads only a few of them; a card on its own assumes a PC with 64 GB. A dense model never counts as running with offload: it reads every weight for every token.
4. How fast. Generating text is limited by memory bandwidth, so the speed comes from the chip's published bandwidth, calibrated against published llama.cpp measurements. The figure is for llama.cpp's standard benchmark (Llama 2 7B, 4-bit), the file the public scoreboards use (Llama 2 7B, Q4_0, llama-bench tg128). A chip whose family has no matching current measurement gets no speed, only what fits.
5. What it is not. Not a test of any unit. Real speed varies with the software, its settings and the context length; prompt processing, splitting one model across cards for speed, and image generation are not modelled.

The public benchmark threads the speeds are checked against: llama.cpp's CUDA results thread (NVIDIA cards), llama.cpp's Vulkan results thread (AMD and Intel cards), llama.cpp's Apple Silicon results thread. Cards and computers in our Local AI view show this same estimate on their listings.