Model fit

Which GPU runs which model

How much video memory a model needs, and the cheapest chip that holds it at today's prices.

10models

32,768Context

Weights plus attention cache, with 15% added for activations and allocator overhead. Everything here is arithmetic — we do not measure speed, so we do not rank chips by it. Memory figures come from each provider's own listing. We report what they publish and do not correct it.

Llama 3.1 8B

8 B params

16-bit

22GB

RTX 2080 Ti$0.07/h

8-bit

11GB

RTX 3060 laptop$0.03/h

4-bit

5.4GB

RTX 3060 laptop$0.03/h

Llama 3.1 70B

70 B params

16-bit

161GB

MI300X$2.39/h

8-bit

81GB

GB10$0.24/h

4-bit

40GB

GB10$0.24/h

Llama 3.1 405B

405 B params

16-bit

886GB

GB10 × 8$1.92/h

8-bit

443GB

GB10 × 4$0.96/h

4-bit

221GB

MI350X$5.49/h

Qwen 2.5 7B

7.6 B params

16-bit

18GB

RTX 2080 Ti$0.07/h

8-bit

9.1GB

RTX 3060 laptop$0.03/h

4-bit

4.6GB

RTX 3060 laptop$0.03/h

Qwen 2.5 32B

32.5 B params

16-bit

79GB

GB10$0.24/h

8-bit

39GB

GB10$0.24/h

4-bit

20GB

RTX 2080 Ti$0.07/h

Qwen 2.5 72B

72.7 B params

16-bit

167GB

MI300X$2.39/h

8-bit

84GB

GB10$0.24/h

4-bit

42GB

GB10$0.24/h

Mistral 7B

7.2 B params

16-bit

20GB

RTX 2080 Ti$0.07/h

8-bit

10GB

RTX 3060 laptop$0.03/h

4-bit

5.0GB

RTX 3060 laptop$0.03/h

Gemma 2 9B

9.2 B params

16-bit

32GB

GB10$0.24/h

8-bit

16GB

Tesla P100$0.04/h

4-bit

7.9GB

RTX 3060 laptop$0.03/h

Gemma 2 27B

27.2 B params

16-bit

71GB

GB10$0.24/h

8-bit

36GB

GB10$0.24/h

4-bit

18GB

RTX 2080 Ti$0.07/h

Mixtral 8x7B

46.7 B paramsMoE

16-bit

105GB

GB10$0.24/h

8-bit

52GB

GB10$0.24/h

4-bit

26GB

GB10$0.24/h

Mixture of experts: only part of the weights runs per token, but all of them must sit in memory.

Quantizing halves the weights at each step. It also changes output quality, which this page does not measure.

See all GPU prices