LLM VRAM Calculator: GPU Memory and Self-Hosting Cost

Use this free LLM VRAM calculator to estimate how much GPU memory a model needs, see how many GPUs it takes, and compare the monthly cost of self-hosting with an API. It gives you a planning estimate to use before you buy or rent hardware. Your numbers stay in your browser.

How the VRAM estimate works

Running an LLM needs memory for two main things: the model weights and the context cache for each request. The calculator adds them and allows 10% overhead:

weights in GB = billions of parameters x bits per parameter / 8

A 70 billion parameter model at Q4, about 4.85 bits per parameter, needs about 42 GB for its weights. The same model at FP16 needs 140 GB, which is why precision is usually the biggest lever. Lower precision saves memory but can reduce output quality, so test it on your own tasks.

The context cache grows with context length and with the number of requests running at once. Its size depends on the model’s architecture, and the calculator uses a rough estimate based on Llama-style models. Real use also varies with your serving software, so treat the total as a starting point and confirm it with a load test.

When self-hosting beats an API

A rented or owned GPU costs money whether it is busy or idle, while an API charges only for tokens used. Self-hosting therefore tends to make sense when usage is high and steady, when an open model is good enough for the task, or when data control requires it. Low or uneven traffic usually favours an API. The comparison in the tool shows the volume you would need to break even at your own GPU quote.

The calculator compares cost only. Check three more things before you decide: whether the GPUs can serve your peak traffic at an acceptable speed, whether the open model passes your evaluation, and who will run the system day to day.

Read more on API models versus self-hosting, how to host an AI model locally and when private LLMs justify the cost. To price the API side in detail, use the LLM cost calculator. To test the business case, use the AI ROI calculator.

Frequently asked questions

How much VRAM do I need to run an LLM?

As a starting point, multiply the model's parameters in billions by its bits per parameter and divide by 8 to get the weights in GB, then allow extra for the context cache and overhead. A 70 billion parameter model at Q4 needs about 42 GB for its weights alone.

Does quantization reduce VRAM?

Yes. Fewer bits per parameter means smaller weights, so Q4 needs far less memory than FP16. It can also lower output quality, so test the quantized model on your own tasks.

Can I split a model across several GPUs?

Often, yes, if your serving software supports it. Splitting can reduce speed compared with a single GPU that has enough memory, so test throughput as well as memory.

Is self-hosting an LLM cheaper than using an API?

It depends on your volume, your GPU cost and how busy the GPUs stay. Enter your own quote and token volume in the calculator to see the break-even point for your case.

Last updated: 05 November 2026