Sizing worksheet · language models on bare metal · since 2004
Filled in from the top, the way the decision goes: the model sets the weights, the precision scales them, the head-room covers the runtime and the conversation, and the total names the card. Nothing else on the order form moves the answer — a card two gigabytes short does not run slower, it fails to load.
Fill in the worksheet Why a fit can still fail What I install on it
Worksheet
70B: 38 GB × 1.5 = 57 GB
Smallest card on the schedule that clears it (3 larger ones also do):
Instinct MI21064 GB HBM2eThe link opens the machine it goes in. What it costs is on /gpu — not on this sheet.
Line 1 × line 2 is the weights; line 3 is the site’s published quarter-to-a-half head-room rule as arithmetic — a quarter again for one user at a time on llama.cpp or Ollama, half again for serving traffic with continuous batching on vLLM or TGI. The verdict names the smallest card in the order catalogue that clears the total, and links to the machine it goes in. What that machine costs is deliberately not on this sheet: every figure is on /gpu, against the exact machine it belongs to.
Fig. 1
Parameters multiplied by bytes per parameter, and that is the whole calculation. Sixteen-bit weights are two bytes each, eight-bit are one, four-bit are a half. Quantising is what turns a machine that cannot hold a model into one that can; it costs quality, and how much depends on the model and the method far more than the bit count alone. If accuracy is the point of the exercise, test both before committing to hardware — the difference is cheaper to discover now than after delivery.
These figures are the weights and nothing else. They are the floor, not the requirement — Fig. 2 is the part that catches people out.
| Model | 16-bit | 8-bit | 4-bit |
|---|---|---|---|
| 7-8 billion parameters | 16 GB | 8 GB | 5 GB |
| 13-14 billion parameters | 28 GB | 14 GB | 8 GB |
| 30-34 billion parameters | 68 GB | 34 GB | 19 GB |
| 70 billion parameters | 140 GB | 70 GB | 38 GB |
Fig. 2 · interactive
The most common way a GPU purchase goes wrong, every time the same way: the model is thirty-eight gigabytes, the card is forty, it loads — and holds roughly one short conversation. The weights are not alone in there. Slide the workload and watch what the conversation does.
One user, short context: the tight multiplier (×1.25) covers this.
Fig. 3
Read down to your model, across to the precision. Each cell names the smallest card on the parts schedule that holds those weights with room left to serve, and links to the machine it goes in. Where nothing clears it alone, the cell says so rather than leaving a blank.
| If you want to run | at 16-bit | at 8-bit | at 4-bit |
|---|---|---|---|
| 7-8 billion parameters | TESLA P40 / QUADRO P600024 GB16 GB of weights | TITAN V12 GB HBM28 GB of weights | TESLA P48 GB GDDR55 GB of weights |
| 13-14 billion parameters | RTX A600048 GB GDDR6 ECC28 GB of weights | TESLA P40 / QUADRO P600024 GB14 GB of weights | TITAN V12 GB HBM28 GB of weights |
| 30-34 billion parameters | More than one cardNothing we fit holds it alone68 GB of weights | Instinct MI21064 GB HBM2e34 GB of weights | Tesla V100 32GB32 GB HBM219 GB of weights |
| 70 billion parameters | More than one cardNothing we fit holds it alone140 GB of weights | More than one cardNothing we fit holds it alone70 GB of weights | Instinct MI21064 GB HBM2e38 GB of weights |
Schedule A
Largest first — this sheet answers “what is the biggest thing I can run”, the reverse of GPU dedicated servers, which sorts the same catalogue by the cheapest complete machine because it is answering “what does this cost”. A card whose memory the manufacturer does not publish is left off rather than guessed at.
Several parts appear on more than one chassis, sometimes at different prices, and the bus generation of the machine underneath changes what a modern card can actually do — both are on GPU dedicated servers. What is standing racked and ready this minute is a different question again, answered by instant servers.
Reference
The open-weight families people bring here, with the published parameter count that says which row of Fig. 1 applies. Parameter counts are the publishers’ own; nothing else about a model matters to the fit question. A model that is not on this list sizes exactly the same way — parameters × bytes per parameter, plus head-room. New families ship monthly; the arithmetic does not move.
Note 1 · mixture-of-experts
A “30B with 3B active” model needs a 30B model’s memory and runs nearer a 3B model’s speed: every expert must be resident, because different tokens wake different ones. Size from the total parameter count, always.
| Family | Sizes | Row |
|---|---|---|
| Llama 3.1 / 3.3 Meta | 8B · 70B | 7–8B, and 70B for the 3.3 release |
| Qwen3 Alibaba | 8B · 14B · 32B | The first three rows |
| DeepSeek-R1 distills | 7B – 70B | One per row — pick the distill your card holds |
| Mistral Small 3 Mistral AI | 24B | Between rows 2 and 3; size it from the arithmetic, not the label |
| Gemma 3 Google | 12B · 27B | Row 2, and just under row 3 |
| Phi-4 Microsoft | 14B | Row 2 |
| gpt-oss OpenAI | 20B · 120B | Published at 4-bit natively: a 16 GB card, and one 80 GB card |
Installation notes
The machine arrives with a clean operating system and root. No vendor inference service in front of your model, no managed runtime that upgrades itself the week before a deadline. vLLM and Ollama both answer on an OpenAI-compatible endpoint — code written against an API provider points at your own machine by changing one base URL. Pin the driver, the CUDA or ROCm release and the framework to the versions your code was tested against; nothing moves underneath you. If you would rather the driver were installed before handover, say so on the order and name the version.
Site conditions
Weights, prompts, retrieved documents and every generated token stay on hardware only you can log into. Amsterdam or Bucharest and the data sits in the EU; New York, Miami or San Francisco and it sits in the US. For workloads answering to a privacy officer, that sentence is the whole business case.
A per-token API bills every token, and an agent or a batch pipeline generates tokens all day. A dedicated machine is the same figure every month at any volume, with unmetered bandwidth under it — weights in and tokens out cost nothing beyond the port. Which figure it is, is /gpu’s line to fill in, not this sheet’s.
The model you pin answers unchanged in six months. No requests-per-minute ceiling, no queue at somebody else’s busy hour, no sunset notice. The trade is honest: upgrades are yours, on your schedule.
Retrieval wants an embedding model, a vector store and fast disk; agents want room to run tools. NVMe for the store, processor cores for the pipeline, the card for the tokens — one machine carries all of it.
Note 2 · ratings
Deliberately. Every such number depends on the model, the quantisation, the runtime and its version, the batch size, the prompt length, the reply length and the concurrency — change any one and it moves by a multiple. A headline number that does not carry all of that is a number chosen to look good, and we would be quoted on it. What governs it: reading the prompt is compute-bound and happens once per request; generating the reply is memory-bandwidth-bound, because each token requires reading the weights; batching amortises that reading across concurrent requests, until the key-value cache runs out of room. Head-room again.
Tell us the model, the quantisation you intend, the context length and roughly how many concurrent requests. If we have run something comparable we will say what we saw and on what. If we have not, we will say that too.
Every hour of every day — telephone, chat and tickets. Support has the response times in writing.
Note 3 · fine-tuning
Serving needs the weights. Training needs the weights, the gradients and the optimiser’s state for every parameter it updates, all resident at once, plus the activations kept for the backward pass — a model that serves comfortably on one card can be several times too large to fully fine-tune on it. Parameter-efficient methods update a small fraction of the parameters and need a correspondingly small fraction of the extra memory, which is why most fine-tuning here uses one. The training loop also streams batches across the processor-to-card link continuously, so the PCIe generation of the machine underneath is a real constraint there in a way it is not for serving — which platform gives a card which link is on GPU dedicated servers. Tell us the model, the method, the sequence length and the dataset size and we will size it rather than have you find out after delivery; more than one card is multi-GPU and multi-node’s sheet.
Adjacent sheets
This sheet owns one question: what it takes to run a model. Everything adjacent has its own page, and this is all of them.
Queries raised
Taken from what people actually ask us in chat and on tickets, in roughly the order they ask it. The first three are the ones that decide whether a purchase works.
For the record
Card memory is the manufacturers’ published figure; which cards exist is read from the order catalogue when this page was served. No monthly price appears here on purpose — GPU dedicated servers holds every one, against the exact machine it belongs to.
