Workload · Hosting a language model
LLM hosting at Primcast is a whole server with a dedicated GPU, running the model you choose on the runtime you choose, for the same price every month. Pick your model size and precision on the worksheet and it names the smallest GPU that holds it with room to serve.
Get the memory wrong and nothing else matters: a model two gigabytes too big for its GPU either refuses to start or crawls along on the processor. That is why the worksheet comes first.
70B: 38 GB × 1.5 = 57 GB
Fits on one GPU. The smallest that clears it (3 larger also do):
Instinct MI21064 GB HBM2eThe link opens the machine it goes in. Its price is on GPU dedicated servers.
01 GPU memory
Multiply the parameter count by the bytes each parameter takes. At 16-bit that is two bytes, at 8-bit one, at 4-bit half a byte. A 70B model is about 140 GB of weights at 16-bit and about 38 GB at 4-bit.
Quantising to fewer bits is what lets a big model fit a smaller GPU. It costs some quality, and how much depends on the model and the method more than on the bit count. If accuracy matters to you, test both versions before you choose hardware.
Expect the real download to weigh a bit more than this sum, since popular 4-bit formats leave some layers at higher precision. Ollama’s default Llama 3.1 70B download at 4-bit comes to 43 GB rather than 38. Once you have picked the file, size from what it really weighs.
These are the weights alone. Treat them as the floor, never the requirement: the next section is the part that catches people out.
| Model | 16-bit | 8-bit | 4-bit |
|---|---|---|---|
| 7-8 billion parameters | 16 GB | 8 GB | 5 GB |
| 13-14 billion parameters | 28 GB | 14 GB | 8 GB |
| 30-34 billion parameters | 68 GB | 34 GB | 19 GB |
| 70 billion parameters | 140 GB | 70 GB | 38 GB |
The parameter count is the publisher’s own and it is all the fit question needs. A model missing from this list sizes the same way, and new ones arrive every month without changing the arithmetic.
MoEMixture of experts
A model described as “30B with 3B active” needs the memory of a 30B model and runs closer to the speed of a 3B one. Every expert has to stay loaded, because each token can wake a different one. Always size from the total.
| Model | Parameters | Sizing row |
|---|---|---|
| Llama 3.1 / 3.3 Meta | 8B · 70B | 7–8B, and 70B |
| Llama 4 Scout Meta | 109B total · 17B active | Above the 70B row: size from 109B |
| Qwen3 Alibaba | 8B · 14B · 32B | The first three rows |
| Qwen3 30B-A3B Alibaba | 30B total · 3B active | 30–34B, sized from the total |
| DeepSeek-R1 distilled | 7B – 70B | One per row: pick the one your GPU holds |
| Mistral Small 3.2 Mistral AI | 24B | Between rows two and three: do the arithmetic |
| Gemma 3 Google | 12B · 27B | 13–14B, and just under 30–34B |
| Phi-4 Microsoft | 14B | 13–14B |
| gpt-oss OpenAI | 20B · 120B | Shipped at 4-bit: a 16 GB GPU, and one 80 GB GPU |
02 Head-room
Because the weights share GPU memory with the runtime and with the conversation. The usual story: the model is 38 GB, the GPU has 40, it loads, and it manages about one short chat before requests start to queue. Move the slider and watch the conversation grow.
One user, short context: the ×1.25 multiplier covers this.
03 GPU server for LLM
Find your model on the left and your precision across the top. Each cell names the smallest GPU we fit that holds those weights with room to serve, and links to the machine it goes in. Where no single GPU is enough, the cell says so.
| If you want to run | at 16-bit | at 8-bit | at 4-bit |
|---|---|---|---|
| 7-8 billion parameters | TESLA P40 / QUADRO P600024 GB16 GB of weights | TITAN V12 GB HBM28 GB of weights | TESLA P48 GB GDDR55 GB of weights |
| 13-14 billion parameters | RTX A600048 GB GDDR6 ECC28 GB of weights | TESLA P40 / QUADRO P600024 GB14 GB of weights | TITAN V12 GB HBM28 GB of weights |
| 30-34 billion parameters | More than one GPUNothing we fit holds it alone68 GB of weights | Instinct MI21064 GB HBM2e34 GB of weights | Tesla V100 32GB32 GB HBM219 GB of weights |
| 70 billion parameters | More than one GPUNothing we fit holds it alone140 GB of weights | More than one GPUNothing we fit holds it alone70 GB of weights | Instinct MI21064 GB HBM2e38 GB of weights |
04 The GPUs
We fit 56 GPUs with a published memory figure, and the largest holds 96 GB. This list starts at the top because the question here is “what is the biggest model I can run”. GPU dedicated servers lists the same GPUs from the cheapest machine up, with prices. A GPU whose maker does not publish its memory is left off rather than guessed.
Some GPUs come in more than one chassis, and the PCIe generation of the machine underneath changes what a modern GPU can do. Both are on GPU dedicated servers. If you would rather not wait for a build, see the finished machines on instant servers.
05 Self-hosted LLM
You get the server with a fresh operating system and root. There is no inference service of ours sitting in front of your model and nothing upgrades itself the week before your deadline. Keep the GPU driver, CUDA or ROCm and your framework on exactly the versions you tested with. If you want the driver installed before hand-over, say so on the order and name the version.
vLLM and Ollama both answer on an OpenAI-compatible endpoint, so code written for a hosted API points at your own server by changing one base URL.
Continuous batching and paged attention let one GPU answer many requests at once. That is the difference between a demo and a service. It wants a reasonably modern GPU.
Its quantised formats are why a large model fits a small GPU at all, and it can split the work between processor and GPU when the weights do not quite fit.
Production servers in the same family, with different strengths around structured output, prefix reuse and serving several models. They run here unchanged.
The shortest path from an empty machine to a model answering on a port. Ten minutes with it tells you whether your sizing was right.
06 Rent a server for your LLM
A hosted API is easy to start with. These are the four reasons people move their model onto a machine of their own.
Weights, prompts, retrieved documents and every generated token stay on a machine only you can log into. You also choose the jurisdiction: Amsterdam or Bucharest keeps the data in the EU, San Francisco, Miami or New York keeps it in the US. Which GPUs each city carries is on the order page.
An API bills every token, and an agent or a batch job produces tokens all day. Your own server costs the same each month however busy it gets, and nothing meters the port either, so pulling weights in and sending answers out adds nothing to the bill. The monthly figures are on GPU dedicated servers.
The model you pin today answers the same way in six months. No requests-per-minute ceiling, no queue at someone else’s busy hour, no retirement notice. The fair trade is that upgrades are yours to do, when it suits you.
Retrieval needs an embedding model, a vector store and fast disks. Agents need room to run their tools. NVMe for the store, processor cores for the pipeline and the GPU for the tokens: one machine carries all of it.
07 Speed
It depends on too many things for one honest number, so we do not publish a tokens-per-second figure. The model, the quantisation, the runtime and its version, the batch size, the prompt and reply lengths and the number of users at once can each move it several times over.
What drives it: reading your prompt is limited by the GPU’s arithmetic and happens once per request. Writing the reply is limited by memory bandwidth, because every new token reads the weights again. Batching spreads that reading across many requests, until the key-value cache runs out of room. Head-room again.
Tell us the model, the quantisation, the context length and roughly how many requests arrive at once. If we have run something similar, we will say what we saw and on which GPU. If we have not, we will say that too.
Live chat gets you a reply within seconds from an AI that looks into your account and your server, and from a colleague whenever you ask. Response times are on the support page.
08 LLM training servers
Serving needs the weights. Training needs the weights, the gradients and the optimiser state for every parameter it updates, all loaded at once, plus the activations kept for the backward pass. A model that serves comfortably on one GPU can be several times too big to fully fine-tune on it.
Parameter-efficient methods such as LoRA update a small share of the parameters and need a matching small share of the extra memory, which is why most fine-tuning on our machines uses one. Training also streams batches over the link between processor and GPU all the time, so the PCIe generation of the machine matters here in a way it does not for serving. GPU dedicated servers shows which platform gives each GPU which link.
Send us the model, the method, the sequence length and the dataset size and we will size it with you. For more than one GPU, see multi-GPU and cluster servers.
+ From the marketplace
Owners list these GPU machines on our marketplace and set their own monthly prices. One marked as listed through our sister company, Server Room, is rented on serverroom.net.
09 In short
10 Questions
Collected from live chat and tickets. The first four decide whether an order works out.
LLM hosting means running a large language model on a server you control instead of calling someone else’s API. With us, that means you rent an entire physical server with its own dedicated GPU, root access and a fresh operating system, then install the model and runtime you want. It is production-grade in the plain sense: the GPU is yours alone and the versions are yours to choose.
Take the parameter count and multiply it by the bytes per parameter, which is 2 for 16-bit, 1 for 8-bit and 0.5 for 4-bit. A 7–8B model is about 16 GB of weights at 16-bit, 8 GB at 8-bit and 5 GB at 4-bit. A 70B model is about 140 GB, 70 GB and 38 GB. That is only the weights, so plan for 25 to 50 percent more on top for the runtime, the activations and the cache that holds the conversation.
At 4-bit the weights are about 38 GB, so with room to serve you need a GPU with 64 GB or more: one GPU, one machine. At 8-bit the weights are 70 GB, which only our largest single GPUs hold, and only for one user at a time, so plan on more than one GPU if you serve traffic. At 16-bit no single GPU we fit holds it and it becomes a multi-GPU build we quote. The table above names the exact GPUs, and the prices are on GPU dedicated servers.
Because the weights are not alone in GPU memory. The runtime and its buffers sit there too, and every token of a conversation leaves an entry in the key-value cache so it is not recomputed. That cache grows with the context length and again with each request served at the same time, so a model that only just fits holds roughly one short conversation. Size the GPU at one and a quarter to one and a half times the weights.
Any of it. You get root on a clean operating system, with no inference service of ours in the way. vLLM is the usual choice for serving real traffic, llama.cpp for tight memory or older GPUs, and TGI and SGLang run unchanged. Ollama is the quickest way to a model answering on a port. vLLM and Ollama both offer an OpenAI-compatible endpoint, so existing client code only needs a new base URL.
It depends on how much you use it. An API charges per token and costs nothing when idle; your own server costs the same every month and nothing more when it is busy. Steady traffic, agents that run all day, batch jobs and anything with privacy rules tend to come out ahead on their own machine, while a model called ten times a day does not. The monthly figures for that sum are on GPU dedicated servers.
We do not publish a figure, because the model, the quantisation, the runtime version, the batch size, the prompt and reply lengths and the number of users at once can each change it several times over. Reading the prompt is limited by arithmetic and writing the reply by memory bandwidth, and batching spreads the cost across requests until the cache fills. Tell us your model, quantisation and expected users and we will share what we have actually seen.
Then it is a multi-GPU build, which we quote rather than sell from a checkout, because how many GPUs fit depends on their width and the power budget of the chassis. Two GPUs give you the sum of their memory only if the runtime splits the model across both, which every serious runtime can do at some cost in speed. Our multi-GPU page shows what that looks like, and the DGX Spark offers 128 GB of shared memory in one machine.
Yes, and plan for much more memory than serving the same model. Training holds the weights, the gradients and the optimiser state for every parameter it updates, plus the activations for the backward pass, so a model that serves easily on one GPU can be several times too big to fully fine-tune there. Parameter-efficient methods such as LoRA need only a fraction of that extra memory, which is why most fine-tuning on our machines uses one.
Yes. The AMD Instinct GPUs are among the largest-memory GPUs we fit and often the best value per gigabyte, but your software has to support ROCm rather than CUDA. Most current inference runtimes do, and some tools around them still do not. Tell us your runtime before ordering and we will check it for you.
11 Next
Pick the GPU the worksheet named, choose a city and order. No long-term contract, and if your numbers are close, we will size it with you first.