DeepSeek self-hosted: a private reasoning model on a GPU server you control

Which DeepSeek can you actually run?

The one you can run is a distill. DeepSeek took R1's reasoning and taught it to smaller, ordinary models from the Qwen and Llama families, then published those as separate downloads. They think out loud before answering, like R1 does, but they are dense models of a normal size, so one graphics card can hold them.

• Trying it out: the 7B or 8B. Ollama's default deepseek-r1 tag is R1-0528-Qwen3-8B, a 5.2 GB download at 4-bit.
• A daily assistant for a small team: the 14B or 32B, Qwen2.5-based, 9 GB and 20 GB at 4-bit.
• As close to the big model as one box gets: the 70B, based on Llama 3.3 70B Instruct, 43 GB at 4-bit and 75 GB at 8-bit.

A note on licences, because it matters if you build a product on top: DeepSeek's weights are MIT-licensed, but each distill was trained from someone else's model and keeps that model's licence too: Apache 2.0 on the Qwen side, Meta's Llama 3.1 or 3.3 licence on the Llama side. DeepSeek's R1 card spells this out.

How much card memory each download wants

These are the file sizes Ollama's library lists. The file is the floor, not the whole bill: your context, the conversation so far, needs its own room on the card. Our LLM page suggests budgeting an extra 25 percent beyond the file for one user at a time, and 50 percent when serving many users with vLLM; the last column is our arithmetic using that rule.

Smaller numbers mean coarser rounding of the model's weights, and some quality goes with it. If your card can take the 8-bit file of a smaller model or the 4-bit file of a bigger one, try both on your own prompts before you decide. For the same sizing logic across other model families, see running an LLM.

Why the full R1, V3 and V4 will not fit in one box

The headline DeepSeek models are Mixture-of-Experts: lots of specialist sub-networks, only a few of which work on each word. R1 and V3 are 671B parameters in total with 37B active at a time, according to DeepSeek's model cards. As we read it, the serving software still needs every one of those 671B parameters loaded and ready, so the small active share saves arithmetic on each word rather than memory.

That puts the full R1 at 404 GB in Ollama's 4-bit build and 713 GB at 8-bit. The newer V4-Flash is smaller, 284B in total with 13B active, yet even if each of its weights took only half a byte, our sums give 142 GB or more; DeepSeek stores most of the weights outside the experts in FP8, so the actual download is bigger. The biggest single card on our GPU server list holds 96 GB. DeepSeek's own V3 instructions launch the model across two machines with eight GPUs in each.

If you genuinely need the full model on your own hardware, that is several cards in one machine or several machines working together, and it is quoted rather than ordered off the shelf. Start with more than one card or machine.

Up and running in an evening with Ollama

Ollama is the shortest road: one install script, one command to fetch the model, and a local API at port 11434. Its docs say it binds to 127.0.0.1 by default, which is what you want. On a fresh Ubuntu server with the NVIDIA driver installed, logged in as yourself:

1. Close the doors first, so only SSH gets in.
2. Install Ollama with the one-line script from its Linux docs.
3. Pull the model and talk to it once to check the card is doing the work.
4. Raise the context window: Ollama's docs give 4,096 tokens as the default, and a reasoning model can use that up before it reaches its answer. The docs set service variables with systemctl edit.

sudo ufw allow OpenSSH
sudo ufw enable

curl -fsSL https://ollama.com/install.sh | sh
ollama run deepseek-r1:14b

# raise the default context for the service
sudo systemctl edit ollama.service
#   add under [Service]:
#   Environment="OLLAMA_CONTEXT_LENGTH=32768"
sudo systemctl daemon-reload
sudo systemctl restart ollama

Our Ollama server page covers the engine itself in more depth. If you would rather serve the original Hugging Face weights to an application with an API key, vLLM is the better fit, and its reasoning parser hands back the thinking separately from the answer.

Getting good answers out of it

DeepSeek is unusually specific about how R1 models like to be asked, and following it makes a visible difference.

• Keep the temperature between 0.5 and 0.7; DeepSeek recommends 0.6. Higher and the model can loop or ramble.
• With the original distills, put everything in your message and leave the system prompt empty. The 0528 generation, including the 8B that Ollama serves by default, now accepts a system prompt.
• For maths, ask for step-by-step working and a boxed final answer.
• Expect a thinking section before every reply, and give the context window room for it. The original distills sometimes skip it, so DeepSeek advises beginning their replies with the opening think tag and a line break; the 0528 models have dropped that need.

Letting the team use it without opening it to the internet

Leave Ollama on 127.0.0.1 and reach it through SSH from your own laptop: ssh -N -L 11434:127.0.0.1:11434 you@your-server makes the server's API appear on your laptop's port 11434. For colleagues who want a chat window rather than an API, Open WebUI sits in front of Ollama; make its first account, which becomes the admin, through the tunnel while you are the only one who can see it. Colleagues reach it the same way, each through their own SSH tunnel to the Open WebUI port, or through an HTTPS reverse proxy with sign-in that our Open WebUI page walks through. Never publish the port bare.

Two traps. Changing OLLAMA_HOST to 0.0.0.0 puts the API on every address the server has. And if you run anything in Docker, write ports as -p 127.0.0.1:3000:8080: a plain -p 3000:8080 listens publicly on IPv4 and IPv6 and slips past ufw.

The server that fits

Choose the card by the table above, then look for it on the GPU servers page, which filters the whole list by memory. A 24 GB card such as an RTX 3090 or L4 runs the 14B well; a 48 GB L40S or RTX A6000 takes the 32B at 4-bit with plenty of room, or at 8-bit for one user at a time; an 80 GB A100 or H100, or the 96 GB RTX PRO 6000 Blackwell, holds the 70B at 4-bit.

Whichever you choose, the card is yours alone, passed through whole to the system you install, unsliced, with no one else's jobs on it, inside a machine where you have every core, root and an iLO or iDRAC console. Nothing meters the port, so downloading a 75 GB model twice adds nothing to the bill. It is monthly with no contract. Timing depends on the card: about 30 minutes once the order is approved if a machine already carries it, 4 to 24 hours when we fit a card we hold, 5 to 10 working days when one has to be bought in, and 10 to 15 when it is a current-generation accelerator on allocation.

When renting a server is not the right call

If you only ask a few questions a week and nothing you send is sensitive, DeepSeek's own hosted service is simpler than any machine. If you need the full 671B model, plan for a multi-card build. Self-hosting pays off when your prompts should never leave a machine you run, when usage is steady, or when you want to pin one model version and keep it.

Questions people ask

Is the DeepSeek I run myself the same as the one on DeepSeek's website?

Not unless you run the full 671B model. The distills are smaller Qwen and Llama models trained on R1's reasoning, so they inherit its style of thinking but not all of its knowledge. The 32B and 70B distills come closest.

What is the smallest GPU that runs DeepSeek usefully?

The 7B and 8B distills are around 5 GB at 4-bit, so an 8 GB card loads them. For answers you would trust day to day, the 14B on a 16 or 24 GB card is a better starting point.

Do my prompts go back to DeepSeek when I host it myself?

Ollama and vLLM run the model entirely on your server; the weights are downloaded once and nothing needs to call DeepSeek afterwards. Keep the API on 127.0.0.1 and your conversations stay on that machine.

Can I add a second card later for a bigger model?

More than one card in one machine is available and is quoted rather than picked at checkout. Ask for a quote with the model and precision you are aiming for.

Can I sell a product built on a DeepSeek distill?

DeepSeek's R1 card says the R1 series supports commercial use under MIT. Each distill also keeps the licence of the model it was trained from, so read that one as well before you launch.