Open WebUI self-hosted: a private ChatGPT-style chat for your team

Two pieces: the chat page and the engine

Open WebUI is only the part people see: logins, the conversation list, file uploads, the model picker. It does not come with any models. The answers come from an engine that holds the model in the graphics card's memory: Ollama is the easy choice, and anything that speaks the OpenAI API works as well.

That split tells you where the money goes. The chat page is light on its own; in the project's own test the slim image sat at about 300 MiB of memory while idle, with nobody signed in. The engine is what needs a real card.

How much graphics card do you need?

Enough card memory to hold the model you want, with room to spare. A model's weights take its parameter count times the bytes each one uses, so at 4-bit:

• a 7-8 billion parameter model needs about 5 GB
• 13-14 billion, about 8 GB
• 30-34 billion, about 19 GB
• 70 billion, about 38 GB

Those are floors. The engine and the conversation history share the card, so plan on roughly a quarter more for one person at a time with Ollama, and half again when many people chat at once through a batching engine like vLLM. A model that only just fits will load and then fail when people use it. Running an LLM walks through the sizing in more detail.

Up and running in an evening

1. Put the card's NVIDIA driver on the server, then Docker, then NVIDIA's Container Toolkit. Follow NVIDIA's guide to register the toolkit with Docker and restart Docker, so containers can see the card.
2. Make a folder such as ~/open-webui and create a private secrets file in it (first two lines below).
3. Save the Compose file below as docker-compose.yml in the same folder.
4. Start it with docker compose up -d and follow docker compose logs -f until it says Application startup complete.

umask 077
printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > .env

# docker-compose.yml
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:ollama
    container_name: open-webui
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - ollama:/root/.ollama
      - open-webui:/app/backend/data
    env_file: .env
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
volumes:
  ollama:
    name: ollama
  open-webui:
    name: open-webui

Why 127.0.0.1 in the ports line: written as plain 3000:8080, Docker opens the page to the whole internet and its rules skip past the ufw firewall. Why the .env file: the secret key signs everyone's logins, and typed on the command line it would sit in your shell history. Keep this file; a new key signs everybody out.

Make the admin account yours before anyone else can

The first person to register becomes the administrator, so do it while the page is still reachable only from the server. On your own computer run ssh -L 8080:127.0.0.1:3000 root@your-server, open http://localhost:8080 in your browser and pick Create Admin Account. Write the password down somewhere safe.

Then download a model: start a new chat, type the model's name into the model picker and confirm. Ollama keeps the files in the ollama volume, so they survive restarts.

Letting your colleagues in

Creating that first account switches sign-ups off, which is the safe default. When you are ready for others, go to your avatar, Settings > Admin > Authentication and turn on New Sign Ups. Each new account waits as Pending until you approve it from the Admin Panel, unless you change Default User Role to user. For a bigger organisation, sign-in through Google, Microsoft, Okta or Keycloak saves you approving people one by one.

Settings you save in the admin screens are stored in the database and beat environment variables from then on. The DEFAULT_USER_ROLE and ENABLE_SIGNUP variables only set the starting values, so change these in the admin screens rather than in the file. To keep conversations from being shared to the Open WebUI community site, switch Community Sharing off in Settings > Admin > General.

Putting it on your own web address

Point a name like chat.yourcompany.com at the server, install Caddy, and give it this short config. Caddy fetches the HTTPS certificate on its own and passes through the live connection Open WebUI uses to stream answers. Let ports 80 and 443 through your firewall; port 3000 stays closed.

chat.yourcompany.com {
    reverse_proxy 127.0.0.1:3000
}

If replies hang or come back empty, add CORS_ALLOW_ORIGIN=https://chat.yourcompany.com to .env, then recreate the container with docker compose up -d; Open WebUI's troubleshooting guide traces most of those WebSocket problems to that setting.

Already serving a model with vLLM or llama.cpp?

Then switch the image to :main (or the much smaller :slim), drop the ollama volume and the deploy block, and add extra_hosts: ["host.docker.internal:host-gateway"] so the container can find the server underneath it. In Settings > Admin > Connections, add an OpenAI connection such as http://host.docker.internal:8000/v1; the /v1 on the end matters. Your engine must listen on more than 127.0.0.1 for the container to reach it. Close its port to the internet, allow it only from the Compose network's subnet (find it with docker network inspect, for example ufw allow from 172.18.0.0/16 to any port 8000), and set an API key on the engine. Our guides to a vLLM server and a llama.cpp server cover the engine side, and Ollama on its own if you would rather keep it separate.

Updates without losing a single chat

Everything people have typed and uploaded lives in the open-webui volume, so an update is a new container on the same data. Copy the volume first: some releases change the database in a way an older version cannot read, and going back to the previous image will not undo that.

docker run --rm -v open-webui:/data -v "$HOME/backups":/backup \
  alpine tar czf /backup/openwebui-$(date +%Y%m%d).tar.gz /data

docker compose pull
docker compose up -d

Skim the release notes before you update, and refresh your browser with the cache cleared afterwards. If you want a version that only changes when you decide, swap the image tag for a pinned release, vX.Y.Z-ollama with a number from the project's releases page, as its docs suggest for production.

The server that fits

On our GPU servers the card is yours alone. It sits in a physical server that only your account rents, handed directly to the system you install, and no sharing scheduler decides when it is your turn. The driver and CUDA version are your pick, and the rest of the machine comes with it: every core, all the memory, every disk, root access, and a remote console through the box's iLO or iDRAC, so you can still see the screen if a driver update knocks the network over. Nothing counts the traffic, so model downloads cost nothing extra.

Timing depends on the card. A machine that already carries it is ready about 30 minutes after your order is approved; fitting a card we hold takes 4-24 hours, and one we have to buy in takes 5-10 working days. Billing is monthly, with no contract.

Questions people ask

Is this the same as using ChatGPT?

It looks similar, but the model is one you choose and run yourself, and every conversation stays on your server. How good the answers are depends on that model and on how much card memory you give it. You can also connect a hosted provider's API key alongside your own models.

Does the chat page itself need the graphics card?

Not for chatting. The card does its work inside the engine. The :cuda image can use it for Open WebUI's own extras, such as turning speech into text and indexing documents, but a plain chat setup does not need that.

How many people can share one server?

It depends on the model size, how long the conversations get and how many people type at the same moment, so no fixed number holds. Leave memory headroom on the card, and if many people chat at once, an engine that batches requests, like vLLM, is built for that.

What happens if I forget the admin password?

You can reset it from the server. Open WebUI's documentation shows how to write a new password hash straight into its database inside the volume with a short-lived helper container. Keep a note of the password so you never need to.

Can I turn logins off for just myself?

Only on a brand-new install: setting WEBUI_AUTH=False before anyone registers gives a single-user mode with no login, and the project warns you cannot switch back to accounts later. On a server reachable from the internet, keep logins on.