Workload · Running a shared JupyterHub

A JupyterHub Server for Your Class or Team, With a GPU in It

For a class or a small team on one machine, install The Littlest JupyterHub (TLJH) on an Ubuntu or Debian LTS server with a GPU: one command gives you a login page, and every person gets a notebook server and a Unix account of their own. Reach for Zero to JupyterHub on Kubernetes only when you need containers, several machines, or a hard rule about who holds the card.

The short version
  1. TLJH: one machine, one install command, no containers
  2. Size memory for the busiest hour, not the whole roster
  3. TLJH limits memory and CPU per user, never the GPU
  4. Whole server, card passed straight to your own OS
  5. Monthly term, no contract, unmetered port

Which JupyterHub do you need?

If everything fits on one server and nobody needs containers, you want TLJH; the project itself draws the line at those two questions. Zero to JupyterHub (Z2JH) is the Kubernetes distribution: each user session runs in a container, and the cluster can grow or shrink across many machines.

The Littlest JupyterHubZero to JupyterHub
Runs onOne Debian or Ubuntu LTS server, amd64 or arm64A Kubernetes cluster
Users areUnix accounts prefixed jupyter-Containers (pods)
Per-user limitsMemory and CPUMemory, CPU, and GPUs as a resource
Setup effortOne bootstrap commandA cluster, Helm and a config file
Good forA class, a lab, a small teamLarge or growing groups, mixed hardware

TLJH depends on systemd, so it will not install inside Docker, and its documentation warns against putting it on a laptop or personal computer. A rented server with root is the setting it was written for.

How big should the machine be?

Size for the number of people working at the same moment, which TLJH's own guide puts at roughly 40 to 60 percent of a class. Measure one person's typical notebook session first, take its peak and add 20 to 40 percent; that is your memory per user.

ResourceThe TLJH ruleCounts
Memoryconcurrent users × memory per user (peak + 20-40%) + 128 MBConcurrent users
CPUconcurrent users × CPU per user + 20%Concurrent users
Diskall users × disk per user + 2 GBEveryone on the roster
Card memoryNot covered by TLJH; add up what notebooks load at onceWhoever is on the GPU

Memory is the hard limit. A user who passes the allowance has their kernel restarted, while a CPU shortfall only makes things slower. Install needs at least 1 GB, but a class needs the sum above. For card memory, our guide to running language models shows how model size turns into gigabytes; the same arithmetic applies to a notebook that loads one.

Will every user get the GPU?

On TLJH, nothing divides the card: its documented per-user limits are limits.memory and limits.cpu, and there is no GPU setting at all. Whoever's notebook uses the card shares all of its memory with everyone else on it, so one large model loaded in one notebook can leave the rest with an out-of-memory error. For a class doing short exercises that is usually fine; for a team training overnight, agree who runs what, or move to Z2JH.

On Z2JH, a profile can ask Kubernetes for a whole GPU per user, once NVIDIA's device plugin is deployed on the node:

singleuser:
  profileList:
    - display_name: "GPU Server"
      description: "Spawns a notebook server with access to a GPU"
      kubespawner_override:
        extra_resource_limits:
          nvidia.com/gpu: "1"

With one card in the machine, that profile serves one user at a time. NVIDIA's plugin can advertise one card several times (time-slicing), but its README says those replicas share the card's memory and fault domain: if one workload crashes, they all do.

Why a whole machine suits a shared hub

A hub is many people leaning on one box at the same hour, so the box has to be all yours. On our servers the card sits in a physical machine rented to one account and is handed to your operating system directly: your driver, your CUDA or ROCm, no hypervisor, no MIG partition, no time-slicing scheduler you did not choose. Every core, all the memory and the disks come with it, plus root.

  • Nothing counts the bytes. Datasets in and checkpoints out cross an unmetered port.
  • A screen when things break. The server's HPE iLO or Dell iDRAC gives you a browser console, so a driver that takes the network down is still fixable.
  • No contract. One month is the default term, which fits a semester or a project.

The server that fits

Every card we fit is listed on our GPU dedicated servers page, sorted into three shelves. Pick by what the class or team does, not by the newest name:

  • Consumer cards (GeForce): no ECC, but the parts most consumer software is written against. Right for teaching and coursework, where a wrong bit means rerunning a cell.
  • Professional cards: certified drivers and ECC on most of the line. A fit for a lab that also renders or encodes.
  • Accelerators: large memory, ECC throughout. For a research team fine-tuning models, where a run has to finish.

The standard host builds on that page carry 16 or 32 GB of system memory, and memory can be added later to the machine you already have, with no migration and no new address. If your hub needs no card at all, any of our dedicated servers will run TLJH. Delivery depends on the card: about 30 minutes once approved when a machine already carries it, 4 to 24 hours when we fit one we hold, 5 to 10 working days when it has to be bought in, and 10 to 15 when it is a current-generation accelerator on allocation.

Up and running in an evening

  1. Choose a current Ubuntu or Debian LTS image; TLJH supports no other distribution. Point a DNS name at the server's address now, because HTTPS needs it.
  2. On Ubuntu, install the NVIDIA driver with sudo ubuntu-drivers install --gpgpu, then the utilities package Ubuntu's guide pairs with it, sudo apt install nvidia-utils-BRANCH-server with BRANCH replaced by the driver branch number that ubuntu-drivers list --gpgpu shows, reboot, and confirm nvidia-smi lists the card. On Debian, use Debian's own NVIDIA driver packages instead.
  3. Open the firewall with SSH allowed first, so enabling it cannot cut your session: sudo ufw allow 22 && sudo ufw allow 80 && sudo ufw allow 443 && sudo ufw enable. ufw allows IPv6 by default, so the same rules cover both.
  4. Install the prerequisites and TLJH, putting your admin name in place of the placeholder (code below).
  5. Turn on HTTPS before anyone logs in (second block below).
  6. Straight away, log in at your hub's address as the admin name you chose. The password you type now becomes that account's password, and until you do it the name is unclaimed, so do steps 4 to 6 in one sitting.
  7. Set the per-user ceilings, for example sudo tljh-config set limits.memory 4G and sudo tljh-config set limits.cpu 2, then sudo tljh-config reload.
  8. From a Jupyter terminal as admin, install shared libraries for everyone with sudo -E pip install or sudo -E conda install; they go into the shared environment in /opt/tljh/user. Take your framework's CUDA build from its own install instructions.
sudo apt install python3 python3-dev git curl
curl -L https://tljh.jupyter.org/bootstrap.py | sudo -E python3 - --admin <admin-user-name>
sudo tljh-config set https.enabled true && sudo tljh-config set https.letsencrypt.email you@example.com && sudo tljh-config add-item https.letsencrypt.domains yourhub.yourdomain.edu && sudo tljh-config reload proxy

TLJH's default login, FirstUseAuthenticator, stores whatever password a user types on their first attempt. Accounts must be created by an admin first; create_users true lets anyone with the address register, so leave it off for a public hub and add people from the admin panel.

Keeping it healthy all term

  • Backups are yours. We do not sell a backup product. Before an upgrade TLJH suggests a disk snapshot or a copy of /opt/tljh; the user database is in /opt/tljh/state. Copy home directories too, since that is where the work lives.
  • Upgrades. Re-run the bootstrap with --version=latest from an SSH session, never from a notebook terminal.
  • Idle servers. The culler stops a notebook server after an hour idle (services.cull.timeout, in seconds), which frees memory between lessons.
  • Admins are root. Anyone in jupyterhub-admins gets passwordless sudo on the whole machine. Keep that list short.
  • Removing people. Deleting a user from JupyterHub leaves their Unix account behind, so remove it on the server as well.

When this is the wrong place for it

If you need hard GPU isolation between many users, TLJH cannot give it, and one card can only if it supports NVIDIA's MIG partitioning, which you would set up yourself; otherwise plan a Kubernetes cluster with several GPU nodes. You can run Kubernetes yourself on bare metal, or have us run the control plane in New York with managed Kubernetes, labelling GPU nodes with kubectl to keep notebooks apart from other jobs. We sell no private network between servers, so nodes talk over their public addresses. And a hub for three people who never use a GPU is better on a small machine without a card.

Questions people ask

Can students create their own accounts?
Only if you allow it. With TLJH's default login, an admin creates each account and the user sets a password on first login. Setting auth.FirstUseAuthenticator.create_users to true lets anyone who finds the address in, which is rarely what a class wants.
How many people can one hub serve?
It depends on what each person's notebook uses, which is why TLJH asks you to measure one typical session. Take that session's peak memory, add 20 to 40 percent, multiply by the number working at the same moment and add 128 MB. If the result is more than the server holds, add memory or set a lower limits.memory.
Can two people use the GPU at the same time?
Yes on TLJH, but they share its memory with no limit between them, so one heavy notebook can starve the other. Z2JH can give each user a whole card through the device plugin. Splitting one card by time-slicing shares memory and crashes too.
What happens when a notebook sits idle?
After an hour without activity TLJH's culler stops that user's notebook server. The account stays, and notebooks saved to disk stay in their home folder. Variables held only in memory are gone, so ask users to save before a break.
Can I rent the server for one course and stop?
Yes. Billing is monthly with no contract, so you keep it for the term and cancel. A server already racked and ready can also be ordered by the hour exactly as it stands, which suits a short workshop.

Get a GPU server for your class's hub

Pick the card that matches the coursework, install TLJH with one command, and hand out logins the same week.

Choose your GPU server

Read next