Self-hosted OpenSearch for your site search and your logs

What will OpenSearch need from your server?

Mostly memory and quick disks. OpenSearch is the engine behind a search box on your shop or docs site, or the place your servers send their logs so you can dig through them later. Either way it keeps working data in memory and reads index files constantly, so those two parts of the machine decide how it feels.

• Memory. OpenSearch runs on Java, and the Java heap is the memory it manages itself. The Debian and RPM guides tell you to set the starting and maximum heap to half the available memory as a starting point, and say a dedicated host can go higher if your workload needs it. Leave it unset and you get just 1 GB.
• Disks. The docs recommend SSDs installed in the machine and warn against a network file system for the data, because latency and throughput across the network slow the whole cluster.
• Swap. Turn it off. The docs say swapping can badly hurt speed and stability, and the Docker examples lock the heap in memory with bootstrap.memory_lock=true.
• Processor. The Linux packages come for x64 and arm64 with the Java runtime bundled, so there is nothing else to install.

OpenSearch does not publish a RAM-per-gigabyte-of-data figure, and neither do we. Load a slice of your real data, watch heap use in Dashboards, and size from what you see.

The server that fits

Pick by memory, then by disk. Every machine on our dedicated servers page is a whole computer with nobody else on it, so the memory you hand to OpenSearch is really there. The port is unmetered, which means log shippers, bulk imports and snapshot copies never add a bandwidth line to the invoice.

Ready servers are handed over within 30 minutes of payment approval, and you can reinstall the operating system yourself from the control panel. On rack servers and blades you also get a remote console, so a firewall mistake does not lock you out for good. For more disk choices see NVMe servers or storage servers.

Two host settings before anything else

OpenSearch fails its startup checks when the host's vm.max_map_count is below 262144, and the log says the max virtual memory areas value is too low. Containers use the host's value, so set it on the server itself, not in the container. Then create a folder for the project and store the admin password in a .env file only you can read. Since version 2.12 that password is required. It needs 8 to 100 characters mixing capitals, small letters, numbers and symbols; it must also score strong on the zxcvbn checker and not resemble the word admin.

echo 'vm.max_map_count=262144' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
sudo swapoff -a

mkdir -p ~/opensearch && cd ~/opensearch
# read the password without echoing it or leaving it in shell history
read -rs PW && printf "OPENSEARCH_INITIAL_ADMIN_PASSWORD='%s'\n" "$PW" > .env && unset PW
chmod 600 .env

Pick symbols other than $ and quote marks; Compose would try to expand a $ in .env.

swapoff -a lasts until the next reboot; to keep swap off, also comment out the swap line in /etc/fstab.

Up and running with Docker Compose

This is a one-node version of the official sample, with two changes that matter when your machine sits on the open internet: both ports are published on 127.0.0.1 only, and the image versions are pinned so OpenSearch and Dashboards stay on the same release, which Dashboards requires. Set the heap to half of your server's memory; the example assumes 32 GB.

# ~/opensearch/docker-compose.yml
services:
  opensearch:
    image: opensearchproject/opensearch:3.9.0
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms16g -Xmx16g"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
    ulimits:
      memlock: { soft: -1, hard: -1 }
      nofile: { soft: 65536, hard: 65536 }
    volumes:
      - opensearch-data:/usr/share/opensearch/data
    ports:
      - "127.0.0.1:9200:9200"
  dashboards:
    image: opensearchproject/opensearch-dashboards:3.9.0
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch:9200"]'
    ports:
      - "127.0.0.1:5601:5601"
    depends_on:
      - opensearch
volumes:
  opensearch-data:

1. Start it from the project folder with docker compose up -d, then check with docker compose ps.
2. Give it a minute, then test from the server: curl -k -u admin https://127.0.0.1:9200. curl asks for the password, so it never sits on the command line.
3. If a container stops, docker compose logs opensearch shows why; a weak password and a low vm.max_map_count are the usual two.

Keeping strangers out of your data

Never publish port 9200 to the internet. Here is why the Compose file looks the way it does:

• Docker skips your firewall. A mapping written as 9200:9200 listens on every address, and Docker inserts its own firewall rules, so ufw does not block it. Writing 127.0.0.1: in front keeps it local.
• The demo security setup is for testing. Security is switched on out of the box, but a fresh install ships with demo certificates it signed itself and demo users whose passwords are public knowledge. The docs say that configuration is for testing only. Before real customer data or logs go in, replace the certificates with your own, set up proper users and roles, and delete or re-password the demo accounts.
• Dashboards stays private too. Open it through an SSH tunnel: run ssh -L 5601:127.0.0.1:5601 with your user and server address on your own computer, then browse to http://localhost:5601 on your own computer; the login is admin and your password. If your whole team needs it, put it behind a proxy with HTTPS and its own login rather than opening 5601.
• Port 9600 is the Performance Analyzer. You do not need it from outside, so the file above does not publish it at all.

Backups you can actually restore

Use snapshots. The OpenSearch docs say copying the data folder does not give you a consistent backup, and restoring such a copy can fail or quietly lose data; snapshots are the reliable way. They are incremental, so taking them often costs little extra space.

1. Mount storage that lives somewhere else, another server or a network share, on the host, and map it into the container, for example /mnt/snapshots.
2. Tell OpenSearch it may write there with the path.repo setting, then register it as a file system repository through the _snapshot API.
3. Set up Snapshot Management so snapshots are taken on a schedule and old ones are deleted for you.
4. Delete snapshots only through OpenSearch, never by removing files, because each snapshot shares data with the others.

Plan upgrades with this in mind: a snapshot's indexes go back only into the same major release they were made in or the one after it, never an older release.

When you will want more than one server

A single server is a single node, and the docs' production checklist asks you to run multiple nodes, with dedicated cluster-manager nodes among them, so search keeps working when one machine fails. One box is a good fit for site search you can rebuild from your database, logs you can afford to lose a few hours of, or a staging copy. When the search has to stay up through a hardware fault, plan for a small cluster of servers in the same city. There is no private network between our servers, so the nodes talk over their public addresses: keep TLS on, and let port 9300 accept connections only from the other nodes' IPs. If what you really store is rows of business data, a PostgreSQL server is the better home.

Questions people ask

How much memory should I give OpenSearch?

Start the Java heap at half of the server's RAM, with the minimum and maximum set to the same value: that is the starting point in OpenSearch's install guides. On a dedicated machine you can raise it later if your workload needs it. Leave the rest for the operating system and everything else on the box.

Can I install it without Docker?

Yes. OpenSearch publishes Debian and RPM packages and APT and YUM repositories, which install it as a systemd service; on Ubuntu you use the Debian package. A package install binds to localhost by default, and you still set the admin password and vm.max_map_count the same way.

Do I need OpenSearch Dashboards?

Not to run searches: your application talks to the REST API on port 9200. Dashboards is the web interface for exploring logs, building charts and managing security and snapshots, and most people running a log store want it. It must be the same version as OpenSearch.

Why won't OpenSearch start?

Check the container or service log. The common causes are vm.max_map_count below 262144 on the host, an admin password that fails the strength check, and, once you bind to a non-local address, missing discovery settings: for one server set discovery.type to single-node.

Is one server enough?

For site search, staging, or logs with snapshots stored elsewhere, often yes. If search must survive a hardware failure, you need several nodes, which means several servers.