Installing Elasticsearch and Kibana using Docker Compose

TLDR

• When deploying Elasticsearch and Kibana, ensure the data directory permissions are set to UID 1000:1000, otherwise the container will fail to start.
• It is recommended not to mount the logs directory as a Volume to avoid startup failures caused by permission conflicts during upgrades or migrations.
• Use discovery.type=single-node to quickly set up a development environment.
• Use a setup service container to automate the password configuration for the kibana_system user.
• If memory is insufficient, adjust MEM_LIMIT and JVM Heap settings in the .env file.

Environment Setup and Permission Management

When deploying an Elasticsearch container, the most common issue is insufficient data directory permissions, which causes the container to fail to start because it cannot write data.

Creating Data Directories

When does this issue occur: When mounting a Volume to a container on a Linux host.

Execute the following commands to create the directory and assign the correct permissions (1000 is the default user ID inside the Elasticsearch container):

# Create data directory
mkdir -p volumes/elasticsearch/data

# Set permissions
sudo chown -R 1000:1000 volumes

TIP

• It is recommended not to mount the logs directory, as permission issues with log files can easily cause startup failures when upgrading or migrating Elasticsearch.
• Kibana does not store critical data itself, so there is no need to create a data Volume for it.

Configuration File Examples

Environment Variable Settings (.env)

Centralize settings using a .env file for easier maintenance.

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=YourPassword

# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=Wing1205

# Version of Elastic products
STACK_VERSION=9.1.4

# Set the cluster name
CLUSTER_NAME=docker-cluster

# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic

# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200

# Port to expose Kibana to the host
KIBANA_PORT=5601

# Increase or decrease based on the available host memory (in bytes)
# 1GB = 1073741824 bytes
MEM_LIMIT=2147483648

Docker Compose Configuration

Create a docker-compose.yml file to integrate Elasticsearch, Kibana, and the automated setup service.

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    container_name: elasticsearch
    restart: always
    environment:
      - node.name=elasticsearch
      - cluster.name=${CLUSTER_NAME}
      - discovery.type=single-node
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=false
      - xpack.security.transport.ssl.enabled=false
      - xpack.license.self_generated.type=${LICENSE}
      - ES_JAVA_OPTS=-Xms512m -Xmx512m -Xlog:gc:stdout:time,level,tags
    volumes:
      - ./volumes/elasticsearch/data:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test: ["CMD-SHELL", "curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health | grep -q 'yellow\\|green'"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 60s

  kibana:
    depends_on:
      elasticsearch:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    container_name: kibana
    restart: always
    environment:
      - SERVER_NAME=kibana
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
    ports:
      - ${KIBANA_PORT}:5601
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test: ["CMD-SHELL", "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 60s

  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    container_name: elasticsearch-setup
    user: "0"
    command: >
      bash -c '
        echo "Waiting for Elasticsearch to start...";
        until curl -s -u elastic:${ELASTIC_PASSWORD} http://elasticsearch:9200/_cluster/health?pretty | grep -q "yellow\|green"; do
          echo "Elasticsearch is not ready yet, waiting...";
          sleep 10;
        done;
        echo "Setting kibana_system user password...";
        curl -s -X POST -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" \
          http://elasticsearch:9200/_security/user/kibana_system/_password \
          -d "{\"password\":\"${KIBANA_PASSWORD}\"}";
        echo "Initialization complete!";
      '
    depends_on:
      elasticsearch:
        condition: service_healthy

Troubleshooting

Insufficient Memory

When does this issue occur: When host memory resources are limited, causing the Elasticsearch container to fail to start or restart frequently.

• It is recommended to adjust MEM_LIMIT in the .env file.
• Adjust the JVM Heap size in ES_JAVA_OPTS; it is recommended to set it to about 50% of the host's memory.

Permission Issues

When does this issue occur: When the container displays a "Permission Denied" error after starting.

• Please re-verify that the owner of volumes/elasticsearch/data is UID 1000:1000.
• If the issue persists, you can try running sudo chmod -R 777 volumes/elasticsearch/data for testing (development environments only).

Changelog

• 2025-09-24 Initial document creation.
• 2025-11-04 Fixed missing restart configuration in the YAML file, which prevented automatic startup after a reboot.