Deploy Elasticsearch & Kibana with Dokku

I really love to use Dokku as it drastically simplifies the deployment of my nodejs and python apps. I also try to deploy other popular open-source apps with it. This is my attempt to deploy ELK stack with Dokku. Enjoy.

Install Dokku

Install latest Dokku (mid-December 2021):

wget https://raw.githubusercontent.com/dokku/dokku/v0.26.6/bootstrap.sh;
sudo DOKKU_TAG=v0.26.6 bash bootstrap.sh

Finish the installation by providing your public ssh key cat .ssh/id_rsa.pub:

# usually your key is already available under the current user's `~/.ssh/authorized_keys` file
cat ~/.ssh/authorized_keys | dokku ssh-keys:add admin

# you can use any domain you already have access to
dokku domains:set-global yourdomain.com

Install Elasticsearch with Dokku

Install the dokku elasticsearch plugin:

sudo dokku plugin:install https://github.com/dokku/dokku-elasticsearch.git elasticsearch

Find the version that you want to deploy on the official site. I take the latest - 7.16.1

export ELASTICSEARCH_IMAGE_VERSION="7.16.1"

Create Elasticsearch:

dokku elasticsearch:create elastic

If you received an error from dokku - just copy-paste the suggested bash command, run it and try to create Elastic again.

You may also want to expose ports of elasticsearch container to make it accessible outside:

dokku elasticsearch:expose elastic

-----> Service elastic exposed on port(s) [container->host]: 9200->27701 9300->16897

Let's check that everything is installed and run (notice that I used not 9200 port but the exposed one).

curl -X GET http://localhost:27701/

{
  "name" : "dokku.elasticsearch.elastic-1",
  "cluster_name" : "dokku.elasticsearch.elastic-cluster",
  "cluster_uuid" : "8T5-w6S2345KYUON66J6Q",
  "version" : {
    "number" : "7.16.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "5b38441b16b14444447c107a4c3865776e20c53",
    "build_date" : "2021-12-11T00:29:38.865893768Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Install Kibana with Dokku

Create Dokku app and add essential environment variables.

dokku apps:create kibana

dokku elasticsearch:link elastic kibana
# take ELASTICSEARCH_URL variable and put it below
# for my setup it looks like 'http://dokku-elasticsearch-elastic:9200'

dokku config:set kibana ELASTICSEARCH_HOSTS=http://dokku-elasticsearch-elastic:9200

You need to make your Kibana storage persistent (for future config editing and kibana-keystore saving across deployments):

dokku storage:ensure-directory kibana
# created folder at path '/var/lib/dokku/data/storage/kibana'

mkdir /var/lib/dokku/data/storage/kibana/config/
mkdir /var/lib/dokku/data/storage/kibana/data/

dokku storage:mount kibana /var/lib/dokku/data/storage/kibana/config/:/usr/share/kibana/config/
dokku storage:mount kibana /var/lib/dokku/data/storage/kibana/data/:/usr/share/kibana/data/

echo "server.host: "0.0.0.0"" > /var/lib/dokku/data/storage/kibana/config/kibana.yml


echo "0000-0000-0000-0000" > /usr/share/kibana/data/uuid

sudo chmod a+rwx /root/kibana/

The latest lines create an empty kibana.yml and uuid files - somehow Kibana doesn't automatically create all required files so we have to create them from scratch (our new shiny folders are empty).

Now we are ready to deploy an official Kibana docker image:

dokku git:from-image kibana docker.elastic.co/kibana/kibana:7.16.1

Additional Setup

Let your Kibana have a pretty URL and HTTPS. Proxy its port to 80:

dokku proxy:ports-set kibana http:80:5601

Specify a subdomain for Kibana:

dokku domains:set kibana kibana.yourdomain.com
dokku config:set kibana SERVER_HOST=kibana.yourdomain.com SERVER_PORT=80

If you want to enable HTTPS on your Kibana, use letsencrypt plugin:

sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
dokku letsencrypt:enable kibana

Now open your browser on kibana.yourdomain.com. It should work 🤗


Enable basic security for Elasticsearch & Kibana

Note that by default your Elasticsearch and Kibana are not secured: anyone can access your web address and start to operate with your data. So you need to setup authentication on Elasticsearch and Kibana.

dokku elasticsearch:info elastic

# find path to config dir
cd /var/lib/dokku/services/elasticsearch/elastic/config

Enable security in config file (official docs are large and quite misleading). Just add two lines (you can open a file like this vim elasticsearch.yml:

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

Enter the container with Elasticsearch and create passwords for default services:

dokku elasticsearch:enter elastic

./bin/elasticsearch-setup-passwords auto

Copy-paste them somewhere.

Now add creadentials to Kibana:

dokku config:set kibana ELASTICSEARCH_USERNAME="kibana_system"

Enter Kibana container and add generated password:

dokku enter kibana

./bin/kibana-keystore create
./bin/kibana-keystore add elasticsearch.password

<copy-paste generate password now>

And on that point, I decided to use Elastic.co cloud version because I spent almost 8 hours figuring out how to deploy it. And I failed to secure the elasticsearch & kibana. And I don't know if is it even possible to do without paid license. Sorry, guys.