Deploy Elasticsearch & Kibana with Dokku
I really love to use Dokku as it drastically simplifies the deployment of my nodejs and python apps. I also try to deploy other popular open-source apps with it. This is my attempt to deploy ELK stack with Dokku. Enjoy.
Install Dokku
Install latest Dokku (mid-December 2021):
wget https://raw.githubusercontent.com/dokku/dokku/v0.26.6/bootstrap.sh;
sudo DOKKU_TAG=v0.26.6 bash bootstrap.sh
Finish the installation by providing your public ssh key cat .ssh/id_rsa.pub
:
# usually your key is already available under the current user's `~/.ssh/authorized_keys` file
cat ~/.ssh/authorized_keys | dokku ssh-keys:add admin
# you can use any domain you already have access to
dokku domains:set-global yourdomain.com
Install Elasticsearch with Dokku
Install the dokku elasticsearch plugin:
sudo dokku plugin:install https://github.com/dokku/dokku-elasticsearch.git elasticsearch
Find the version that you want to deploy on the official site. I take the latest - 7.16.1
export ELASTICSEARCH_IMAGE_VERSION="7.16.1"
Create Elasticsearch:
dokku elasticsearch:create elastic
If you received an error from dokku - just copy-paste the suggested bash command, run it and try to create Elastic again.
You may also want to expose ports of elasticsearch container to make it accessible outside:
dokku elasticsearch:expose elastic
-----> Service elastic exposed on port(s) [container->host]: 9200->27701 9300->16897
Let's check that everything is installed and run (notice that I used not 9200 port but the exposed one).
curl -X GET http://localhost:27701/
{
"name" : "dokku.elasticsearch.elastic-1",
"cluster_name" : "dokku.elasticsearch.elastic-cluster",
"cluster_uuid" : "8T5-w6S2345KYUON66J6Q",
"version" : {
"number" : "7.16.1",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "5b38441b16b14444447c107a4c3865776e20c53",
"build_date" : "2021-12-11T00:29:38.865893768Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Install Kibana with Dokku
Create Dokku app and add essential environment variables.
dokku apps:create kibana
dokku elasticsearch:link elastic kibana
# take ELASTICSEARCH_URL variable and put it below
# for my setup it looks like 'http://dokku-elasticsearch-elastic:9200'
dokku config:set kibana ELASTICSEARCH_HOSTS=http://dokku-elasticsearch-elastic:9200
You need to make your Kibana storage persistent (for future config editing and kibana-keystore saving across deployments):
dokku storage:ensure-directory kibana
# created folder at path '/var/lib/dokku/data/storage/kibana'
mkdir /var/lib/dokku/data/storage/kibana/config/
mkdir /var/lib/dokku/data/storage/kibana/data/
dokku storage:mount kibana /var/lib/dokku/data/storage/kibana/config/:/usr/share/kibana/config/
dokku storage:mount kibana /var/lib/dokku/data/storage/kibana/data/:/usr/share/kibana/data/
echo "server.host: "0.0.0.0"" > /var/lib/dokku/data/storage/kibana/config/kibana.yml
echo "0000-0000-0000-0000" > /usr/share/kibana/data/uuid
sudo chmod a+rwx /root/kibana/
The latest lines create an empty kibana.yml
and uuid
files - somehow Kibana doesn't automatically create all required files so we have to create them from scratch (our new shiny folders are empty).
Now we are ready to deploy an official Kibana docker image:
dokku git:from-image kibana docker.elastic.co/kibana/kibana:7.16.1
Additional Setup
Let your Kibana have a pretty URL and HTTPS. Proxy its port to 80:
dokku proxy:ports-set kibana http:80:5601
Specify a subdomain for Kibana:
dokku domains:set kibana kibana.yourdomain.com
dokku config:set kibana SERVER_HOST=kibana.yourdomain.com SERVER_PORT=80
If you want to enable HTTPS on your Kibana, use letsencrypt plugin:
sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
dokku letsencrypt:enable kibana
Now open your browser on kibana.yourdomain.com. It should work 🤗
Enable basic security for Elasticsearch & Kibana
Note that by default your Elasticsearch and Kibana are not secured: anyone can access your web address and start to operate with your data. So you need to setup authentication on Elasticsearch and Kibana.
dokku elasticsearch:info elastic
# find path to config dir
cd /var/lib/dokku/services/elasticsearch/elastic/config
Enable security in config file (official docs are large and quite misleading). Just add two lines (you can open a file like this vim elasticsearch.yml
:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
Enter the container with Elasticsearch and create passwords for default services:
dokku elasticsearch:enter elastic
./bin/elasticsearch-setup-passwords auto
Copy-paste them somewhere.
Now add creadentials to Kibana:
dokku config:set kibana ELASTICSEARCH_USERNAME="kibana_system"
Enter Kibana container and add generated password:
dokku enter kibana
./bin/kibana-keystore create
./bin/kibana-keystore add elasticsearch.password
<copy-paste generate password now>
And on that point, I decided to use Elastic.co cloud version because I spent almost 8 hours figuring out how to deploy it. And I failed to secure the elasticsearch & kibana. And I don't know if is it even possible to do without paid license. Sorry, guys.