Analyze the data collected by the Elasticsearch search engine software visually by installing Kibana Dashboard on Ubuntu 22.04 Jammy Jellyfish or 20.04 Focal Fossa.
Elasticsearch is a popular enterprise search engine software to collect data, index, and analyze it. The software is open-source and hence distributed to use free of cost. The key feature is it can examine and index a wide variety of data types that can be structured or unstructured. For example, it can be text-based data, numerical data, data with time information, or data with geographic information.
But this Elastic Seach doesn’t offer itself a graphical option to analyze data, hence to get that we use Kibana, an open-source analysis, and visualization platform. Together with Elasticsearch and Logstash, it forms the Elastic Stack and enables the data collected by Elasticsearch to be visualized. Users can have various types of visualizations representation of their data such as line diagrams, pie charts, donut charts, or histograms. And allows the display of time series or geographical data. Kibana can be operated both on-premises and cloud-based.
Along with Elasticsearch and Logstash in an ELK stack, Kibana is used for various application areas, here are a few examples:
• Quickly identify hindrances appearing in the performance
• To get location-based search results on a map
• Helps in finding security threats
• Real-time analysis of data
• Monitor applications and IT infrastructure or networks
Steps to install Kibana on Ubuntu 22.04 or 20.04 LTS Linux
As per the developers of Kibana before installing the user should have already installed the Elastic Search instance. Hence, in this article we also show the steps to configure Elasticsearch. The steps given here will be the same for other Debian or Ubuntu-based systems such as Linux Mint, POP OS, and MX Linux…
1. Requirements
Ubuntu 20.04 or 22.04 server
4GB RAM and 2 CPUs
A non-root sudo user
OpenJDK 11
2. APT Update & Install OpenJDK
Before going further to know the steps for setting up Elastic search, let’s update our system and install Java required to follow this tutorial.
sudo apt update && sudo apt upgrade
sudo apt install default-jdk
3. Add Elastic Search GPG key and Repository
Note: If you already have ElasticSearch installed server then you can directly plunge to step 6 of this article but don’t forget to add the Elastic repo.
Well, the packages to install Elastic Search are not available to install on Ubuntu 22.04/20.04 using the default system repository. Hence, we have to add the one offered by the Elastic developers for Debian-based systems.
First, add the GPG key:
sudo apt-get install apt-transport-https
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Add Repository
Next, we will add the repository, the current stable version is 7. x whereas the current development one is 8.x. we have given both the repository here, add the one as per your choice,
For Version 7.x
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
For Version 8.x
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x-prerelease/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
Run system update
Again refresh the system and let it know what are packages available through the newly added repository.
sudo apt update
4. Install ElasticSearch on Ubuntu 22.04 | 20.04 Linux
Once you have added the repository, we can simply use the APT package manager command to install the latest available version of ElasticSearch on Ubuntu 20.04 or 22.04 Server using the command given below.
sudo apt install elasticsearch
5. Start ElasticSearch Services
ElasticSearch services will not start automatically on the system, we have to do that manually. Whereas, to start the service automatically with system boot or in case of a service crash, we have to enable it as well.
Reload Daemon:
sudo systemctl daemon-reload
Enable
sudo systemctl enable elasticsearch
Start
sudo systemctl start elasticsearch
Check Status
sudo systemctl status elasticsearch
Note– In the future to stop the same service you can use this:
sudo systemctl stop elasticsearch
Verify Elasticsearch is working properly
Now everything is up and running on your system for ElasticSearch, it’s time to check whether it is working fine or not. So, to test it we use CURL.
sudo apt-get install curl
Now test the Elasticsearch by sending an HTTP request with port number 9200
curl -X GET "localhost:9200/"
6. Install Kibana Dashboard on Ubuntu 22.04 | 20.04
For installing and configuring Kibana Dashboard, we don’t need to add any other repository because the packages are available through the already added ElasticSearch. Hence, just use the given command:
sudo apt install kibana
7. Enable and start the Kibana service
Now, we have the Kibana Dashboard installed on our Ubuntu 22.04 or 20.04, let’s start and enable its services just like we did for Elastic search.
Reload daemon
sudo systemctl daemon-reload
Start and Enable:
sudo systemctl enable kibana sudo systemctl start kibana
To check the status:
sudo systemctl status kibana
8. Access the Kibana Dashboard web interface
Let’s access the Kibana Dashboard’s web interface to start monitoring and analyzing data generated by Elastic Search visually. For that open your system browser and point to localhost with port number 5601.
http://localhost:5601
To Check system Status, use:
http://localhost:5601/status
9. Access Remotely
If you want to access Kibana Dashboard remotely not on the same local machine where you have installed then we have to install Nginx to create a reverse proxy to easily but securely access the Kibana from remotely a PC.
Install Nginx
We need Nginx for setting up a Reverse proxy, so install it:
sudo apt install nginx
For security create a username and password for Kibana Dashboard:
Create a password and username to access securely Kibana over the network. using htpasswd file.
echo "youruser:`openssl passwd -apr1`" | sudo tee -a /etc/nginx/.htpasswd
Note: Replace youruser with some strong username that you want to use. The password (generated by OpenSSL) will save inside the file /etc/nginx/.htpasswd
.
Create an Nginx server block file:
To let instruct the Nginx server on where to look for the files to serve when the user calls for it over port HTTP/HTTPS, we create an Nginx configuration file:
sudo nano /etc/nginx/sites-available/kibana.conf
Add the following files. Here we are also adding the path to htpasswd
the file to authenticate the user when he or she trying to access Kibana Dashboard.
Note: If you are using Apache or port 80 is bound to some other application then change that in the following file accordingly.
server { listen 80; server_name your.domain.com; #server_name server-ip-address; #For autherntication remove the # from below two line: #auth_basic "Restricted Access"; #auth_basic_user_file /etc/nginx/.htpasswd; location / { proxy_pass http://localhost:5601; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; #upgrade to WebSocket protocol when requested proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } }
Save the file by pressing Ctrl+O, hitting the Enter key, and then Ctrl+X to exit it.
Allow port 80 in the firewall:
sudo ufw allow 'Nginx Full'
Conclusion:
so, far we have learned how to use Ubuntu 20.04 or 22.04 command for installing ElasticSearch and Kibana to evaluate location data and visualize it in various forms such as maps or vector diagrams. Time series can be analyzed, transformed, and visualized including exploring and visualizing the relationships of Elasticsearch data to one another using graphing functions and can be used for machine learning to detect anomalies.