Install Graylog on Ubuntu 20.04 LTS – A log management software

This tutorial will help the users install and use Graylog on the Ubuntu 20.04 LTS server to collect and analyze systems log data centrally in one place.

Graylog is an open-source tool that offers an integrated platform for collecting, indexing, and analyzing log data. The system essentially consists of the Graylog web interface, the Graylog servers, the Elasticsearch nodes, and a Mongo database.

The nodes can be scaled as required. A system in which everything is combined in one node is sufficient for testing. The Graylog server is the central element of the architecture, which takes care of the management of the Elasticsearch indices and forms an abstraction layer. Therefore, it would be possible to swap Elasticsearch for another system that is particularly suitable for analyzing the log data.

Graylog supports various input mechanisms. By default, four different formats or protocols are supported: Syslog, GELF, JSON / REST-URLs, and RAW. syslog is a standard for the transmission of log messages and is often used by system components.

Things we require to perform this tutorial:

• MongoDB
• ElasticSearch
• Graylog server
• A non-root user with sudo rights
• A Ubuntu server with  4 CPU Cores and  8 GB RAM

Steps to Install Graylog Ubuntu 20.04 LTS

1. Install required dependencies

There are a few things required by the Graylog server to be installed on Ubuntu 20.04 LTS which are Java and, a password generator along with some common ones. Run the below commands to install all of them.

First, run the system update command

sudo apt update

Then install the following packages…

sudo apt-get install apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen

2. Setup MongoDB on Ubuntu 20.04 for Graylog

Graylog uses MongoDB to store data, hence we need to install it on our server so that later the generated logs can be saved there for further analyses.

While writing this article, the latest version of the MongoDB was “7.0“, hence we are installing that. In your case, if it is different or you want to get some older version then change the version number – 7.0 in the given commands with the one that you want to install:

curl -sSL https://www.mongodb.org/static/pgp/server-7.0.asc  -o mongoserver.asc

gpg --no-default-keyring --keyring ./mongo_key_temp.gpg --import ./mongoserver.asc

gpg --no-default-keyring --keyring ./mongo_key_temp.gpg --export > ./mongoserver_key.gpg

sudo mv mongoserver_key.gpg /etc/apt/trusted.gpg.d/

Add MongoDB Repository:

echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu $(lsb_release -cs)/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list 

Run the System Update:

Let the system know about your newly added repository by running the system update command:

sudo apt update

Graylog uses MongoDB to store data, hence now we can install it on our server so that later the generated logs can be saved there for further analyses.

sudo apt install mongodb-org

Enable and start the Database Server services:

sudo systemctl enable --now mongod

sudo systemctl restart mongod.service

To check whether it is running properly without any errors you can run:

sudo systemctl status mongod

3. Install Elastic Search on the Ubuntu 20.04 LTS server

Elasticsearch is an open-source full-text search and analytics engine. It is also highly scalable and allows users to store, search, and analyze big volumes of data quickly and in near real-time which will be helpful in Graylog to deal  & analyze with a large number of logs.

This system is not available in Ubuntu 20.04’s base repo, hence we manually need to add the official Elastic Search repository.

Add GPG Key:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Add Elastic Search repository:

8.x was the latest version of ElasticSearch while doing this article, hence you should check what is at the time you are following this article. Here is the link to its official page.

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.li

Command to Install ElasticSearch open-source version on Ubuntu 20.04:

sudo apt-get update && sudo apt-get install elasticsearch

Modify the Elasticsearch configuration file to set the cluster name to graylog and add action.auto_create_index: false

For this simply copy-paste the below given command block and hit the Enter key.

sudo tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOT
cluster.name: graylog
action.auto_create_index: false
EOT

Enable and start Elastic search service:

sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch
sudo systemctl restart elasticsearch.service

4. Command to Install Graylog Server on Ubuntu 20.04

Download the repository of Graylog that is available as a deb package.

wget https://packages.graylog2.org/repo/packages/graylog-5.0-repository_latest.deb

Install it:

sudo dpkg -i graylog-5.0-repository_latest.deb

Now, update your system, so that it can recognize the newly added repository to download the packages for Graylog:

sudo apt-get update

Finally, install it

sudo apt-get install graylog-server

Extra: If you also want to install the Integrations Plugins or the Enterprise Plugins, then run:

sudo apt install graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins

5. Edit the Graylog configuration file to set the admin Password

There are two password values-  password_secret and root_password_sha2, we need to configure them otherwise Graylog on Ubuntu 20.04 LTS will not start at all.

These two values are present in the Graylog configuration file and what we set for them will use to secure user passwords and log in to the admin user on its web interface. But we cannot set a plain text value for them instead we have to generate a hash. So, run:

Set password_secret key

pwgen -N 1 -s 96

The above command will generate a secret key to secure user passwords, so copy that and edit the configuration file using:

sudo nano /etc/graylog/server/server.conf

Now, find password_secret = in the file and paste the copied secret key in front of it. As shown in the below screenshot.

Save the file by pressing Ctrl + X, Y, and hit the Enter key.

Set root_password_sha2 hash

The default username to log in Graylog web interface is admin, whereas the password needs to be set, that’s what we are doing here. Generate a hash for the password you want to set using the below-given command:

echo -n MyPassword | sha256sum

Note: Change the MyPassword in the above command with the password you want to set to log in Graylog web interface.

As you hit the Enter key after using the above command, a hash sum will be generated. Copy it.

Now, again edit the configuration file:

sudo nano /etc/graylog/server/server.conf

Find the line:  root_password_sha2 and paste the hash sum in front of it, as shown in the below screenshot:

Also, by default, the Graylog is only accessible using localhost IP i.e 127.0.0.1 thus in case you are planning to access its web interface remotely, then change it with your server IP address in the configuration file.

Find the line: http_bind_address, uncomment it and change 127.0.0.1 with the IP address of your system where you are installing graylog.

Save the file– Ctrl + X, Y and hit the Enter key.

6. Enable and Restart the Graylog Server

We already have done all the essential configurations, and now enable this log system service to start automatically.

sudo systemctl daemon-reload
sudo systemctl enable --now graylog-server

sudo systemctl restart graylog-server

Check whether it is running without any error or not:

sudo systemctl status graylog-server

If you are planning to access the Graylog web interface remotely then also open port 9000 in the Ubuntu firewall:

sudo ufw allow 9000

7. Access the Web interface

Open a browser on your local system or remote that can access the Ubuntu 20.04 server IP address. And type the http://your-server-ipaddress:9000

Replace your-server-ip-address with the actual IP address of your Server where Graylog has been installed.

The default username is admin whereas the password is what you have set in step 5 of this article for root_password. For example in the command, we have used MyPassword.

8. Send Sys logs of the host system to Graylog

Create a config file under /etc/rsyslog.d/ to tell the system where to send the logs.

sudo nano /etc/rsyslog.d/90-graylog.conf

Add the following line:

*.* @your-server-ip:5140;RSYSLOG_SyslogProtocol23Format

Replace the your-server-IP with the IP address of the system from where you are sending the logs. If it is a host system where you have installed the Graylog then use the IP address of that.

Save the file by typing Ctrl+X, Y, and hit the Enter key.

Now, add Input for Node in Graylog. 

On the Dashboard of Graylog click on the System -> Inputs.

Select Syslog UDP and hit the Launch new input button.

Select the node from the drop-down box, give some title (whatever you want) to Input, and then set the port number to 5140 after that scroll down and save the configuration.

Now, click on the “Start Input” button to start the server input.

9. Metrics Dashboard

Once the Input from the server started, click on the Search given in the Graylog menu and you will start getting metrics and logs in real-time from your server. Also, you can set the frequency of metrics update.

To know more about this log management tool and other configuration tasks refer to official documentation where you will also find the way to use Nginx/Apache as a reverse proxy and HTTPS in Graylog.