Elastic Stack Setup

Overview

^[1]

Setting up the Basics

In this guide, based on the Elastic tutorials ^{[2] [3]}, we'll set up the following:

Step 1: Set up the First Elasticsearch Node

1. Create the following VM:

   NAME: "Target-IT-Elasticsearch1"
   DISK: "500 GiB"
   CORES: "8 Threads"
   MEMORY: "32768 MiB"
   NETWORK: "TarIT"
   

2. Installing Elasticsearch from the APT repository
   Note

   Make sure you have the correct IP address for your server before installing Elasticsearch

   => Force DHCP Client Release

   wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
   sudo apt-get install apt-transport-https
   echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
   sudo apt-get update && sudo apt-get install elasticsearch
   

   ^[4]
3. Copy the terminal output from the installation command to a local file (you'll need that info later)
4. Store the elastic password in your shell:

   export ELASTIC_PASSWORD="your_password"
   

5. Enable Elasticsearch to run as a service:

   sudo systemctl daemon-reload
   sudo systemctl enable elasticsearch.service
   

Step 2: Configure the First Elasticsearch Node for Connectivity

1. Copy the host's IP somewhere (using ip a)
2. Uncomment and edit the following lines of the config:

   sudo vim /etc/elasticsearch/elasticsearch.yml
   

   cluster.name: elasticsearch-infosec
   network.host: 10.2.2.50
   transport.host: 0.0.0.0
   

Step 3: Start Elasticsearch

Now, it's time to start the service and make sure it's running properly:

sudo systemctl start elasticsearch.service
sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
sudo systemctl status elasticsearch

Step 4: Set up Additional Elasticsearch Nodes

Elastic scales very well horizontally, but not very well vertically. So if you are experiencing performance issues, you need more nodes.^[5]

To set up additional nodes, please have a look at steps 4 and 5 in the article this notes is based on:

• Step 4: Set up a second Elasticsearch node
• Step 5: Set up additional Elasticsearch nodes

Step 5: Install Kibana

1. Create the following VM:

   NAME: "Target-IT-Kibana1"
   DISK: "16 GiB"
   CORES: "1 Threads"
   MEMORY: "4096 MiB"
   NETWORK: "TarIT"
   

2. Installing Kibana from the APT repository

   wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
   sudo apt-get install apt-transport-https
   echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
   sudo apt-get update && sudo apt-get install kibana
   

   ^[6]
3. Back on the Kibana host, enable Kibana:

   sudo systemctl daemon-reload
   sudo systemctl enable kibana.service
   

4. Copy the host's IP somewhere (using ip a)
5. Generate a new encryption keys:

   sudo /usr/share/kibana/bin/kibana-encryption-keys generate
   

6. Uncomment and edit the following lines of the config:

   sudo vim /etc/kibana/kibana.yml
   

   server.host: 10.2.2.51
   xpack.encryptedSavedObjects.encryptionKey: "min-32-byte-long-strong-encryption-key"
   xpack.reporting.encryptionKey: "min-32-byte-long-strong-encryption-key"
   xpack.security.encryptionKey: "min-32-byte-long-strong-encryption-key"
   

   ^[7]
7. Start the service and make sure it's running properly:

   sudo systemctl start kibana.service
   sudo systemctl status kibana
   

8. In the status, copy out the URL with a 6 digit verification code to get started. It should look something like this:

   Go to http://10.2.2.51:5601/?code=000000 to get started.
   

9. Generate a Kibana enrollment token on the first Elasticsearch Node:

   sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
   

10. Open the website and paste the enrolment token we generated previously on the Elasticsearch Node
11. When you see the Welcome to Elastic page, provide the username elastic the password that you copied when we installed the first Elasticsearch Node (the ELASTIC_PASSWORD one)

Step 6: Securing the Elastic Stack

Step 7: Install Fleet Server

1. Create the following VM:

   NAME: "Target-IT-FleetServer1"
   DISK: "16 GiB"
   CORES: "1 Threads"
   MEMORY: "4096 MiB"
   NETWORK: "TarIT"
   

2. Create a working directory:

   mkdir fleet-install-files
   cd fleet-install-files
   

3. Copy the host's IP somewhere (using ip a)
4. Back to your web browser, open the Kibana menu and go to Management -> Fleet
5. Click Add Fleet Server
6. Specify a name for your Fleet Server host, for example FleetServer1.
7. Specify the host URL where Elastic Agents will reach Fleet Server, for example: https://10.2.2.52:8220 (use the IP from the host we just set up)
8. Click Generate Fleet Server policy
9. On the Install Fleet Server to a centralized host step, select the Linux Tar tab
10. Copy and paste the generated commands in the terminal on your Fleet Server host
11. At the prompt, enter Y to install Elastic Agent and run it as a service. Wait for the installation to complete
12. In the Kibana Add a Fleet Server flyout, wait for confirmation that Fleet Server has connected
13. For now, ignore the Continue enrolling Elastic Agent option and close the flyout

Step 8: Install Elastic Agent

1. Log in to the host where you'd like to set up Elastic Agent
2. Create a working directory for the installation package:

   mkdir agent-install-files
   cd agent-install-files
   

3. Open Kibana and go to Management -> Fleet
4. Open the Settings tab
5. Reopen the Agents tab and select Add agent
6. In the flyout, choose a policy name, for example Demo Agent Policy.
7. Leave Collect system logs and metrics enabled.
8. Click Create policy.
9. For the "Enroll in Fleet?" step, leave Enroll in Fleet selected
10. On the Install Elastic Agent on your host step, for this example we select the Linux Tar tab
11. Copy and paste the generated commands in the terminal on your Fleet Server host
12. Append an --insecure flag at the end.
    Note

    If you want to set up secure communications using SSL certificates, refer to Tutorial 2: Securing a self-managed Elastic Stack.

13. At the prompt, enter Y to install Elastic Agent and run it as a service. Wait for the installation to complete
14. In the Kibana Add a Fleet Server flyout, wait for confirmation that Elastic Agent has connected

Setting up the Security

Step 1: Add Elastic Defend Integration

1. Go to the Integrations page, by going to Kibana: Management -> Integrations
2. Search for and select Elastic Defend, then select Add Elastic Defend
3. Configure the Elastic Defend integration with an Integration name, for example: elastic-defend-policy
4. Select Traditional Endpoints as the type of environment you want to protect
5. Select Complete EDR as the configuration preset
6. Click the Existing hosts tab and select the Demo one we just created.
7. Save and continue

^[8]

Step 2: Test if the Integration is Working

1. Log into an Endpoint where the Elastic Agent has been deployed
2. Download a EICAR file: wget https://secure.eicar.org/eicar.com.txt
3. Under Security -> Alerts you should now see some alerts:
   

Step 3: Enable Detection Rules

1. Go to the Rules page, by going to Kibana: Security -> Rules -> Detection rules
2. Click on Add Elastic rules
3. Click on Install all
4. Now go back and click on Select all 1171 rules
5. Under Bulk actions, select Enable

Step 4: Configure Auditd & Windows Integrations

Auditd Manager

Install Auditd on Your Clients

sudo apt update
sudo apt install auditd

Setup the Auditd Manager Integration

In this lab I'm going to use Audit Deamon Rules, so we're going to copy and paste those rule into the Audit rules field:

Make sure to use the existing Demo Agent Policy:

Make Sure Your Agents Are Still Healthy

If not you might need to change the Auditd rules.

^[9]

Windows

1. Head to Managment -> Integrations and search for an Integration called Windows:
   
2. Set the toggles you want and install it to the Demo Agent Policy we've already created:
   

Step 5: Configure Osquery Manager Integration

Make sure to use the existing Demo Agent Policy:

Once installed, enable all existing packs under Managment -> Osquery:

^{[10] [11]}

(Optional) Step 6: Enable pfSense/OPNsense Integration

Install the Integration on Kibana

1. Go to the Integrations page, by going to Kibana: Management -> Integrations
2. Search for and select pfSense.
3. Give the Integration a name, for example: opnsense-1
4. Set the Syslog Host to 0.0.0.0
5. Set the Syslog Port to 9001
6. Click the Existing hosts tab and select Fleet Server Policy

Configure Remote Logging on OPNsense

1. Navigate to System -> Settings -> Logging
2. Click on the Remote tab
3. Add a new Logging/Target (Click the plus icon)
   • Transport = UDP
   • Applications = Nothing Selected (this will forwar all logs)
   • Levels = Nothing Selected
   • Facilities = Nothing Selected
   • Hostname = IP of Fleet Server 10.2.2.52
   • Port = 9001
   • Description = Syslog to Elasticsearch
4. Click Save
5. And don't forget to hit Apply

(Optional) Step 7: Enable Additional Integrations

• AbuseCH
• Network Packet Capture
• Defend for Containers
• Lateral Movement Detection
• Data Exfiltration Detection
• Living off the Land Attack Detection
• Domain Generation Algorithm Detection

Relevant Note(s):