Each week, a new ‘Do You Know’ post will be added to our Elastic Technical Knowledge Hub to share valuable insights for improving observability using Elasticsearch. These topics stem from day-to-day challenges we have solved for our clients. Stepwise descriptions help you successfully implement solutions to enhance the performance of your deployment and achieve optimal monitoring of your applications using dashboards and alerting.
This week, I will discuss: how to use Elastic API Key in Logstash code.
Background
Integrating Elastic API keys into Logstash enhances the security and efficiency of data pipelines. This process involves configuring Logstash to authenticate with Elasticsearch using API keys instead of traditional credentials. By enabling API authentication and creating API keys through Kibana, users can securely manage access and streamline their data workflows. This article provides a guide on setting up and implementing Elastic API keys within Logstash for a robust and secure integration.
What are the security best practices that can be used instead of exposing Elastic credentials in Logstash pipeline?
Solution
Step 1 – API authentication
First, enable it by using the parameter below in the elasticsearch.yml file:
xpack.security.authc.api_key.enabled: true
According to Elastic documentation, it is enabled by default, but it is best to verify this and set it to true if not done already.
Step 2 – Creating API key
Go to Kibana → Stack Management → API Keys → Create API Key.
- Name the API key
- Update the values if required
- Create it
- Change the key format to JSON and copy the entire JSON file
- Sample shown below:
{
"id":"Nr0UroABgN_gBm6G",
"name":"logstash_key",
"api_key":"1XgAtLFuSk2lNb7rXHg",
"encoded":"TnIwVXJvQUJnTlVWX2dCbUhtNkc6QXRMRnVTazJtRnpsTmI3clhIZw="
}
![](https://nl.devoteam.com/wp-content/uploads/sites/13/2024/07/9466be2c-5a1c-4f39-832d-3e95e1e4042d.png)
Created API Key
This can also be copied directly in Logstash format like “Nr0UroABgV_gBmHm6G:1XgAtLFuSkzlNb7rXHg”.
![](https://nl.devoteam.com/wp-content/uploads/sites/13/2024/07/69692a45-669b-4dac-a917-4e92cdf941fc.png)
Logstash Format
Step 3 – Logstash implementation
Finally, paste this into the pipeline output with other parameters:
output {
elasticsearch {
apikey => "Nr0UroABgN_gBm6G:1XgAtLFuSk2lNb7rXHg"
}
}
![](https://nl.devoteam.com/wp-content/uploads/sites/13/2024/07/f7333c6c-be18-4e3b-a549-682ffc675ccd.png)
Elastic output
Using a plain text username and password in the configuration file to connect to Elasticsearch is not the recommended way for a secure Logstash output connection to Elasticsearch. Using an API key is the preferred method.
Need help with your Elastic challenges? Contact our experts.
![](https://nl.devoteam.com/wp-content/uploads/sites/13/2023/12/elastic-logo-V-full-color-628x660.png)
With our 25+ Elastic certified consultants, Devoteam is your partner for developing and implementing Monitoring & Observability solutions that facilitate optimal IT control, from Business & IT Operations dashboards to centralized logging and proactive alerting.