Do you know: how to calculate response time for a certain transaction in your logging with the use of Elasticsearch?

Each week, a new “Do you know” will be posted on our Elastic Technical Knowledge Hub to share useful knowledge to improve the observability using Elasticsearch. These topics originate from day-to-day challenges we solved for our clients. A stepwise description helps you to successfully implement solutions on improving the performance of your deployment and get the best monitoring of your applications using dashboards and alerting.

This week I will discuss: how to calculate response time for a certain transaction in your logging with the use of Elasticsearch.

Background

In order to proactively monitor the performance of your applications, it can be wise to check the response time of a certain transaction that passes through a number of applications. If the response time is higher than normal, it might be due to low performance of one of your applications.

Solution

Step 1

In Kibana go to the menu and choose: Stack Management → Transforms → Create a Transform.

Step 2

Choose your index in which you want to calculate the response time and apply the following configuration in the pivot:

• Group by: terms(TransacionId) – you can add more variables to specify your transaction if needed.
• Aggregation: (min)@timestamp, (max)@timestamp
• TransformID: <given name>
• Destination index: <given index time like response_times>
• Time field for Kibana data view: @timestamp.max
• Date field for continuous mode: @timestamp
• Date field for retention policy: @timestamp.max

Step 3

Navigate to the Dev Tool in Kibana and perform a mapping to calculate the response time. In this mapping the field: duration, gives the response time.

PUT <given index time like response_times>/_mapping
{
  "runtime":{
    "duration": {
      "type": "long",
      "script": {
        "source": """
          if (!doc['@timestamp.max'].empty && !doc['@timestamp.min'].empty ) {
            emit(doc['@timestamp.max'].value.toInstant().toEpochMilli() - doc['@timestamp.min'].value.toInstant().toEpochMilli())
          }
          else if (!doc['@timestamp.min'].empty) {
            emit(System.currentTimeMillis() - doc['@timestamp.min'].value.toInstant().toEpochMilli())
          }
        """
      }
    }
  }
}

Step 4

Go to the Discover page in Kibana. Choose your response_times data view and check the field with the response times.

Need help with your Elastic challenges? Contact our experts.

With our 25+ Elastic certified consultants, Devoteam is your partner for developing and implementing Monitoring & Observability solutions that facilitate optimal IT control, from Business & IT Operations dashboards to centralized logging and proactive alerting.