Do you know: how to reindex an index in Elasticsearch with a composable template?

Each week, a new “Do You Know” will be posted on our Elastic Technical Knowledge Hub to share useful knowledge to improve the observability using Elasticsearch. These topics originate from day-to-day challenges we solved for our clients. A stepwise description helps you to successfully implement solutions on improving the performance of your deployment and get the best monitoring of your applications using dashboards and alerting.

This week I will discuss: how to reindex an index with a composable template.

Background

It may occur that you need to reindex your index. For instance, to modify data types requires a new index as you cannot change data types in place. Once the new mapping has been created for the modified data type, then the data needs reindexing. These structural changes require reindexing due to Elasticsearch’s schema-on-write approach. Another reason could be the necessity to resend the document in the index through an ingest pipeline.

Solution

Step 1

Perform a reindex call in Kibana’s Dev Tools and send the documents through the ingest pipeline.

POST _reindex
{
  "source": {
    "index": "<index name>"
  },
  "dest": {
    "index": "<index name_reindexed>",
    "pipeline": "<ingest pipeline name>"
  }
}

In case your index uses a composable template you will get the following 404 error message:

{
  "error": {
    "root_cause": [
      {
        "type": "index_not_found_exception",
        "reason": "no such index [composable template [<index>*] forbids index auto creation]",
        "index_uuid": "_na_",
        "index": "composable template [<index>] forbids index auto creation"
      }
    ],
    "type": "index_not_found_exception",
    "reason": "no such index [composable template [<index>] forbids index auto creation]",
    "index_uuid": "_na_",
    "index": "composable template [<index>] forbids index auto creation"
  },
  "status": 404
}

Step 2

In order to solve this problem, you should first create an empty index in Kibana. Go to the Kibana menu → Stack Management → Index Management → Create Index. The index name must be the same as the destination index that is used in the re-index call.

Step 3

Perform the same Post call in the Dev Tool to perform a reindex of your data. Now the reindexing will be successful.

Step 4

Go to the Kibana menu → Stack Management → Index Management  and remove the source index.

Need help with your Elastic challenges? Contact our experts.

With our 25+ Elastic certified consultants, Devoteam is your partner for developing and implementing Monitoring & Observability solutions that facilitate optimal IT control, from Business & IT Operations dashboards to centralized logging and proactive alerting.