Do you know: how to solve a mapping conflict in your Elasticsearch index?

Each week, a new “Do You Know” will be posted on our Elastic Technical Knowledge Hub to share useful knowledge to improve the observability using Elasticsearch. These topics originate from day-to-day challenges we solved for our clients. A stepwise description helps you to successfully implement solutions on improving the performance of your deployment and get the best monitoring of your applications using dashboards and alerting.

This week I will discuss: how to solve a mapping conflict in your Elasticsearch index.

Background

When a new document is indexed, Elasticsearch automatically infers the data types of the fields based on the first document received. If subsequent documents contain the same field but with a different data type, Elasticsearch will generate a mapping exception. To resolve this issue, a mapping of the field must be applied, and subsequently re-indexing your data will remove the mapping conflict.

Solution

Step 1

In Kibana, go to the menu and choose Stack Management → Data View. Check your data view on mapping conflicts. When a mapping conflict occurs, you can retrieve the index name and the conflicting field type by clicking on the conflicting field type.

Step 2

Create your mapping by updating your index template.

In Kibana go to: Stack Management → Index Management → Index Template.

Search for your index template and choose Edit. At Tab 4: Mappings, choose Add field. Map the Field Name and Field Type. Save the template and perform an index rollover:

POST <index name>/_rollover

In case you do not have an index template, just perform the following query in the Dev Tool in Kibana:

PUT /my_index
{
  "mappings": {
    "properties": {
      "<field name>": {
        "type": "<choose your type, like text or object>"
      }
}
}
}

Step 3

Perform a re-index of your data that contains a mapping conflict.

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
  }
}

Step 4

After the re-indexing took place, go to Kibana to Stack Management → Index Management → Indices and remove the source index.  Your mapping conflict  does not longer exist.

Need help with your Elastic challenges? Contact our experts.

With our 25+ Elastic certified consultants, Devoteam is your partner for developing and implementing Monitoring & Observability solutions that facilitate optimal IT control, from Business & IT Operations dashboards to centralized logging and proactive alerting.