Recently, I noticed a problem with my Index Lifecycle Management (ILM) not appropriately rotating indexes. The error was not super clear, but I did notice that the existing index had just reached 90 days without closing and that was the first move in the ILM. It was clear that the 30-day rollover wasn't happening.

The primary problem was easy to solve, which was to make sure that the write index was set correctly and the index was attached to the template with the alias:

PUT filebeat-8.0.0-2023-04-01-000001
{
  "aliases": {
    "filebeat-8.0.0": {
      "is_write_index": true
    }
  }
}

That resolved part of the problem, but the roll-over then occurred and it created filebeat-8.0.0-2023-04-01-000002, which definitely wasn't what I wanted (although in truth that date is just for the humans, the ILM uses the write dates).

To fix this, I needed to:

1. Stop the ILM

   POST _ilm/stop

2. Create a new index using the date fields

   PUT %3Cfilebeat-8.0.0-%7Bnow%2Fd%7D-000001%3E

3. Set the write index correctly for both indexes:

   POST /_aliases

   {
   "actions" : [
       {
           "add" : {
               "index" : "filebeat-8.0.0-2023-07-03-000001",
               "alias" : "filebeat-8.0.0",
               "is_write_index" : true
           }
       },
       {
           "add" : {
               "index" : "filebeat-8.0.0-2023-04-01-000002",
               "alias" : "filebeat-8.0.0",
               "is_write_index" : false
           }
       }
   ]
   }
   

4. Turn ILM back on

   POST _ilm/start

Relatively straightforward. The only hiccup was that the interim index was now out of sync with the ILM program, showing:

illegal_argument_exception: index [filebeat-8.0.0-2023-04-01-000002] is not the write index for alias [filebeat-8.0.0]

Since the temporary rollover index was small and didn't contain anything essential, I decided to delete it. There were some postings for older versions of ES that suggested ways of fixing this, but with 8+ they didn't seem to work.

Also of note: last year, I described Renaming Elasticsearch indexes when the situation arose to change the name of an index template.