Kibana shard errors after update to 2.3.100

[RyanITGuy]

Hello, I just upgraded to 2.3.100 this morning and Kibana loads but then dumps a random number of shard errors each time the page is loaded. All point to the _id from various indexes. I've attached the error here but i'm not sure where to actually go to correct the error. Looking through the Stack Management interface, I don't see anything showing _id as being disallowed. Any direction is appreciated. I'm running one manager and one forwarder node if that helps.

{
"took": 1639,
"timed_out": false,
"_shards": {
"total": 531,
"successful": 526,
"skipped": 509,
"failed": 5,
"failures": [
{
"shard": 0,
"index": "mymanagernamehere:so-zeek-2022.02.01",
"node": "hyhz86b1T4WY3e5yCqXfrQ",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.index.mapper.IdFieldMapper$IdFieldType.fielddataBuilder(IdFieldMapper.java:144)",
"org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:112)",
"org.elasticsearch.index.query.SearchExecutionContext.lambda$lookup$2(SearchExecutionContext.java:512)",
"org.elasticsearch.search.lookup.SearchLookup.getForField(SearchLookup.java:109)",
"org.elasticsearch.search.lookup.LeafDocLookup$2.run(LeafDocLookup.java:107)",
"org.elasticsearch.search.lookup.LeafDocLookup$2.run(LeafDocLookup.java:104)",
"java.base/java.security.AccessController.doPrivileged(AccessController.java:318)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:104)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:28)",
"'https://192.168.7.47/soctopus/thehive/case/' + doc['_id'].value",
" ^---- HERE"
],
"script": "'https://192.168.7.47/soctopus/thehive/case/' + doc['_id'].value",
"lang": "painless",
"position": {
"offset": 52,
"start": 0,
"end": 64
},
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"
}
}
}
]
},
"_clusters": {
"total": 1,
"successful": 1,
"skipped": 0
},
"hits": {
"total": 2796359,
"max_score": null,
"hits": []
}
}

[TOoSmOotH]

Try running this on the manager:

salt-call state.apply kibana.so_savedobjects_defaults -l info

[RyanITGuy]

That worked. Thank you very much. Just for my own curiosity, should I have seen an error in the soup.log for this? I didn't see errors in there.

[palitech100]

Thank you so much for posting this!!!

[dougburks]

I've added this command to https://docs.securityonion.net/en/2.3/soup.html#kibana

[cerberus10748]

I had this same issue, and this was a perfect fix. Thank you all.

[golfreeze]

Thank you working done after run command

salt-call state.apply kibana.so_savedobjects_defaults -l info queue=True

[m0duspwnens]

@RyanITGuy Would you mind posting your soup.log?

[RyanITGuy]

temp_souplog.txt
here you go, thanks for taking a look. You guys are great.