Using a temp IndexService for template validation

[masseyke]

We currently create and then quickly delete a real IndexService for template validation in MetadataIndexTemplateService::validateCompositeTemplate. If a cluster state update happens to come in while the IndexService exists, then we can see an assertion failure like this:

java.lang.AssertionError: index [VnipvFiyQtGvaLjGyo4xiQ/osjlH3fjQeuz66QRpH04vg] does not exist in the cluster state, it should either have been deleted or the cluster must be new
	at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.removeIndicesAndShards(IndicesClusterStateService.java:491)
	at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.doApplyClusterState(IndicesClusterStateService.java:318)
	at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:276)
	at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:572)
	at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:558)
	at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:531)
	at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:460)
	at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:159)
	at [email protected]/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:977)
	at [email protected]/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:218)
	at [email protected]/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:184)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
	at java.base/java.lang.Thread.run(Thread.java:1447)

This PR uses a temp IndexService instead to avoid this problem.
Closes #129473

[elasticsearchmachine]

Hi @masseyke, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[nielsbauman]

LGTM, thanks for the quick fix, Keith!

[elasticsearchmachine]

💚 Backport successful

Status	Branch	Result
✅	8.19