Allow storing scripts in multiple projects #130578
Conversation
This commit makes the `ScriptService`, the `ScriptCache`, and the stored script APIs project-aware. By adding the project ID to the cache key in the `ScriptCache`, we avoid data leakage between projects. The method `ScriptService#compile` has a lot of usages, so we introduce a temporary method that hard-codes the default project ID. Follow-up PRs will update the remaining usages to pass the correct project ID.
nielsbauman
added
the
:Core/Infra/Scripting
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
Compilation should normally be in the context of a user request. Shouldn't it then get the project id from the thread context? |
@rjernst, thanks for chiming in! We usually try to explicitly pass the project ID around and minimize the reliance on the thread context, as it can be difficult to track down where the project ID is supposed to be set in a call chain. Since there are a lot of usages all over the codebase here, I'm open to considering using the thread context. Can you think of any usages where compilation happens outside of the context of a user request? A perhaps counterintuitive example is cluster state update tasks; those are run without the thread context of the request. Do you happen to know if we do any script compilation in cluster state update tasks, for example? I can do some investigation if you don't already happen to know. |
One I'm not sure of is async search. I assume the search happens in the user context, even after being forked to the async threadpool as a task, but if not then that would be one. Otherwise, scripts are a way for users to customize functionality, so it wouldn't make sense to compile a script outside a user request. |
|
Thanks, @rjernst. I pushed some changes to use the project resolver instead of explicitly passing a project ID to |
| return; | ||
| } | ||
| for (var project : clusterState.metadata().projects().values()) { | ||
| ScriptMetadata scriptMetadata = project.custom(ScriptMetadata.TYPE); |
There was a problem hiding this comment.
What happens with non MP clusters here? Will existing cluster states have their script metadata moved into project state?
There was a problem hiding this comment.
Yep, that's exactly right. ScriptMetadata extends Metadata.ProjectCustom and will thus always be put in ProjectMetadata#customs. See the Metadata deserialization for instance, we read the metadata from an old node here:
and put that in the
ProjectMetadata.Bulider#putCustom here: Does that answer your question?
3 participants
This commit makes the
ScriptService, theScriptCache, and the storedscript APIs project-aware. By adding the project ID to the cache key in
the
ScriptCache, we avoid data leakage between projects. We resolve theproject ID from the thread context inside
ScriptService#compileinstead ofpassing the project ID explicitly, to avoid making a large number of changes.
Since scripts should always be compiled in a user context, resolving the
project ID this way should be fine.