Skip to content

Use LogsDB for profiling events to reduce storage footprint #122034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 20, 2025
Merged

Use LogsDB for profiling events to reduce storage footprint #122034

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
022bae0
Use LogsDB for profiling events to reduce storage footprint
rockdaboot Feb 7, 2025
a8dcff1
Amend number for storage footprint
rockdaboot Feb 10, 2025
a31abf0
Merge branch 'main' into profiling-events-logsdb
rockdaboot Feb 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions .../template-resources/src/main/resources/profiling/component-template/profiling-events.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"template": {
"settings": {
"mode": "logsdb",
"index": {
"number_of_shards": 4,
"auto_expand_replicas": "0-1",
Expand All @@ -11,15 +12,22 @@
"profiling.project.id",
"k8s.namespace.name",
"orchestrator.resource.name",
"service.name",
"host.name",
"container.name",
"process.thread.name"
"process.thread.name",
"@timestamp"
],
"order": [
"asc",
"asc",
"asc",
"asc",
"asc",
"asc",
"asc",
"desc"
]
},
"mapping": {
"source": {
"mode": "synthetic"
}
}
},
"codec": "best_compression"
Expand Down
6 changes: 4 additions & 2 deletions ...in/java/org/elasticsearch/xpack/profiling/persistence/ProfilingIndexTemplateRegistry.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,12 @@ public class ProfilingIndexTemplateRegistry extends IndexTemplateRegistry {
// version 11: Added 'profiling.agent.protocol' keyword mapping to profiling-hosts
// version 12: Added 'profiling.agent.env_https_proxy' keyword mapping to profiling-hosts
// version 13: Added 'container.id' keyword mapping to profiling-events
public static final int INDEX_TEMPLATE_VERSION = 14;
// version 14: Stop using using _source.mode attribute in index templates
// version 15: Use LogsDB mode for profiling-events-* (~30% smaller storage footprint)
public static final int INDEX_TEMPLATE_VERSION = 15;

// history for individual indices / index templates. Only bump these for breaking changes that require to create a new index
public static final int PROFILING_EVENTS_VERSION = 5;
public static final int PROFILING_EVENTS_VERSION = 6;
public static final int PROFILING_EXECUTABLES_VERSION = 1;
public static final int PROFILING_METRICS_VERSION = 2;
public static final int PROFILING_HOSTS_VERSION = 2;
Expand Down