[FEATURE] Request: Adopt Proper Elastic Common Schema (ECS) Field Mappings

[camAtGitHub]

Is your feature request related to a problem?

Yes! The current Linux log type mappings claim to use "ECS" fields, but most of them don't actually align with the Elastic Common Schema specification. This creates confusion and compatibility issues for anyone familiar with actual ECS or trying to work with standard schemas, and can lead to query underperformance, incorrect search results, or aggregation errors when the wrong field types or mappings are used.

Looking at the published mappings, only 3 out of 14 fields (21%) actually use proper ECS field names. The rest use:

• Vendor-specific prefixes (auditd.log.*, system.auth.*, rsa.web.*) that don't exist in ECS
• Non-existent fields (user.filesystem.name)
• Incorrect shortened names (process.exe instead of process.executable)
• Wrong base fields (timestamp instead of @timestamp)

More fundamentally, there's significant confusion about what these fields even represent:

• What is user.filesystem.name? Is it supposed to be user.name?
• If so, then why does system.auth.user also exist - wouldn't that also be user.name?
• Why do we have multiple different raw fields (exe, Image, ParentImage) all potentially mapping to what should be process.executable?
• The whole point of ECS is standardization - there should be ONE canonical mapping per concept, not multiple vendor-specific variations

This creates real uncertainty about what the current fields are even supposed to represent, making it nearly impossible to write reliable queries or build dashboards that work consistently across different log sources.

What solution would you like?

Please align the mappings with the actual ECS specification available at: https://github.com/elastic/ecs/blob/main/generated/csv/fields.csv

Here are the specific changes needed:

Current "ECS" Field	Should Be	Reasoning
user.filesystem.name	user.name	The field user.filesystem.name doesn't exist in ECS
auditd.log.a0	Custom field or proper ECS field	Vendor namespaces aren't ECS - if this is the first argument, consider mapping to relevant process/command fields
auditd.log.comm	process.name	The command name should use standard process fields
auditd.log.exe	process.executable	Path to executable belongs in process.executable
auditd.log.uid	user.id or process.user.id	User IDs should use standard user/process.user fields
system.auth.user	user.name	Username should use standard user.name field
process.exe	process.executable	ECS uses full word executable, not abbreviation
rsa.web.remote_domain	destination.domain or url.domain	Domain names should use standard destination/url fields, not vendor-specific ones
timestamp	@timestamp	ECS standard timestamp field is @timestamp with the @ prefix

What alternatives have you considered?

1. Keep current approach - But this defeats the purpose of claiming ECS compatibility, creates confusion and promotes hair loss.
2. Provide examples - Show example logs and their fields.
3. Offer Assistance - Help map to proper ECS fields where possible.

Do you have any additional context?

The confusion is particularly problematic for fields like:

• LogonId → process.real_user.id - A session/logon identifier isn't semantically the same as a user ID. This should probably be a custom field or possibly event.id depending on what it represents
• DestinationHostname → rsa.web.remote_domain - Why route through an RSA-specific namespace instead of using ECS destination.domain?

The ECS specification is vendor-agnostic and well-documented. Using it properly would:

• Make OpenSearch more compatible with existing ECS tooling
• Reduce confusion for security analysts familiar with ECS
• Improve interoperability with other security platforms
• Make the documentation more trustworthy

Could we get clarification on:

1. Whether there's a plan to adopt proper ECS mappings?
2. What the rationale was for the current non-standard field names?
3. If there are specific auditd fields that genuinely don't have ECS equivalents, could we document those separately?
4. Can example logs be added to documentation?

Thanks for considering this! Happy to help with suggesting proper ECS mappings if needed.

[andrross]

Catch All Triage - 1 2 3

@sbcd90 Can a maintainer take a look at this?