Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions specification/entities/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ weight: 2
- [Resource and Entities](#resource-and-entities)
* [Attribute Referencing Model](#attribute-referencing-model)
* [Placement of Shared Descriptive Attributes](#placement-of-shared-descriptive-attributes)
- [Merging of Entities](#merging-of-entities)
- [Examples of Entities](#examples-of-entities)

<!-- tocstop -->
Expand Down Expand Up @@ -149,6 +150,42 @@ different values, then **only** the `k8s.node` entity can reference this key
Other entities (e.g., `k8s.cluster`) can report this attribute in a separate
telemetry channel (e.g., entity events) where full ownership context is known.

## Merging of Entities

Entities MAY be merged if and only if their types are the same, their
identity attributes are exactly the same AND their schema_url is the same.
This means both Entities MUST have the same identity attribute keys and
for each key, the values of the key MUST be the same.

Here's an example algorithm that will check compatibility:

```
can_merge(e, e') {
e.type == e'.type &&
e.schema_url == e'.schema_url &&
has_same_attributes(e.identity, e'.identity)
}
```

When merging entities, all attributes in description are merged together, with
one entity acting as "primary" where any conficting attribute values will be
chosen from the "primary" entity.

Here's an example algorithm that will merge:

```
merge(e, e') {
if can_merge(e, e') {
for attribute in e'.description {
if !e.description.contains(attribute.key) {
e.description.insert(attribute)
}
// Ignore otehrwise.
}
}
}
```

## Examples of Entities

_This section is non-normative and is present only for the purposes of
Expand Down
34 changes: 34 additions & 0 deletions specification/resource/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ weight: 2
<!-- toc -->

- [Identity](#identity)
- [Merging Resources](#merging-resources)
* [Merging Entities into a Resource](#merging-entities-into-a-resource)

<!-- tocstop -->

Expand Down Expand Up @@ -47,3 +49,35 @@ different if one contains an entity not found in the other.
Some resources include raw attributes in additon to Entities. Raw attributes are
considered identifying on a resource. That is, if the key-value pairs of
raw attributes are different, then you can assume the resource is different.

## Merging Resources

Note: The current SDK specification outlines a [merge algorithm](sdk#merge).
This specification updates the algorithm to be compliant with entities. This
section will replace that section upon stabilization of entities. SDKs SHOULD
NOT update their merge algorithm until full Entity SDK support is provided.

Merging resources is an action of joining together the context of observation.
That is, we can look at the resource context for a signal and *expand* that
context to include more details (see
[telescoping identity](README.md#telescoping)). As such, a merge SHOULD preserve
any identity that already existed on a Resource while adding in new identifying
information or descriptive attributes.

### Merging Entities into a Resource

We define the following algorithm for merging entities into an existing
resource.

- Construct a set of existing entities on the resource, `E`.
- For each entity, `new_entity`, in priority order (highest first),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does an SDK determine the relative priority of entities?

do one of the following:
- If an entity `e'` exsits in `E` with the same entity type as `new_entity`:
- Perform a [Entity DataModel Merge](../entities/data-model.md#merging-of-entities), if applicable, otherwise ignore `new_entity`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if applicable

nit: maybe rephrase to "if they are mergeable"? If applicable confused me here.

otherwise ignore new_entity.

Do we consider this an error (e.g. surfaced using the SDK error handler or logged as a warning in the collector)? This makes it sound like these cases should be silently ignored.

- Otherwise, add the entity `new_entity` to set `E`
- Update the Resource to use the set of entities `E`.
- If all entities within `E` have the same `schema_url`, set the
resources `schema_url` to match.
- Otherwise set the Resource `schema_url` blank.
- Remove any attribute from `Attributes` which exists in either the
description or identity of an entity in `E`.
Comment on lines +82 to +83
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is unclear to me. Does it mean that any attributes of the new_entity can potentially drop attributes from other existing entities in E? Shouldn't we make resolving any cross-entity attribute conflicts a requirement for adding an entity to a resource?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more that they override them.

Remember, in the datamodel, attributes are a separate thing from entities vs. in OTLP. So what this says, is in the datamodel, you remove it from the "loose" attributes, because it exists in the new entity's attributes. The new entity will provide a new value that lands in OTLP resource.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this deserves a clarifying note, as I think most readers (for better or worse) are more familiar with OTLP than the data model document, and assume data model == OTLP. Consider:

Note: For cases where Entity Attributes are encoded as references to Resource Attributes, as is the case for OTLP, this implies Entity Attributes take precedence over any Resource Attributes which are not associated with an Entity.

Even though entities in the data model have their own set of attributes, I still agree that entity conflict resolution needs to be a part of the merge process. E.g. if two entities claim to have detected different values for the same key, it seems like that still needs to be resolved when merging entities into a resource.

Loading