Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 206 additions & 0 deletions docs/proposals/namespaces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
# Namespaces

This document is a design proposal to add the ability to define namespaces in semconv.

## Use cases

There are a few different use cases which namespaces will help us to achieve.

### Describing attribute/signal collections

By introducing namespaces ie `gen-ai` we can define a group which all the attributes can be assigned to.
This linking can occur in 1 of 3 ways which are explicit, implicit and fallback and described below.

Currently in v1 schema this can be achieved to a point via attribute groups however these do not exist in v2 schema.
Hence this proposal becomes a useful way to migrate the information which is contained in the attribute groups to v2
and in the process increase the capabilities of this grouping to include other signals.

#### Explicit linking

This is where the author explicitly states the namespace which this telemetry signal should be associated with.
This is to help with the when the namespace of an attribute changes.
For instance `az` has become `azure`.
Comment on lines +18 to +22
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to have prefix - I wonder what the reasoning was behind its removal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding was prefix was added to the attribute keys etc whereas this explicit namespace can be thought of as aliasing.


#### Implicit linking

This is where the full namespace of the attribute is used to detemine which namespace it belongs to.
An example of this is the `gen_ai.openai.request.service_tier` attribute could be
automatically associated with the `gen_ai` namespace,
if the namespace definition existed.

This approach can be also used for subnamespaces for instance the `gen_ai.openai.request.service_tier` attribute could be
automatically associated with the `gen_ai.openai` namespace,
if the namespace definition existed.

#### Fallback linking

If neither of the 2 previous methods are satisifed,
a namespace is created by weaver which corresponds to the first path segment.
This namespace will use the default values for a namespace.
Comment on lines +35 to +39
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of defining namespaces explicitly. Within Weaver right now there is no concept of namespace whatsoever by design. Hence your proposal.

(I took a wrong turn when developing live-check, I had namespace checking with the . separator built into rust code. This was removed and rewritten in rego policies.)

So,

  1. What if I don't want namespaces at all?
  2. What if I want namespaces but a different separator? Or, just CamelCase (I've seen examples of this in the wild)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So my thought would be that there should be a default namespace "default" which items are assigned to if the namespace can not be determined via either of the methods. I think that would handle the scenario of not wanting to use namespaces.

In an effort to keep it simple I would stick with the dot convention being the namespace seperator however if an author is using another convention they should use the explicit option. We could always extend later.

Comment on lines +37 to +39
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have a configuration option to use the folder as fall back source?


### Migration Guides

By introducing the namespaces we now have a location to define information for migration guides.
In the future these migration guides could be generated by weaver,
by using the output from weaver diff hence further automating documentation and increasing consistency.

This ability to indicate a migration guide is a key benefit as currently these links/descriptiions are manually added into the markdown and the documents manually prepred.

The namespace opt-in option would look something like:

```
version: "2"
namespaces:
- name: gen_ai
migration-guide:
enabled: true
brief:
note:
target: 1.a.x
```

### Revision history

Currently as part of the process of writing/updating semantic conventions,
a changelog file is also needing to be authored.
By having namespaces, we could also be automatically generating a revision history document using the output of weaver diff.

This output could produce a single file as per now and/or namespace revision history.
The namespace option would either need to be opted in on a per namespace basis or possibly a weaver setting to control the default behaviour.

The namespace opt-in option would look something like:

```
version: "2"
namespaces:
- name: gen_ai
revison-history:
enabled: true
brief:
note:
```

## Unresolved schema Defintition (input)

At it's core the namespace definition does not differ much at all to the defintion of any other signal.

As such the defintition would be something like:

```
version: "2"
namespaces:
- name: gen_ai
brief:
note:
annotations:
migration-guide:
enabled: true
brief:
note:
target: 1.b.y
revison-history:
enabled: true
brief:
note:
- name: gen_ai.openai
brief:
note:
annotations:
migration-guide:
enabled: true
brief:
note:
target: 1.c.z
revison-history:
enabled: true
brief:
note:
```

## Resolved schema (output)

### Each [Application Mode]

This application mode is what is utisiled to generate definitions of a single items.

The definition of this would be something like:

```
version: "2"
mode: each
definition:
name: gen_ai.client.token.usage
signal-type: metric
......
namespace:
name: gen_ai
brief:
note:
migration-guide:
enabled: true
brief:
note:
revison-history:
enabled: true
brief:
note:
```

*Subject to change as part of the v2 works

### Namespace [Application Mode]

This application mode is what is utisiled to generate definitions of namespaces
and is also what is produced by the resolve schema command.

The definition of this would be something like:

```
version: "2"
mode: namespace
namespace:
name: gen_ai
brief:
note:
migration-guide:
enabled: true
brief:
note:
history:
- version: x.y.z
.......
revison-history:
enabled: true
brief:
note:
history:
- version: x.y.z
.......
metrics:
- name: gen_ai.client.token.usage
type: histogram
......
attributes:
......
spans:
......
events:
......
```

*Subject to change as part of the v2 works

### Registry/Single [Application Mode]

This is used to produce index pages in the registry which covers multiple namespaces.

The structure is the same as Namespace with the only difference being an array of namespaces are outputted rather than a single namespace.

```
version: "2"
mode: registry
registry:
namespaces:
- name: gen_ai
......
```