Skip to content

Conversation

@ivomurrell
Copy link
Contributor

Description

This package allows us to send metrics from Tool Kit invocations to a backend server, currently planned to be an instance of Vector running in AWS. We can use these metrics to get visibility for how people use Tool Kit, and where things may be going wrong. For now, I've only added one metric, which tracks what Tool Kit tasks are run and whether they completed successfully.

The telemetry package uses a child process to allow us to send the metrics asynchronously and in the background once the main Tool Kit process exits, meaning there is minimal impact on the time it takes for Tool Kit commands to run on developer machines.

Children of the metrics class can be created with arbitrary attributes overridden, and these classes are passed around between functions and stored as fields in Task and Hook classes, allowing important metadata, such as whether we're in CI or the name of the task currently running, to always be included in recorded events and scoped logically.

It doesn't make sense for this to be released yet as the backend server isn't ready but we can either put this in the dev branch if we manage to get that working or leave this unmerged for now.

Checklist:

  • My branch has been rebased onto the latest commit on main (don't merge main into your branch)
  • My commit messages are conventional commits, for example: feat(circleci): add support for nightly workflows, fix: set Heroku app name for staging apps too

@ivomurrell ivomurrell force-pushed the metrics branch 2 times, most recently from 26dc084 to 35a387b Compare December 15, 2025 18:33
@ivomurrell ivomurrell force-pushed the metrics branch 3 times, most recently from d5fa839 to 5b591d0 Compare January 8, 2026 13:10
Copy link
Member

@apaleslimghost apaleslimghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noice! i have a few suggestions on how the classes are structured but feel free to ignore

Copy link
Contributor

@olusoladavid olusoladavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellente!

This package allows us to send metrics from Tool Kit invocations to a
backend server, currently planned to be an instance of Vector running in
AWS. We can use these metrics to get visibility for how people use Tool
Kit, and where things may be going wrong. For now, I've only added one
metric, which tracks what Tool Kit tasks are run and whether they
completed successfully.

The telemetry package uses a child process to allow us to send the
metrics asynchronously and in the background once the main Tool Kit
process exits, meaning there is minimal impact on the time it takes for
Tool Kit commands to run on developer machines.

Children of the metrics class can be created with arbitrary attributes
overridden, and these classes are passed around between functions and
stored as fields in Task and Hook classes, allowing important metadata,
such as whether we're in CI or the name of the task currently running,
to always be included in recorded events and scoped logically.
This allows us to create a TelemetryProcess and start tracking metrics
early on in a Tool Kit invocation before we've decided whether we should
actually send the telemetry or not (i.e., whether the telemetry has been
opted into or not.) The child process is only forked once the telemetry
has been enabled to avoid unnecessary work.
This is opt-in for now whilst we set up and test integration with a
backend server (mostly with our own projects) but will be an opt-out
option in a future (non-breaking) version.
@ivomurrell ivomurrell merged commit e265e93 into main Jan 13, 2026
14 checks passed
@ivomurrell ivomurrell deleted the metrics branch January 13, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants