Public Vector roadmap

[zamazan4ik]

Hi!

Does Vector have a publicly accessible roadmap? It would help a lot to people to see, in which direction(s) Vector will evolve in the (near) future. Also, it helps with estimation regarding the most important/fixes (without a commitment ofc - because that's an open source :).

Another use-case for the public roadmap - helping the new contributors concentrate their work on the most important stuff from Vector's point of view.

I am almost sure that Vector dev team already has some plans, they are just discussed internally. From time to time these plans are mentioned in multiple issues, but that's just a piece of the whole picture.

I understand that for some reason it could be unacceptable for your current workflow (because workflows are complex systems itself with multiple traps) - I just want to start a discussion about it and how it could be implemented.

I think (and hope) @jszwedko knows more about the topic.

[zamazan4ik]

Just found this issue: #1538

And it has a link https://roadmap.vector.dev - seems a really good source of truth but it's outdated. Maybe would be a good idea to place the information regarding the roadmap somewhere in a more searchable place. Maybe in Vector documentation?

[spencergilbert]

Yep, definitely outdated 😞 we've been in a bit of an awkward space trying to continue using the public docs/workflows Vector originally had while becoming more streamlined with Datadog's existing internal pieces.

[jszwedko]

Yeah, it is out of date. Unfortunately, any public roadmap we've published, to-date, hasn't had a ton of accuracy more than a quarter out so I hesitate to even try to revive it.

[zamazan4ik]

And there is a related question. Is there a way to get statistics, which Vector features are mostly used? I am interested in it since it could influence a decision on which stuff I will work next. Since bringing the most valuable changes to the users is better :)

[spencergilbert]

We've talked about this in the past, and we don't have a good way of tracking this aside from anecdotes and conversations with users. We've discussed a variety of implementation that could report such telemetry - with the understanding that's a touchy subject, and a lot of people would prefer the tool not to phone home with metrics.

If we were going to commit to adding that, we'd definitely run it through an RFC process and gather user opinions on the plans there.

[zamazan4ik]

Just an idea: maybe from time to time organize polls somewhere? E.g. in the official Discord channel to get feedback from the users about their use cases, etc. In this case, there is no need to introduce telemetry and other spy-like stuff.

[spencergilbert]

Yep, that's another angle - I think we've done a couple of those in the past, but didn't get much of a response. I'm sure we could do a better job advertising it out and getting responses.

[zamazan4ik]

Yeah. Why I am asking about that?

According to the discussion #15977 now Vector has very inefficient metrics implementation. And seems like migration to the newer register! based implementation helps a lot. And I want to understand, in which priority would be better to convert currently-implemented components to the newer implementation.