docs(logging): would be great to add more docs

[mota-lhd]

I did a small configuration to get caddy logs using syslog and forward them to betterstack. I can document this and explain how one can do it on their VPS.

The setup is pretty simple : vector opens a local syslog -> collects / parses logs to filter only caddy logs -> vector forwards caddy logs to betterstack or any other monitoring destination.

[gregsadetsky]

I would definitely love to hear more about what you're doing re: logging as it relates to caddy and vector!

TLDR: this isn't a very clear issue, but I've definitely been thinking about moving away (or rather, offering a new option to move away) from syslog logging, and specifically the logspout image that we use (which vector should be able to supersede)

to do so, I'd love to see:

• can vector fully supersede logspout ie can it support our current syslog usecase out of the box // what configuration does it need to do so
• which changes does the daemon need to talk to vector
• at this point, the daemon would presumably be at the "same place" (ie syslog works just like before), but we now have more logging/destination options thanks to vector
• do a r&d spike (this can be done in parallel with having logspout btw -- vector doesn't need to replace logspout for this): how much custom yaml/configuration does vector need to send logs to betterstack, and other services? does every service need a custom config or no? (ie is it "just" about changing a destination url? and/or a token? or is there more custom stuff?)
• how would a "vector logging configuration" work - would the daemon store this in its database? would user be able to add/edit/remove configurations? ie would we offer a "betterstack" config out of the box (not super flexible, but easier to use), or would we allow... a yaml config upload?? into the daemon config?? (doesn't sound super great? but also sounds flexible?)

oh! and :-) I was also wondering if vector miiiight be able to connect to docker containers on its own (via this) and... would that actually potentially remove/simplify our logging architecture? ie vector would talk (directly?) to docker, and send its logs wherever it was configured to do so?

these are ~all of my thoughts/ideas/brainstorms on this topic :-) sorry, I know it's a lot! let me know yours!

[mota-lhd]

imho vector can be shipped in disco using a default configuration that runs on the same network.
you can also say that you can forward these logs anywhere and we have an example for betterstack. why ? they offer free metrics for 30 days and free logs for 3 days 🎉

it is also better and much more modern compared to the syslog approch.

also, the other actors do this kind of approach. for example coolify have a default integration with fluentbit. but i hate their config file. vector is yaml and their vrl lang is awesome.

[gregsadetsky]

yeah! vector does seem to be the way forward, I fully agree!!

I really just am trying to get an idea of how much work it would be / how much needs to be yanked out (ie our current logspout integration)

have you used vector and/or betterstack for this on some other setup? did you set it up manually or with coolify? thanks

[mota-lhd]

i did the setup outside of coolify and the docker stack. it is not complicated.

• one config file to get host and docker metrics / logs.
• one command to run to put the vector user within the docker group so vector can talk to dockerd
  et voila ;)

[gregsadetsky]

super cool! yeah I was thinking more of how to integrate it so that people could use the CLI or the UI here to interact/set it up

right now we have "disco syslog:add", "disco syslog:list", etc. commands. we'd have to find a way to integrate the vector settings into that workflow..!