Skip to content

Commit 016c7db

Browse files
author
Joshua Sierles
authored
Merge pull request #27 from dan-cooke/docs/update-readme
Update README to be more detailed
2 parents f4a57a6 + 1f68b5a commit 016c7db

File tree

1 file changed

+56
-10
lines changed

1 file changed

+56
-10
lines changed

README.md

Lines changed: 56 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,27 @@
22

33
Ship logs from fly to other providers using [NATS](https://docs.nats.io/) and [Vector](https://vector.dev/)
44

5-
Here we have some vector configs, alongside a wrapper script to run it all,
6-
that will subscribe to a log stream of your organisation's logs and ship it to various providers.
5+
In this repo you will find various [Vector Sinks](https://vector.dev/docs/reference/configuration/sinks/) along with the required fly config. The end result is a Fly.IO application that automatically reads your organisation logs and sends them to external providers.
76

8-
# Configuration
7+
# Quick start
98

10-
Create a new Fly app based on this [Dockerfile](./Dockerfile) and configure using the following secrets:
9+
1. Create a new fly logger app based on our docker image
10+
11+
```
12+
fly launch --image ghcr.io/superfly/fly-log-shipper:latest
13+
```
14+
15+
2. Set [NATS source secrets](#nats-source-configuration) for your new app
16+
3. Set your desired [provider](#provider-configuration) from below
17+
18+
**Thats it** - no need to setup NATs clients within your apps, as fly apps are already sending monitoring information back to fly which we can read.
19+
20+
However for advanced uses you can still configure a NATs client in your apps to talk to this NATs server. See [NATS](#nats)
1121

1222
## NATS source configuration
1323

1424
| Secret | Description |
15-
| -------------- |------------------------------------------------------------------------------------------------------------------|
25+
| -------------- | ---------------------------------------------------------------------------------------------------------------- |
1626
| `ORG` | Organisation slug (default to `personal`) |
1727
| `ACCESS_TOKEN` | Fly personal access token (required; set with `fly secrets set ACCESS_TOKEN=$(fly auth token)`) |
1828
| `SUBJECT` | Subject to subscribe to. See [[NATS]] below (defaults to `logs.>`) |
@@ -139,16 +149,52 @@ One of these is required for New Relic logs. New Relic recommend the license key
139149

140150
# NATS
141151

142-
The log stream is provided through the [NATS protocol](https://docs.nats.io/nats-protocol/nats-protocol)
143-
and is limited to subscriptions to logs in your organisations.
144-
The NATS source takes some Fly specific environment variables to connect to the stream,
145-
but any NATS client can connect to `fdaa::3` on port `4223` in a Fly vm,
146-
with an organisation slug as the username and a Fly Personal Access Token as the password.
152+
The log stream is provided through the [NATS protocol](https://docs.nats.io/nats-protocol/nats-protocol) and is limited to subscriptions to logs in your organisations.
153+
154+
## Connecting
155+
156+
> Note: You do **not** have to manually connect a NAT Client, see [Quick Start](#quick-start)
157+
158+
If you want to add custom behaviours or modify the subject sent from your app, then you can connect your app to the NATs server manually.
159+
160+
Any fly app can connect to the NATs server on `nats://[fdaa::3]:4223` (IPV6).
161+
162+
**Note: you will need to supply a user / password.**
163+
164+
> **User**: is your Fly organisation slug, which you can obtain from `fly orgs list` > **Pass**: is your fly token, which you can obtain from `fly auth token`
165+
166+
#### Example using the NATs client
167+
168+
Launch a nats client based on the nats-server image
169+
170+
```
171+
fly launch --image="synadia/nats-server:nightly" --name="nats-client"
172+
```
173+
174+
SSH into the new app
175+
176+
```
177+
fly -a nats-client ssh console
178+
```
179+
180+
```
181+
nats context add nats --server [fdaa::3]:4223 --description "NATS Demo" --select \
182+
--user <YOUR FLY ORG SLUG>
183+
--pass <YOUR PAT>
184+
```
185+
186+
```
187+
nats pub "logs.test" "hello world"
188+
```
189+
190+
## Subject
147191

148192
The subject schema is `logs.<app_name>.<region>.<instance_id>` and the standard
149193
[NATS wildcards](https://docs.nats.io/nats-concepts/subjects#wildcards) can be used.
150194
In this app, the `SUBJECT` secret can be used to set the subject and limit the scope of the logs streamed.
151195

196+
## Queue
197+
152198
If you would like to run multiple vm's for high availability, the NATS endpoint supports
153199
[subscription queues](https://docs.nats.io/nats-concepts/queue) to ensure messages are only sent to one
154200
subscriber of the named queue. The `QUEUE` secret can be set to configure a queue name for the client.

0 commit comments

Comments
 (0)