Skip to content

Commit e44bee1

Browse files
committed
Write documentation about database adapters
1 parent 3f5ae2a commit e44bee1

File tree

2 files changed

+171
-13
lines changed

2 files changed

+171
-13
lines changed

README.md

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Public analytics as a Node.js microservice, no sysadmin experience required.
44

55
[![Build Status](https://travis-ci.org/mxstbr/micro-analytics.svg?branch=master)](https://travis-ci.org/mxstbr/micro-analytics)
66

7-
A tiny analytics server with less than 100 lines of code, easy to run and hack around on. It does one thing, and it does it well: count the views of something and making the views publicly accessible via an API.
7+
A tiny analytics server with ~150 lines of code, easy to run and hack around on. It does one thing, and it does it well: count the views of something and making the views publicly accessible via an API.
88

99
(there is currently no frontend to display pretty graphs, feel free to build one yourself!)
1010

@@ -24,12 +24,6 @@ See [`server-setup.md`](./server-setup.md) for instructions on acquiring a serve
2424

2525
> **Note**: You can pass any option to the `micro-analytics` command that you can pass to [`micro`](https://github.com/zeit/micro). As an example, to change the host you'd do `micro-analytics -H 127.0.0.1`
2626
27-
### Database adapters
28-
29-
micro-analytics supports custom database adapters. They can be configured with the environment
30-
variable `DB_ADAPTER`. Setting it to `redis` will make it require `micro-analytics-adapter-redis`.
31-
Leaving it unset will make micro-analytics use the builtin flat-file-db adapter.
32-
3327
## Usage
3428

3529
### Tracking views
@@ -56,17 +50,17 @@ If you just want to get the views for an id and don't want to increment the view
5650

5751
If you want to get all views for all ids, set the `all` query parameter to `true` on a root request. (i.e. `/?all=true`) If you pass the `all` parameter to an id, all ids starting with that pathname will be included. E.g. `/x?all=true` will match views for `/x`, `/xyz` but not `/y`.
5852

59-
## Built with
53+
### Database adapters
6054

61-
- [`micro`](https://github.com/zeit/micro) to create the service.
55+
By default, `micro-analytics` uses `flat-file-db`, a fast in-process flat file database, which makes for easy setup and backups. _(change the path to the database with the `DB_PATH` env variable, e.g. `$ DB_PATH=storage/analytics.db micro-analytics`)_
6256

63-
`micro` is a lightweight wrapper around Nodes `http.Server` which makes it easy to write ultra-high performance, asynchronous microservices. Perfect for our use case!
57+
This works fine for side-project usage, but for a production application with bajillions of visitors you might want to use a real database with a _database adapter_. Install the necessary npm package (e.g. `micro-analytics-adapter-xyz`) and then specify the `DB_ADAPTER` environment variable: `$ DB_ADAPTER=xyz micro-analytics`
6458

65-
- [`flat-file-db`](https://github.com/mafintosh/flat-file-db) to store the data. (and [`promise`](https://github.com/then/promise) to promisify `flat-file-db`)
59+
These are the available database adapters, made by the community:
6660

67-
`flat-file-db` is a fast in-process flat file database that caches all data in memory and persists it to an open file using an append-only algorithm ensuring compact file sizes and strong consistency. By using the filesystem for storage setup is easy and backups are only a copy & paste away. (in case you need more advanced features of a real database, swapping out `flat-file-db` for a real db shouldn't take long)
61+
- [`micro-analytics-adapter-redis`](https://github.com/relekang/micro-analytics-adapter-redis)
6862

69-
*If you want to change the path the database file is saved as pass it as an env variable called `DB_PATH`. E.g. `DB_PATH=storage/analytics.db micro-analytics`.*
63+
Don't see your favorite database here? Writing your own adapter is super easy! See [`writing-adapters.md`](writing-adapters.md) for a simple step-by-step guide.
7064

7165
## License
7266

writing-adapters.md

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Writing database adapters
2+
3+
`micro-analytics` database adapters are simple JavaScript modules which export an object with five methods. They _have_ to be called `micro-analytics-adapter-xyz`, where `xyz` is the name users will pass to the `DB_ADAPTER` environment variable when starting `micro-analytics`.
4+
5+
## Overview
6+
7+
The five methods every adapter has to have are:
8+
9+
- `get(key: string)`: Get a value from the database
10+
- `put(key: string, value: object)`: Put a value into the database
11+
- `has(key: string)`: Check if the database has a value for a certain key
12+
- `getAll(options?)`: Get all values from the database as JSON
13+
14+
All of these methods have to return Promises. On top of that there is one more method, which returns an Observer (based on the [proposed spec](https://github.com/tc39/proposal-observable))
15+
16+
- `subscribe(pathname?: string)`: Subscribe to changes of all keys starting with a certain `pathname`. If no pathname is provided, subscribe to all changes.
17+
18+
This is what the export of an adapter should thusly look like:
19+
20+
```JS
21+
// index.js
22+
const { get, put, getAll, has, subscribe } = require('./adapter')
23+
24+
module.exports = {
25+
get,
26+
put,
27+
getAll,
28+
has,
29+
subscribe,
30+
}
31+
```
32+
33+
Let's dive into the individual methods:
34+
35+
### `get(key: string): Promise`
36+
37+
Should resolve the Promise with the value stored in the database, if there is one, or reject it, if not.
38+
39+
#### Usage
40+
41+
```JS
42+
try {
43+
const value = await adapter.get('/hello')
44+
} catch (err) {/* New record added here */}
45+
```
46+
47+
### `put(key: string, value: object): Promise`
48+
49+
Should resolve the Promise with nothing, if the insertion was successful, and reject with a descriptive error message, if anything went wrong.
50+
51+
#### Usage
52+
53+
```JS
54+
try {
55+
await adapter.put('/hello', { views: [{ time: 123 }] })
56+
} catch (err) {
57+
throw err
58+
}
59+
```
60+
61+
### `has(key: string): Promise`
62+
63+
Should resolve the Promise with `true` if the database contains a value for the `key`, or resolve with `false` if the database does not contain a value for `key`.
64+
65+
Should only reject the Promise if something went wrong with the database while checking.
66+
67+
#### Usage
68+
69+
```JS
70+
const pathExists = await adapter.has('/hello')
71+
```
72+
73+
### `getAll(options?): Promise`
74+
75+
Should resolve the promise with all keys and values currently stored in the database as JSON like so:
76+
77+
```JSON
78+
{
79+
"/hello": {
80+
"views": [{
81+
"time": 123
82+
}, {
83+
"time": 124
84+
}]
85+
},
86+
"/about": {
87+
"views": [{
88+
"time": 122
89+
}, {
90+
"time": 125
91+
}]
92+
}
93+
}
94+
```
95+
96+
Should resolve to `{}` if nothing was found. Should only reject if something went wrong during the database lookup.
97+
98+
#### Options
99+
100+
The passed `options` can contain the following keys:
101+
102+
```JS
103+
{
104+
pathname?: string,
105+
filter?: {
106+
before?: UTCTime,
107+
after?: UTCTime,
108+
}
109+
}
110+
```
111+
112+
All of these are optional. Let's examine what each one does individually.
113+
114+
##### `pathname: string`
115+
116+
If a `pathname` is passed the adapter should return all values of keys that _start with_ the path. Examine this query:
117+
118+
```JS
119+
await adapter.getAll({ pathname: '/car' })
120+
```
121+
122+
This would not only return the values for the record with the key `/car`, but also for `/car2`, `/car/make/toyota`, `/caramba`, etc, essentially for _any key that starts with the string `/car`_.
123+
124+
##### `filter: { before: UTCTime, after: UTCTime }`
125+
126+
The adapter should return filter all records returned to only contain the views before `before` and after `after`. The times are passed in UTC, so a simple `record.views[x].time < filter.before` is good enough.
127+
128+
Both, either one or none of them might be specified. It also has to work in conjunction with `pathname`. These are all valid queries, including any further combination of those:
129+
130+
```JS
131+
// Return all records
132+
await.getAll()
133+
134+
// Return all keys and their views that happened before 1234 UTC
135+
await.getAll({ filter: { before: 1234 }})
136+
137+
// Return all keys and their views that happened before 1234 UTC but after 1200 UTC
138+
await.getAll({ filter: { after: 1200, before: 1234 }})
139+
140+
// Return all keys that start with /car and their views that happened before 1234 UTC but after 1200 UTC
141+
await.getAll({ pathname: '/car', filter: { after: 1200, before: 1234 }})
142+
```
143+
144+
### `subscribe(pathname?: string): Observable`
145+
146+
Should return an Observable (based on the proposed [ECMAScript spec](https://github.com/tc39/proposal-observable)) which pings `micro-analytics` whenever a key changes.
147+
148+
If a `pathname` is passed it should have the same behaviour as `getAll`, where it listens to all keys that _start with_ the passed `pathname`.
149+
150+
#### Usage
151+
152+
```JS
153+
const subscription = adapter.subscribe('/hello')({
154+
next(val) {/* Send new value */},
155+
error(err) {/* Send error */},
156+
complete() {/* Cleanup after unsubscription */},
157+
})
158+
159+
// At any time
160+
161+
subscription.unsubscribe()
162+
```
163+
164+
> **Note:** If your database of choice does not support subscriptions, it is fine not to have `adapter.subscribe` as long as you mention that visibly in your documentation.

0 commit comments

Comments
 (0)