Skip to content

Commit 25e06c8

Browse files
committed
Initial public release
0 parents  commit 25e06c8

File tree

12 files changed

+745
-0
lines changed

12 files changed

+745
-0
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
config.yaml
2+
notifyrss-go
3+
bin
4+
ao3-notifications.atom.xml

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2025 by Eugene Medvedev (eugene@dvedev.me)
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# notifyrss-go
2+
3+
## What is it?
4+
5+
Imagine you have a [Miniflux](https://miniflux.app/) installation and you're tracking stories on [Archive of Our Own](https://archiveofourown.org/). And while that latter sends you email notifications about stories updating, you want them in your Miniflux instead. Or tinyRSS. Or Feedly. Or any other RSS/Atom/JSONFeed feed reader you like.
6+
7+
This tool was born to solve this particular problem, and so far, no others.
8+
9+
In the future, I plan to extend it to do other, similar jobs of transmuting email notifications into a feed of story updates (Major forums like Spacebattles and Sufficient Velocity come to mind), and there is code in there to make it easier, but right now, that's all it is doing.
10+
11+
## Installation
12+
13+
This is a [Go](https://go.dev/) program, so to build from source you just:
14+
15+
```shell
16+
go install github.com/Mihara/notifyrss-go@latest
17+
```
18+
19+
Or you can grab one of the binaries on the releases page. This is pure Go, and should work on any platform Go can compile for. Once you have an executable, it's on you to run it at regular intervals, or whenever a new email message comes in, in whatever way seems more expedient.
20+
21+
```shell
22+
notifyrss [configuration file]
23+
```
24+
25+
If you don't supply the configuration file parameter, it looks for `config.yaml` in the current directory.
26+
27+
You will also need to get the resulting static RSS/Atom/JSONFeed file to a web server, so that your Miniflux/tinyRSS/Feedly can pick it up. If you run your own feed aggregator, you probably already have a web server, or don't really have a problem with setting one up. If you don't, it shouldn't be difficult to set up Github Pages or Neocities or any other free static hoster to serve it, as long as you can run `notifyrss-go` at regular intervals to update your feed file.
28+
29+
Ideally, you want a separate email account to collect notifications and set up a forwarding scheme from your primary account that you used to register with *Archive of Our Own* where you actually receive notification emails. *(That's what I did.)* This is because your configuration file will inevitably contain the password to access this account, in plain text. Having a separate write-only account for the job is inherently more secure. While I have this account on my own email server, which runs on the same machine as my Miniflux installation, it can be anywhere, the only real requirement is to offer IMAP access and accept password login.
30+
31+
## Configuration
32+
33+
The configuration is a [YAML](https://en.wikipedia.org/wiki/YAML) file:
34+
35+
```yaml
36+
mail:
37+
host: example.com
38+
port: 993
39+
connection: ssl
40+
user: notifier
41+
pass: verysecret
42+
folder: INBOX
43+
options:
44+
format: atom
45+
files:
46+
aoo: ao3-notifications.atom.xml
47+
```
48+
49+
+ **mail**: Section pertaining to setting up the email where it will be picking up notifications from.
50+
+ **host**: hostname of the email server. Required.
51+
+ **port**: port of the IMAP server. Default is `993`.
52+
+ **connection**: Connection type. Valid types are `plain`, `ssl`, `starttls`, default is `ssl`.
53+
+ **user**: Username used for logging in. Required.
54+
+ **pass**: Password. Required.
55+
+ **folder**: IMAP folder to check. Default is `INBOX`, which is the primary inbox. You can use some other folder, e.g. set up your email to sort all notifications about story updates into a separate folder, and use that, although I still recommend a separate account. Ideally, there should be no extraneous emails in this folder, although if there are any, they will be ignored.
56+
+ **options**: Section for general options.
57+
+ **format**: Format of the feed to generate. Valid formats are `atom`, `rss`, `json`. Default is `atom`.
58+
+ **files**: Files to be generated.
59+
+ **aoo**: The filename for the Archive of Our Own feed. If not given, the feed will not be generated at all.
60+
61+
## How it works
62+
63+
Given the configuration file, `notifyrss-go` logs into the IMAP account, acquires every *unread* email in the given mailbox which it recognizes as coming from Archive of Our Own email notifier (or potentially, other such notifiers, once I get around to making them) and parses it to make a plausible feed item telling you that a story has a new chapter to read. The feed is then saved to a file. That's it. With Miniflux in particular, you can even configure it to fetch the actual chapter text, which is quite convenient.
64+
65+
It's important to note two things:
66+
67+
+ The emails will *stay* unread. It's on you to decide when you want to mark them read if at all.
68+
+ Only the emails currently present in the mailbox and still unread will appear in the generated feed file as feed entries.
69+
70+
In practice, it will take you years of active reading to rack up enough notifications for the feed generation to start taking more than a second.
71+
72+
## Development
73+
74+
If you wish to preempt me and write a parser for some other kind of email notification, I'm open to pull requests -- take a look at `feed.go` and `parser-aoo.go` where comments should make what you need to do reasonably obvious. There's no reason this tool shouldn't be able to handle any reasonable email notifier service under the sun.
75+
76+
To build release binaries, you may want to use [Task](https://taskfile.dev/), although there's nothing particularly special about what it is doing here, and simple `go build` will build:
77+
78+
```shell
79+
task build
80+
```
81+
82+
## License
83+
84+
This program is released under the terms of MIT license. See the full text in [LICENSE](LICENSE)

Taskfile.yml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# https://taskfile.dev
2+
3+
version: '3'
4+
5+
tasks:
6+
default:
7+
desc: "List available tasks"
8+
cmds:
9+
- task -a
10+
silent: true
11+
12+
build:
13+
desc: "Build executables for release"
14+
vars:
15+
VERSION:
16+
sh: git describe --tags --abbrev=0 || echo \(development\)
17+
env:
18+
CGO_ENABLED: 0
19+
cmds:
20+
- mkdir -p bin
21+
- rm -f bin/*
22+
- for:
23+
matrix:
24+
OS: ["windows", "linux", "darwin"]
25+
ARCH: ["amd64", "arm64"]
26+
cmd: >
27+
GOOS={{.ITEM.OS}} GOARCH={{.ITEM.ARCH}}
28+
go build -o
29+
./bin/notifyrss_{{.ITEM.OS}}_{{.ITEM.ARCH}}{{ ternary ".exe" "" (eq .ITEM.OS "windows")}}
30+
-trimpath -ldflags '-s -w -X main.version={{.VERSION}}"' .
31+

config.example.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
mail:
2+
host: example.com
3+
port: 993
4+
connection: ssl
5+
user: notifier
6+
pass: -------------
7+
folder: INBOX
8+
options:
9+
format: atom
10+
files:
11+
aoo: ao3-notifications.atom.xml

email.go

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
package main
2+
3+
// The most arcane part of the whole thing, because apparently,
4+
// to parse email with golang you need to actually know the IMAP standard
5+
// by heart, because none of this is properly documented.
6+
//
7+
// Oh well.
8+
9+
import (
10+
"bytes"
11+
"fmt"
12+
"io"
13+
"log"
14+
"strings"
15+
"time"
16+
17+
"github.com/emersion/go-imap/v2"
18+
"github.com/emersion/go-imap/v2/imapclient"
19+
"github.com/emersion/go-message/mail"
20+
"golang.org/x/net/html/charset"
21+
)
22+
23+
// Because the actual types are truly mindbending, we consolidate the interesting
24+
// parts of the email into a simpler structure before handing that over to the parsers.
25+
type NotificationEmail struct {
26+
From string
27+
Subj string
28+
Date time.Time
29+
Text string
30+
Html string
31+
}
32+
33+
// The annoying part: taking the message apart into its html and txt bodies.
34+
// It is my intuition, (the documentation is exceedingly lacking)
35+
// that the BodySection[] map always contains exactly one element
36+
// when the message was downloaded with Collect() as above.
37+
// Whose key is a struct.
38+
// And bizarrely, it's not a nil struct.
39+
// So we have to curse to high heavens and loop through sections.
40+
func parseEmail(message *imapclient.FetchMessageBuffer) *NotificationEmail {
41+
42+
notification := NotificationEmail{
43+
From: message.Envelope.From[0].Addr(),
44+
Subj: message.Envelope.Subject,
45+
Date: message.Envelope.Date,
46+
}
47+
48+
for _, bodyPart := range message.BodySection {
49+
50+
mr, err := mail.CreateReader(bytes.NewReader(bodyPart))
51+
if err != nil {
52+
// Looking at the createreader code, if there was
53+
// an error, it's probably a borked message anyway.
54+
return nil
55+
}
56+
57+
partLoop:
58+
for {
59+
p, err := mr.NextPart()
60+
if err == io.EOF {
61+
break
62+
} else if err != nil {
63+
log.Printf("failed to read message part: %v", err)
64+
return nil
65+
}
66+
67+
// We ignore attachments and stuff...
68+
switch h := p.Header.(type) {
69+
case *mail.InlineHeader:
70+
chunkBytes, _ := io.ReadAll(p.Body)
71+
contentType, contentTypeParams, err := h.ContentType()
72+
if err != nil {
73+
continue partLoop
74+
}
75+
76+
// In case we got utf-8, that's where it ends.
77+
text := string(chunkBytes)
78+
79+
// But encodings that are not utf-8 should be converted to utf-8.
80+
// We're assuming they didn't lie to us. (they can)
81+
cs := contentTypeParams["charset"]
82+
if strings.ToUpper(cs) != "UTF-8" {
83+
enc, _, certain := charset.DetermineEncoding(chunkBytes, h.Get("Content-Type"))
84+
if certain && enc != nil {
85+
decodedText, err := enc.NewDecoder().Bytes(chunkBytes)
86+
if err != nil {
87+
log.Printf("failed to decode message, skipping.")
88+
continue partLoop
89+
}
90+
text = string(decodedText)
91+
}
92+
}
93+
94+
switch contentType {
95+
case "text/plain":
96+
notification.Text = text
97+
case "text/html":
98+
notification.Html = text
99+
}
100+
101+
}
102+
}
103+
}
104+
return &notification
105+
}
106+
107+
// Reach into the IMAP server and ask it for all unread emails matching a certain From address.
108+
func fetchMail(client *imapclient.Client, fromAddress string) []*NotificationEmail {
109+
110+
searchResult, err := client.Search(
111+
&imap.SearchCriteria{
112+
Header: []imap.SearchCriteriaHeaderField{
113+
{Key: "From", Value: fromAddress},
114+
},
115+
NotFlag: []imap.Flag{
116+
"\\Seen",
117+
}}, &imap.SearchOptions{}).Wait()
118+
if err != nil {
119+
log.Fatalf("search failed: %v", err)
120+
}
121+
122+
fetchOptions := &imap.FetchOptions{
123+
Flags: true,
124+
Envelope: true,
125+
BodyStructure: &imap.FetchItemBodyStructure{Extended: true},
126+
BodySection: []*imap.FetchItemBodySection{{}},
127+
}
128+
129+
messages, err := client.Fetch(searchResult.All, fetchOptions).Collect()
130+
if err != nil {
131+
log.Fatalf("failed to fetch: %v", err)
132+
}
133+
134+
log.Printf("unread messages from %s: %d", fromAddress, len(messages))
135+
136+
parsedMessages := make([]*NotificationEmail, 0, len(messages))
137+
138+
for _, message := range messages {
139+
result := parseEmail(message)
140+
if result != nil {
141+
parsedMessages = append(parsedMessages, result)
142+
}
143+
}
144+
return parsedMessages
145+
}
146+
147+
// Log into the IMAP server, fetch all interesting emails,
148+
// and produce a slice of structures containing the important parts we're looking for.
149+
func acquireEmail(cfg NotifyRSSConfig) []*NotificationEmail {
150+
var err error
151+
152+
dialTone := fmt.Sprintf("%s:%d", cfg.Mail.Host, cfg.Mail.Port)
153+
var client *imapclient.Client
154+
switch strings.ToLower(cfg.Mail.Connection) {
155+
case "ssl":
156+
client, err = imapclient.DialTLS(dialTone, nil)
157+
case "starttls":
158+
client, err = imapclient.DialStartTLS(dialTone, nil)
159+
case "plain":
160+
client, err = imapclient.DialInsecure(dialTone, nil)
161+
default:
162+
log.Fatalf("ssl parameter must be one of 'plain', 'ssl', 'starttls'")
163+
}
164+
if err != nil {
165+
log.Fatalf("connection failure: %v", err)
166+
}
167+
defer client.Close()
168+
169+
if err := client.Login(cfg.Mail.User, cfg.Mail.Pass).Wait(); err != nil {
170+
log.Fatalf("failed to login: %v", err)
171+
}
172+
173+
mailbox, err := client.Select(cfg.Mail.Folder, &imap.SelectOptions{ReadOnly: true}).Wait()
174+
if err != nil {
175+
log.Fatalf("failed to reach %s: %v", cfg.Mail.Folder, err)
176+
}
177+
178+
log.Printf("%s contains %v messages", cfg.Mail.Folder, mailbox.NumMessages)
179+
180+
var notifications []*NotificationEmail
181+
182+
if mailbox.NumMessages > 0 {
183+
for _, notifier := range SupportedNotifiers {
184+
notifications = append(notifications, fetchMail(client, notifier.From)...)
185+
}
186+
}
187+
188+
if err := client.Logout().Wait(); err != nil {
189+
log.Fatalf("failed to logout: %v", err)
190+
}
191+
192+
return notifications
193+
}

0 commit comments

Comments
 (0)