Skip to content

cmd/bsky-webhook: add multiple words, case insenstivity and word boundary #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

cmd/bsky-webhook: add multiple words, case insenstivity and word boundary #19

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 53 additions & 9 deletions cmd/bsky-webhook/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (
"os"
"os/signal"
"path"
"regexp"
"strconv"
"strings"
"syscall"
Expand All @@ -42,7 +43,10 @@ var (
"https://bsky.social"), "bluesky PDS server URL")
watchWord = flag.String("watch-word", envOr("WATCH_WORD", "tailscale"),
"the word to watch out for. may be multiple words in future (required)")

watchWords = flag.String("watch-words", envOr("WATCH_WORDS", ""), // "word1;word2"
"the words to watch out for. if watch-word is also specified, they are added")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separately: I guess you're adding a new flag rather than extending the existing one to avoid breaking existing use. Should we maybe make them mutually exclusive to make a clean break?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and good point

delimiter = flag.String("word-delimiter", envOr("WORD_DELIMITER", ";"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this really need to be a parameter? That seems like overkill.

If we do want to allow arbitrary substrings rather than "words", I feel like maybe we should define a more robust format, e.g., pick a delimiter and define an escape sequence. Is there a specific use case you have in mind here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's to allow for watching "words" that contain the character in the default delimiter, it's not intended to be changed in most cases. I hear you though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility, if you don't wind up going the regexp route, would be to have a flag that lets unclaimed non-flag arguments act as watch words:

var useWatchArgs = flag.Bool("watchword-args", false, "Use non-flag arguments as watch words")
// ...
var watchWords []string
if *useWatchArgs {
   watchWords = flag.Args()
} else if flag.NArg() != 0 {
   log.Fatalf("extra arguments after command: %q", flag.Args())
} else if *watchWord != "" {
   watchWords = []string{*watchWord}
}

or words to that effect.

Copy link
Member Author

@Erisa Erisa Apr 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point actually, then the shell would be responsible for separating up "words" right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the caller can do any quoting they need to to preserve whitespace, and otherwise it'll get split up before exec.

"the character(s) that multi-word options are split by.")
secretsURL = flag.String("secrets-url", envOr("SECRETS_URL", ""),
"the URL of a secrets server (if empty, no server is used)")
secretsPrefix = flag.String("secrets-prefix", envOr("SECRETS_PREFIX", ""),
Expand All @@ -51,6 +55,10 @@ var (
"the Tailscale hostname the server should advertise (if empty, runs locally)")
tsStateDir = flag.String("ts-state-dir", envOr("TS_STATE_DIR", ""),
"the Tailscale state directory path (optional)")
caseSensitive = flag.Bool("case-sensitive-words", hasEnv("CASE_SENSITIVE_WORDS"),
"make watch words case-sensitive")
enforceWordBoundary = flag.Bool("enforce-word-boundary", hasEnv("ENFORCE_WORD_BOUNDARY"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm.

What do you think instead about having the word matches be regexps? Then the user can specify case (in)sensitivity directly, and the choice to enforce a word boundary can be encoded in the expression.

That would also give us a way to express multiple matches, since the RE2 syntax already has a way to express alternation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it, the only thing I worry is that it gets confusing when you have multiple words to make sure to group them and add (or not) the options for each group

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Especially since I wanted character sensitivity to be disabled by default since that's usually what you want and it's not always obvious unless pointed out that you need to enable it, especially when enabling it requires adding in some characters to your regex

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah, so it might still make sense to have a separate flag for "enable case sensitivity". That could still be grafted with a regexp, though, e.g., https://go.dev/play/p/-LQN-fCPhdN

Something like that would let you have the default case-insensitive match without having to explicitly ask for it, yet still give you the ability to control it finely if you want.

"only match \"whole words\", \\b in regex")
)

// Public addresses of jetstream websocket services.
Expand Down Expand Up @@ -92,8 +100,40 @@ func main() {
log.Fatal("Missing Bluesky account handle (BSKY_HANDLE)")
case *bskyAppKey == "" && *secretsURL == "":
log.Fatal("missing Bluesky app secret (BSKY_APP_PASSWORD)")
case *watchWord == "":
log.Fatal("missing watchword")
case *watchWord == "" && *watchWords == "":
log.Fatal("missing watchWord and watchWords (WATCH_WORD / WATCH_WORDS)")
}

words := strings.Split(*watchWords, *delimiter)
words = append(words, *watchWord)

if len(words) == 0 {
log.Fatal("no words! nothing to do!")
}

// build the regular expression for matching words
var rb strings.Builder
if *enforceWordBoundary {
rb.WriteString("\\b(")
}
if !*caseSensitive {
rb.WriteString("(?i)")
}

// prepare the words for being compiled into regex
for i, v := range words {
words[i] = "(" + regexp.QuoteMeta(v) + ")"
}

rb.WriteString(strings.Join(words, "|"))
if *enforceWordBoundary {
rb.WriteString(")\\b")
}

log.Print(rb.String())
wordRx, err := regexp.Compile(rb.String())
if err != nil {
log.Fatalf("compile regex: %v", err)
}

ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
Expand Down Expand Up @@ -139,14 +179,19 @@ func main() {
}
slog.Info("ws connecting", "url", wsURL.String())

err := websocketConnection(ctx, wsURL)
err := websocketConnection(ctx, wsURL, *wordRx)
slog.Error("ws connection", "url", wsURL, "err", err)

// TODO(erisa): exponential backoff
time.Sleep(2 * time.Second)
}
}

func hasEnv(name string) bool {
_, ok := os.LookupEnv(name)
return ok
}

func envOr(key, defaultVal string) string {
if result, ok := os.LookupEnv(key); ok {
return result
Expand All @@ -171,7 +216,7 @@ func nextWSAddress() func() string {
}
}

func websocketConnection(ctx context.Context, wsUrl url.URL) error {
func websocketConnection(ctx context.Context, wsUrl url.URL, wordRx regexp.Regexp) error {
// add compression headers
headers := http.Header{}
headers.Add("Socket-Encoding", "zstd")
Expand Down Expand Up @@ -206,7 +251,7 @@ func websocketConnection(ctx context.Context, wsUrl url.URL) error {
return err
}

err = readJetstreamMessage(ctx, jetstreamMessage, bsky)
err = readJetstreamMessage(ctx, jetstreamMessage, bsky, wordRx)
if err != nil {
msg := jetstreamMessage[:min(32, len(jetstreamMessage))]
log.Printf("error reading jetstream message %q: %v", msg, err)
Expand All @@ -216,7 +261,7 @@ func websocketConnection(ctx context.Context, wsUrl url.URL) error {
return ctx.Err()
}

func readJetstreamMessage(ctx context.Context, jetstreamMessageEncoded []byte, bsky *bluesky.Client) error {
func readJetstreamMessage(ctx context.Context, jetstreamMessageEncoded []byte, bsky *bluesky.Client, wordRx regexp.Regexp) error {
// Decompress the message
m, err := zstdDecoder.DecodeAll(jetstreamMessageEncoded, nil)
if err != nil {
Expand Down Expand Up @@ -247,7 +292,7 @@ func readJetstreamMessage(ctx context.Context, jetstreamMessageEncoded []byte, b
return nil
}

if strings.Contains(strings.ToLower(bskyMessage.Commit.Record.Text), strings.ToLower(*watchWord)) {
if wordRx.MatchString(bskyMessage.Commit.Record.Text) {
jetstreamMessageStr := string(jetstreamMessage)

go func() {
Expand All @@ -258,7 +303,6 @@ func readJetstreamMessage(ctx context.Context, jetstreamMessageEncoded []byte, b
}

var imageURL string

if len(bskyMessage.Commit.Record.Embed.Images) != 0 {
imageURL = fmt.Sprintf("https://cdn.bsky.app/img/feed_fullsize/plain/%s/%s", bskyMessage.DID, bskyMessage.Commit.Record.Embed.Images[0].Image.Ref.Link)
}
Expand Down