Relay eventually times out all events when using a writePolicy plugin

[jb55]

at first I thought this was some resource exhaustion in an efficient script I was using, so I rewrote my plugins in rust, but still after awhile the relay eventually just times out all incoming notes. I am looking through the plugin code and don't see anything obvious where it could go wrong 🤔

for now I have to pkill strfry every hour to fix it, but its definitely not ideal

[hoytech]

The most likely explanation I can think of is that the plugin isn't responding to a particular event, or its response is malformed somehow. Are there any possible failure conditions in your plugin that would prevent a response from being delivered?

Around when it locks up do you see any logs like Got unparseable line from write policy plugin or maybe id mismatch? I'm going to try to think of any extra logging I can add. Having a timeout at the plugin-level is possible too, but it's slightly annoying to code up because I'm using a blocking fgets() call right now.

noteguard looks pretty sweet BTW!

[jb55]

On Thu, Jul 18, 2024 at 08:25:52PM GMT, Doug Hoyte wrote: The most likely explanation I can think of is that the plugin isn't responding to a particular event, or its response is malformed somehow. Are there any possible failure conditions in your plugin that would prevent a response from being delivered?

Only if the input doesn't parse, then it has no id to put into the output message for serialization. I tried to simplify the loop so we'll see if that helps. Feel free to check the code. I can't see anything wrong with it: https://github.com/damus-io/noteguard/blob/8e1bb0363f1c7729195a560ba1cd91f5b1c657c8/src/main.rs#L140-L170 As you can see noteguard.run() is guaranteed to return and write an output message for each valid input, and all of the filters are pretty simple and would never timeout.

[hoytech]

I'm not sure TBH. I setup noteguard and it's passing all my basic testing. The buffering is correct and I can't see any obvious error cases that would de-sync the stream.

So, I made a debugging script: https://github.com/hoytech/strfry/blob/master/scripts/plugin-debug.pl

Can you please try this? You need the JSON::XS perl package (libjson-xs-perl on Debian-like distros, usually packaged elsewhere too). After it's installed, set this as the relay.writePolicy.plugin param in strfry.conf (paths are probably different):

plugin = "cd /home/user/noteguard && /home/user/strfry/scripts/plugin-debug.pl ./target/debug/noteguard"`

It detects some obvious problems, like mismatched ID, undecodeable JSON, etc. It also times out if it doesn't get a response, by default after 5 seconds. This will restart the plugin, so hopefully you won't need to pkill every hour anymore!

After it's running a while, check the strfry logs for lines starting with PLUGIN-DEBUG. Hopefully this will reveal something!

Thank you!

[hoytech]

Please re-open if still an issue.

[jb55]

i still kill my relay every hour yes lol its fine though

[hoytech]

Are you using the stock noteguard.toml from the noteguard repo? I'm going to try to reproduce this.

[hoytech]

So this issue only happens when using a writePolicy plugin correct?

If so, do you mind trying the branch plugin-timeouts? It's the same as release 1.0.4 except it has the following commit: 816e193

This commit adds a timeout for reading from the plugin, and will print an error (along with the event it's waiting for) to the logs, and then restarts the plugin. Here's an example error line:

2026-03-01 01:55:30.275 (  27.125s) [Writer          ] ERR| Plugin error: Timed out reading response from plugin. Request: {"event":{"content":"hi","created_at":1772347007,"id":"27c885a0d8a3ddd1ce025034775f3391218128efc571c59a8f53282d81432632","kind":1,"pubkey":"4c7535ab86ff6709ba82422c61a3c3f5cd7bb69aaed625f641c387948f02cf7f","sig":"9122fed0b44b6dd31bdc982a75a5d9e328b9aa535844b2d59a9e2885ece0156d521c053b5e7741705c100d26e8d023ac6203c58926c8bf7c11753eeb45e42b04","tags":[]},"receivedAt":1772348129,"sourceInfo":"127.0.0.1","sourceType":"IP4","type":"new"}