Spin trigger-cron is reserving 100% CPU usage.

[seungjin]

$ spin --version
spin 3.2.0 (c9be6d8 2025-03-24)
$ spin plugins list | grep cron
trigger-cron 0.2.0 [installed]

Spin trigger-cron is reserving 100% CPU usage.
The fastest cron job I'm running is every hour, but it keeps using 100% of the CPU

From htop:

    PID USER       PRI  NI  VIRT   RES   SHR S  CPU%▽MEM%   TIME+  Command
 454838 seungjin    20   0 4035G  131M 20740 S 100.5  1.7  0:41.63 /home/seungjin/.local/share/spin/plugins/trigger-cron/trigger-cron --runtime-config-file runtime-config.toml
 454848 seungjin    20   0 4035G  131M 20740 R 100.5  1.7  0:40.46 /home/seungjin/.local/share/spin/plugins/trigger-cron/trigger-cron --runtime-config-file runtime-config.toml

I use the released spin version (downloaded from github) and installed trigger-cron with spin plugins install --url https://github.com/fermyon/spin-trigger-cron/releases/download/canary/trigger-cron.json

It is Debian GNU/Linux 12 (bookworm)

[seungjin]

My laptop that runs spin 3.3.0-pre0 (bde256d2 2025-04-10) does not have that 100% cpu usage problem.

[seungjin]

Tested released spin 3.2.0 (c9be6d8 2025-03-24) with my other VPS server, but I don't see that 100% cpu load issue. Seems my that specific VPS issue.
Or do you know any case trigger-cron can consume 100% cpu load?

[seungjin]

Seems it is related to libc.so.6: version `GLIBC_2.38' (Debian 12).

[itowlson]

Thanks for flagging this. I've transferred it to the spin-trigger-cron repo.

[seungjin]

So I tried the same environment under a Fedora container, which comes with fairly new libraries.

$ spin up --runtime-config-file runtime-config.toml
Using runtime config [key_value_store.mem: spin], [sqlite_database.mem: spin], [sqlite_database.lachuoi: libsql], [sqlite_database.geoname: libsql], [sqlite_database.feedpub: libsql] from "runtime-config.toml"
Logging component stdio to ".spin/logs/"
Using runtime config [key_value_store.mem: spin], [sqlite_database.mem: spin], [sqlite_database.lachuoi: libsql], [sqlite_database.geoname: libsql], [sqlite_database.feedpub: libsql] from "runtime-config.toml"
Logging component stdio to ".spin/logs/"

Serving http://127.0.0.1:3000
Available Routes:
  geoip: http://127.0.0.1:3000/geoip (wildcard)
  webfinger: http://127.0.0.1:3000/.wellknown/webfinger
  root: http://127.0.0.1:3000
  image-description: http://127.0.0.1:3000/image/description (wildcard)
  assets: http://127.0.0.1:3000/static (wildcard)
  random-place: http://127.0.0.1:3000/random-place (wildcard)
  ifconfig: http://127.0.0.1:3000/ifconfig (wildcard)
Preparing Wasm modules is taking a few seconds...

It seems all HTTP triggers are up and running, but the trigger-cron is stuck and not starting, and it’s using 100% CPU. I can use the HTTP-triggered app and get the proper returns

[seungjin]

That only happens my staging(a sort of) server. On my laptop or other my dev vms do not have that issue. Does trigger-cron have some dependency? (I am digging now).

[seungjin]

I have one VM running Debian 12, which comes with an old version of GLIBC. Spin works there, but trigger-cron does not.
I then launched a Fedora Toolbox (https://containertoolbx.org/), but the same issue occurred: Spin works, but trigger-cron does not start.
Next, I created a full OCI container image and ran it on the same machine. That works fine.
Most users are using OCI containers and their images, so they won’t encounter this issue.
However, we still have the case where Spin works, but trigger-cron doesn’t start.
I’m not sure exactly what caused this (even with Fedora Toolbox—this is probably a separate issue, as it’s related to Toolbox).
I won’t investigate further because it seems to be a rare case, and I can simply use an OCI image-based instance.
Just recording this for future reference.

You may close this issue or I will close in 24 hours.
Thanks!

[seungjin]

Oh.. probably,, we can give some nice logging message or timeout when trigger-cron takes too long to start.

[seungjin]

Create an OCI image. The image works on my laptop, but it doesn't work (only spin-trigger-cron) on my staging VM.
Currently, I can say that there is a VM that does not run spin-trigger-cron even with a container.
It’s possibly not an issue with trigger-cron, but with the VM; I can't be sure for now.

[itowlson]

@karthik2804 I think you know the cron trigger better than others - are you tracking this? Do you have any ideas on what might be happening or what @seungjin could try in order to diagnose it? Thanks!

[seungjin]

@karthik2804 Is there any flag (e.g., -v) that can show logs or verbose messages? It seems to be stuck on the message 'Preparing Wasm modules is taking a few seconds...' and I have no idea what's going on inside. There are no logs or messages.

[itowlson]

@seungjin You can set the RUST_LOG environment variable e.g. RUST_LOG=trace spin up. This will likely produce a lot of output though. I think you can filter it to Spin crates via RUST_LOG=spin=trace, but you may need to also trace into the wasmtime engine e.g. RUST_LOG=spin=trace,wasmtime=debug.

(https://docs.rs/env_logger/latest/env_logger/#enabling-logging)

[seungjin]

RUST_LOG=trace How come I completely forgot about this! Thank you!!! 🙇🙇🙇

RUST_LOG=spin=trace,wasmtime=debug

[seungjin]

Found when it happens:

I use SQLite and a key-value store in memory for caching purposes. This module is triggered by an HTTP request.
https://github.com/seungjin/lachuoi/blob/57a44fad1faf8c990d6fee63a010b0709703f49c/runtime-config.toml#L1
https://github.com/seungjin/lachuoi/blob/9c71a6fe35d8692ccc5fb82963819c7ad6b4458f/spin.dev.toml#L135

This issue occurs only when the following conditions are met:
(1) The HTTP trigger module is using SQLite/key-value storage in memory.
(2) spin-trigger-cron is running alongside (1).
(3) This specific VM (VPS) is being used—seems this VM has some peculiar behavior as well.

~When all three conditions (1), (2), and (3) are met simultaneously, spin-trigger-cron gets stuck. ~
It seems that this VM is also experiencing some unusual behavior with its memory management.

Is there any chance that the HTTP trigger's storage in memory could freeze spin-trigger-cron?