Scrape interval

[DeltaTangoLima]

Just wondering how often people are scraping with this exporter? I've got mine set to a 5s interval and am running it alongside the node-exporter community plugin on my OPNsense device itself (also set to 5s in Prometheus).

I'm now seeing a significant uptick in CPU usage since using this exporter. It's averaging ~25% user CPU (up from below 1%) and system CPU is now averaging ~15% (up from low single digits).

I expected an uptick in CPU, but 30%-35% increase seems high for API calls. I'm assuming frequency is a contributor, but thought I'd see what others have experienced.

[kquinsland]

Do you need temporal resolution so precise that you can pin down a specific event to a particular 12th of a minute? I rarely see exporters scraped quicker than 30 seconds. 60 seconds is common but 120 and 300 seconds is also not unheard of.

It largely depends on how reactive you need to be to certain metrics changing.

[DeltaTangoLima]

Do you need temporal resolution so precise

No. not specifically. I'm relatively new to using Prometheus - most of my metrics until now have come from a variety of other sources pushing data into Influx - so thought I'd see if anyone here had a particular view about why/why not.

I did notice that a lot of the exporters I've been playing with have a default interval of 5s in the config samples on their repos, so I took that was a defacto standard, I guess. And, until now, I've not had any issues - even using the node-exporter directly in OPNsense with 5s intervals didn't noticeably increase CPU utilisation.

I guess this particular exporter is a just little more processor intensive when it scrapes the API.

[kquinsland]

so thought I'd see if anyone here had a particular view about why/why not.

Ahh. Push/Pull for metrics is a fun rabbit hole, loads to google / read up on that.

so I took that was a defacto standard, I guess.

For small data sets where you really do need to see changes by sampling every few seconds it's fine especially if you don't retain the data for too long / it's got a reasonable number of labels. 60s is that sweet spot for decent resolution, avoiding staleness gaps (default is 5m, you want each series to have at least one data point every 5m else Prometheus might not graph it) and still keeping prometheus server memory/disk impact manageable and predictable.

I guess this particular exporter is a just little more processor intensive when it scrapes the API.

I'd bet that explains most of your CPU use, yeah; the API is in PHP and probably a few proc_open() calls to get some of the data used in API payloads. If you don't mind the extra load / it's not costing you route/firewall performance then keep it up :).

For prometheus, their docs on storage layout / how samples are compacted and down-sampled is pretty informative. Probably has some clues about what you may want to tune for your application

[DeltaTangoLima]

You make a good point about just taking a step back and asking myself how old is too old for some of this data. My Grafana dashboard updates every 5 minutes, so why would I need 5s slices of CPU history? On my firewall.