N+1 Cache Queries During GQL Initialization

[aaccensi]

Context

Since adopting Solid Cache, several controller actions that trigger a GQL calculation are flagged by Sentry as N+1 queries. This was not visible before because Memcached round-trips are cheap and don't use SQL; with Solid Cache every cache operation is a SQL query. The fix proposed below is cache-backend agnostic.

During GQL initialization, several etsource modules fetch their static data from Rails.cache individually. Each call produces a separate SQL query against solid_cache_entries:

• Fever → fever_data
• Reconciliation → reconciliation_hash, reconciliation_carriers
• Molecules → molecules.from_energy_keys, molecules.from_molecules_keys
• MeritOrder → merit_order_hash, hydrogen_hash, heat_network_lt_hash, heat_network_mt_hash, heat_network_ht_hash, agriculture_heat_hash

This data is entirely static between ETSource imports, yet it is fetched with individual Rails.cache.fetch calls on every request, producing ~11 SQL queries per GQL calculation.

Proposed solution

MeritOrder and Molecules are the two classes where batching is practical, they have multiple independent keys with uniform compute logic. Fever has a single key so there is nothing to batch. Reconciliation has a dependency between its two keys (reconciliation_hash depends on reconciliation_carriers) so they cannot be fetched in parallel.

For MeritOrder and Molecules, the individual Rails.cache.fetch calls are replaced with a single Rails.cache.fetch_multi that reads all keys for that class in one SQL query, computing and writing only missing keys. Thread.current is used to ensure fetch_multi runs at most once per request, since each MeritOrder caller instantiates a fresh object and Molecules methods may be called independently multiple times. Subsequent individual fetch calls within the same request hit the LocalStore (Rails' in-request memory layer populated by fetch_multi) at no additional cost.