Support increment/decrement

[adzap]

Any reason not to support these cache methods? I gather you don't use them so you left them out.

Wondering if you want a PR for it?

[skatkov]

I'm not a maintainer of this gem. but increment/decrement seems like standard interface for rails cache storage. It would be great to have it, just in case.

[dim]

PR would be nice :)

[adzap]

Here is the solution I've been using. Not efficient as I'd like, a read and a write, which could be done with an upsert with conflict adding old and new value.

# Increment an integer value in the cache.
def increment(name, amount = 1, options = nil)
  options = merged_options(options)
  entry = read_entry(name) || Entry.new(options.fetch(:initial, 0))
  new_entry = Entry.new(entry.value.to_i + amount)
  write_entry(name, new_entry, **options)
  new_entry.value
end

# Decrement an integer value in the cache.
def decrement(name, amount = 1, options = nil)
  options = merged_options(options)
  entry = read_entry(name) || Entry.new(options.fetch(:initial, 0))
  new_entry = Entry.new(entry.value.to_i - amount)
  write_entry(name, entry, **options)
  new_entry.value
end

[dim]

That's prone to races, I'm afraid. All DBs support atomic increments.

[skatkov]

AR comes with #increment_counter and #decrement_counter functions, the only problem with those is that they only offer a way to increment and decrement by the value of one.

#update_counters seems like a generic implementation for updating counters, that could be used here.

[adzap]

Yeah it's not great. But anything more needed a bit more plumbing. Ideally we need add an upsert-like atomic increment which uses a raw (non-Marshaled) value and inserts the delta or adds the the delta to existing value on conflict.

ActiveRecord has upsert in v6.0 and 6.1 but there is no on_duplicate option, which v7.0 has, to set how you handle conflicts. Which leaves us needing to use raw SQL as far as I can tell.

Unless there are other ideas.

[adzap]

The existing AR counter methods assume a record exists, which we cannot.

[dim]

We could use on_duplicate but I assume incrementing a Marshal.dump value with pure SQL could be difficult. Transactions maybe?

[adzap]

We can't use on_duplicate if you want to support < AR v7 though. We don't need/want to use marshal dump. There is a convention in AS Cache to support a raw value which is not dumped so that atomic operations can be performed on it at the cache server. Otherwise you need to return existing record to ruby land and unmarshal to perform a simple increment. Seems wasteful and slow. SQL looks like the best option short of backporting the on_duplicate option. Though I'm not sure of the consistency of the upsert SQL statements across database types.

…

On 28 Aug 2023 at 10:26 PM +1000, Dimitrij Denissenko ***@***.***>, wrote: We could use on_duplicate but I assume incrementing a Marshal.dump value with pure SQL could be difficult. Transactions maybe? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[adzap]

Here's a quick hack for what I'm thinking

key = 'test'
value = 1

binary_key   = @model.connection.escape_bytea(key)
binary_value = @model.connection.escape_bytea(value.to_s)

str  = <<~SQL
INSERT INTO "activesupport_cache_entries"
    ("key","value","version","expires_at","created_at")
VALUES
    ('%{key}', '%{value}', '%{version}', '%{expires_at}', '%{created_at}')
ON CONFLICT ("key")
DO 
  UPDATE SET "value"=decode((encode(activesupport_cache_entries.value, 'escape')::int + encode(excluded."value", 'escape')::int)::text, 'escape'), "version"=excluded."version","expires_at"=excluded."expires_at","created_at"=excluded."created_at"
RETURNING
  value
SQL

res = @model.connection.execute(str % { key: binary_key, value: binary_value, version: 1, expires_at: 10.minutes.from_now.to_s(:db), created_at: Time.current.to_s(:db) })
@model.connection.unescape_bytea(res.entries[0]['value']).to_i

[adzap]

The expiry is not honoured so that would need fixing.

[skatkov]

We can't use on_duplicate if you want to support < AR v7 though.

It's not yet released, but support for Rails 6 has been removed.
#35

[adzap]

Oh right. Well I need this for Rails 6.0 and 6.1 projects which are not going to be upgraded for a little while.

I'll go my own way I guess.

[dim]

Ooops, my bad, I assumed Rails 6.1 is now reached EOL too, but it's only 6.0. Will bring it back in a separate PR - https://endoflife.date/rails

[dim]

Ref: #37