Can I store event data with an expiration date in Redis?

[mizoR]

I always find it useful, thank you. I have a question.

Can I store event data with an expiration date in Redis? Or are you interested in being able to do that?

My system does not require a feed for inactive users. Therefore, I want to store the data with an expiration date in Redis.

By storing the data with expiration date, Redis with volatile-ttl enabled can free up memory with eviction.

So being able to set an expiration date on the data will be useful in my use case.

Thanks.

[kigster]

Actually. This is a good idea.

Can you propose an API change that would work for you?

[mizoR]

Thank you for your response.

For example, how about Redis::Provider accepting a validity period as follows:

 SimpleFeed.define(:newsfeed) do |f|
   f.provider   = SimpleFeed.provider(:redis,
                                       redis: -> { ::Redis.new },
+                                      expire_in: 60 * 60 * 24 * 7 * 2, # 2 weeks
                                       pool_size: 10)
   f.per_page   = 50     # default page size
   f.batch_size = 10     # default batch size
   f.namespace  = 'nf'   # only needed if you use the same redis for more than one feed
 end

At least in my use case, this alone will be so helpful, as I don't need to set a different validity period for each activity.

[rromanchuk]

I was just thinking about this to, and with the same resolution as @mizoR

My main reasoning, is to act as safety valve for volatile-lfu. I can rebuild feeds on the fly, so in the situation of a memory pressure event its not a big deal for these keys to start being evicted.

[rromanchuk]

This is kind of an interesting problem, but probably not that useful for most. My strategy would be something like on zadd set a very generous expires (1.year etc, who cares), and on reset_last_read, extend the initial ttl. You can't run expires on individual members, but in my case, i wouldn't want to anyways. The only reason i would do this is to hint to redis that these keys should be treated as volatile, and should be considered for eviction.

It also has the benefit of allowing me to be lazy and let redis perform "spring cleaning" on low value keys (feeds) for users that go dormant or have churned. Rebuilding on the fly is trivial, so persistence is not something i need redis to worry about with these keys.

[kigster]

I think you made a compelling argument that this is indeed useful.

However it looks like to implement it quite a few places must be changed. I am hoping to get to it shortly but if you'd rather take a stab at a PR I'd be happy to review and merge assuming it's approved and offers tests for this functionality.