[Enhancement]: Automatic Balance refill

[leondape]

What features would you like to see added?

I propose an automatic balance refiller.
It basically works as a cronjob which resets all users credits to a certain amount. I've already implemented it (though ugly).
Community, what do you say? Is that a feature you would like to see?

I also created two seperate "Levels":

• All users
• heavy users.

Heavy users can have more credits per day.
All that logic contributes to users to think about trying out new models and testing every model without thinking about the money they spend.

More details

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[rubentalstra]

I already have an idea for this but not using a cronjob.

because then we also update the users that are not that active. maybe do it during login and as well based on a new item when the last daily limit has reached and you send a chat it will then automatically check you balance and if it passes the old daily limit date time it will fill-up your balance with the daily limit.

something like this. is just an idea

[Odrec]

We also implemented this, although for all users equally. Basically this is our code that we ran each month to reset the balance. You run it inside the mongo container.

const batchSize = 500;
let operations = [];
let count = 0;

db.users.find().forEach(function(user) {
    operations.push({
        updateOne: {
            filter: { user: user._id },
            update: { $set: { tokenCredits: {{ librechat_balance }} } },
            upsert: true
        }
    });

    if (operations.length === batchSize) {
        db.balances.bulkWrite(operations);
        operations = [];
        count += batchSize;
        print(`Processed ${count} users...`);
    }
});

if (operations.length > 0) {
    count += operations.length;
    db.balances.bulkWrite(operations);
}

print("Balance set/updated for all users. Total: " + count);

[rubentalstra]

@Odrec uh? but this Is for everyone is same then? and you need to run it manually, and that's not the goal?

[Odrec]

@Odrec uh? but this Is for everyone is same then? and you need to run it manually, and that's not the goal?

We configured to run it as a service like this

[Unit]
Description=Set balance for LibreChat users

[Service]
Type=oneshot
ExecStart=docker compose exec mongodb mongosh LibreChat --file /scripts/set-balance.js
WorkingDirectory=/opt/librechat

and then we set the timer

[Unit]
Description=Run set-balance script monthly

[Timer]
OnCalendar=monthly
Persistent=true

[Install]
WantedBy=timers.target

But yeah is to set the balance equally to everyone.

[leondape]

I currently have the following workflow:

on container level

1. a cron.js file which uses node-cron.
2. when starting services it starts cron.js detached npm run backend & cron.js

inside container level

Cron.js runs on a specified time (in .env), im my case daily, and does the following, all inside the librechat container and having access to the npm scripts.

1. iterate over every user
2. get current balance
3. set every users balance to 0
4. add-balance to pre-defined amount. There are currently two tiers of users, so users who get the baseline, lets say 3$ and others who get 5$ based on their usage.

likes and dislikes

I see there are a LOT of unnessecary transaction going on there, especially for all the users who have not touched their balance.
We could do the following:

1. iterate over every user
2. get current balance and calculate difference to defined amount.
3. add missing balance to users which have a difference
4. This would not touch any user who hasn't consumed any credits.

[rubentalstra]

@leondape Thank you for sharing. But I prefer not doing it like this.

Here my toughest:

• have an extra data object what represents what there limit is:
• another extra data object with its retention period. When it needs to be reset, like every day or every 2 days or per week or per month.
• This refill should happen when it checks it balance so we don't need to add any cronjob and any other logic to keep it clean and only do this extra task when it's really needed on user bases
• When the user does not have enough balance and when the user tries again and its retention period is over and you send another chat it will top up its balance with the limit set by the "admin"

Issues to be solved:

• this new balance is fetched regularly. So when it reaches the data retention time of the user it will also top up. For the users that are active but not chatting so their balance will also update.

🤓

[leondape]

Hmmm. The solution seems a bit counterintuitive but I see the point.
I'm just asking myself: On one side we are adding complexity with new data objects. On the other hand we have the cron and need one additional maintenance script.
Regarding implementation complexity I don't see a big difference between the two solutions.

For seem reason I have the feeling that the cron is the more "natural" approach.

I am not sure.. what do you think?

[rubentalstra]

@leondape I'm sorry but I'm not going to implement a cron job here. Because it's definitely not the best way to do it. Imagine having 10.000 users and then updating it all at once somewhere as night?

System will say.............

So based on time and retention period it is. Then you eliminate all users that do not use it active.

[leondape]

I just saw your solution and aplaude. Very nicely done!
That is good engineering vs crobar craftsmanship :)

[rubentalstra]

@leondape #6452

works great 👌