[Feature Request] Multiple DB support

[TylorS]

I have been reading through the reddit post and a common thread I see is not taking to the use of mongodb. I'm not currently aware of the featureset of mongo currently in use, but I'd suspect the usage isn't too far away from the operations of other databases.

I feel creating an abstraction around Odin's database operations, in order to provide the ability to plug in implementations of that abstraction for different databases would be a net win and would cast a larger net on the community.

[theycallmemac]

I agree on this, I think abstraction around DB operations is the right way to go. It offer a larger potential for community customization. You're definitely right in saying that the usage isn't too far away from the operations of other databases, this will be a significant rewrite of some hardcoded MongoDB specific logic, but this is something which can also be abstracted over.

Adding this to the v2.0.0 milestone.

[theycallmemac]

Based on some chatter in the Discord today we have defined the following:

• When MongoDB goes down currently, Odin itself preserves the queue at the time of failure. Jobs cannot be added or removed from the queue at this time but execution still takes place. This, however, is not done in memory. If Odin itself was to fail and then restart, the queue would be lost until the database was running again.

In turn we have come up with the following plan of action for this issue:

• The user can plug in their database of choice. If said database fails, Odin resorts to Plan B - an in memory store of the job queue. This way, if Odin were also to fail and restart, the queue would be preserved in memory. When the database is back online, the database is synced with the in memory queue

[theycallmemac]

@hiepd has begun work on this - issue is being built on this branch.

[tmkontra]

Glad to see this on the roadmap.

@theycallmemac is the v2.0.0 requirement simply to provide the interface + mongoDB implementation, or should there be some more built-in implementations (the usual supects i.e. psql, redis, dynamodb)?

[theycallmemac]

@ttymck with v2.0.0 we are looking to provide abstraction which will allow for a wider range of data stores to be used with Odin, both nosql and sql based :)