Core Data Store

This reffers to the data store (data base) we use on the core part of the system (not necessarily the data store that all the collectors or data analizers will use).

Basic design principles

The most likely way to go here will by with a NoSQL solution. SQL-based datastores may be needed for specific components further "out" of the core system, but for the core speed and and the abillity to scale well and fast are of the essence.

Open issues

Software choice

• Cassandra - I'm using this in production in a work-related project and I like it for it's incredible write speed and simplicity. On the other hand it is simply a dumb store, with no structure associated with the datd it stores, so it will be an extra burden on the applications to use it;

• Redis - The all singing, all dancing, swiss army-like data store. I like it very much for it's features and the way it handles data (things like lists and sets and their associated native functions are amazingly useful). It also provides a messaging solution that might eventually be usefull for the core messaging service. Still need to read up on scalability, though;