Securing the sharding data

A developer created a multi-tenant application that uses sharding and, by mistake he deleted the FileStore Cache file (see issue #115 how that happens). At that point he lost the sharding information and he couldn't get it back. While the deletion of a FileStore Cache file is very rare, the consequences of loosing the sharding information on an active application would be very bad. Therefore I added code in AuthP version 8.1.0 a new feature to backup the sharding information, plus some features to ensure the sharding information is correct. This page shows what you need to do to use this new feature.

NOTE: You may ask why I use the FileStore Cache if it has this problem?. Its all about performance. Every access to the tenant's data needs the specific sharding data for that tenant, and typically more that 90% of all the requests needs the sharding data. In terms of speed the FileStore Cache get the sharding in ~25 ns, while using a SQL Server distributed cache would take at least 0.1 ms – that means FileStore cache is >4,000 faster.

Here are the sections in this page:

• Overview of the "Secure your sharding data" feature
• What the CheckTwoShardingSources method does
• Things to do if there are errors

Overview of the "Secure your sharding data" feature

The solution in AuthP version 8.1.0 adds a new database called ShardingEntryBackup (referred to as 'ShardingBackup database') to the AuthP's AuthPermissionsDbContext and extra code is added to the GetSetShardingEntriesFileStoreCache so any change to a sharding information in the the FileStore Cache will also update the database ShardingEntryBackup.

For new multi-tenant applications built with AuthP version 8.1.0 the backup is automatic, but if you are updating an existing multi-tenant application, then you need to run the method called CheckTwoShardingSources in the GetSetShardingEntriesFileStoreCache which will copy the existing FileStore Cache's sharding data to the ShardingEntryBackup. See the next section for how you do this.

NOTE: If you want see how to add the 'Check' feature to your application, then look at either the Example6 or Example7 project - the code are the same in the two Examples. Most of the code is in their Controller.ShardingController class.

What the CheckTwoShardingSources method does

When the CheckTwoShardingSources method is run it goes through four stages. The first three stages make the two sharding sources, i.e FileStore Cache and the ShardingBackup database, have sharding data in them. The final stage checks that the two sources are the same. The four stages have names and the list below describes what they do:

1. EMPTY-OK: no sharding data found

The EMPTY-OK stage is run when both sharding sources are empty. This happens when there are no sharding databases (apart of the default database if the HybridMode option is true). For instance, the first deploy of the application there are no tenants, so there wouldn't any sharding entries. This stage returns an success message, starting with "EMPTY-OK".

2. BACKUP-SHARDINGS: The ShardingBackup database needs to be filled

The BACKUP-SHARDINGS stage is run when the FileStore Cache has entries, but the ShardingBackup database is empty. This happens when you have updated to AuthP version 8.1.0 or above and it will copy over the shardings from the FileStore Cache into the ShardingBackup database. Once this stage is had executed you have secured your sharding data.

NOTE: once your is updated to AuthP version 8.1.0, then any change to a sharding are written to both the FileStore Cache AND the ShardingBackup database.

3. RESTORE-SHARDINGS: The FileStore Cache is restored

The RESTORE-SHARDINGS stage is run when the FileStore Cache is empty, but the ShardingBackup database had . This happens when the FileStore Cache file is deleted (this was the problem the developer had), and it copy the data from the ShardingBackup database into the FileStore Cache.

NOTE: For examples of using the 'Check' feature see Example6 project (hybrid mode) and Example7 (ShardingOnly mode). Most of the code is in their Controller.ShardingController classes.

4. CHECK-SHARDINGS: checks the sharding data

If the first three stages aren't triggered, then the sharding entries in both the two sharding sources should have the same data. So this stage checks that the properties in each sharding entry are the same in both sharding sources.

NOTE: It is very rare that the two sharding sources could be different, but it could happen. For instance, if the FileStore Cache update works but the ShardingBackup database update has an error. Therefore I added this check which provides details information on any differences.

Things to do if there are errors