Skip to content

[OBSOLETE] Don't download unnecessary records when syncing

stefffdev edited this page Dec 15, 2020 · 1 revision

UPDATE 2020-12-15:

Using this feature in production can be problematic, as it relies on a full history of all operations - making a cleanup of older operations impossible. It is not recommended to implement this feature, as the advantages do not outweigh the complications this implies.


When a client syncs changes to the server, the server marks these records as "changed" by setting the ServerUpdatedAt field, so that clients, that perform an incremental sync only download records changed since the last sync.

However, the client that just pushed those changes does not need to download this records, because it already has the latest version in the local storage.

Therefore which every sync a "InstallationId" is passed to the server, so that the server can be configured to only returns records that where changed by a different InstallationId.

The InstallationId is unique "per installation" and is stored in the clients database. We cannot use the user id for this, as one user could possibly install the same app on multiple devices and have different local caches on each device.

Setup the InstallationId on the server

The server has to store the InstallationId for every operation it processes:

[HttpPost]
public async Task<IActionResult> PostOperationsAsync(List<NubeOperation> operations)
{
    HttpContext.VerifyUserHasAnyAcceptedScope(_authentication.ScopeRequiredByApi);
    var userId = _authentication.GetUserIdentifier(User);
    var installationId = Request.GetInstallationId();

    try
    {
        operations.ForEach(o =>
        {
            o.UserId = userId;
            o.InstallationId = installationId;
        });

        await _operationService.ProcessOperationsAsync(_context, operations);
    }
    catch (Exception ex)
    {
        while (ex.InnerException != null)
        {
            ex = ex.InnerException;
        }

        return BadRequest(ex.Message);
    }

    return Ok();
}

When querying the database on GET requests the NubeOperationService can be utilized to filter the records before they are returned to the client:

[HttpGet]
public async Task<ActionResult<IEnumerable<Customer>>> GetItems(DateTimeOffset? laterThan)
{
    HttpContext.VerifyUserHasAnyAcceptedScope(_authentication.ScopeRequiredByApi);
    var userId = _authentication.GetUserIdentifier(User);
    var installationId = Request.GetInstallationId();

    string tableName = typeof(Customer).Name;

    if (laterThan.HasValue)
    {
        var allItems = await _context.Customers.Where(
            i => i.UserId == userId &&
            i.ServerUpdatedAt >= laterThan).ToListAsync();
        return allItems.Where(
            i => _operationService.LastChangedByOthers(_context, tableName, i.Id, installationId, laterThan.Value)).ToList();
    }
    else
    {
        return await _context.Customers.Where(i => i.UserId == userId).ToListAsync();
    }
}
Clone this wiki locally