-
Notifications
You must be signed in to change notification settings - Fork 365
Object Policy
By now you understand serializers and deserializers. And you understand how to achieve concurrency using multiple connections. So now its time to learn about YapDatabasePolicy rules. And how you can use them to achieve an extra performance boost.
Consider the following common scenario:
- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
__block IMMessage *message = nil;
// Access IMMessage on main thread (via uiConnection)
[uiConnection readWithBlock:^(YapDatabaseReadTransaction *transaction){
message = [ [transaction ext:@"view"] objectAtIndex:indexPath.row inGroup:conversationId];
}];
// configure and return cell...
}
- (void)didReceiveDeliveryReceiptForMessageId:(NSString *)messageId
{
// Update IMMessage on background thread (via bgConnection)
[bgConnection readWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){
IMMessage *message = [transaction objectForKey:messageId inCollection:@"messages"];
// As per the recommended best practice, we copy the object before modifying it.
message = [message copy];
message.delivered = YES;
[transaction setObject:message forKey:messageId inCollection:@"messages"];
}];
}In the example code above, an object is being updated on 'bgConnection', but it will immediately be needed on the 'uiConnection' in order to redraw the corresponding cell in the tableView. As you might imagine, this is a very common scenario.
So what exactly happens? That is, how does the 'IMMessage' object go from 'bgConnection' to 'uiConnection'?
The exact answer depends upon how YapDatabase is configured. Specifically, it depends on what objectPolicy is set. If you look in YapDatabaseConnection.h you'll see this:
typedef enum {
YapDatabasePolicyContainment = 0,
YapDatabasePolicyShare = 1,
YapDatabasePolicyCopy = 2,
} YapDatabasePolicy;
/**
* YapDatabase can use various optimizations to reduce overhead and memory footprint.
* The policy properties allow you to opt in to these optimizations when ready.
*
* The default value is YapDatabasePolicyContainment.
* It is the slowest, but also the safest policy.
* The other policies are faster, but require a little more work, and little deeper understanding.
**/
@property (atomic, assign, readwrite) YapDatabasePolicy objectPolicy;
@property (atomic, assign, readwrite) YapDatabasePolicy metadataPolicy;So if the databaseConnections are using the default objectPolicy (YapDatabasePolicyContainment), then here's what happens:
- The 'bgConnection' serializes the object and writes it to disk
- The 'uiConnection' then reads the serialized data from disk, and deserializes it
But YapDatabase can do better. You can do better. We can be much more efficient than this.
The default policy, YapDatabasePolicyContainment, is a policy to contain all objects to their connection. In this respect, it works conceptually like Core Data, where every NSManagedObject is tied to its NSManagedObjectContext.
An alternative to this is YapDatabasePolicyCopy. Here's how the above example plays out under YapDatabasePolicyCopy:
- The 'bgConnection' serializes the object and writes it to disk (same as before)
- The 'uiConnection' receives a copy of the object ([object copy]) automatically
- The 'uiConnection' does NOT need to re-read it from disk (or deserialize it) if the object was previously in the cache
When a read-write transaction completes, a change-set is posted internally to the other connections. This allows other connections to automatically update their cache. This is how connections manage to keep their cache in-sync as they move from commit to commit.
So if we use YapDatabasePolicyCopy, then we can move objects from connection to connection without forcing a re-read from disk.
There is still the memory overhead of having multiple copies of the object, one per connection. And if the object doesn't support the NSCopying protocol, then it falls back to YapDatabasePolicyContainment for that particular object. But the disk overhead is reduced, and that's the biggest bottleneck.
The policy settings can be configured for objects and metadata separately. And they can be configured for each connection separately.
The easiest way to configure your policy is to set it as the default policy when initializing your database. That way all connections inherit the correct policy automatically when they're created:
YapDatabase *database = [ [YapDatabase alloc] initWithPath:databasePath];
database.defaultObjectPolicy = YapDatabasePolicyCopy;
database.defaultMetadataPolicy = YapDatabasePolicyCopy;
// All connections going forward will default to YapDatabasePolicyCopy when created.You may have noticed in the example above that the code is copying objects before modifying them. This is the recommended practice from the Performance Primer article.
That is, other threads/connections have promised to never directly modify an object received from the database. Instead they promise to first make copies, and then modify the copy before writing it back to the database. Thus the updated object, once saved to the database, is thread-safe.
If this promise is kept (or if YapDatabase has cool tools to enforce this promise), then YapDatabase can give itself an additional performance boost by directly passing the updated object to other connections. No copies, and no re-reading from disk if the object was already in the connection's cache. Thus multiple connections can share the single instance of this object.
This is how YapDatabasePolicyShare works.
Continuing our example, here's how it plays out under YapDatabasePolicyShare:
- The 'bgConnection' serializes the object and writes it to disk (same as before)
- The 'uiConnection' receives the object automatically
- The 'uiConnection' does NOT need to re-read it from disk (or deserialize it) if the object was previously in the cache
This is similar to the copy policy, but without the overhead of making copies, or the memory cost of having multiple copies in RAM.
I'm a little confused still. What do you mean by "sharing objects between connections" ? Can you give an example?
Say there are 2 connections: connectionA and connectionB.
Both connectionA and connectionB have their own separate caches. And say both connections have, in their cache, the object for key @"abc123". Then connectionA executes a read-write transaction, makes a copy of said object, modifies it, and then saves it back into the database. What will happen is that, connectionB, when it updates to this latest commit, will internally process a "change-set" from connectionA. And in doing so, it will update the contents of its own cache. Specifically, it will replace the object for key @"abc123" with the object from the "change-set" that was passed from connectionA. Thus both connectionA and connectionB will have, in their separate caches, the same exact object instance for key @"abc123". (Both connections will have a reference (in their cache) to the same object in memory for key @"abc123".)
Of course, this policy can be a little "dangerous" if you start breaking "promises". Luckily YapDatabase has tools to help you keep your promise. Break your promise and it breaks your kneecaps throws an exception.
As noted, the recommended best practice is to makes copies of database objects before modifying them. There are multiple reasons this is consistently recommended:
- it's straight-forward and easy to do
- it allows all your database objects to be safely passed among threads
- it allows your database objects to be shared between connections
- it's enforceable using a few tricks
In addition to having a serializer & deserializer, YapDatabase supports a sanitizer:
typedef id (^YapDatabaseSanitizer)(NSString *key, id object);
- (id)initWithPath:(NSString *)path
serializer:(YapDatabaseSerializer)serializer
deserializer:(YapDatabaseDeserializer)deserializer
sanitizer:(YapDatabaseSanitizer)sanitizer;Whenever you invoke setObject:forKey:inCollection:, the sanitizer will automatically be run on the given object, before it enters the database system.
For a simple example, assume all objects you put into the database are NSString's. For example:
[databaseConnection readWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction){
NSString *fortune = [transaction objectForKey:fortuneCookieId inCollection:@"fortunes"];
NSMutableString *updatedFortune = [fortune mutableCopy];
[updatedFortune appendString:@" in bed"];
[transaction setObject:updatedFortune forKey:fortuneCookieId inCollection:@"fortunes"];
}];You'll notice that we're actually storing an NSMutableString. We could use the sanitizer to ensure that all string objects we store into the database are immutable. This ensures that we won't later forget about thread-safety, and start casting and mutating the string.
YapDatabaseSanitizer sanitizer = NULL;
#if DEBUG
sanitizer = ^(NSString *collection, NSString *key, id object){
if ([object isKindOfClass:[NSString class]])
return [object copy]; // NSMutableString => copy, NSString => retain
else
return object;
};
#endifYou'll notice that we only enabled the sanitizer for DEBUG builds. This is because we really only need this for debugging purposes. In production we can turn it off, confident that any mutation attempts would have failed during testing.
Of course, this was a simple example. In real life, you will likely be storing your own custom objects. And it may be overkill to have separate mutable vs immutable versions! Not to mention the extra work involved. All we really want to do is mark an object as immutable. And again, we really only need this for debugging purposes.
Here is a template you can use to mark custom objects as immutable.
Just have your custom objects extend this class instead of NSObject.
MyDatabaseObject.h
#import <Foundation/Foundation.h>
@interface MyDatabaseObject : NSObject <NSCopying>
@property (nonatomic, readonly) BOOL isImmutable;
- (void)makeImmutable;
- (NSException *)immutableExceptionForKey:(NSString *)key;
+ (NSMutableSet *)immutableProperties;
@endMyDatabaseObject.m
#import "MyDatabaseObject.h"
#import <objc/runtime.h>
@implementation MyDatabaseObject {
@private
BOOL isImmutable;
}
@synthesize isImmutable = isImmutable;
#pragma mark NSCopying
/**
* All copies are automatically mutable.
**/
- (id)copyWithZone:(NSZone *)zone
{
// You can optionally call this method via [super copyWithZone:zone].
//
// But since this method only sets isImmutable to NO (the default value),
// doing so is optional.
id copy = [[[self class] alloc] init];
((MyDatabaseObject *)copy)->isImmutable = NO;
return copy;
}
#pragma mark Logic
- (void)makeImmutable
{
if (!isImmutable)
{
// Set immutable flag
isImmutable = YES;
// Turn on KVO for object.
// We do this so we can get notified if the user is about to make changes to one of
// the object's properties.
//
// Don't worry, this doesn't create a retain cycle.
[self addObserver:self forKeyPath:@"isImmutable" options:0 context:NULL];
}
}
+ (BOOL)automaticallyNotifiesObserversForKey:(NSString *)key
{
if ([key isEqualToString:@"isImmutable"])
return YES;
else
return [super automaticallyNotifiesObserversForKey:key];
}
+ (NSSet *)keyPathsForValuesAffectingIsImmutable
{
// In order for the KVO magic to work, we specify that the isImmutable property is dependent
// upon all other properties in the class that should become immutable..
//
// The code below ** attempts ** to do this automatically.
// It does so by creating a list of all the properties in the class.
//
// Obviously this will not work for every situation.
// In particular:
//
// - if you have custom setter methods that aren't specified as properties
// - if you have other custom methods that modify the object
//
// To cover these edge cases, simply add code like the following at the beginning of such methods:
//
// - (void)recalculateFoo
// {
// if (self.isImmutable) {
// @throw [self immutableExceptionForKey:@"foo"];
// }
//
// // ... normal code ...
// }
return [self immutableProperties];
}
+ (NSMutableSet *)immutableProperties
{
// This method returns a list of all properties that should be considered immutable once
// the makeImmutable method has been invoked.
//
// By default this method returns a list of all properties in each subclass in the
// hierarchy leading to "[self class]".
//
// However, this is not always exactly what you want.
// For example, if you have any properties which are simply used for caching.
//
// @property (nonatomic, strong, readwrite) UIImage *avatarImage;
// @property (atomic, strong, readwrite) UIImage *cachedTransformedAvatarImage;
//
// In this example, you store the user's plain avatar image.
// However, your code transforms the avatar in various ways for display in the UI.
// So to reduce overhead, you'd like to cache these transformed images in the user object.
// Thus the 'cachedTransformedAvatarImage' property doesn't actually mutate the user object.
// It's just temporary. (Not stored to disk either.)
//
// So your subclass would override this method like so:
//
// + (NSMutableSet *)immutableProperties
// {
// NSMutableSet *immutableProperties = [super immutableProperties];
// [immutableProperties removeObject:@"cachedTransformedAvatarImage"];
//
// return immutableProperties;
// }
NSMutableSet *dependencies = nil;
Class rootClass = [MyDatabaseObject class];
Class subClass = [self class];
while (subClass != rootClass)
{
unsigned int count = 0;
objc_property_t *properties = class_copyPropertyList(subClass, &count);
if (properties)
{
if (dependencies == nil)
dependencies = [NSMutableSet setWithCapacity:count];
for (unsigned int i = 0; i < count; i++)
{
const char *name = property_getName(properties[i]);
NSString *property = [NSString stringWithUTF8String:name];
[dependencies addObject:property];
}
free(properties);
}
subClass = [subClass superclass];
}
return dependencies;
}
- (void)observeValueForKeyPath:(NSString *)keyPath
ofObject:(id)object
change:(NSDictionary *)change
context:(void *)context
{
// Nothing to do
}
- (void)willChangeValueForKey:(NSString *)key
{
if (isImmutable)
{
@throw [self immutableExceptionForKey:key];
}
[super willChangeValueForKey:key];
}
- (void)dealloc
{
if (isImmutable)
{
[self removeObserver:self forKeyPath:@"isImmutable" context:NULL];
}
}
- (NSException *)immutableExceptionForKey:(NSString *)key
{
NSString *reason;
if (key)
reason = [NSString stringWithFormat:@"Attempting to mutate immutable object. Class = %@, property = %@",
NSStringFromClass([self class]), key];
else
reason = [NSString stringWithFormat:@"Attempting to mutate immutable object. Class = %@",
NSStringFromClass([self class])];
NSDictionary *userInfo = @{ NSLocalizedRecoverySuggestionErrorKey:
@"To make modifications you should create a copy via [object copy]."
@" You may then make changes to the copy before saving it back to the database."};
return [NSException exceptionWithName:@"MyDatabaseObjectException" reason:reason userInfo:userInfo];
}
@endThe template class works rather simply. It observes changes to itself via KVO, and throws an exception if modifications are attempted on an object which has been marked immutable. For example:
#import "MyDatabaseObject.h"
@interface Car : MyDatabaseObject
@property (nonatomic, copy, readwrite) NSString *make;
@property (nonatomic, copy, readwrite) NSString *model;
@end
@implementation Car
@synthesize make;
@synthesize model;
@end
@implementation AppDelegate
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
Car *car = [[Car alloc] init];
car.make = @"Tesla";
car.model = @"Model S";
[car makeImmutable];
car.make = @"Ford"; // Throws exception: Attempting to mutate immutable object...
}So, using this base class, we can easily mark all of our objects as immutable when they go into the database. We can do this via the sanitizer. And we can similarly mark the objects as immutable when they're coming out of the database via the deserializer. And we can do all this conditionally, only for DEBUG builds.
YapDatabaseDeserializer deserializer;
YapDatabaseSanitizer sanitizer;
#if DEBUG
deserializer = ^(NSString *collection, NSString *key, NSData *data){
id object = [NSKeyedUnarchiver unarchiveObjectWithData:data];
if ([object isKindOfClass:[MyDatabaseObject class]])
[object makeImmutable];
return object;
};
sanitizer = ^(NSString *collection, NSString *key, id object){
if ([object isKindOfClass:[MyDatabaseObject class]])
[object makeImmutable];
return object;
};
#elif
deserializer = NULL; // Use default deserializer (NSCoding)
sanitizer = NULL; // Don't use a sanitizer
#endifAs noted in the comments of the template class, if you have custom setters then you'll need to add checks for immutability. This handles mostly every case except one: mutable ivars.
For example:
#import "MyDatabaseObject.h"
@interface Car : MyDatabaseObject
@property (nonatomic, copy, readwrite) NSString *make;
@property (nonatomic, copy, readwrite) NSString *model;
@property (nonatomic, strong, readwrite) NSMutableArray *passengers; // danger...
@end
@implementation Car
@synthesize make;
@synthesize model;
@synthesize passengers;
@end
@implementation AppDelegate
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
Car *car = [[Car alloc] init];
car.make = @"Tesla";
car.model = @"Model S";
car.passengers = [NSMutableArray arrayWithObjects:@"Harry", @"Hermione", nil];
[car makeImmutable];
[car.passengers addObject:@"Ron"]; // No exception!
}The solution is straight-forward: use immutable properties when possible. It's good defensive programming anyway, especially in a concurrent system.
Note: A copy operation on an immutable object just returns 'self'. That is, it doesn't actually copy any bytes.
#import "MyDatabaseObject.h"
@interface Car : MyDatabaseObject
@property (nonatomic, copy, readwrite) NSString *make;
@property (nonatomic, copy, readwrite) NSString *model;
@property (nonatomic, copy, readwrite) NSArray *passengers; // Fixed to be Thread-Safe
@end