Custom Extensions

It's extensible.

The key/value architecture of YapDatabase is just the foundation for an advanced plugin system. These plugins are called "extensions". YapDatabase ships with a number of helpful extensions. And, naturally, you can write your own.

Overview

Extensions are integrated deep into the architecture:

• they have access to sqlite internals
• they can create their own dedicated sqlite tables
• they take part in transactions
• they implement hook methods which are called whenever items are inserted, modified, removed, etc

In order to implement your own custom extension, you'll be overriding 3 base classes:

• YapDatabaseExtension
• YapDatabaseExtensionConnection
• YapDatabaseExtensionTransaction

All the methods that you need to override are private / internal. So they are listed & documented in the implementation files (.m not .h).

Hooks

The "meat & potatoes" of extensions are the hook methods. So it's instructive to start by listing & discussing these methods first, and then building up on top of this foundation.

If you consider the key/value nature of YapDatabase, the ReadWrite API is rather sparse. There is a method to set/update an object ([transaction setObject:forKey:inCollection:]). And a few methods to remove objects (such as [transaction removeObjectForKey:inCollection:]). And that's mostly it. This simplicity makes YapDatabase easy to use, and also simplifies life for the extension developer.

So let's start with the 2 most basic hook methods:

/**
 * YapDatabaseReadWriteTransaction Hook, invoked post-op.
 * Corresponds to [transaction setObject:object forKey:key inCollection:collection] &
 *                [transaction setObject:object forKey:key inCollection:collection withMetadata:metadata]
 * where the object is being inserted (value for collection/key does NOT exist at the moment this method is called).
**/
- (void)handleInsertObject:(id)object
          forCollectionKey:(YapCollectionKey *)collectionKey
              withMetadata:(id)metadata
                     rowid:(int64_t)rowid;

/**
 * YapDatabaseReadWriteTransaction Hook, invoked post-op.
 * Corresponds to [transaction setObject:object forKey:key inCollection:collection] &
 *                [transaction setObject:object forKey:key inCollection:collection withMetadata:metadata]
 * where the object is being updated (value for collection/key DOES exist, and is being updated/changed).
**/
- (void)handleUpdateObject:(id)object
          forCollectionKey:(YapCollectionKey *)collectionKey
              withMetadata:(id)metadata
                     rowid:(int64_t)rowid;

These 2 methods are defined for YapDatabaseExtensionTransaction, and you'll need to override them in your subclass. You'll notice that the hook methods pass you a bit of information that was discovered as the core layer was performing the operation. That is, whether or not this is an insert or an update. Your extension class may be able to take advantage of this knowledge, and optimize around it.

You'll also notice that the extension methods are passed a YapCollectionKey tuple object, and a rowid.

The YapCollectionKey class is a simple tuple class, and is used heavily within the YapDatabase internals. In particular, YapCollectionKey instances are used as the 'key' for the cache. So it ends up being much faster (and simpler) to pass around these immutable tuples, verses separate collection & key strings.

The rowid is the primary key of the row within sqlite. That is, if you inspect the 'database' table within the sqlite file, you'll see there is an INTEGER PRIMARY KEY rowid. There are a few implications to understand here (if you're not already familiar with sqlite):

• the rowid will never change (even if you vacuum the database)
• the rowid can be used to fetch all other information (including collection, key, object & metadata)
• its faster to fetch information from the database using the rowid, vs using just the collection/key

Basically, the collection/key is external facing information. But the rowid is what should be used internally. Thus, for developing extensions, you should likely be storing rowid's. Not only are they smaller (int64_t vs long string pairs), but they're faster.

Additionally, there are a LOT of heavily optimized methods that accept a rowid parameter. These were designed for internal use (which includes extensions). They're listed in YapDatabaseInternal.h. Here's a small sample. See the header for the full list.

- (YapCollectionKey *)collectionKeyForRowid:(int64_t)rowid;

- (BOOL)getCollectionKey:(YapCollectionKey **)collectionKeyPtr object:(id *)objectPtr forRowid:(int64_t)rowid;
- (BOOL)getCollectionKey:(YapCollectionKey **)collectionKeyPtr metadata:(id *)metadataPtr forRowid:(int64_t)rowid;

- (BOOL)getCollectionKey:(YapCollectionKey **)collectionKeyPtr
				  object:(id *)objectPtr
				metadata:(id *)metadataPtr
				forRowid:(int64_t)rowid;

- (BOOL)hasRowid:(int64_t)rowid;

- (BOOL)getObject:(id *)objectPtr
		 metadata:(id *)metadataPtr
 forCollectionKey:(YapCollectionKey *)collectionKey
		withRowid:(int64_t)rowid;

- (void)_enumerateKeysInCollection:(NSString *)collection
                        usingBlock:(void (^)(int64_t rowid, NSString *key, BOOL *stop))block;

// Many more methods like this...

Lifecycle

If you're familiar with the general YapDatabase architecture, then the 3 extension classes should be familiar as well.