runtime: add custom storage backend registration API

[alex60217101990]

Problem Statement

Organizations embedding OPA often need custom storage implementations that differ from the built-in inmem and disk backends. While performance optimization PRs targeting core storage are often rejected (as noted in #8245), there's currently no clean way to plug in custom storage backends when using OPA as a server via opa run.

Current State

OPA provides several extension points:

• runtime.RegisterPlugin() for custom plugins
• topdown.RegisterBuiltinFunc() for custom built-in functions
• bundle.RegisterStoreFunc() for bundle storage (used by rego, tester, sdk)
• runtime.Params.StoreBuilder for programmatic runtime storage configuration

However, there are gaps:

1. runtime.Params.StoreBuilder requires direct access to Params at runtime creation. This works for SDK users but not for CLI users running opa run, since you can't set a function pointer via command-line flags.

2. bundle.BundleExtStore (set via bundle.RegisterStoreFunc()) is used in specific contexts (v1/rego/rego.go:1400, v1/tester/runner.go:1179, v1/sdk/opa.go:187) but is not consulted by the main runtime storage initialization in v1/runtime/runtime.go:447-461:

switch {
case params.DiskStorage != nil:
    store, err = disk.New(ctx, logger, metrics, *params.DiskStorage)
case params.StoreBuilder != nil:
    store, err = params.StoreBuilder(ctx, logger, metrics, config, params.ID)
default:
    store = inmem.NewWithOpts(...)
}

Proposed Solution

Add a global storage backend registration mechanism similar to runtime.RegisterPlugin(), following the existing pattern:

package runtime

// StorageFactory creates a storage.Store instance
type StorageFactory func(
    ctx context.Context,
    logger logging.Logger,
    metrics prometheus.Registerer,
    config []byte,
    id string,
) (storage.Store, error)

// RegisterStorageBackend registers a custom storage factory.
// When called before creating a Runtime, the registered factory will be used
// instead of the default inmem/disk backends (unless overridden by Params.StoreBuilder).
func RegisterStorageBackend(factory StorageFactory) {
    registeredStorageMux.Lock()
    defer registeredStorageMux.Unlock()
    registeredStorage = factory
}

The runtime initialization logic would check for registered storage:

switch {
case params.DiskStorage != nil:
    store, err = disk.New(ctx, logger, metrics, *params.DiskStorage)
case params.StoreBuilder != nil:
    store, err = params.StoreBuilder(ctx, logger, metrics, config, params.ID)
case registeredStorage != nil:
    store, err = registeredStorage(ctx, logger, metrics, config, params.ID)
default:
    store = inmem.NewWithOpts(...)
}

Use Case

This allows embedding users to create custom main.go that registers storage before invoking the standard CLI:

package main

import (
    "github.com/open-policy-agent/opa/cmd"
    "github.com/open-policy-agent/opa/v1/runtime"
    "mycompany/optimized-storage"
)

func main() {
    runtime.RegisterStorageBackend(optimizedstorage.NewStore)

    // Now users can run: ./opa run -s ...
    // and get custom storage without modifying OPA core
    if err := cmd.RootCommand.Execute(); err != nil {
        os.Exit(1)
    }
}

Benefits

1. Preserves OPA core maintainability - keeps experimental/specialized storage out of the main repository
2. Follows established patterns - mirrors RegisterPlugin, RegisterBuiltin, etc.
3. Minimal implementation complexity - ~30 lines of code, no breaking changes
4. Enables innovation - teams can experiment with storage optimizations without waiting for upstream approval
5. Works with CLI - unlike Params.StoreBuilder, this approach works with opa run

Implementation Notes

• The registration should be optional and non-breaking (defaults to existing behavior)
• The precedence order should be: Params.StoreBuilder > DiskStorage > RegisterStorageBackend() > default inmem
• Thread-safety should follow the same pattern as registeredPluginsMux in v1/runtime/runtime.go:67
• Consider whether to unify this with bundle.RegisterStoreFunc() or keep them separate (the former is bundle-specific, this would be runtime-global)

Alternative Considered

Extending bundle.RegisterStoreFunc() to also affect runtime storage initialization. However, this conflates bundle storage with runtime storage, which may have different lifecycle and configuration requirements.

[srenatus]

EOPA uses the existing mechanisms to inject a custom store. Have you had a look there?

[alex60217101990]

Thanks for the EOPA reference! I examined their code and found that they use runtime.Params.StoreBuilder (cmd/run.go:315).

However, to set this field, EOPA overrides the entire run command implementation - their cmd/run.go replaces OPA's RunE handler and manually parses all CLI flags (~400 lines total). This works but requires maintaining significant duplicate logic.

The proposed RegisterStorageBackend() API would achieve the same result with ~10 lines of user code instead of 400+:

func main() {
    runtime.RegisterStorageBackend(customStorage.New)
    cmd.RootCommand.Execute()
}

While EOPA proves the functionality is achievable, I believe a registration API would significantly lower the barrier for teams needing custom storage while following OPA's established extensibility patterns (RegisterPlugin, RegisterBuiltin, etc.).

[anderseknert]

Organizations embedding OPA often need
...
performance optimization PRs targeting core storage are often rejected