Enable rolling upgrade from LocalGrainDirectory to DistributedGrainDirectory

[ReubenBond]

This PR enables basic support for rolling upgrades from the legacy DHT-based LocalGrainDirectory to the DistributedGrainDirectory (enabled by calling ISiloBuilder.AddDistributedGrainDirectory()). When performing an upgrade, users must follow the procedure detailed under the 'Rolling Upgrade Path' header.

There are caveats to this rolling uprade support. In particular, there will be inconsistency during the rollout: there can be duplicate activations created for a given grain, since different hosts have different ideas of what the correct directory for a given grain is. After the rollout is complete and all hosts are running the DistributedGrainDirectory, there is no more chance for inconsistency and there will be no orphaned (unregistered) or duplicate activations.

A different strategy could include attempts to reduce the chance for inconsistency during the rollout window. For now, that is not something I intend to pursue since it greatly increases complexity for dubious benefit.

Changes

• Added DelegatingRemoteGrainDirectory - A system target that handles IRemoteGrainDirectory requests from old silos still using LocalGrainDirectory. This allows old silos to communicate with new silos during a rolling upgrade. It implements the same grain types (Constants.DirectoryServiceType, Constants.DirectoryCacheValidatorType) that LocalGrainDirectory uses.
• Changed service registration - DistributedGrainDirectory and DirectoryMembershipService are now registered on all silos by default (not just when explicitly enabled). This ensures IGrainDirectoryClient is available for recovery queries during rolling upgrades. When AddDistributedGrainDirectory() is called:
  • Removes LocalGrainDirectory lifecycle participation
  • Swaps IGrainLocator from DhtGrainLocator to CachedGrainLocator
  • Registers the delegating system targets
• Removed LocalGrainDirectoryPartition from DI - It's now created directly by LocalGrainDirectory instead of being injected.
• Added UseTestClusterGrainDirectory option to InProcTestClusterOptions for testing.
• Added GrainDirectoryMigrationTests - Tests covering rolling upgrade scenarios.

Rolling Upgrade Path

To take advantage of this new functionality, you will need to perform two complete rollouts. The first adds support for the new directory on all hosts. The second switches to it by default.

1. Rollout 1: Upgrade all hosts to Orleans v10.0.0.
2. Rollout 2: Add a call to AddDistributedGrainDirectory() in your silo configuration.

• Old silos send IRemoteGrainDirectory messages to new silos, which are handled by DelegatingRemoteGrainDirectory.
• New silos use DistributedGrainDirectory for all directory operations.
• You may see duplicate activations during the rollout.

Fixes #9356

[Copilot]

Pull request overview

This PR enables rolling upgrades from the legacy DHT-based LocalGrainDirectory to the new DistributedGrainDirectory. The change ensures that old silos using LocalGrainDirectory can communicate with new silos using DistributedGrainDirectory during a rolling upgrade.

Changes:

• Added DelegatingRemoteGrainDirectory system targets to handle IRemoteGrainDirectory requests from old silos
• Changed service registration to register DistributedGrainDirectory and DirectoryMembershipService on all silos by default
• Added test infrastructure option UseTestClusterGrainDirectory and comprehensive migration tests

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Orleans.Runtime/GrainDirectory/DelegatingRemoteGrainDirectory.cs	New system target that delegates IRemoteGrainDirectory calls from old silos to DistributedGrainDirectory
src/Orleans.Runtime/Hosting/CoreHostingExtensions.cs	Updated AddDistributedGrainDirectory to remove LocalGrainDirectory lifecycle participation and register delegating system targets
src/Orleans.Runtime/Hosting/DefaultSiloServices.cs	Changed to register DistributedGrainDirectory on all silos and moved IGrainLocator registration
src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs	Updated constructor to create LocalGrainDirectoryPartition directly instead of via DI
src/Orleans.Runtime/GrainDirectory/LocalGrainDirectoryPartition.cs	Changed constructor to accept ILogger instead of ILoggerFactory
src/Orleans.Runtime/GrainDirectory/GrainDirectoryHandoffManager.cs	Removed unused Factory dependency
src/Orleans.Runtime/GrainDirectory/GrainLocatorResolver.cs	Changed to use IGrainLocator instead of DhtGrainLocator directly
src/Orleans.Runtime/GrainDirectory/GrainDirectoryPartition.cs	Fixed indentation and changed logging level from Error to Warning for transient failures
src/Orleans.Runtime/GrainDirectory/DistributedGrainDirectory.cs	Made _localActivations readonly and reordered fields
src/Orleans.Runtime/GrainDirectory/CachedGrainLocator.cs	Sealed the class
src/Orleans.Runtime/Core/InternalGrainRuntime.cs	Removed ILocalGrainDirectory dependency
src/Orleans.Core/Configuration/ServiceCollectionExtensions.cs	Added TaggedServiceDescriptor to track and remove service registrations by implementation type
src/Orleans.TestingHost/InProcTestClusterOptions.cs	Added UseTestClusterGrainDirectory property
src/Orleans.TestingHost/InProcTestClusterBuilder.cs	Set UseTestClusterGrainDirectory default to true
src/Orleans.TestingHost/InProcTestCluster.cs	Added validation and conditional registration of test grain directory
test/TesterInternal/LocalGrainDirectoryPartitionTests.cs	Updated test to use NullLogger instead of LoggerFactory and renamed class
test/TesterInternal/GrainDirectory/GrainDirectoryMigrationTests.cs	New comprehensive test suite for rolling upgrade scenarios