VirtualPathContainer logs "trying to remove non-child" warnings under high Ask load

[Aaronontheweb]

Describe the bug

Under high load with many concurrent Ask operations, the /temp actor (a VirtualPathContainer) logs numerous warnings:

{0} trying to remove non-child {1}

Where {0} is akka://[SystemName]/temp and {1} is the temporary actor name (e.g., 71R).

The underlying behavior is correct - the temporary actor is properly cleaned up. The issue is that we're logging a Warning for what is expected, benign behavior in a concurrent data structure.

Environment

• Akka.NET version: 1.5.58
• Observed during: Load testing

Evidence

The logs show:

• Pattern: akka://OperatorAkka/temp trying to remove non-child
• Level: Warning
• LogSource: LocalActorRefProvider(akka://OperatorAkka)
• Multiple occurrences during load test

Analysis

The /temp container uses a ConcurrentDictionary and is specifically designed for high-concurrency access without an ActorCell:

https://github.com/akkadotnet/akka.net/blob/dev/src/core/Akka/Actor/ActorRef.cs#L799

The warning is logged when TryRemove returns false:

https://github.com/akkadotnet/akka.net/blob/dev/src/core/Akka/Actor/ActorRef.cs#L877-L884

This can happen when:

1. Multiple code paths call Stop() on the same PromiseActorRef concurrently
2. The actor was never registered (path was never queried before stop)

In both cases, this is fine - the actor is gone, which is the desired outcome. TryRemove returning false just means "already removed" - not an error condition.

The Real Problem

We're logging a Warning for expected behavior in a lock-free concurrent data structure, which:

• Alarms users during load testing
• Creates log noise
• Suggests something is wrong when it isn't

Suggested Fix

Change the log level from Warning to Debug (or remove entirely) in VirtualPathContainer.RemoveChild:

public void RemoveChild(string name)
{
    if (!_children.TryRemove(name, out _))
    {
        // This is expected under concurrent access - the actor is already gone
        Log.Debug("{0} trying to remove non-child {1}", Path, name);
    }
}

Additional Context

• Customer reports this was not observed on production with older Akka.NET versions, but likely just wasn't hitting the concurrency threshold to trigger it frequently
• The /temp actor is intentionally designed for concurrent modifications for Ask performance