Skip to content

Assert.That: fix Expression.Assign double-evaluation and assorted cleanups#8363

Open
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/assert-that-followup
Open

Assert.That: fix Expression.Assign double-evaluation and assorted cleanups#8363
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/assert-that-followup

Conversation

@Evangelink
Copy link
Copy Markdown
Member

Fix a single-pass guarantee violation for Expression.Assign (and compound assignments) plus a few related cleanups in Assert.That. Also addresses leftover review feedback from #8308.

Bug: Expression.Assign ran the RHS twice

Empirical repro on main:

static int Counter_Calls;
static int ComputeValue() { Counter_Calls++; return 42; }

// container.Inner.Value = ComputeValue() < 0   (fails)
Assert.That(lambda);
// Counter_Calls == 2   ← bug: ComputeValue ran twice

Root cause

The bottom-up evaluator walked Left and Right, cached the RHS value, then went through the generic "rebuild and compile" path. That replaces children with constants — but the Left of an Assign is a writable target. The rebuilt Assign(Constant, Constant) was invalid, the outer try/catch sentineled it, and the parent expression's later rebuild fell back to invoking the original assignment from scratch — running the RHS side effects a second time.

The same hazard affects every compound assignment (AddAssign, MultiplyAssign, …) and the unary assignment forms (PreIncrementAssign, PostIncrementAssign, …).

Fix

Special-case assignment node types in EvaluateAllSubExpressions:

  1. Walk Left to capture diagnostics and cache any side-effecting sub-children.
  2. Walk Right exactly once.
  3. Rebuild Left via ReplaceSubExpressionsWithConstants so the writable wrapper (MemberExpression / IndexExpression / ParameterExpression) is preserved while its sub-children become constants. This also fixes double-evaluation of side-effecting Left sub-children (e.g., provider.GetBox().Value = 42).
  4. Substitute Right with its cached constant and invoke the rebuilt assignment exactly once.

RequiresSinglePassEvaluation now also recognizes assignment node types, so a compound assignment whose only side effect is the assignment itself can't slip into the fast path and double-execute.

Cleanups

  • Removed dead IsSimpleExpression / IsVariableReferenceIsSimpleExpression only returned true for ConstantExpression, which is handled before it's called. The cached-value lookup branch in GetIndexArgumentDisplay was unreachable. Simplified GetIndexArgumentDisplay and dropped its unused evaluationCache parameter. Removed the orphaned TranslateFailureSentinel helper as well.
  • Cached the netstandard/net462 Regex instances for the display-class cleanup and list-init cleanup pipelines (previously a fresh compiled Regex was allocated per failed assertion). On NET, list-init patterns are now GeneratedRegex instead of inline string patterns.
  • Reduced substring allocations in RemoveAnonymousTypeWrappers / TryMatchListInitPattern via a new non-allocating HasSubstringAt(input, start, value) helper backed by string.CompareOrdinal + bounds.
  • Simplified IsFuncOrActionType to call GetGenericTypeDefinition() at most once.

PR #8308 review feedback

The Copilot AI review left two comments on #8308 that landed after merge:

  1. "This new test method is missing a test attribute" — Not actionable: the file uses TestContainer, which treats any public parameterless method as a test (test/Utilities/TestFramework.ForTestingMSTest/TestContainer.cs:6-11). No attribute needed.
  2. "...sub-expressions are not object-typed" — Already addressed by the rename to That_TwoCallsReturningSameMutatedReference_DeduplicatesInDetails in the merged version. The test exercises the actual scenario its name describes (reference-identity dedup after mutation) regardless of static type.

Tests

Added three regression tests:

  • That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce — the main bug.
  • That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure — exercises the RequiresSinglePassEvaluation fix for compound assignments without other side-effecting nodes.
  • That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce — exercises the ReplaceSubExpressionsWithConstants Left-rebuild fix for side-effecting receivers.

All 1,157 TestFramework.UnitTests pass on net48 / net8.0 / net9.0.

…anups

Fixes a single-pass guarantee violation for `Expression.Assign` (and compound

assignment forms): the RHS of the assignment was evaluated once during the

bottom-up walk, but the generic 'rebuild and compile' path then produced an

invalid `Assign(Constant, Constant)` that the outer catch sentineled, and the

parent expression's later rebuild re-invoked the original assignment from

scratch — running the RHS side effects a second time.

Concretely:

- `EvaluateAllSubExpressions` now special-cases assignment node types

  (`Assign` and the compound forms `AddAssign` / `SubtractAssign` / etc.,

  plus the unary `PreIncrementAssign` / `PostIncrementAssign` family).

  It walks Left to capture diagnostics, walks Right exactly once, then

  rebuilds via `ReplaceSubExpressionsWithConstants` so the writable wrapper

  node (`MemberExpression` / `IndexExpression` / `ParameterExpression`) is

  preserved while its sub-children become constants. This also avoids double

  evaluation of side-effecting Left sub-children (e.g.,

  `provider.GetBox().Value = 42`).

- `RequiresSinglePassEvaluation` now recognizes assignment node types so

  expressions containing assignments always take the single-pass evaluator,

  closing a fast-path leak where an assignment with constant RHS would run

  once in the fast path and again on the failure-diagnostic path.

Other cleanups:

- Removed dead `IsSimpleExpression` and `IsVariableReference` (every case in

  `IsSimpleExpression` returned `false` for any input that reached it, so

  the cached-value lookup branch in `GetIndexArgumentDisplay` was

  unreachable). Simplified `GetIndexArgumentDisplay` and dropped its unused

  `evaluationCache` parameter. Removed the orphaned `TranslateFailureSentinel`

  helper as well.

- Cached the netstandard/net462 `Regex` instances for the display class

  cleanup and list-init cleanup pipelines (previously a fresh compiled

  `Regex` was allocated per failed assertion). On NET, the list-init

  patterns are now `GeneratedRegex` instead of inline string patterns.

- Reduced substring allocations in `RemoveAnonymousTypeWrappers` and

  `TryMatchListInitPattern` via a new `HasSubstringAt` helper that uses

  `string.CompareOrdinal` with bounds.

- Simplified `IsFuncOrActionType` to call `GetGenericTypeDefinition()` at

  most once.

Added regression tests:

- `That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce` — the main bug.

- `That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure` — exercises

  the `RequiresSinglePassEvaluation` fix for compound assignments without

  any other side-effecting nodes.

- `That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce` —

  exercises the `ReplaceSubExpressionsWithConstants` Left rebuild.

All 1,157 TestFramework.UnitTests pass on net48 / net8.0 / net9.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 19, 2026 15:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Assert.That’s single-pass evaluation guarantee for assignment expressions (Expression.Assign, compound assignments, and unary assignment forms) by special-casing how those nodes are evaluated and rebuilt so side effects aren’t accidentally re-executed during diagnostic extraction. It also includes targeted cleanups in the diagnostic rendering pipeline and adds regression tests to lock in the corrected behavior.

Changes:

  • Special-cases assignment expression node types in EvaluateAllSubExpressions and RequiresSinglePassEvaluation to prevent invalid rebuilds and double execution.
  • Simplifies index-argument display formatting and removes dead/unreachable helper logic.
  • Optimizes diagnostic text cleanup via cached/source-generated regex and reduced substring allocations; adds regression tests for assignment single-evaluation.
Show a summary per file
File Description
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.That.cs Adds regression tests covering RHS single-evaluation, compound assignment single-application, and side-effecting receiver evaluation.
src/TestFramework/TestFramework/Assertions/Assert.That.cs Implements assignment-aware single-pass evaluation and diagnostic cleanup performance improvements.

Copilot's findings

Comments suppressed due to low confidence (1)

src/TestFramework/TestFramework/Assertions/Assert.That.cs:278

  • Similarly for unary assignment nodes, EvaluateAllSubExpressions(unaryExpr.Operand, cache) evaluates the operand expression (including property/indexer getters) and then the rebuilt Pre/PostIncrementAssign will evaluate it again as part of its semantics. This can duplicate side effects on writable operands. A dedicated walk that evaluates only receiver/index-argument subexpressions (but not the writable wrapper itself) would avoid double getter execution.
                case UnaryExpression unaryExpr when IsAssignmentUnaryNodeType(unaryExpr.NodeType):
                    EvaluateAllSubExpressions(unaryExpr.Operand, cache);
                    Expression rebuiltUnaryOperand = ReplaceSubExpressionsWithConstants(unaryExpr.Operand, cache);
                    UnaryExpression rebuiltUnary = unaryExpr.Update(rebuiltUnaryOperand);
                    cache[unaryExpr] = Expression.Lambda(rebuiltUnary).Compile().DynamicInvoke();
                    return;
  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment on lines +244 to +251
case BinaryExpression binaryExpr when IsAssignmentBinaryNodeType(binaryExpr.NodeType):
EvaluateAllSubExpressions(binaryExpr.Left, cache);
EvaluateAllSubExpressions(binaryExpr.Right, cache);
Expression rebuiltAssignLeft = ReplaceSubExpressionsWithConstants(binaryExpr.Left, cache);
Expression rebuiltAssignRight = ReplaceChildWithConstant(binaryExpr.Right, cache);
BinaryExpression rebuiltAssign = binaryExpr.Update(rebuiltAssignLeft, binaryExpr.Conversion, rebuiltAssignRight);
cache[binaryExpr] = Expression.Lambda(rebuiltAssign).Compile().DynamicInvoke();
return;
Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# Dimension Verdict
13 Test Completeness 🔴 BLOCKING — 3 tests missing [TestMethod]
10 Test Isolation 🟡 MAJOR — static mutable CountingComputeValue.Calls

✅ 19/21 dimensions clean.

  • Test Completeness (#13)That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce, That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure, and That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce all lack [TestMethod]. MSTest will never discover them; the three regression scenarios they document are completely unguarded in CI.
  • Test Isolation (#10)CountingComputeValue.Calls is a non-atomic static mutable field. Reset (= 0) and assertion (Should().Be(1)) are not atomic under parallel execution. Prefer an instance-based counter or Interlocked operations.

The production-side changes (assignment special-casing in EvaluateAllSubExpressions, RequiresSinglePassEvaluation, HasSubstringAt, regex caching) look correct and well-reasoned.

Generated by Expert Code Review (on open) for issue #8363 · ● 19.5M

// path. The fast path ran the assignment once, then EvaluateAllSubExpressions ran it
// again on the failure-diagnostic path. RequiresSinglePassEvaluation now recognizes
// assignment node types so these expressions always take the single-pass evaluator.
public void That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure()
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKING] Test Completeness — Missing [TestMethod]

That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure is also missing the [TestMethod] attribute — it will never be discovered or run by MSTest.

// produced an invalid Assign(Constant, Constant) that the catch sentineled, and the
// parent LessThan was re-invoked from the original tree — re-running ComputeValue a
// second time. Assign is now special-cased so the RHS runs exactly once.
public void That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce()
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKING] Test Completeness — Missing [TestMethod]

This method has no [TestMethod] attribute (same applies to That_CompoundAssignmentOnField_AppliesExactlyOnceOnFailure and That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce).

Without [TestMethod], MSTest will never discover or execute any of the three new regression tests. The regressions they guard against are completely unprotected — reverting the production fix in Assert.That.cs would not be caught by CI.

Recommendation: Add [TestMethod] above each of the three method declarations:

[TestMethod]
public void That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce()

// rebuild kept the original (unevaluated) Left in place. The Left's sub-children are
// now substituted with cached constants while the writable MemberExpression wrapper is
// preserved, so the receiver runs exactly once.
public void That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce()
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKING] Test Completeness — Missing [TestMethod]

That_AssignToFieldOfSideEffectingReceiver_EvaluatesReceiverExactlyOnce is also missing the [TestMethod] attribute — it will never be discovered or run by MSTest.

// Receiver method (GetBox) must run exactly once.
provider.Calls.Should().Be(1);
provider.Box.Value.Should().Be(42);
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Test Isolation — Static mutable counter

CountingComputeValue.Calls is a static mutable field reset with Calls = 0 inside That_ManuallyConstructedAssign_EvaluatesRhsExactlyOnce. If this test class ever runs with parallelism enabled (e.g., via [Parallelize] at assembly level or via a future .runsettings change), the reset and the assertion Calls.Should().Be(1) are not atomic, and another test's Get() call could corrupt the count.

Also, Calls++ is a non-atomic read-modify-write; concurrent increments (even from a single test if the expression tree were ever evaluated on multiple threads) could silently undercount.

Recommendation: Move the counter to an instance field, or use Interlocked.Increment and Interlocked.Exchange for the increment and reset:

// Reset
Interlocked.Exchange(ref CountingComputeValue.Calls, 0);
// Increment
Interlocked.Increment(ref CountingComputeValue.Calls);

Or, simpler, make CountingComputeValue non-static and instantiate it fresh per test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants