Filtered Entity Dml Update Throws Collection was modified

[gokhanabatay]

Hi,
When entity has "Filter Condition" in mapping dml update throws Collection was modified; enumeration operation may not execute.

Mapping:

ApplyFilter<MemberConditionFilter>()

This query fails(async and sync test cases are implemented)

           await session.Query<TxnIssuer>()
                   .Where(x => guids.Contains(x.Guid))
                   .UpdateAsync(x => new TxnIssuer
                   {
                           MrcDailyMoved = "Y",
                           MrcDailyMovedDate = DateTime.Now
                    }).ConfigureAwait(false);

Unit Test Case 1: MultiThread_FilteredEntity_DmlStyleUpdateFails

Integration Test Case 1: MultiThread_DmlStyleQueryUpdateFails

I personally opened some issues about NHibernate Filters and some of them is related with dml style update with filters and I know those bugs are resolved. This issue could be regression.

Stack 1

InvalidOperationException
    Enumerator.MoveNextRare()
    Enumerator.MoveNext()
    BasicExecutor.Execute(QueryParameters parameters, ISessionImplementor session)
    QueryTranslatorImpl.ExecuteUpdate(QueryParameters queryParameters, ISessionImplementor session)
    HQLQueryPlan.PerformExecuteUpdate(QueryParameters queryParameters, ISessionImplementor session)
    SessionImpl.ExecuteUpdate(IQueryExpression queryExpression, QueryParameters queryParameters)
    AbstractQueryImpl2.ExecuteUpdate()
    DefaultQueryProvider.ExecuteDml[T](QueryMode queryMode, Expression expression)
    DmlExtensionMethods.ExecuteUpdate[TSource](IQueryable`1 source, Expression updateExpression, Boolean versioned)
    DmlExtensionMethods.Update[TSource](IQueryable`1 source, Expression`1 expression)

Stack 2

InvalidOperationException
    Enumerator.MoveNextRare()
    Enumerator.MoveNext()
    SelectListIterator`2.MoveNext()
    Enumerable.SelectManyIterator[TSource,TCollection,TResult](IEnumerable`1 source, Func`2 collectionSelector, Func`3 resultSelector)+MoveNext()
    Lookup`2.Create(IEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer)
    GroupedEnumerable`2.ToArray()
    Buffer`1.ctor(IEnumerable`1 source)
    OrderedEnumerable`1.GetEnumerator()+MoveNext()
    SelectIPartitionIterator`2.MoveNext()
    SelectManySingleSelectorIterator`2.ToArray()
    Enumerable.ToArray[TSource](IEnumerable`1 source)
    ParametersBackTrackExtensions.GetQueryParameterTypes(IEnumerable`1 parameterSpecs, IList`1 sqlQueryParametersList, ISessionFactoryImplementor factory)
    BasicExecutor.Execute(QueryParameters parameters, ISessionImplementor session)
    QueryTranslatorImpl.ExecuteUpdate(QueryParameters queryParameters, ISessionImplementor session)
    HQLQueryPlan.PerformExecuteUpdate(QueryParameters queryParameters, ISessionImplementor session)
    SessionImpl.ExecuteUpdate(IQueryExpression queryExpression, QueryParameters queryParameters)
    AbstractQueryImpl2.ExecuteUpdate()
    DefaultQueryProvider.ExecuteDml[T](QueryMode queryMode, Expression expression)
    DmlExtensionMethods.ExecuteUpdate[TSource](IQueryable`1 source, Expression updateExpression, Boolean versioned)
    DmlExtensionMethods.Update[TSource](IQueryable`1 source, Expression`1 expression)

Image:

[hazzik]

NHibernate is not and never was thread safe.

[gokhanabatay]

NHibernate is not and never was thread safe.

@hazzik really, did you look at test case different sessions/transactions must be thread safe? Its absolutely bug, could be related with filter improvments.

#1921 @fredericDelaporte

#2053 @maca88

#2298 @bahusoid

If there is no filters everything works fine.

[bahusoid]

Is it still reproducible if the following line is changed to:

List<long> guids = pagedGuids[i].ToList();

[gokhanabatay]

Is it still reproducible if the following line is changed to:

List<long> guids = pagedGuids[i].ToList();

Yes @bahusoid its still reproducible and below method already returns List[]
when you add .ToList() it’ unnecessary casting.

Just Copy TxnIssuer class and mapping with filter then you see test case fails.

List<long>[] pagedGuids = GroupBy(listOfGuids, 1);

[bahusoid]

and below method already returns List[]. when you add .ToList() it’ unnecessary casting.

My hunch was - it's related to closured variable. But if you really checked and it still fails - it's something else. I don't really have time to investigate it though..

[maca88]

This is a regression caused by #1931, I will provide a fix it.

[gokhanabatay]

Thank you @maca88 next time I will reproduce issue via NHibernate test cases.
The issue writing test case with NHibernate, FluentNHibernate mapping is not available.

After some research I finally done with Mapping By code.

[TestFixture]
public class Fixture : TestCaseMappingByCode
{
	protected override HbmMapping GetMappings()
	{
		var mapper = new ModelMapper();
		var filterDef = new FilterDefinition("MemberFilter",
			null, // or your default condition
			new Dictionary<string, IType> { { "MbrId", NHibernateUtil.Int16 } },
			false);
		cfg.AddFilterDefinition(filterDef);

		mapper.Class<Entity>(rc =>
		{
			rc.Id(x => x.Id, m => m.Generator(Generators.Native));
			rc.Property(x => x.Name);
			rc.Property(x => x.IsProcessed);
			rc.Property(x => x.MbrId);
			rc.Filter("MemberFilter", x => x.Condition("mbrid = :MbrId"));
		});
		return mapper.CompileMappingForAllExplicitlyAddedEntities();
	}

	protected override void OnSetUp()
	{
		using (var session = OpenSession())
		using (var transaction = session.BeginTransaction())
		{
			for (int i = 0; i < 500; i++)
			{
				var e1 = new Entity
				{
					Name = $"Abatay {i}",
					IsProcessed = 0,
					MbrId = 1
				};
				session.Save(e1);
			}
			transaction.Commit();
		}
	}

	protected override void OnTearDown()
	{
		using (var session = OpenSession())
		using (var transaction = session.BeginTransaction())
		{
			session.CreateQuery("delete from System.Object").ExecuteUpdate();
			transaction.Commit();
		}
	}

	[Test]
	//Multi Thread concurrent update fails when filter exists in entity mapping
	public void MultiThread_FilteredEntity_DmlStyleUpdate()
	{
		List<long> listOfGuids;
		using (var session = OpenSession())
		using (var transaction = session.BeginTransaction())
		{
			listOfGuids = session.Query<Entity>()
				.Select(x => x.Id)
				.ToList();

			transaction.Commit();
		}

		var pagedGuids = GroupBy(listOfGuids, 2);

		Parallel.For(1, 50, i =>
		{
			List<long> onePagedGuids = pagedGuids[i].ToList();
			UpdatePaged(onePagedGuids);
		});

		void UpdatePaged(List<long> guids)
		{
			using (var session = OpenSession())
			using (var transaction = session.BeginTransaction())
			{
				session.EnableFilter("MemberFilter").SetParameter("MbrId", 1);
				session.Query<Entity>()
					.Where(x => guids.Contains(x.Id))
					.Update(x => new Entity()
					{
						IsProcessed = 1
					});

				transaction.Commit();
			}
		}
	}

	public static List<T>[] GroupBy<T>(IEnumerable<T> items, int itemCount)
	{
		List<List<T>> pagedItems = new List<List<T>>();
		List<T> pageItem = new List<T>();
		foreach (var item in items)
		{
			if (pageItem.Count == itemCount)
			{
				pagedItems.Add(pageItem);
				pageItem = new List<T>();
			}

			pageItem.Add(item);
		}
		if (pageItem.Count > 0)
		{
			pagedItems.Add(pageItem);
		}

		return pagedItems.ToArray();
	}
}

[fredericDelaporte]

Fixed by #2712