TestPlan migrator uses TARGET work item ID to locate SOURCE Work Item

[sonofhammer]

{
      "$type": "TestPlansAndSuitesMigrationConfig",
      "Enabled": true,
      "PrefixProjectToNodes": false,
      "OnlyElementsWithTag": null,
      "TestPlanQueryBit": "PlanName IN ('Regression Plan 3.6')",
      "RemoveAllLinks": false,
      "MigrationDelay": 0,
      "UseCommonNodeStructureEnricherConfig": false,
      "NodeBasePaths": [],
      "AreaMaps": {
        "^OldProj.*": "NewProj\\",
      },
      "IterationMaps": {
        "^OldProj.*": "NewProj\\Descoped",
      },
      "RemoveInvalidTestSuiteLinks": false,
      "FilterCompleted": false
    }

Relevant log output

[14:43:29 INF] [v14.3.4] == Plan[1/1] Suite[0/0] Cases[0/0]  - planid[190209] | ApplyConfigurationsAndAssignTesters Regression Plan 3.6
[14:43:30 DBG] [v14.3.4] TfsWorkItemMigrationClient::GetWorkItem(12402)
[14:43:30 FTL] [v14.3.4] Error while running TestPlansAndSuitesMigrationContext
System.NullReferenceException: Object reference not set to an instance of an object.
   at VstsSyncMigrator.Engine.TestPlansAndSuitesMigrationContext.ProcessTestPlan(ITestPlan sourcePlan) in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\TestPlansAndSuitesMigrationContext.cs:line 917
   at VstsSyncMigrator.Engine.TestPlansAndSuitesMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\TestPlansAndSuitesMigrationContext.cs:line 137
   at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools\_EngineV1\Processors\MigrationProcessorBase.cs:line 46
[14:43:30 ERR] [v14.3.4] TestPlansAndSuitesMigrationContext The Processor MigrationEngine entered the failed state...stopping run

the second fetch of the target plan here

https://github.com/nkdAgility/azure-devops-migration-tools/blob/2c5de98077fbf8ca85cc10bd3121bad97a35f310/src/VstsSyncMigrator.Core/Execution/MigrationContext/TestPlansAndSuitesMigrationContext.cs#L915C1-L916C1

Isn't working, which results in the error you see logged above.

It looks like the sourceWI in

        private ITestPlan FindTestPlan(TestManagementContext tmc, string name)
        {
            ITestPlan testPlan = (from p in tmc.Project.TestPlans.Query("Select * From TestPlan") where p.Name == name select p).SingleOrDefault();
            if (testPlan != null)
            {
                //Check test plan is in fact the right one
                var sourceWI = Engine.Source.WorkItems.GetWorkItem(testPlan.Id.ToString());
                string expectedReflectedId = Engine.Source.WorkItems.CreateReflectedWorkItemId(sourceWI).ToString();
                string workItemReflectedId = (string)sourceWI.Fields[Engine.Target.Config.AsTeamProjectConfig().ReflectedWorkItemIDFieldName].Value;
                if (workItemReflectedId != expectedReflectedId)
                {
                    testPlan = null;
                }
            }
            return testPlan;
        }

Is using testPlan.Id of the TARGET TestPlan it just found instead of the Source Test plan, Resulting in a mismatch of workItem Id's.

This also applies to re-running the tool,. FindTestPlan can't find the test plan and a duplicate test plan gets created.

Debug in Visual Studio

• Visual Studio Debug

[MrHinsh]

This is 100% the correct behaviour.

var sourceWI = Engine.Source.WorkItems.GetWorkItem(testPlan.Id.ToString());

This line gets the WorkItem from the Source so that we can validate that the target item is the correct item.

string expectedReflectedId = Engine.Source.WorkItems.CreateReflectedWorkItemId(sourceWI).ToString();

We then use the source work item to create the reflected work item id string. This is a string that uniquely identifies the Source work item and is stored inside the Target Work item so that we can validate this.

string workItemReflectedId = (string)sourceWI.Fields[Engine.Target.Config.AsTeamProjectConfig().ReflectedWorkItemIDFieldName].Value;

We then get the value of the ReflectedWorkItemId field from the Target work item, so that we can compare it to the expected source.

if (workItemReflectedId != expectedReflectedId)

If these two values are not the same, then we do not have the correct item.

[MrHinsh]

You should check the field value of ReflectedWorkItemId manually and see if this is the same work item.

[MrHinsh]

I have added additional logging for this and skip the ApplyConfigurationsAndAssignTesters when the value is null.

 ITestPlan targetPlan2 = FindTestPlan(_targetTestStore, targetPlan.Name);
 if (targetPlan2 !=  null)
 {
     ApplyConfigurationsAndAssignTesters(sourcePlan.RootSuite, targetPlan2.RootSuite);
     //////////////////////////////
     TagCompletedTargetPlan(targetPlan.Id);
 }
 else
 {
     Log.LogError("Unable to ApplyConfigurationsAndAssignTesters: When loading the test plan again with the name it was found that they do not match!");
 }

[sonofhammer]

Thank you for responding so quickly.

I still believe the behavior is incorrect.

This line finds the target test Plan and sets it to the variable of testPlan

ITestPlan testPlan = (from p in tmc.Project.TestPlans.Query("Select * From TestPlan") where p.Name == name select p).SingleOrDefault();

This line uses the ID of the target work item to find an item in source, thereby returning the wrong work item.

 var sourceWI = Engine.Source.WorkItems.GetWorkItem(testPlan.Id.ToString());

Observe how in my case the id of 12422 finds a Test Plan in the target, but a Bug work item in the source

[sonofhammer]

It looks like the two calls of FindTestPlan both pass in the target test store.

[MrHinsh]

Im looking at:

private ITestPlan FindTestPlan(TestManagementContext targetTmc, string name, int sourcePlanId)
{
    Log.LogDebug("TestPlansAndSuitesMigrationContext::FindTestPlan");
    ITestPlan testPlan = (from p in targetTmc.Project.TestPlans.Query("Select * From TestPlan") where p.Name == name select p).SingleOrDefault();
 
    if (testPlan != null)
    {
        Log.LogDebug("TestPlansAndSuitesMigrationContext::FindTestPlan:: FOUND Test Plan with {name}", name);
        //Check test plan is in fact the right one
        var sourceWI = Engine.Source.WorkItems.GetWorkItem(sourcePlanId);
        string expectedReflectedId = Engine.Source.WorkItems.CreateReflectedWorkItemId(sourceWI).ToString();
        var targetWI = Engine.Source.WorkItems.GetWorkItem(testPlan.Id.ToString());
        string workItemReflectedId = (string)targetWI.Fields[Engine.Target.Config.AsTeamProjectConfig().ReflectedWorkItemIDFieldName].Value;
        if (workItemReflectedId != expectedReflectedId)
        {
            Log.LogDebug("TestPlansAndSuitesMigrationContext::FindTestPlan:: Found test plan with name {name} does not match {workItemReflectedId} ", name, workItemReflectedId);
            testPlan = null;
        }
    }
    else
    {
        Log.LogDebug("TestPlansAndSuitesMigrationContext::FindTestPlan:: NOT FOUND Test Plan with {name}", name);
    }
    return testPlan;
}

Which should fix it

[sonofhammer]

I was thinking something similar, let me test it on my end real quick.

[sonofhammer]

Looks good save for targetWI needs to use the Target engine instead of source.

Would you like me to make a PR with these changes?

[MrHinsh]

Already have #1773 ready to merge