Turkish characters (ğ, ü, ş, ç) are inconsistent in Title and User fields during migration

[fatihaydnrepo]

Description

We are facing an issue with Turkish characters (ğ, ü, ş, ç, İ, ö) when migrating work items using Azure DevOps Migration Tools.

• In some work items, the Title field shows characters correctly.
• In others, the same characters appear broken or replaced with symbols (e.g. ?, ğ, Cagri instead of Çağrı).
• The same behavior happens with User fields (CreatedBy, AssignedTo, etc.), where names containing Turkish characters are not migrated consistently.

It looks like an encoding issue (UTF-8 vs legacy encodings) because some old work items were stored differently in the source system.

We tried forcing "Encoding": "UTF-8" in the configuration, but the behavior still occurs.

---

Expected behavior

All Turkish characters should be migrated correctly and appear the same in both Title and User fields.

---

Actual behavior

• Some characters are correct.
• Some characters appear corrupted (like ?, ğ, Cagri).

---

Steps to reproduce

1. Use a project that contains work items with Turkish characters in Title or User fields.
2. Run migration with the configuration below.
3. Check Target project — some characters are broken.

---

Configuration used

{
  "$schema": "https://raw.githubusercontent.com/nkdAgility/azure-devops-migration-tools/main/src/MigrationTools/_configuration/configuration.schema.json",
  "MigrationTools": {
    "Version": "16.0",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/xyz",
        "Project": "devops",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "",
          "Encoding": "UTF-8"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "AllowCrossProjectLinking": false
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://ado.xyz.com.tr/xyz",
        "Project": "devops",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "",
          "Encoding": "UTF-8"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "AllowCrossProjectLinking": false
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "SourceName": "Source",
        "TargetName": "Target",
        "Enabled": true,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.WorkItemType] IN ('Issue', 'Task', 'Epic', 'Bug', 'Error') ORDER BY [System.Id]",
        "LinkMigration": true,
        "AttachmentMigration": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "GenerateMigrationComment": true,
        "BypassRules": true
      }
    ]
  }
}

[MrHinsh]

You can use the StringManipulatorTool to control this. The default behaviour is to strip any non aski characters as some are not supported.