Skip to content

Commit b1c5fb5

Browse files
20250617 Azure SQL Database virtualization
1 parent 04fe6dc commit b1c5fb5

File tree

2 files changed

+32
-125
lines changed

2 files changed

+32
-125
lines changed

docs/relational-databases/import-export/use-unicode-character-format-to-import-or-export-data-sql-server.md

Lines changed: 31 additions & 124 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,7 @@ title: "Use Unicode Character Format to Import & Export Data"
33
description: The Unicode character data format allows data to be exported from a SQL Server instance by using a code page that differs from the code page used by the client.
44
author: rwestMSFT
55
ms.author: randolphwest
6-
ms.reviewer: wiassaf
7-
ms.date: 06/17/2025
6+
ms.date: 05/27/2025
87
ms.service: sql
98
ms.subservice: data-movement
109
ms.topic: concept-article
@@ -31,11 +30,9 @@ When using Unicode character format, consider:
3130

3231
- The [sql_variant](../../t-sql/data-types/sql-variant-transact-sql.md) data that is stored in a Unicode character-format data file operates in the same way it operates in a character-format data file, except that the data is stored as [nchar](../../t-sql/data-types/nchar-and-nvarchar-transact-sql.md) instead of [char](../../t-sql/data-types/char-and-varchar-transact-sql.md) data. For more information about character format, see [Collation and Unicode support](../collations/collation-and-unicode-support.md).
3332

34-
<a id="special_considerations"></a>
35-
3633
## Special considerations for Using Unicode character format, bcp, and a format file
3734

38-
Unicode character format data files follow the conventions for Unicode files. The first two bytes of the file are hexadecimal numbers, `0xFFFE`. These bytes serve as byte-order marks (BOM), specifying whether the high-order byte is stored first or last in the file. The [bcp Utility](../../tools/bcp-utility.md) might misinterpret the BOM and cause part of your import process to fail; you might receive an error message similar as follows:
35+
Unicode character format data files follow the conventions for Unicode files. The first two bytes of the file are hexadecimal numbers, 0xFFFE. These bytes serve as byte-order marks (BOM), specifying whether the high-order byte is stored first or last in the file. The [bcp Utility](../../tools/bcp-utility.md) might misinterpret the BOM and cause part of your import process to fail; you might receive an error message similar as follows:
3936

4037
```csharp
4138
Starting copy...
@@ -48,51 +45,6 @@ The BOM might be misinterpreted under the following conditions:
4845
- The [bcp Utility](../../tools/bcp-utility.md) is used and the `-w` switch is used to indicate Unicode character
4946

5047
- A format file is used
51-
52-
- The first field in the data file is non-character
53-
54-
Consider whether any of the following workarounds are available for your *specific* situation:
55-
56-
- Don't use a format file. An example of this workaround is provided below, see [Using bcp and Unicode Character Format to Import Data without a Format File](#bcp_widechar_import).
57-
58-
- Use the `-c` switch instead of `-w`.
59-
60-
- Re-export the data using a native format.
61-
62-
- Use [BULK INSERT](../../t-sql/statements/bulk-insert-transact-sql.md) or [OPENROWSET BULK](../../t-sql/functions/openrowset-bulk-transact-sql.md). For more information, see [Using BULK INSERT and Unicode Character Format with a Non-XML Format File](#bulk_widechar_fmt) and [Using OPENROWSET and Unicode Character Format with a Non-XML Format File](#openrowset_widechar_fmt).
63-
64-
- Manually insert first record in destination table and then use `-F 2` switch to have import start on second record.
65-
66-
- Manually insert dummy first record in data file and then use `-F 2` switch to have import start on second record. An example of this workaround is provided below, see [Using bcp and Unicode Character Format to Import Data with a Non-XML Format File](#bcp_widechar_import_fmt).
67-
68-
- Use a staging table where the first column is a character data type.
69-
70-
- Re-export the data and change the data field order so that the first data field will be character. Then use a format file to remap the data field to the actual order in the table. For an example, see [Use a format file to map table columns to data-file fields (SQL Server)](use-a-format-file-to-map-table-columns-to-data-file-fields-sql-server.md).
71-
72-
<a id="command_options"></a>
73-
74-
## Command options for unicode character format
75-
76-
You can import Unicode character format data into a table using [bcp](../../tools/bcp-utility.md), [BULK INSERT](../../t-sql/statements/bulk-insert-transact-sql.md), or [OPENROWSET BULK](../../t-sql/functions/openrowset-bulk-transact-sql.md). For a [bcp](../../tools/bcp-utility.md) command or [BULK INSERT](../../t-sql/statements/bulk-insert-transact-sql.md) statement, you can specify the data format in the statement. For an [OPENROWSET BULK](../../t-sql/functions/openrowset-bulk-transact-sql.md) statement, you must specify the data format in a format file.
77-
78-
Unicode character format is supported by the following command options:
79-
80-
|Command|Option|Description|
81-
|-------------|------------|-----------------|
82-
| `bcp` |`-w`|Uses the Unicode character format.|
83-
| `BULK INSERT` |`DATAFILETYPE ='widechar'`|Uses Unicode character format when bulk importing data.|
84-
| `OPENROWSET` |N/A|Must use a format file.|
85-
86-
> [!NOTE]
87-
> Alternatively, you can specify formatting on a per-field basis in a format file. For more information, see [Format files to import or export data (SQL Server)](format-files-for-importing-or-exporting-data-sql-server.md).
88-
89-
<a id="etc"></a>
90-
91-
## Example test conditions
92-
93-
<a id="sample_table"></a>
94-
95-
The examples are based on the sample `myWidechar` table and format file. The script creates a test database, a table named `myWidechar` and populates the table with some initial values. Execute the following Transact-SQL in Microsoft [!INCLUDE[ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
9648
- The first field in the data file is non-character
9749

9850
Consider whether any of the following workarounds might be available for your *specific* situation:
@@ -141,12 +93,14 @@ CREATE DATABASE TestDatabase;
14193
GO
14294

14395
USE TestDatabase;
144-
CREATE TABLE dbo.myWidechar (
145-
PersonID smallint NOT NULL,
146-
FirstName nvarchar(25) NOT NULL,
147-
LastName nvarchar(30) NOT NULL,
148-
BirthDate date,
149-
AnnualSalary money
96+
97+
CREATE TABLE dbo.myWidechar
98+
(
99+
PersonID SMALLINT NOT NULL,
100+
FirstName NVARCHAR (25) NOT NULL,
101+
LastName NVARCHAR (30) NOT NULL,
102+
BirthDate DATE,
103+
AnnualSalary MONEY
150104
);
151105

152106
-- Populate table
@@ -172,52 +126,32 @@ REM Review file
172126
Notepad D:\BCP\myWidechar.fmt
173127
```
174128

175-
> [!IMPORTANT]
176-
> Ensure your non-XML format file ends with a carriage return\line feed. Otherwise you will likely receive the following error message:
177-
>
129+
> [!IMPORTANT]
130+
> Ensure your non-XML format file ends with a carriage return\line feed. Otherwise you might receive the following error message:
131+
>
178132
> `SQLState = S1000, NativeError = 0`
179133
> `Error = [Microsoft][ODBC Driver 13 for SQL Server]I/O error while reading BCP format file`
180134

181-
<a id="examples"></a>
182-
183135
## Examples
184136

185-
The examples use [the database and format files created in this article](#example-test-conditions).
186-
187-
<a id="bcp_widechar_export"></a>
137+
The following examples use the database, and format files created previously.
188138

189-
<a id="using-bcp-and-unicode-character-format-to-export-data"></a>
139+
### Use bcp and Unicode character format to Export Data
190140

191-
### Use bcp and Unicode Character Format to Export Data
141+
`-w` switch and `OUT` command. The data file created in this example is used in all subsequent examples. At a command prompt, enter the following commands:
192142

193-
Use the `-w` switch and `OUT` command.
194-
195-
The data file `myWidechar.bcp` created in this example will be used in subsequent examples.
196-
197-
Change the location of the code sample from `D:\BCP\myWidechar.bcp` to a file location on your machine.
198-
199-
At a command prompt, enter the following commands:
200-
201-
```cmd
143+
```batch
202144
bcp TestDatabase.dbo.myWidechar OUT D:\BCP\myWidechar.bcp -T -w
203145

204146
REM Review results
205147
NOTEPAD D:\BCP\myWidechar.bcp
206148
```
207149

208-
<a id="bcp_widechar_import"></a>
209-
210-
<a id="using-bcp-and-unicode-character-format-to-import-data-without-a-format-file"></a>
211-
212-
### Use bcp and Unicode Character Format to Import Data without a Format File
150+
### Use bcp and Unicode character format to Import Data without a format file
213151

214-
The `-w` switch and `IN` command.
152+
`-w` switch and `IN` command. At a command prompt, enter the following commands:
215153

216-
Change the location of the code sample from `D:\BCP\myWidechar.bcp` to a file location on your machine.
217-
218-
At a command prompt, enter the following commands:
219-
220-
```cmd
154+
```batch
221155
REM Truncate table (for testing)
222156
SQLCMD -Q "TRUNCATE TABLE TestDatabase.dbo.myWidechar;"
223157

@@ -227,19 +161,13 @@ bcp TestDatabase.dbo.myWidechar IN D:\BCP\myWidechar.bcp -T -w
227161
REM Review results is SSMS
228162
```
229163

230-
<a id="bcp_widechar_import_fmt"></a>
231-
232-
<a id="using-bcp-and-unicode-character-format-to-import-data-with-a-non-xml-format-file"></a>
233-
234-
### Use bcp and unicode character format to import data with a non-XML format file
235-
236-
The `-w` and `-f` switches, and the `IN` command.
164+
### Use bcp and Unicode character format to Import Data with a non-XML format file
237165

238-
[A workaround will need to be used](#special_considerations) since this example involves bcp, a format file, Unicode character, and the first data field in the data file is non-character. The data file `myWidechar.bcp` will be altered by adding an additional record as a "dummy" record which will then be skipped with the `-F 2` switch.
166+
`-w` and `-f` switches and `IN` command. A workaround needs to be used since this example involves bcp, a format file, Unicode character, and the first data field in the data file is non-character. See [Special Considerations for Using Unicode character format, bcp, and a format file](#special-considerations-for-using-unicode-character-format-bcp-and-a-format-file) earlier in the article. The data file `myWidechar.bcp` is altered by adding an extra record as a "dummy" record which is then skipped with the `-F 2` switch.
239167

240168
At a command prompt, enter the following commands and follow the modification steps:
241169

242-
```cmd
170+
```batch
243171
REM Truncate table (for testing)
244172
SQLCMD -Q "TRUNCATE TABLE TestDatabase.dbo.myWidechar;"
245173

@@ -257,15 +185,9 @@ REM Return data file to original state for usage in other examples
257185
bcp TestDatabase.dbo.myWidechar OUT D:\BCP\myWidechar.bcp -T -w
258186
```
259187

260-
<a id="bulk_widechar"></a>
261-
262-
<a id="using-bulk-insert-and-unicode-character-format-without-a-format-file"></a>
263-
264-
### Use BULK INSERT and unicode character format without a format file
188+
### Use BULK INSERT and Unicode character format without a format file
265189

266-
The `DATAFILETYPE` argument.
267-
268-
Execute the following Transact-SQL in Microsoft [!INCLUDE[ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
190+
`DATAFILETYPE` argument. Execute the following Transact-SQL in Microsoft [!INCLUDE [ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
269191

270192
```sql
271193
TRUNCATE TABLE TestDatabase.dbo.myWidechar; -- for testing
@@ -277,36 +199,23 @@ BULK INSERT TestDatabase.dbo.myWidechar FROM 'D:\BCP\myWidechar.bcp'
277199
SELECT * FROM TestDatabase.dbo.myWidechar;
278200
```
279201

280-
<a id="bulk_widechar_fmt"></a>
281-
282-
<a id="using-bulk-insert-and-unicode-character-format-with-a-non-xml-format-file"></a>
283-
284-
### Use BULK INSERT and unicode character format with a non-XML format file
202+
### Use BULK INSERT and Unicode character format with a non-XML format file
285203

286-
The `FORMATFILE` argument.
287-
288-
Execute the following Transact-SQL in Microsoft [!INCLUDE[ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
204+
`FORMATFILE` argument. Execute the following Transact-SQL in Microsoft [!INCLUDE [ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
289205

290206
```sql
291207
TRUNCATE TABLE TestDatabase.dbo.myWidechar; -- for testing
292208

293-
BULK INSERT TestDatabase.dbo.myWidechar
294-
FROM 'D:\BCP\myWidechar.bcp'
295-
WITH (FORMATFILE = 'D:\BCP\myWidechar.fmt');
209+
BULK INSERT TestDatabase.dbo.myWidechar FROM 'D:\BCP\myWidechar.bcp'
210+
WITH (FORMATFILE = 'D:\BCP\myWidechar.fmt');
296211

297212
-- review results
298213
SELECT * FROM TestDatabase.dbo.myWidechar;
299214
```
300215

301-
<a id="openrowset_widechar_fmt"></a>
302-
303-
<a id="using-openrowset-and-unicode-character-format-with-a-non-xml-format-file"></a>
216+
### Use OPENROWSET and Unicode character format with a non-XML format file
304217

305-
### Use OPENROWSET and unicode character format with a non-XML format file
306-
307-
The `FORMATFILE` argument.
308-
309-
Execute the following Transact-SQL in Microsoft [!INCLUDE[ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
218+
`FORMATFILE` argument. Execute the following Transact-SQL in Microsoft [!INCLUDE [ssManStudioFull](../../includes/ssmanstudiofull-md.md)] (SSMS):
310219

311220
```sql
312221
TRUNCATE TABLE TestDatabase.dbo.myWidechar; -- for testing
@@ -321,8 +230,6 @@ SELECT * FROM OPENROWSET (
321230
SELECT * FROM TestDatabase.dbo.myWidechar;
322231
```
323232

324-
<a id="RelatedTasks"></a>
325-
326233
## Related tasks
327234

328235
To use data formats for bulk import or bulk export

docs/t-sql/statements/create-database-scoped-credential-transact-sql.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Specifies the name of the account to be used when connecting outside the server.
5656
- To import a file from Azure Blob Storage using a managed identity, the identity name must be `MANAGED IDENTITY`.
5757
- When using Kerberos (Windows Active Directory or MIT KDC) do not use the domain name in the `IDENTITY` argument. It should just be the account name.
5858
- In a SQL Server instance, if creating a database scoped credential with a Storage Access Key used as the `SECRET`, `IDENTITY` is ignored.
59-
- `WITH IDENTITY` is not required if the container in Azure Blob storage is enabled for anonymous access. For an example querying Azure Blob storage with `OPENROWSET BULK`, see [Import into a table from a file stored on Azure Blob storage](../functions/openrowset-bulk-transact-sql.md#j-use-openrowset-to-access-several-delta-files-from-azure-data-lake-gen2).
59+
- `WITH IDENTITY` is not required if the container in Azure Blob storage is enabled for anonymous access. For an example querying Azure Blob storage with `OPENROWSET BULK`, see [Import into a table from a file stored on Azure Blob storage](../functions/openrowset-bulk-transact-sql.md#j-use-openrowset-to-access-several-delta-tables-from-azure-data-lake-gen2).
6060
- In [!INCLUDE [sssql22-md](../../includes/sssql22-md.md)] and later versions, the REST-API connector replaces HADOOP. For Azure Blob Storage and Azure Data Lake Gen 2, the only supported authentication method is shared access signature. For more information, see [CREATE EXTERNAL DATA SOURCE (Transact-SQL)](create-external-data-source-transact-sql.md).
6161
- In [!INCLUDE [sssql19-md](../../includes/sssql19-md.md)], the only PolyBase external data source that supports Kerberos authentication is Hadoop. All other external data sources (SQL Server, Oracle, Teradata, MongoDB, generic ODBC) only support Basic Authentication.
6262
- SQL pools in Azure Synapse Analytics include the following notes:

0 commit comments

Comments
 (0)