You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/metadata/table.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ ms.custom: devx-track-csharp
17
17
18
18
Azure Synapse Analytics allows the different workspace computational engines to share databases and tables between its Apache Spark pools and serverless SQL pool.
19
19
20
-
Once a database has been created by a Spark job, you can create tables in it with Spark that use Parquet or CSV as the storage format. Table names will be converted to lower case and need to be queried using the lower case name. These tables will immediately become available for querying by any of the Azure Synapse workspace Spark pools. They can also be used from any of the Spark jobs subject to permissions.
20
+
Once a database has been created by a Spark job, you can create tables in it with Spark that use Parquet, Delta, or CSV as the storage format. Table names will be converted to lower case and need to be queried using the lower case name. These tables will immediately become available for querying by any of the Azure Synapse workspace Spark pools. They can also be used from any of the Spark jobs subject to permissions.
21
21
22
22
The Spark created, managed, and external tables are also made available as external tables with the same name in the corresponding synchronized database in serverless SQL pool. [Exposing a Spark table in SQL](#expose-a-spark-table-in-sql) provides more detail on the table synchronization.
23
23
@@ -43,17 +43,17 @@ Spark provides two types of tables that Azure Synapse exposes in SQL automatical
43
43
44
44
Spark also provides ways to create external tables over existing data, either by providing the `LOCATION` option or using the Hive format. Such external tables can be over a variety of data formats, including Parquet.
45
45
46
-
Azure Synapse currently only shares managed and external Spark tables that store their data in Parquet or CSV format with the SQL engines. Tables backed by other formats are not automatically synced. You may be able to sync such tables explicitly yourself as an external table in your own SQL database if the SQL engine supports the table's underlying format.
46
+
Azure Synapse currently only shares managed and external Spark tables that store their data in Parquet, DELTA, or CSV format with the SQL engines. Tables backed by other formats are not automatically synced. You may be able to sync such tables explicitly yourself as an external table in your own SQL database if the SQL engine supports the table's underlying format.
47
47
48
48
> [!NOTE]
49
-
> Currently, only Parquet and CSV formats are synced to serverless SQL pool. A Spark delta table metadata will not sync to the SQL engine, even though Delta table uses Parquet as the snapshot's storage format. External tables from Spark are currently not synchronizing into dedicated SQL pool databases.
49
+
> Currently, only Parquet and CSV formats are fully supported in serverless SQL pool. Spark Delta tables are also available in the serverless SQL pool, but this feature is in **public preview**. External tables created in Spark are not available in dedicated SQL pool databases.
50
50
51
51
### Share Spark tables
52
52
53
53
The shareable managed and external Spark tables exposed in the SQL engine as external tables with the following properties:
54
54
55
55
- The SQL external table's data source is the data source representing the Spark table's location folder.
56
-
- The SQL external table's file format is Parquet or CSV.
56
+
- The SQL external table's file format is Parquet, Delta, or CSV.
57
57
- The SQL external table's access credential is pass-through.
58
58
59
59
Since all Spark table names are valid SQL table names and all Spark column names are valid SQL column names, the Spark table and column names will be used for the SQL external table.
@@ -78,7 +78,7 @@ Spark tables provide different data types than the Synapse SQL engines. The foll
78
78
|`array`, `map`, `struct`|`varchar(max)`|**SQL**: Serializes into JSON with collation `Latin1_General_100_BIN2_UTF8`. See [JSON Data](/sql/relational-databases/json/json-data-sql-server).|
79
79
80
80
>[!NOTE]
81
-
>Database level collation is `Latin1_General_100_CI_AS_SC_UTF8`.
81
+
>Database level collation is `Latin1_General_100_CI_AS_SC_UTF8`.
82
82
83
83
## Security model
84
84
@@ -112,7 +112,7 @@ This command creates the table `myparquettable` in the database `mytestdb`. Tabl
112
112
Verify that `myparquettable` is included in the results.
113
113
114
114
>[!NOTE]
115
-
>A table that is not using Parquet or CSV as its storage format will not be synchronized.
115
+
>A table that is not using Delta, Parquet or CSV as its storage format will not be synchronized.
116
116
117
117
Next, insert some values into the table from Spark, for example with the following C# Spark statements in a C# notebook:
0 commit comments