Merge pull request #41728 from jingyanjingyan/usql-extract-script

usql-create-extract-script

Author: rjagiewich

articles/data-lake-analytics/data-lake-analytics-data-lake-tools-for-vscode.md

@@ -28,30 +28,32 @@ For MacOS and Linux:
 - [.NET Core SDK 2.0](https://www.microsoft.com/net/download/core). 
 - [Mono 5.2.x](http://www.mono-project.com/download/).
 
-## Install Data Lake Tools
+## Install Azure Data Lake Tools
 
-After you install the prerequisites, you can install Data Lake Tools for VS Code.
+After you install the prerequisites, you can install Azure Data Lake Tools for VS Code.
 
-**To install Data Lake Tools**
+**To install Azure Data Lake Tools**
 
 1. Open Visual Studio Code.
-2. Click **Extensions** in the left pane. Enter **Azure Data Lake** in the search box.
-3. Click **Install** next to **Azure Data Lake tools**. After a few seconds, the **Install** button will be changed to **Reload**.
-4. Click **Reload** to activate the **Azure Data Lake tools** extension.
-5. Click **Reload Window** to confirm. You can see **Azure Data Lake tools** in the Extensions pane.
+2. Click **Extensions** in the left pane. Enter **Azure Data Lake Tools** in the search box.
+3. Click **Install** next to **Azure Data Lake Tools**. After a few seconds, the **Install** button will be changed to **Reload**.
+4. Click **Reload** to activate the **Azure Data Lake Tools** extension.
+5. Click **Reload Window** to confirm. You can see **Azure Data Lake Tools** in the Extensions pane.
 
     ![Data Lake Tools for Visual Studio Code Extensions pane](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-extensions.png)
 
 
 ## Activate Azure Data Lake Tools
 Create a new .usql file or open an existing .usql file to activate the extension. 
 
-## Open the sample script
-Open the command palette (Ctrl+Shift+P) and enter **ADL: Open Sample Script**. It opens another instance of this sample. You can also edit, configure, and submit script on this instance.
 
 ## Work with U-SQL
 
-You need open either a U-SQL file or a folder to work with U-SQL.
+To work with U-SQL, you need open either a U-SQL file or a folder.
+
+**To open the sample script**
+
+Open the command palette (Ctrl+Shift+P) and enter **ADL: Open Sample Script**. It opens another instance of this sample. You can also edit, configure, and submit script on this instance.
 
 **To open a folder for your U-SQL project**
 
@@ -120,7 +122,7 @@ You need open either a U-SQL file or a folder to work with U-SQL.
         ![Data Lake Tools for Visual Studio Code configuration file](./media/data-lake-analytics-data-lake-tools-for-vscode/default-context-setting.png)
 
         > [!NOTE] 
-        > After the configuration is saved, The account, database, and schema information appear on the status bar at the bottom-left corner of the corresponding .usql file file if you don’t have default context set up.
+        > After the configuration is saved, the account, database, and schema information appear on the status bar at the bottom-left corner of the corresponding .usql file file if you don’t have default context set up.
 
 **Set Git Ignore**
 
@@ -132,10 +134,15 @@ You need open either a U-SQL file or a folder to work with U-SQL.
 
     ![Data Lake Tools for Visual Studio Code configuration file](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-gitignore.png)
 
-## Use Python, R, and CSharp code-behind file
+
+## Work with code-behind file: CSharp, Python, and R
+
 Azure Data Lake Tool supports multiple custom codes, the instructions see [Develop U-SQL with Python, R, and CSharp for Azure Data Lake Analytics in VSCode](data-lake-analytics-u-sql-develop-with-python-r-csharp-in-vscode.md).
 
-## Use assemblies
+## Work with assemblies
+
+For information on developing assemblies, see [Develop U-SQL assemblies for Azure Data Lake Analytics jobs](data-lake-analytics-u-sql-develop-assemblies.md).
+
 You can use Data Lake Tools to register custom code assemblies in the Data Lake Analytics catalog.
 
 **To register an assembly**
@@ -200,42 +207,80 @@ The following U-SQL code demonstrates how to call an assembly. In the sample, th
             USING Outputters.Tsv();
 
 
+## U-SQL local run and local debug for windows users
+U-SQL local run tests your local data and validates your script locally before your code is published to Data Lake Analytics. The local debug feature enables you to complete the following tasks before your code is submitted to Data Lake Analytics: 
+- Debug your C# code-behind. 
+- Step through the code. 
+- Validate your script locally.
+
+For instructions on local run and local debug, see [U-SQL local run and local debug with Visual Studio Code](data-lake-tools-for-vscode-local-run-and-debug.md).
+
+
 ## Connect to Azure
 
 Before you can compile and run U-SQL scripts in Data Lake Analytics, you must connect to your Azure account.
 
-**To connect to Azure**
+<b id="sign-in-by-command">To connect to Azure using command</b>
 
 1.	Select Ctrl+Shift+P to open the command palette. 
-2.  Enter **ADL: Login**. The login information appears on the top area.
+2.  Enter **ADL: Login**. The login information appears at the right-bottom.
 
     ![Data Lake Tools for Visual Studio Code command palette](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-extension-login.png)
+
     ![Data Lake Tools for Visual Studio Code device login information](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-login-info.png)
-3.  Click **Copy & Open** to open the login webpage with URL: https://aka.ms/devicelogin. Paste the code **G567LX42V** into the text box, and then select **Continue**.
 
-   ![Data Lake Tools for Visual Studio Code login paste code](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-extension-login-paste-code.png )   
+3.  Click **Copy & Open** to open the login webpage with URL: https://aka.ms/devicelogin. Paste the code into the text box, and then select **Continue**.
+
+    ![Data Lake Tools for Visual Studio Code login paste code](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-extension-login-paste-code.png )  
+     
 4.  Follow the instructions to sign in from the webpage. When you're connected, your Azure account name appears on the status bar in the lower-left corner of the **VS Code** window. 
 
     > [!NOTE] 
-    >- Data Lake Tool automatically signs in next time If you have signed in before, but you have not logged out yet.
+    >- Data Lake Tool automatically signs in next time if you have signed in before, but you have not logged out yet.
     >- If your account has two factors enabled, we recommend that you use phone authentication rather than using a PIN.
 
 
 To sign out, enter the command **ADL: Logout**.
 
-## List your Data Lake Analytics accounts
+**To connect to Azure from explorer**
 
-To test the connection, get a list of your Data Lake Analytics accounts.
+To sign in from explorer, expand **AZURE DATALAKE**, click **Sign in to Azure...**, then follow the step3 and step4 of [**To connect to Azure using command**](#sign-in-by-command).
 
-**To list the Data Lake Analytics accounts under your Azure subscription**
+![Connect to Azure from explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-sign-in-from-explorer.png )  
+
+You cannot sign out from explorer. To sign out, see [**To connect to Azure using command**](#sign-in-by-command).
 
-1. Select Ctrl+Shift+P to open the command palette.
-2. Enter **ADL: List Accounts**. The accounts appear in the **Output** pane.
 
+## Create extract script 
+You can create EXTRACT script for .csv, .tsv, .txt files using the command **ADL: Create EXTRACT Script** or from **AZURE DATALAKE** explorer.
 
-## Access the Data Lake Analytics catalog
+**To create EXTRACT script using command**
 
-After you have connected to Azure, you can use the following steps to access the U-SQL catalog.
+1. Select Ctrl+Shift+P to open the command palette, enter **ADL: Create EXTRACT Script**.
+2. Specify the full path for an Azure storage file, press **Enter**.
+3. Select one account.
+4. For .txt file, select a delimiter to extract the file. 
+
+    ![Create extract script process](./media/data-lake-analytics-data-lake-tools-for-vscode/create-extract-script-process.png)
+
+The extract script is generated based on your entries. For the script that cannot detect the columns, choose one from the two options. If not, only one script will be generated.
+
+![Create extract script result](./media/data-lake-analytics-data-lake-tools-for-vscode/create-extract-script-result.png)
+
+**To create EXTRACT script from explorer**
+
+Another way to create EXTRACT script is through the right-click menu on the .csv, .tsv, .txt file in data lake storage or blob storage.
+
+![Create extract script from context menu](./media/data-lake-analytics-data-lake-tools-for-vscode/create-extract-script-from-context-menu.png)
+
+## Integrate with Azure Data Lake Analytics through command
+
+You can access Azure Data Lake Analytics resources including list accounts, access metadata, and view analytics jobs. 
+
+**To list the Azure Data Lake Analytics accounts under your Azure subscription**
+
+1. Select Ctrl+Shift+P to open the command palette.
+2. Enter **ADL: List Accounts**. The accounts appear in the **Output** pane.
 
 **To access Azure Data Lake Analytics metadata**
 
@@ -244,17 +289,15 @@ After you have connected to Azure, you can use the following steps to access the
 3.	Select one of the Data Lake Analytics databases.
 4.	Select one of the schemas. You can see the list of tables.
 
-## View Data Lake Analytics jobs
-
-**To view Data Lake Analytics jobs**
+**To view Azure Data Lake Analytics jobs**
 1.  Open the command palette (Ctrl+Shift+P) and select **ADL: Show Jobs**. 
 2.	Select a Data Lake Analytics or local account. 
 3.  Wait for the jobs list for the account to appear.
 4.	Select a job from job list, Data Lake Tools opens the job view in the right pane and displays some information in VS Code **OUTPUT**.
 
     ![Data Lake Tools for Visual Studio Code IntelliSense object types](./media/data-lake-analytics-data-lake-tools-for-vscode/data-lake-tools-for-vscode-show-job.png)
 
-## Azure Data Lake Storage integration
+## Integrate with Azure Data Lake Storage through command
 
 You can use Azure Data Lake Storage-related commands to:
  - Browse through the Azure Data Lake Storage resources. [List the storage path](#list-the-storage-path). 
@@ -306,7 +349,7 @@ As the same time, you can monitor the [uploading status](#check-storage-tasks-st
 
 
 ### Download file 
-You can download files by entering the commands **ADL: Download File** or **ADL: Download File (Advanced)**.
+You can download files using the commands **ADL: Download File** or **ADL: Download File (Advanced)**.
 
 **To download files though the ADL: Download File (Advanced)**
 1. Right-click the script editor, and then select **Download File (Advanced)**.
@@ -339,13 +382,10 @@ The status displays on the bottom of the status bar when completed downloading a
 
    ![Data Lake Tools for Visual Studio Code Check Storage status](./media/data-lake-analytics-data-lake-tools-for-vscode/storage-status.png)
 
-## VSCode Explorer integration with Azure Data Lake
-
-**Azure Integration** 
 
-- Before login to Azure, you can always expand **AZURE DATALAKE**, then click **Sign in to Azure** to login to Azure. After login, you will see all the subscriptions under your Azure account are listed in the left panel of the **AZURE DATALAKE**. 
+## Integrate with Azure Data Lake Analytics from explorer
 
-   ![DataLake explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/sign-in-datalake-explorer.png)
+- After login, you will see all the subscriptions under your Azure account are listed in the left panel of the **AZURE DATALAKE**. 
 
    ![DataLake explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/datalake-explorer.png)
 
@@ -367,33 +407,33 @@ The status displays on the bottom of the status bar when completed downloading a
 
     ![DataLake explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/datalake-explorer-register-assembly.png)
 
-**ADLS Integration** 
+## Integrate with Azure Data Lake Storage from explorer
 
 Navigate to **Data Lake Store**
 
- - On the folder node, you can **Refresh**, **Delete**, **Upload**, **Upload Folder**, **Copy Relative Path**, **Copy Full Path** in the right-click context menu.
+ - On the folder node, you can **Refresh**, **Delete**, **Upload**, **Upload Folder**, **Copy Relative Path**, and **Copy Full Path** in the right-click context menu.
 
    ![DataLake explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/storage-account-folder-menu.png)
 
- - On the file node, you can **Download**, **Preview**, **Delete**, **Copy Relative Path**, **Copy Full Path** in the right-click context menu. 
+- On the file node, you can **Preview**, **Download**, **Delete**, **Create EXTRACT Script** (only available for CSV, TSV and TXT files), as well as **Copy Relative Path**, and **Copy Full Path** in the right-click context menu.
 
-   ![DataLake explorer](./media/data-lake-analytics-data-lake-tools-for-vscode/storage-account-download-preview-file.png)
+    ![DataLake explorer - extract](./media/data-lake-analytics-data-lake-tools-for-vscode/storage-account-extract.png)
 
-**WASB Integration**
+## Integrate with Azure Blob Storage from explorer
 
 Navigate to **Blob Storage**
 
-- On the blob container node, you can **Refresh**, **Delete Blob Container**, **Upload Blob** in the right-click context menu.
+- On the blob container node, you can **Refresh**, **Delete Blob Container**, and **Upload Blob** in the right-click context menu.
 
     ![Blob Storage blob container node](./media/data-lake-analytics-data-lake-tools-for-vscode/blob-storage-blob-container-node.png)
 
-- On the folder node, you can **Refresh**, **Upload Blob** in the right-click context menu.
+- On the folder node, you can **Refresh**, and **Upload Blob** in the right-click context menu.
 
     ![Blob Storage folder node](./media/data-lake-analytics-data-lake-tools-for-vscode/blob-storage-folder-node.png)
 
-- On the file node, you can **Preview/Edit**, **Download**, **Delete**, **Copy Relative Path**, **Copy Full Path** in the right-click context menu.
+- On the file node, you can **Preview/Edit**, **Download**, **Delete**, **Create EXTRACT Script** (only available for CSV, TSV and TXT files), as well as **Copy Relative Path**, and **Copy Full Path** in the right-click context menu.
 
-    ![Blob Storage file node](./media/data-lake-analytics-data-lake-tools-for-vscode/blob-storage-file-node.png)
+    ![Create extract script from context menu](./media/data-lake-analytics-data-lake-tools-for-vscode/create-extract-script-from-context-menu-2.png)
 
 ## Open ADL storage explorer in portal
 1. Select Ctrl+Shift+P to open the command palette.
@@ -402,14 +442,6 @@ Navigate to **Blob Storage**
 
 Data Lake Tools opens the Azure storage path in the Azure portal. You can find the path and preview the file from the web.
 
-## Local run and local debug for Windows users
-U-SQL local run tests your local data and validates your script locally before your code is published to Data Lake Analytics. The local debug feature enables you to complete the following tasks before your code is submitted to Data Lake Analytics: 
-- Debug your C# code-behind. 
-- Step through the code. 
-- Validate your script locally.
-
-For instructions on local run and local debug, see [U-SQL local run and local debug with Visual Studio Code](data-lake-tools-for-vscode-local-run-and-debug.md).
-
 ## Additional features
 
 Data Lake Tools for VS Code supports the following features: