Skip to content

Commit 2c1725a

Browse files
(AzureCXP) fixes MicrosoftDocs/azure-docs#120145
we have done changes from "empLocation" and "deptLocation" To:"emp_Location" and "dept_Location"
1 parent c6b6618 commit 2c1725a

File tree

1 file changed

+25
-25
lines changed

1 file changed

+25
-25
lines changed

articles/synapse-analytics/spark/apache-spark-performance-hyperspace.md

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ res3: org.apache.spark.sql.SparkSession = org.apache.spark.sql.SparkSession@297e
9191

9292
To prepare your environment, you'll create sample data records and save them as Parquet data files. Parquet is used for illustration, but you can also use other formats such as CSV. In the subsequent cells, you'll see how you can create several Hyperspace indexes on this sample dataset and make Spark use them when running queries.
9393

94-
The example records correspond to two datasets: department and employee. You should configure the "empLocation" and "deptLocation" paths so that on the storage account they point to your desired location to save generated data files.
94+
The example records correspond to two datasets: department and employee. You should configure the "emp_Location" and "dept_Location" paths so that on the storage account they point to your desired location to save generated data files.
9595

9696
The output of running the following cell shows contents of our datasets as lists of triplets followed by references to dataFrames created to save the content of each dataset in our preferred location.
9797

@@ -129,10 +129,10 @@ import spark.implicits._
129129
val empData: DataFrame = employees.toDF("empId", "empName", "deptId")
130130
val deptData: DataFrame = departments.toDF("deptId", "deptName", "location")
131131

132-
val empLocation: String = "/<yourpath>/employees.parquet" //TODO ** customize this location path **
133-
val deptLocation: String = "/<yourpath>/departments.parquet" //TODO ** customize this location path **
134-
empData.write.mode("overwrite").parquet(empLocation)
135-
deptData.write.mode("overwrite").parquet(deptLocation)
132+
val emp_Location: String = "/<yourpath>/employees.parquet" //TODO ** customize this location path **
133+
val dept_Location: String = "/<yourpath>/departments.parquet" //TODO ** customize this location path **
134+
empData.write.mode("overwrite").parquet(emp_Location)
135+
deptData.write.mode("overwrite").parquet(dept_Location)
136136
```
137137

138138
::: zone-end
@@ -217,10 +217,10 @@ var employeeSchema = new StructType(new List<StructField>()
217217
DataFrame empData = spark.CreateDataFrame(employees, employeeSchema);
218218
DataFrame deptData = spark.CreateDataFrame(departments, departmentSchema);
219219

220-
string empLocation = "/<yourpath>/employees.parquet"; //TODO ** customize this location path **
221-
string deptLocation = "/<yourpath>/departments.parquet"; //TODO ** customize this location path **
222-
empData.Write().Mode("overwrite").Parquet(empLocation);
223-
deptData.Write().Mode("overwrite").Parquet(deptLocation);
220+
string emp_Location = "/<yourpath>/employees.parquet"; //TODO ** customize this location path **
221+
string dept_Location = "/<yourpath>/departments.parquet"; //TODO ** customize this location path **
222+
empData.Write().Mode("overwrite").Parquet(emp_Location);
223+
deptData.Write().Mode("overwrite").Parquet(dept_Location);
224224

225225
```
226226

@@ -234,8 +234,8 @@ employees: Seq[(Int, String, Int)] = List((7369,SMITH,20), (7499,ALLEN,30), (752
234234

235235
empData: org.apache.spark.sql.DataFrame = [empId: int, empName: string ... 1 more field]
236236
deptData: org.apache.spark.sql.DataFrame = [deptId: int, deptName: string ... 1 more field]
237-
empLocation: String = /your-path/employees.parquet
238-
deptLocation: String = /your-path/departments.parquet
237+
emp_Location: String = /your-path/employees.parquet
238+
dept_Location: String = /your-path/departments.parquet
239239
```
240240

241241
Let's verify the contents of the Parquet files we created to make sure they contain expected records in the correct format. Later, we'll use these data files to create Hyperspace indexes and run sample queries.
@@ -245,9 +245,9 @@ Running the following cell produces an output that displays the rows in employee
245245
:::zone pivot = "programming-language-scala"
246246

247247
```scala
248-
// empLocation and deptLocation are the user defined locations above to save parquet files
249-
val empDF: DataFrame = spark.read.parquet(empLocation)
250-
val deptDF: DataFrame = spark.read.parquet(deptLocation)
248+
// emp_Location and dept_Location are the user defined locations above to save parquet files
249+
val empDF: DataFrame = spark.read.parquet(emp_Location)
250+
val deptDF: DataFrame = spark.read.parquet(dept_Location)
251251

252252
// Verify the data is available and correct
253253
empDF.show()
@@ -276,9 +276,9 @@ dept_DF.show()
276276

277277
```csharp
278278

279-
// empLocation and deptLocation are the user-defined locations above to save parquet files
280-
DataFrame empDF = spark.Read().Parquet(empLocation);
281-
DataFrame deptDF = spark.Read().Parquet(deptLocation);
279+
// emp_Location and dept_Location are the user-defined locations above to save parquet files
280+
DataFrame empDF = spark.Read().Parquet(emp_Location);
281+
DataFrame deptDF = spark.Read().Parquet(dept_Location);
282282

283283
// Verify the data is available and correct
284284
empDF.Show();
@@ -781,8 +781,8 @@ The following cell enables Hyperspace and creates two DataFrames containing your
781781
// Enable Hyperspace
782782
spark.enableHyperspace
783783

784-
val empDFrame: DataFrame = spark.read.parquet(empLocation)
785-
val deptDFrame: DataFrame = spark.read.parquet(deptLocation)
784+
val empDFrame: DataFrame = spark.read.parquet(emp_Location)
785+
val deptDFrame: DataFrame = spark.read.parquet(dept_Location)
786786

787787
empDFrame.show(5)
788788
deptDFrame.show(5)
@@ -814,8 +814,8 @@ dept_DF.show(5)
814814
// Enable Hyperspace
815815
spark.EnableHyperspace();
816816

817-
DataFrame empDFrame = spark.Read().Parquet(empLocation);
818-
DataFrame deptDFrame = spark.Read().Parquet(deptLocation);
817+
DataFrame empDFrame = spark.Read().Parquet(emp_Location);
818+
DataFrame deptDFrame = spark.Read().Parquet(dept_Location);
819819

820820
empDFrame.Show(5);
821821
deptDFrame.Show(5);
@@ -1391,9 +1391,9 @@ val extraDepartments = Seq(
13911391
(60, "Human Resources", "San Francisco"))
13921392

13931393
val extraDeptData: DataFrame = extraDepartments.toDF("deptId", "deptName", "location")
1394-
extraDeptData.write.mode("Append").parquet(deptLocation)
1394+
extraDeptData.write.mode("Append").parquet(dept_Location)
13951395

1396-
val deptDFrameUpdated: DataFrame = spark.read.parquet(deptLocation)
1396+
val deptDFrameUpdated: DataFrame = spark.read.parquet(dept_Location)
13971397

13981398
deptDFrameUpdated.show(10)
13991399

@@ -1431,9 +1431,9 @@ var extraDepartments = new List<GenericRow>()
14311431
};
14321432

14331433
DataFrame extraDeptData = spark.CreateDataFrame(extraDepartments, departmentSchema);
1434-
extraDeptData.Write().Mode("Append").Parquet(deptLocation);
1434+
extraDeptData.Write().Mode("Append").Parquet(dept_Location);
14351435

1436-
DataFrame deptDFrameUpdated = spark.Read().Parquet(deptLocation);
1436+
DataFrame deptDFrameUpdated = spark.Read().Parquet(dept_Location);
14371437

14381438
deptDFrameUpdated.Show(10);
14391439

0 commit comments

Comments
 (0)