File tree Expand file tree Collapse file tree 1 file changed +13
-3
lines changed
bootcamp/materials/3-spark-fundamentals Expand file tree Collapse file tree 1 file changed +13
-3
lines changed Original file line number Diff line number Diff line change 1
- # Week 6 Infrastructure Track Spark Training
1
+ # Week 3 Spark Fundamentals training
2
2
3
+ ## Unit Testing PySpark Course Getting Started
3
4
5
+ You need to install the required dependencies in ` requirements.txt `
4
6
5
- ## Usage
7
+ Running ` pip install -r requirements.txt ` will install them.
6
8
7
- To launch the spark and iceberg docker containers, run:
9
+ Running pytest is easy. You just need to run ` pytest ` and you're good to go!
10
+
11
+
12
+
13
+ ## Spark Fundamentals and Advanced Spark Setup
14
+
15
+ To launch the Spark and Iceberg Docker containers, run:
8
16
9
17
``` bash
10
18
make up
11
19
```
12
20
21
+ Or ` docker compose up ` if you're on Windows!
22
+
13
23
Then, you should be able to access a Jupyter notebook at ` localhost:8888 ` .
14
24
15
25
The first notebook to be able to run is the ` event_data_pyspark.ipynb ` inside the ` notebooks ` folder.
You can’t perform that action at this time.
0 commit comments