@@ -9,9 +9,8 @@ scientists to run frequent and large workloads against a large Apache Spark clus
9
9
several requirements that were not currently available in the Jupyter open source ecosystem. We tried
10
10
to use the Jupyter Kernel Gateway project, but we quickly realized that the JKG server became the
11
11
bottleneck because the co-located Spark driver application for these kinds of workloads (in this case,
12
- the kernel process running on behalf of notebook cells) were extremely resource intensive. Toss in a team
13
- or organization of data scientists, and you quickly saturate the compute resources of the Kernel Gateway
14
- server.
12
+ the kernel process running on behalf of notebook cells) were extremely resource intensive. In organizations
13
+ with multiple data scientists, you can quickly saturate the compute resources of the Kernel Gateway server.
15
14
16
15
Jupyter Enterprise Gateway enables Jupyter Notebook to launch and manage remote kernels in a distributed cluster,
17
16
including Apache Spark managed by YARN, IBM Spectrum Conductor or Kubernetes. New platforms can be added via an
@@ -88,7 +87,7 @@ Other then that, below are some stats that have been collected from the Jupyter
88
87
- 10 contributors
89
88
- 5 different organizations (based on current employment)
90
89
- 205 commits (16,551 additions, 9,616 removals)
91
- - 59 Stars
90
+ - 60 Stars
92
91
- 26 Forks
93
92
- 10K+ pulls of primary docker image
94
93
@@ -131,7 +130,9 @@ The Enterprise Gateway is packaged using setup tools, released in both source an
131
130
## Pros and Cons
132
131
133
132
Pro: Extend Jupyter Stack to support distributed/remote Kernels
133
+
134
134
Pro: The runtime can easily be extensible to support new cluster resource managers
135
+
135
136
Con: Still requires couple extensions (e.g. NB2KG) to connect to the gateway
136
137
137
138
## Interested Contributors
0 commit comments