History of jupyter start document

history_of_jupyter.md

@@ -0,0 +1,103 @@
+# History of Jupyter
+
+It's early 2000's -- Python 2 was still fresh, SciPy had just had its debut,
+Matplotlib was under preparation: the IPython went through its first release.
+That is when the history of Jupyter starts.
+
+Let's go through the timeline development of the project to understand
+*why* and *when* the different components of Jupyter came to be;
+from a tunned-up Python shell, to language-agnostic code notebooks,
+to extendable data analysis web interfaces.
+
+## **IPython**
+
+> 2001
+
+First released in 2001, IPython -- short for *Interactive Python* --
+started from a set of enhancements to the basic interactive Python shell
+(ref: ipython-paper, 2007).
+
+During the following 13 years, IPython evolved in the open and established
+many of the fundamental components we use today in our workflows,
+such as the notebook document format.
+
+### The Big Split
+
+> 2014
+
+Given the growth experienced by IPython in those years, it was decided
+in 2014 that spliting the then monolithic repository was necessary
+not only to improve maintenance but also to allow the largely independent
+components of that code base to be used and evolve independetly
+(ref: big-split, 2015).
+
+The big split generated many repositories, most of them would be the seeds
+of the Jupyter's Github organization:
+
+- [Jupyter-Notebook](https://github.com/jupyter/notebook)
+- [Jupyter-Console](https://github.com/jupyter/jupyter_console)
+- [Jupyter-Client](https://github.com/jupyter/jupyter_client)
+- [NbConvert](https://github.com/jupyter/nbconvert)
+- [NbFormat](https://github.com/jupyter/nbformat)
+- [QtConsole](https://github.com/jupyter/qtconsole)
+
+And some repositories, libraries for the Python language, would remain under
+IPython's Github organization:
+
+- [IPyParallel](https://github.com/ipython/ipyparallel)
+- [IPyKernel](https://github.com/ipython/ipykernel)
+- [Traitlets](https://github.com/ipython/traitlets)
+
+From this point in time, IPython and Jupyter would still share many jargons
+and resources but each would live in separate houses:
+
+- https://ipython.org
+- https://jupyter.org
+
+
+## **Jupyter**
+
+> 2014
+
+Not only IPython's codebase was being reorganized but the community itself:
+the Jupyter project spin-off ["to support interactive data science and
+scientific computing across all programming languages"](https://jupyter.org/about).
+
+The new structure of the project Jupyter encouraged the formation of
+community-managed subprojects by establishing guidelines and a body of
+expertise to support new developers and their ideas.
+
+## JupyterHub
+
+> 2014
+
+As soon as Jupyter (the project) is born, JupyterHub takes place as the
+first argely independent sub-project.
+
+## JupyterLab
+
+> 2016
+
+As handful as Notebook server was, there was demand for a more concise user
+interface, and an UI that allow for greater integration with visualization
+libraries and trends such as data science dashboards that were now popping-up
+in the Python and Javascript communities.
+
+To handle the growing demand for a modern UI to (executable) notebooks,
+JupyterLab was created.
+As an independent project, Jupyter-Lab could recreate a whole new interface
+around Jupyter-Server without breaking Jupyter-Notebook.
+
+## References
+
+- *"IPython: A System for Interactive Scientific Computing"*,
+F. Perez and B.E. Granger (2007)
+https://dl.acm.org/doi/10.1109/MCSE.2007.53
+
+- *"The Big Split™"*,
+Jupyter Blog (2015)
+https://blog.jupyter.org/the-big-split-9d7b88a031a7
+
+- *"Jupyter receives the ACM Software System Award"*,
+Jupyter Blog (2018)
+https://blog.jupyter.org/jupyter-receives-the-acm-software-system-award-d433b0dfe3a2

subprojects.md

@@ -0,0 +1,156 @@
+# What is Jupyter
+
+> Jupyter "notebook" is a term used to express both: the *file* with containing the notebook
+> code and the classic *application* (Jupyter-Notebook) for creating and running those files.
+> There are other applications providing similar functionalities, Jupyter-Lab is one of them.
+
+We can group the subprojects in the following categories. (The subprojects are described below)
+
+### Applications and Interfaces
+- Jupyter Notebook
+- JupyterLab
+  - JupyterHub
+- Binder
+  - BinderHub
+- Voilà
+- IPython
+
+### Server and Gateway
+- Jupyter Server
+  - Jupyter Console
+- Enterprise Gateway
+- Kernel Gateway
+
+### Widgets and Interactive Tools
+- Jupyter Widgets
+- QtConsole
+
+### Kernels and Computational Engines
+- Jupyter Kernels
+- jupyter-xeus
+- IPykernel
+- ipyparallel
+
+### Security and Accessibility
+- Jupyter Security
+- Jupyter Accessibility
+
+### Foundations and Standards
+- Jupyter Notebook Format (nbformat)
+- Jupyter Core
+- Jupyter Client
+- Jupyter-packaging
+- Terminado
+- Telemetry
+- Traitlets
+
+### NB tools
+- nbclient
+- nbconvert
+- nbdime
+- nbgrader
+- nbviewer
+
+### Miscellaneous
+- JEPs repo (Jupyter Enhancement Proposals)
+- Docker stacks
+
+
+# Jupyter Subprojects
+
+Project Jupyter is composed by many subprojects, each subproject focuses on different components
+or aspects of the Jupyter ecosystem.
+Subprojects differ in size, activity, and governance.
+
+Jupyter products span from the definition of the Notebook *file 
+format* all the way to multi-user deployment of Notebook or Lab *applications*.
+Subprojects will take ownership of the different components or take over broader aspects such
+as accessibility and security.
+
+Regarding their governance, some subprojects have their own Council and SSC (Software Steering Committee), and the others are governed directly by the Project SSC.
+
+The major subprojects with their own formal Subproject Council are:
+
+- JupyterLab: An extensible environment for interactive and reproducible computing.
+- JupyterHub and Binder: Focused on multi-user capabilities and sharing of interactive notebooks.
+  * JupyterHub: Multi-user version of the notebook.
+  * Binder: Allows turning a Git repository into a collection of interactive notebooks.
+  * BinderHub: Related to Binder, focusing on the deployment aspect.
+- Voilà: Turns Jupyter notebooks into interactive web applications.
+- Jupyter Server: Provides the backend (core services, APIs, and REST endpoints) for Jupyter web applications.
+  * Enterprise Gateway: Enhances Jupyter Server for enterprise needs.
+  * Kernel Gateway: Simplifies the process of running Jupyter kernels.
+- Jupyter Widgets: Interactive widgets for the Jupyter notebook.
+- Jupyter Notebook: The traditional web application for creating and sharing Jupyter notebooks.
+- Jupyter Kernels: Focused on the computational engines for various programming languages.
+- jupyter-xeus: A collection of Jupyter kernels.
+- IPykernel: The IPython kernel for Jupyter.
+- IPython: Provides a rich toolkit to help you make the most out of using Python interactively.
+- Jupyter Foundations and Standards: Focuses on foundational components and standards across Jupyter projects.
+- Jupyter Security: Addresses security-related aspects of Jupyter projects.
+- Jupyter Accessibility: Dedicated to ensuring Jupyter projects are accessible to all users.
+
+Smaller, less active subprojects include:
+
+- nbdime: Tools for diffing and merging Jupyter notebooks.
+- nbgrader: A system for assigning and grading Jupyter notebooks.
+- nbviewer: Shares Jupyter notebooks as static HTML on the web.
+- ipyparallel: A package for parallel and distributed computing in Python using IPython.
+- QtConsole: A rich Qt-based console for working with Jupyter kernels, maintained by the Spyder team.
+
+Additionally, there are various other repositories that form part of the broader Jupyter ecosystem, including services like Binder and nbviewer. These services are generally managed by working groups that report to the Executive Council of Project Jupyter.
+
+For a complete and detailed list, you can refer to the official Jupyter documentation and the Jupyter governance repository on GitHub.
+
+## Official list diagram
+
+- https://jupyter.org/governance/list_of_subprojects.html
+
+## Jupyter Project Subprojects (website)
+
+### Official Subprojects with SSC Representation
+
+- **JupyterLab**
+- **JupyterHub and Binder**
+  - JupyterHub
+  - Binder
+  - BinderHub
+- **Voilà**
+- **Jupyter Server**
+  - Jupyter Server
+  - Enterprise Gateway
+  - Kernel Gateway
+- **Jupyter Widgets**
+- **Jupyter Notebook**
+- **Jupyter Kernels**
+  - jupyter-xeus
+  - IPykernel
+  - IPython
+- **Jupyter Foundations and Standards**
+- **Jupyter Security**
+- **Jupyter Accessibility**
+
+### Official Subprojects without SSC Representation
+
+- nbdime (jupyter/nbdime)
+- nbgrader (jupyter/nbgrader)
+- nbviewer (jupyter/nbviewer)
+- ipyparallel (ipython/ipyparallel)
+- QtConsole (jupyter/qtconsole)
+
+### Part of Jupyter Foundations and Standards
+
+- Jupyter Client (jupyter/jupyter_client)
+- Jupyter Notebook Format (jupyter/nbformat)
+- JEPs repo (jupyter/enhancement-proposals)
+- nbconvert (jupyter/nbconvert)
+- Jupyter Core (jupyter/jupyter_core)
+- Jupyter-packaging (jupyter/jupyter-packaging)
+- Terminado (jupyter/terminado)
+- nbclient (jupyter/nbclient)
+- Telemetry (jupyter/telemetry)
+- Traitlets (ipython/traitlets)
+- Jupyter Console (jupyter/jupyter_console)
+- Docker stacks (jupyter/docker-stacks)
+
+

what_is_jupyter.md

@@ -0,0 +1,94 @@
+# What is Jupyter
+
+[jupyter.org/about]: https://jupyter.org/about
+[subprojects]: https://jupyter.org/governance/list_of_subprojects.html
+
+There is a lot in the name *Jupyter*; the project, the software, the notebooks.
+It is not unusual to fill confused with the many terms around "Jupyter".
+But confusion is never nice so let's try to clarify them.
+
+If you are completely new to *Jupyter*, the [`jupyter.org/about`][jupyter.org/about]
+page is good reading for an overview of the **project**.
+Completely open-source, the software developed by the project grew substantially
+as to be composed by many pieces; those *pieces* (Notebook, Lab, Hub)
+are developed by the different Jupyter [**subprojects**][subprojects]
+
+Let's break "Jupyter" in its components so we can clear them apart
+and have a better view of the *software*.
+
+
+## *Notebook* vs *notebook*
+
+Let's start from the most popular concept, what most of the discussions
+about *Jupyter* talk about: the "notebook".
+
+"Notebook" has two meanings: the notebook file format -- the `.ipynb` files --,
+and the Notebook graphical user interface -- the application -- we use to create
+our notebooks.
+
+
+## *Notebook* and *Lab*
+
+There are two graphical user interface (software) to edit notebooks:
+Jupyter-Notebook and Jupyter-Lab.
+
+As presented in the [*History of Jupyter*](history_of_jupyter.md), Jupyter-Lab
+is an evolution of Jupyter-Notebook to provide a more concise interface to
+the many activities in that environment such as files management, simultaneous
+view of multiple notebooks or parallel views of the notebook.
+
+The Notebook keeps providing the classic interface, that may be advantageous
+for some communities.
+
+## The *Hub*
+
+To many users not used to the management of software systems,
+setting up a Jupyter Notebook or Lab may be an obstacle to the final goal,
+that of data analysis.
+Furthermore, a it is desirable in many scenarios to have a common software
+setup shared among a team of data analists; a multi-user system.
+
+Jupyter-Hub is a multi-user Jupyter system, responsible for managing
+multiple Jupyter Notebook or Lab servers.
+
+## Jupyter Server
+
+In the previous sections we learned the difference -- or similarity -- of
+Jupyter Notebook and Jupyter Lab, and the role of Jupyter Hub in managing
+those *servers*.
+
+The slightly different user interfaces provided by Notebook and Lab are
+backed by the same engine: the Jupyter Server.
+Jupyter Server is responsible for rendering the (`.ipynb`) notebooks,
+managing configurations, and integrating the extensions.
+
+## Kernels
+
+Kernels are at the very core of the Jupyter software ecosystem, they are
+responsible for interpreting the code cells in a notebook.
+When a Jupyter application -- Notebook or Lab, for instance -- is installed,
+a Python kernel is provided by default.
+But, as you may already know, other programming languages can be used in
+notebooks, such as R and Julia, provided the corresponding kernels are
+installed.
+
+Kernels are the abstraction layer between the Jupyter application and
+different programming languages.
+
+## IPython
+
+If you use Python, you know you can run Python code directly in the command-line
+interface through the `python` interactive interpreter (aka, Python *shell*).
+
+IPython is the Python shell with super-powers.
+IPython is where Jupyter started (see [History of Jupyter](history_of_jupyter.md)),
+a command-line interface with high-level functionalities such as the
+[*magic commands*](https://ipython.readthedocs.io/en/stable/interactive/magics.html)
+(available in Jupyter Notebook and Lab).
+
+As a matter of fact, the extension used for Jupyter notebook files, `ipynb`,
+is reminiscent from when Jupyter was simply IPython Notebooks.
+Besides the files extension and magic commands, IPython is very much alive and integrated
+in the Jupyter ecosystem through `ipykernel`, the Python kernel engine.
+(To be more precise and technically correct, the magic commands are
+part of ipykernel's job.)