imports to add to pyforest

.gitignore

@@ -1,5 +1,9 @@
 .DS_Store
 
+# as long as the pyforest extensions don't need to be build, we can ignore the JS build folders:
+src/pyforest/node_modules
+src/pyforest/yarn.lock
+
 # Temporary and binary files
 *~
 *.py[cod]
@@ -53,3 +57,6 @@ MANIFEST
 .vscode/*
 examples/*.py
 anaconda_credentials.txt
+
+# User-specific folders
+devstuff

CHANGELOG.rst

@@ -2,6 +2,31 @@
 Changelog
 =========
 
+Version 1.1.2
+=============
+
+- https://github.com/8080labs/pyforest/pull/56 Add "random" package to lazy imports
+
+Version 1.1.1
+=============
+
+- https://github.com/8080labs/pyforest/pull/49 disable non-jupyter console from being polluted with lazy import print statements.
+
+Version 1.1.0
+=============
+
+- import statements are now automatically added to the first *CODE* cell when a package is used in the notebook
+
+Version 1.0.0
+=============
+
+- import statements are now automatically added to the first cell when a package is used in the notebook
+
+Version 0.1.4
+=============
+
+- Users can now add own lazy imports by writing their import statements to ~/.pyforest/user_imports.py.
+
 Version 0.1
 ===========
 

MANIFEST.in

@@ -0,0 +1,2 @@
+include src/pyforest/static/*
+include src/pyforest/*.json

README.md

@@ -1,46 +1,68 @@
-# pyforest - lazy-import of all popular Python Data Science libraries. Stop writing the same imports over and over again.
+# pyforest - feel the bliss of automated imports
+
+### From the makers of [bamboolib](https://bamboolib.com)
+
+Writing the same imports over and over again is below your capacity. Let pyforest do the job for you.
+
+
+With pyforest you can use all your favorite Python libraries without importing them before.
+If you use a package that is not imported yet, pyforest imports the package for you and adds the code to the first Jupyter cell. If you don't use a library, it won't be imported.
 
-pyforest lazy-imports all popular Python Data Science libraries so that they are always there when you need them. If you don't use a library, it won't be imported. When you are done with your script, you can export the Python code for the import statements.
 
 - [Demo in Jupyter Notebook](#demo-in-jupyter-notebook)
-- [Demo in Python Shell](#demo-in-python-shell)
+- [Scenario](#scenario)
 - [Using pyforest](#using-pyforest)
 - [Installation](#installation)
 - [FAQs](#frequently-asked-questions)
 - [Contributing](#contributing)
-- [Using pyforest as Package Developer](#using-pyforest-as-package-developer)
 - [About](#about)
-- [Join our community and grow further](#join-our-community-and-grow-further)
 
 
 
 ## Demo in Jupyter Notebook
-![demo](examples/assets/pyforest_demo_in_jupyter.gif)
+![demo](examples/assets/pyforest_demo_in_jupyter_notebook.gif)
+
+
+## Scenario
+
+You are a Data Scientist who works with Python. Every day you start multiple new Jupyter notebooks because you want to explore some data or validate a hypothesis.
+
+During your work, you use many different libraries like `pandas`, `matplotlib`, `seaborn`, `numpy` or `sklearn`. However, before you can start with the actual work, you always need to import your libraries.
+
+
+There are several __problems__ with this. Admittedly, they are small but they add up over time.
+- It is boring because the imports are mostly the same. This is below your capacity.
+- Missing imports disrupt the natural flow of your work.
+- Sometimes, you may even need to look up the exact import statements. For example, `import matplotlib.pyplot as plt` or `from sklearn.ensemble import GradientBoostingRegressor`
 
+__What if you could just focus on using the libraries?__
 
-## Demo in Python Shell
-![demo](examples/assets/pyforest_demo_in_python_shell.png)
+pyforest offers the following __solution__:
+- You can use all your libraries like you usually do. If a library is not imported yet, pyforest will import it and add the import statement to the first Jupyter cell.
+- If a library is not used, it won't be imported.
+- Your notebooks stay reproducible and sharable without you wasting a thought on imports.
 
 
 ## Using pyforest
-pyforest lazy-imports all popular Python Data Science libraries with a single line of code:
 
-```python
-from pyforest import *
-```
+After you [installed](#installation) pyforest and its Jupyter extension, you can __use your favorite Python Data Science commands like you normally would - just without writing imports__.
 
-And if you use Jupyter or IPython, you can even skip this line because pyforest adds itself to the autostart.
+For example, if you want to read a CSV with pandas:
 
-When you are done with your script, you can export all import statements via:
+```python
+df = pd.read_csv("titanic.csv")
+```
 
+pyforest will automatically import pandas for you and add the import statement to the first cell:
 ```python
-active_imports()
+import pandas as pd
 ```
 
-Which libraries are available?
-- We aim to add all popular Python Data Science libraries which should account for >99% of your daily imports. For example, `pandas` as `pd`, `numpy` as `np`, `seaborn` as `sns`, `matplotlib.pyplot` as `plt`, or `OneHotEncoder` from `sklearn` and many more. In addition, there are also helper modules like `os`, `re`, `tqdm`, or `Path` from `pathlib`.
-- You can see an overview of all available lazy imports if you type `lazy_imports()` in Python.
-- If you are missing an import, you can add it to the [pyforest imports](src/pyforest/_imports.py).
+
+__Which libraries are available?__
+- We aim to add all popular Python Data Science libraries which should account for >99% of your daily imports. For example, we already added `pandas` as `pd`, `numpy` as `np`, `seaborn` as `sns`, `matplotlib.pyplot` as `plt`, or `OneHotEncoder` from `sklearn` and many more. In addition, there are also helper modules like `os`, `re`, `tqdm`, or `Path` from `pathlib`.
+- You can see an overview of all currently available imports [here](src/pyforest/_imports.py)
+- If you are missing an import, you can either __add the import to your user specific pyforest imports__ as described in the [FAQs](#frequently-asked-questions) or you can open a pull request for the official [pyforest imports](src/pyforest/_imports.py)
 
 > In order to gather all the most important names, we need your help. Please open a pull request and add the [imports](src/pyforest/_imports.py) that we are still missing.
 
@@ -49,19 +71,22 @@ Which libraries are available?
 
 > You need Python 3.6 or above because we love f-strings.
 
-From the terminal, enter:
+From the terminal (or Anaconda prompt in Windows), enter:
 
-`pip install pyforest`
+```bash
+pip install --upgrade pyforest
+python -m pyforest install_extensions
+```
 
-And you're ready to go.
+Please make sure to restart any running Jupyter server so that the javascript extension can be loaded properly.
 
-Please note, that this will also add pyforest to your IPython default startup settings.
+Also, please note that this will add pyforest to your IPython default startup settings. If you do not want this, you can disable the auto_import as described in the [FAQs](#frequently-asked-questions) below.
 
 
 ## Frequently Asked Questions
 
-- __"I need to always explicitly write down the libraries I used at the top of my scripts."__
-    - Of course, you can export the import statements for all used libraries with `active_imports()`.
+- __"How to add my own import statements without adding them to the package source code?"__
+    - pyforest creates a file in your home directory at `~/.pyforest/user_imports.py` in which you can type any **explicit** import statements you want (e.g. `import pandas as pd`). Your own custom imports take precedence over any other pyforest imports. **Please note:** implicit imports (e.g. `from pandas import *`) won't work.
 
 - __"Doesn't this slow down my Jupyter or Python startup process?"__
     - No, because the libraries will only be imported when you actually use them. Until you use them, the variables like `pd` are only pyforest placeholders.
@@ -73,63 +98,29 @@ Please note, that this will also add pyforest to your IPython default startup se
     - Tensorflow is included in pyforest but pyforest does not install any dependencies. You need to install your libraries separately from pyforest. Afterwards, you can access the libraries via pyforest if they are included in the [pyforest imports](src/pyforest/_imports.py).
 
 - __"Will the pyforest variables interfere with my own local variables?"__
-    - Please make sure that you import pyforest at the beginning of your script. Then you will always be safe. You can use your variables like you would without pyforest. The worst thing that can happen is that you overwrite a pyforest placeholder and thus cannot use the placeholder any more (duh).
+    - No, never. pyforest will never mask or overwrite any of your local variables. You can use your variables like you would without pyforest. The worst thing that can happen is that you overwrite a pyforest placeholder and thus cannot use the placeholder any more (duh).
 
 - __"What about auto-completion on lazily imported modules?"__
     - It works :) As soon as you start the auto-completion, pyforest will import the module and return the available symbols to your auto-completer.
 
 - __"How to (temporarily) deactivate the auto_import in IPython and Jupyter?"__
-    - Go to the directory `~/.ipython/profile_default/startup` and adjust or delete the `pyforest_autoimport.py` file. You will find further instructions in the file.
+    - Go to the directory `~/.ipython/profile_default/startup` and adjust or delete the `pyforest_autoimport.py` file. You will find further instructions in the file. If you don't use the auto_import, you will need to import pyforest at the beginning of your notebook via `import pyforest`
 
 - __"How to (re)activate the pyforest auto_import?"__
     - Execute the following Python command in Jupyter, IPython or Python: `from pyforest.auto_import import setup; setup()`. Please note that the auto_import only works for Jupyter and IPython.
 
-- __"Why is pandas_profiling also imported in the demo?"__
-    - pyforest supports complementary, optional imports. For example, `pandas_profiling` patches the `pd.DataFrame` with the convenience function `df.profile_report`. Therefore, pyforest also imports `pandas_profiling` if you have it installed. If you don't have `pandas_profiling` installed, the optional import will be skipped.
-
-- __"I don't want to copy complementary import statements to the top of my file."__
-    - Please note, that the complementary imports will always appear at the bottom of the import_statements list. So, you can just copy all statements above. Alternatively, you can deactivate complementary imports.
-
-- __"How to deactivate complementary imports?"__
-    - You can uncomment the statements `*.__on_import__()` at the bottom of the [pyforest imports](src/pyforest/_imports.py) file.
-
-- __"How to add my own import statements without adding them to the package source code?"__
-    - pyforest creates a file `~/.pyforest/user_imports.py`, in which you can type any **explicit** import statements you want. **Please note:** implicit imports (e.g. `from pandas import *`) won't work. Besides, you shouldn't write implicit imports anyway. Those are only for bad programmers like the authors of pyforest :)
+- __"Can I use pyforest outside of the Jupyter Notebook or Lab?"__
+    - Technically, yes. However, this is not the intended use case. pyforest is aimed primarily for the use in a Jupyter Notebook or Lab. If you want to use pyforest in IPython or a Python script etc, please import it as follows `import pyforest`. Afterwards, you can get the currently active imports via `pyforest.active_imports()`
 
 - __"Why is the project called pyforest?"__
-    - In which ecosystem do pandas live?
+    - pyforest is created to be the home for all Data Science packages - including pandas. And in which ecosystems do pandas live? :)
 
 
 ## Contributing
-In order to gather all the most important names, we need your help. Please open a pull request and add the imports that we are still missing to the [pyforest imports](src/pyforest/_imports.py). You can also find the guidelines in the [pyforest imports file](src/pyforest/_imports.py)
-
-
-## Using pyforest as Package Developer
-pyforest helps you to minimize the (initial) import time of your package which improves the user experience. If you want your package imports to become lazy, rewrite your imports as follows:
-
-Replace
-
-```python
-import pandas as pd
-```
-
-with
+If you'd like to contribute, a great place to look is the [issues marked with help-wanted](https://github.com/8080labs/pyforest/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22).
 
-```python
-from pyforest import LazyImport
-pd = LazyImport("import pandas as pd")
-```
+In order to gather all the most important names, we need your help. Please open a pull request and add the imports that we are still missing to the [pyforest imports](src/pyforest/_imports.py). You can also find the guidelines in the [pyforest imports file](src/pyforest/_imports.py)
 
 
 ## About
-pyforest is developed by Florian, Tobias and Guido from [8080 Labs](https://8080labs.com). Our goal is to improve the productivity of Python Data Scientists. Other projects that we are working on are [edaviz](https://edaviz.com) and [bamboolib](https://bamboolib.com)
-
-
-## Join our community and grow further
-If you
-- like our work or
-- want to become a faster Python Data Scientist or
-- want to discuss the future of the Python Data Science ecosystem or
-- are just interested in mingling with like-minded fellows
-
-then, you are invited to [join our slack](https://join.slack.com/t/fasterpyds/shared_invite/enQtNzYxMTMzMDQ4MDk3LTYyNGRiNTE0OGJkNDEzZGRjNjg2Y2I0YWRlNTlmOGUxMjY5MDY5Yjg1MjliM2QwNmNhZmI3N2MxMmY3MGNiODA).
+pyforest is developed by [8080 Labs](https://8080labs.com). Our goal is to make Python Data Scientists 10x faster. If you like the speedup to your workflow, you might also be interested in our other project [bamboolib](https://bamboolib.com)

README_for_devs.md

@@ -0,0 +1,54 @@
+
+## How to install the local python version during development
+```bash
+pip install -e .  # alternatively, use pip3
+```
+
+## How to install the extensions during development
+
+### JupyterLab development
+
+Via terminal, it's the same procedure as normal installation:
+
+```bash
+python -m pyforest install_labextension
+```
+
+It is also possible via Python:
+```python
+import pyforest
+pyforest.install_labextension()  # takes 30-60s due to jupyter lab build
+```
+
+Run JupyterLab in watch mode
+```
+jupyter lab --watch
+```
+
+When you make changes on the javascript side, refresh the browser (clear cache) for changes to take effect.
+### Jupyter Notebook
+
+Via terminal, it's the same procedure as normal installation:
+
+```bash
+python -m pyforest install_nbextension
+```
+
+It is also possible via Python:
+```python
+import pyforest
+pyforest.install_nbextension()
+```
+
+Run Notebook
+```
+jupyter notebook
+```
+
+When you make changes on the javascript side, you need to install nbextension again.
+```bash
+python -m pyforest install_nbextension
+```
+
+## Syntax formatting
+We use `black` for formatting the Python code

dockerfiles/pyforest_sandbox/Dockerfile

@@ -0,0 +1,14 @@
+FROM jupyter/minimal-notebook
+
+USER root
+
+COPY ./test.ipynb ./
+
+# Install Python 3 packages
+RUN conda install --quiet --yes \
+    'Cython' \
+    && \
+    conda clean --all -f -y
+
+RUN pip install pyforest==1.0.2 pandas numpy seaborn
+RUN python -m pyforest install_extensions

dockerfiles/pyforest_sandbox/commands.sh

@@ -0,0 +1,3 @@
+docker build -t 8080labs/pyforest_sandbox . && \
+say "docker update jupypter notebook sandbox ready" && \
+docker run --rm -p 8888:8888 8080labs/pyforest_sandbox

dockerfiles/pyforest_sandbox/test.ipynb

@@ -0,0 +1,46 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 2,
+ "metadata": {
+  "language_info": {
+   "name": "python",
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "version": "3.7.3-final"
+  },
+  "orig_nbformat": 2,
+  "file_extension": ".py",
+  "mimetype": "text/x-python",
+  "name": "python",
+  "npconvert_exporter": "python",
+  "pygments_lexer": "ipython3",
+  "version": 3,
+  "kernelspec": {
+   "name": "python37364bitbaseconda9dbe7a38796a4966923d0aec411ee3e8",
+   "display_name": "Python 3.7.3 64-bit ('base': conda)"
+  }
+ },
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import pyforest"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df = pd.DataFrame(dict(a=np.arange(10)))\n",
+    "sns.distplot(df.a)"
+   ]
+  }
+ ]
+}