Skip to content

Commit d39ff21

Browse files
deploy: 05d15eb
0 parents  commit d39ff21

File tree

174 files changed

+77367
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

174 files changed

+77367
-0
lines changed

.nojekyll

Whitespace-only changes.

CNAME

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nbdev.fast.ai

api/clean.html

Lines changed: 1048 additions & 0 deletions
Large diffs are not rendered by default.

api/clean.html.md

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# clean
2+
3+
4+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
5+
6+
To avoid pointless conflicts while working with jupyter notebooks (with
7+
different execution counts or cell metadata), it is recommended to clean
8+
the notebooks before committing anything (done automatically if you
9+
install the git hooks with
10+
[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks)).
11+
The following functions are used to do that.
12+
13+
## Trust
14+
15+
------------------------------------------------------------------------
16+
17+
<a
18+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L25"
19+
target="_blank" style="float:right; font-size:smaller">source</a>
20+
21+
### nbdev_trust
22+
23+
> nbdev_trust (fname:str=None, force_all:bool=False)
24+
25+
*Trust notebooks matching `fname`*
26+
27+
- `fname:str=None`   *A notebook name or glob to trust*
28+
- `force_all:bool=False`   *Also trust notebooks that haven’t changed*
29+
- `return`
30+
31+
## Clean
32+
33+
------------------------------------------------------------------------
34+
35+
<a
36+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L86"
37+
target="_blank" style="float:right; font-size:smaller">source</a>
38+
39+
### clean_nb
40+
41+
> clean_nb (nb, clear_all=False, allowed_metadata_keys:list=None,
42+
> allowed_cell_metadata_keys:list=None, clean_ids=True)
43+
44+
*Clean `nb` from superfluous metadata*
45+
46+
- `nb`   *The notebook to clean*
47+
- `clear_all:bool=False`   *Remove all cell metadata and cell outputs?*
48+
- `allowed_metadata_keys:list=None`   *Preserve the list of keys in the
49+
main notebook metadata*
50+
- `allowed_cell_metadata_keys:list=None`   *Preserve the list of keys in
51+
cell level metadata*
52+
- `clean_ids:bool=True`   *Remove ids from plaintext reprs?*
53+
- `return`
54+
55+
Jupyter adds a trailing <code></code> to images in cell outputs.
56+
Vscode-jupyter does not.
57+
Notebooks should be brought to a common style to avoid unnecessary
58+
diffs:
59+
60+
``` python
61+
test_nb = read_nb('../../tests/image.ipynb')
62+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] == "\n" # Make sure it was not converted by acccident
63+
clean_nb(test_nb)
64+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] != "\n"
65+
```
66+
67+
The test notebook has metadata in both the main metadata section and
68+
contains cell level metadata in the second cell:
69+
70+
``` python
71+
test_nb = read_nb('../../tests/metadata.ipynb')
72+
73+
assert {'meta', 'jekyll', 'my_extra_key', 'my_removed_key'} <= test_nb.metadata.keys()
74+
assert {'meta', 'hide_input', 'my_extra_cell_key', 'my_removed_cell_key'} == test_nb.cells[1].metadata.keys()
75+
```
76+
77+
After cleaning the notebook, all extra metadata is removed, only some
78+
keys are allowed by default:
79+
80+
``` python
81+
clean_nb(test_nb)
82+
83+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
84+
assert {'hide_input'} == test_nb.cells[1].metadata.keys()
85+
```
86+
87+
We can preserve some additional keys at the notebook or cell levels:
88+
89+
``` python
90+
test_nb = read_nb('../../tests/metadata.ipynb')
91+
clean_nb(test_nb, allowed_metadata_keys={'my_extra_key'}, allowed_cell_metadata_keys={'my_extra_cell_key'})
92+
93+
assert {'jekyll', 'kernelspec', 'my_extra_key'} == test_nb.metadata.keys()
94+
assert {'hide_input', 'my_extra_cell_key'} == test_nb.cells[1].metadata.keys()
95+
```
96+
97+
Passing `clear_all=True` removes everything from the cell metadata:
98+
99+
``` python
100+
test_nb = read_nb('../../tests/metadata.ipynb')
101+
clean_nb(test_nb, clear_all=True)
102+
103+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
104+
test_eq(test_nb.cells[1].metadata, {})
105+
```
106+
107+
Passing `clean_ids=True` removes `id`s from plaintext repr outputs, to
108+
avoid notebooks whose contents change on each run since they often lead
109+
to git merge conflicts. For example:
110+
111+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FB4F8979690>
112+
113+
becomes:
114+
115+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28>
116+
117+
------------------------------------------------------------------------
118+
119+
<a
120+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L109"
121+
target="_blank" style="float:right; font-size:smaller">source</a>
122+
123+
### process_write
124+
125+
> process_write (warn_msg, proc_nb, f_in, f_out=None, disp=False)
126+
127+
------------------------------------------------------------------------
128+
129+
<a
130+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L132"
131+
target="_blank" style="float:right; font-size:smaller">source</a>
132+
133+
### nbdev_clean
134+
135+
> nbdev_clean (fname:str=None, clear_all:bool=False, disp:bool=False,
136+
> stdin:bool=False)
137+
138+
*Clean all notebooks in `fname` to avoid merge conflicts*
139+
140+
- `fname:str=None`   *A notebook name or glob to clean*
141+
- `clear_all:bool=False`   *Remove all cell metadata and cell outputs?*
142+
- `disp:bool=False`   *Print the cleaned outputs*
143+
- `stdin:bool=False`   *Read notebook from input stream*
144+
- `return`
145+
146+
By default (`fname` left to `None`), all the notebooks in
147+
`config.nbs_path` are cleaned. You can opt in to fully clean the
148+
notebook by removing every bit of metadata and the cell outputs by
149+
passing `clear_all=True`.
150+
151+
If you want to keep some keys in the main notebook metadata you can set
152+
`allowed_metadata_keys` in `settings.ini`. Similarly for cell level
153+
metadata use: `allowed_cell_metadata_keys`. For example, to preserve
154+
both `k1` and `k2` at both the notebook and cell level adding the
155+
following in `settings.ini`:
156+
157+
...
158+
allowed_metadata_keys = k1 k2
159+
allowed_cell_metadata_keys = k1 k2
160+
...
161+
162+
------------------------------------------------------------------------
163+
164+
<a
165+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L147"
166+
target="_blank" style="float:right; font-size:smaller">source</a>
167+
168+
### clean_jupyter
169+
170+
> clean_jupyter (path, model, **kwargs)
171+
172+
*Clean Jupyter `model` pre save to `path`*
173+
174+
This cleans notebooks on-save to avoid unnecessary merge conflicts. The
175+
easiest way to install it for both Jupyter Notebook and Lab is by
176+
running
177+
[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks).
178+
It works by implementing a `pre_save_hook` from Jupyter’s [file save
179+
hook
180+
API](https://jupyter-server.readthedocs.io/en/latest/developers/savehooks.html).
181+
182+
## Hooks
183+
184+
------------------------------------------------------------------------
185+
186+
<a
187+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L189"
188+
target="_blank" style="float:right; font-size:smaller">source</a>
189+
190+
### nbdev_install_hooks
191+
192+
> nbdev_install_hooks ()
193+
194+
*Install Jupyter and git hooks to automatically clean, trust, and fix
195+
merge conflicts in notebooks*
196+
197+
See
198+
[`clean_jupyter`](https://nbdev.fast.ai/api/clean.html#clean_jupyter)
199+
and [`nbdev_merge`](https://nbdev.fast.ai/api/merge.html#nbdev_merge)
200+
for more about how each hook works.

0 commit comments

Comments
 (0)