Merge remote-tracking branch 'origin/master' into scrub-magics-squash

Author: Eugene Wolfson

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,28 @@
+---
+name: Bug report
+about: Create a minimal reproducible example to help us improve
+labels: ["bug"]
+
+---
+
+# Provide a minimally reproducible example
+
+To maximize the chances of your issue being fixed, you should share a minimally reproducible example that allows us to reproduce your error.  A good way to do this is to setup a new nbdev project which containts the minimal amount of code that reproduces your error.  
+
+If we are not able to reproduce your error, we may close your issue.
+
+## Some background on minimal reproducible examples [^1]
+
+The first benefit of a public reproduction is to prove that the problem is not caused by your environment or by a setting you left out of your description, thinking it wasn't relevant. If there were any doubts about whether you'd found a genuine problem before, they are usually removed once a reproduction is made.
+
+Next, when a reproduction has minimal config and code, it can often let us narrow down or even identify the root cause, suggest workarounds, etc. This means we can often help you from code inspection alone.
+
+Finally, by making the code/dependencies minimal, it usually makes the problem feasible to step through using a debugging if code inspection wasn't sufficient. Production repositories or non-minimal reproductions are often very difficult to debug because break points get triggered dozens or hundreds or times.
+
+The basic idea of a minimal reproduction is to use the least amount of both code and config to trigger missing or wrong behavior. A minimal reproduction helps the developers see where the bug or missing feature is, and allows us to verify that the new code meets the requirements.
+
+## Where to host your mimimal reproducible example
+
+A new, **public** GitHub repo is the best place to host your minimal reproducible example. 
+
+[^1]: the below guidance was adapted [from this language](https://github.com/renovatebot/renovate/blob/main/docs/development/minimal-reproductions.md)

.github/workflows/deploy.yaml

@@ -8,4 +8,4 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: fastai/workflows/quarto-ghp@master
-        with: {pre: 1}
+#         with: {pre: 1}

.pre-commit-hooks.yaml

@@ -3,7 +3,7 @@
   name: nbdev_clean
   entry: nbdev_clean
   description: "Clean metadata from notebooks to avoid merge conflicts"
-  language: system
+  language: python
   always_run: true
   pass_filenames: false
 

CHANGELOG.md

@@ -2,6 +2,26 @@
 
 <!-- do not remove -->
 
+## 2.3.11
+
+### New Features
+
+- add agplv3 license to project options ([#1268](https://github.com/fastai/nbdev/pull/1268)), thanks to [@joel-lbth](https://github.com/joel-lbth)
+- add 0 - Pre-planning status to project options ([#1262](https://github.com/fastai/nbdev/pull/1262)), thanks to [@ddobrinskiy](https://github.com/ddobrinskiy)
+- add conda installation docs to end-to-end tutorial ([#1215](https://github.com/fastai/nbdev/pull/1215)), thanks to [@restlessronin](https://github.com/restlessronin)
+
+### Bugs Squashed
+
+- Fix `nbdev_conda` error when there are prior release tarballs ([#1280](https://github.com/fastai/nbdev/pull/1280)), thanks to [@restlessronin](https://github.com/restlessronin)
+- delete build dir before pypi upload ([#1277](https://github.com/fastai/nbdev/pull/1277)), thanks to [@restlessronin](https://github.com/restlessronin)
+- copy contents of `_extensions` folder from nbs ([#1266](https://github.com/fastai/nbdev/pull/1266)), thanks to [@restlessronin](https://github.com/restlessronin)
+- Fix `nbdev_update` when `lib_name` and `lib_path` are not the same ([#1254](https://github.com/fastai/nbdev/pull/1254)), thanks to [@BirkhoffG](https://github.com/BirkhoffG)
+- Make `nbdev_create_config` write the `black_formatting` setting ([#1235](https://github.com/fastai/nbdev/pull/1235)), thanks to [@dmose](https://github.com/dmose)
+- fix order of categories for migration ([#1221](https://github.com/fastai/nbdev/pull/1221)), thanks to [@hamelsmu](https://github.com/hamelsmu)
+- make anaconda description links consistent with pypi ([#1214](https://github.com/fastai/nbdev/pull/1214)), thanks to [@restlessronin](https://github.com/restlessronin)
+- .get instead of acessing attr ([#1023](https://github.com/fastai/nbdev/pull/1023)), thanks to [@hamelsmu](https://github.com/hamelsmu)
+
+
 ## 2.3.9
 
 ### New Features

nbdev/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "2.3.10"
+__version__ = "2.3.12"
 
 from .doclinks import nbdev_export
 from .showdoc import show_doc

nbdev/_modidx.py

@@ -6,6 +6,7 @@
                 'git_url': 'https://github.com/fastai/nbdev',
                 'lib_path': 'nbdev'},
   'syms': { 'nbdev.clean': { 'nbdev.clean._add_jupyter_hooks': ('api/clean.html#_add_jupyter_hooks', 'nbdev/clean.py'),
+                             'nbdev.clean._add_trailing_n': ('api/clean.html#_add_trailing_n', 'nbdev/clean.py'),
                              'nbdev.clean._clean_cell': ('api/clean.html#_clean_cell', 'nbdev/clean.py'),
                              'nbdev.clean._clean_cell_output': ('api/clean.html#_clean_cell_output', 'nbdev/clean.py'),
                              'nbdev.clean._clean_cell_output_id': ('api/clean.html#_clean_cell_output_id', 'nbdev/clean.py'),
@@ -232,12 +233,22 @@
                            'nbdev.qmd.meta': ('api/qmd.html#meta', 'nbdev/qmd.py'),
                            'nbdev.qmd.tbl_row': ('api/qmd.html#tbl_row', 'nbdev/qmd.py'),
                            'nbdev.qmd.tbl_sep': ('api/qmd.html#tbl_sep', 'nbdev/qmd.py')},
-            'nbdev.quarto': { 'nbdev.quarto._ensure_quarto': ('api/quarto.html#_ensure_quarto', 'nbdev/quarto.py'),
+            'nbdev.quarto': { 'nbdev.quarto._SidebarYmlRemoved': ('api/quarto.html#_sidebarymlremoved', 'nbdev/quarto.py'),
+                              'nbdev.quarto._SidebarYmlRemoved.__enter__': ( 'api/quarto.html#_sidebarymlremoved.__enter__',
+                                                                             'nbdev/quarto.py'),
+                              'nbdev.quarto._SidebarYmlRemoved.__exit__': ( 'api/quarto.html#_sidebarymlremoved.__exit__',
+                                                                            'nbdev/quarto.py'),
+                              'nbdev.quarto._SidebarYmlRemoved.__init__': ( 'api/quarto.html#_sidebarymlremoved.__init__',
+                                                                            'nbdev/quarto.py'),
+                              'nbdev.quarto._copytree': ('api/quarto.html#_copytree', 'nbdev/quarto.py'),
+                              'nbdev.quarto._ensure_quarto': ('api/quarto.html#_ensure_quarto', 'nbdev/quarto.py'),
                               'nbdev.quarto._install_linux': ('api/quarto.html#_install_linux', 'nbdev/quarto.py'),
                               'nbdev.quarto._install_mac': ('api/quarto.html#_install_mac', 'nbdev/quarto.py'),
                               'nbdev.quarto._nbglob_docs': ('api/quarto.html#_nbglob_docs', 'nbdev/quarto.py'),
                               'nbdev.quarto._pre': ('api/quarto.html#_pre', 'nbdev/quarto.py'),
                               'nbdev.quarto._pre_docs': ('api/quarto.html#_pre_docs', 'nbdev/quarto.py'),
+                              'nbdev.quarto._readme_mtime_not_older': ('api/quarto.html#_readme_mtime_not_older', 'nbdev/quarto.py'),
+                              'nbdev.quarto._save_cached_readme': ('api/quarto.html#_save_cached_readme', 'nbdev/quarto.py'),
                               'nbdev.quarto._sort': ('api/quarto.html#_sort', 'nbdev/quarto.py'),
                               'nbdev.quarto._sprun': ('api/quarto.html#_sprun', 'nbdev/quarto.py'),
                               'nbdev.quarto.fs_watchdog': ('api/quarto.html#fs_watchdog', 'nbdev/quarto.py'),

nbdev/clean.py

@@ -1,9 +1,9 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/api/clean.ipynb.
+# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/api/11_clean.ipynb.
 
 # %% auto 0
 __all__ = ['nbdev_trust', 'clean_nb', 'process_write', 'nbdev_clean', 'clean_jupyter', 'nbdev_install_hooks']
 
-# %% ../nbs/api/clean.ipynb 2
+# %% ../nbs/api/11_clean.ipynb 2
 import ast,warnings,stat
 from astunparse import unparse
 from textwrap import indent
@@ -18,7 +18,7 @@
 from .sync import *
 from .process import first_code_ln
 
-# %% ../nbs/api/clean.ipynb 6
+# %% ../nbs/api/11_clean.ipynb 6
 @call_parse
 def nbdev_trust(
     fname:str=None,  # A notebook name or glob to trust
@@ -44,7 +44,7 @@ def nbdev_trust(
         if not NotebookNotary().check_signature(nb): NotebookNotary().sign(nb)
     check_fname.touch(exist_ok=True)
 
-# %% ../nbs/api/clean.ipynb 9
+# %% ../nbs/api/11_clean.ipynb 9
 _repr_id_re = re.compile('(<.*?)( at 0x[0-9a-fA-F]+)(>)')
 
 _sub = partial(_repr_id_re.sub, r'\1\3')
@@ -54,21 +54,26 @@ def _skip_or_sub(x): return _sub(x) if "at 0x" in x else x
 def _clean_cell_output_id(lines):
     return _skip_or_sub(lines) if isinstance(lines,str) else [_skip_or_sub(o) for o in lines]
 
-# %% ../nbs/api/clean.ipynb 11
+# %% ../nbs/api/11_clean.ipynb 11
+def _add_trailing_n(img):
+    if not isinstance(img,str): return [ _add_trailing_n(o) for o in img ]
+    return img + "\n" if img[-1] != "\n" else img
+
+# %% ../nbs/api/11_clean.ipynb 12
 def _clean_cell_output(cell, clean_ids):
     "Remove `cell` output execution count and optionally ids from text reprs"
     outputs = cell.get('outputs', [])
     for o in outputs:
         if 'execution_count' in o: o['execution_count'] = None
         data = o.get('data', {})
         data.pop("application/vnd.google.colaboratory.intrinsic+json", None)
-        if clean_ids:
-            for k in data:
-                if k.startswith('text'): data[k] = _clean_cell_output_id(data[k])
-            if 'text' in o: o['text'] = _clean_cell_output_id(o['text'])
+        for k in data:
+            if k.startswith('text') and clean_ids: data[k] = _clean_cell_output_id(data[k])
+            if k.startswith('image'): data[k] = _add_trailing_n(data[k])
+        if 'text' in o and clean_ids: o['text'] = _clean_cell_output_id(o['text'])
         o.get('metadata', {}).pop('tags', None)
 
-# %% ../nbs/api/clean.ipynb 12
+# %% ../nbs/api/11_clean.ipynb 13
 def _clean_cell(cell, clear_all, allowed_metadata_keys, clean_ids):
     "Clean `cell` by removing superfluous metadata or everything except the input if `clear_all`"
     if 'execution_count' in cell: cell['execution_count'] = None
@@ -79,7 +84,7 @@ def _clean_cell(cell, clear_all, allowed_metadata_keys, clean_ids):
     cell['metadata'] = {} if clear_all else {
         k:v for k,v in cell['metadata'].items() if k in allowed_metadata_keys}
 
-# %% ../nbs/api/clean.ipynb 13
+# %% ../nbs/api/11_clean.ipynb 14
 def clean_nb(
     nb, # The notebook to clean
     clear_all=False, # Remove all cell metadata and cell outputs?
@@ -88,6 +93,7 @@ def clean_nb(
     clean_ids=True, # Remove ids from plaintext reprs?
 ):
     "Clean `nb` from superfluous metadata"
+    assert isinstance(nb, AttrDict)
     metadata_keys = {"kernelspec", "jekyll", "jupytext", "doc", "widgets"}
     if allowed_metadata_keys: metadata_keys.update(allowed_metadata_keys)
     cell_metadata_keys = {"hide_input"}
@@ -97,33 +103,33 @@ def clean_nb(
         nb['metadata']['kernelspec']['display_name'] = nb.metadata.kernelspec.name
     nb['metadata'] = {k:v for k,v in nb['metadata'].items() if k in metadata_keys}
 
-# %% ../nbs/api/clean.ipynb 24
+# %% ../nbs/api/11_clean.ipynb 27
 def _reconfigure(*strms):
     for s in strms:
         if hasattr(s,'reconfigure'): s.reconfigure(encoding='utf-8')
 
-# %% ../nbs/api/clean.ipynb 25
+# %% ../nbs/api/11_clean.ipynb 28
 def process_write(warn_msg, proc_nb, f_in, f_out=None, disp=False):
     if not f_out: f_out = f_in
     if isinstance(f_in, (str,Path)): f_in = Path(f_in).open()
     try:
         _reconfigure(f_in, f_out)
-        nb = loads(f_in.read())
+        nb = dict2nb(loads(f_in.read()))
         proc_nb(nb)
         write_nb(nb, f_out) if not disp else sys.stdout.write(nb2str(nb))
     except Exception as e:
         warn(f'{warn_msg}')
         warn(e)
 
-# %% ../nbs/api/clean.ipynb 26
+# %% ../nbs/api/11_clean.ipynb 29
 def _nbdev_clean(nb, path=None, clear_all=None):
     cfg = get_config(path=path)
     clear_all = clear_all or cfg.clear_all
     allowed_metadata_keys = cfg.get("allowed_metadata_keys").split()
     allowed_cell_metadata_keys = cfg.get("allowed_cell_metadata_keys").split()
     return clean_nb(nb, clear_all, allowed_metadata_keys, allowed_cell_metadata_keys, cfg.clean_ids)
 
-# %% ../nbs/api/clean.ipynb 27
+# %% ../nbs/api/11_clean.ipynb 30
 @call_parse
 def nbdev_clean(
     fname:str=None, # A notebook name or glob to clean
@@ -139,15 +145,15 @@ def nbdev_clean(
     if fname is None: fname = get_config().nbs_path
     for f in globtastic(fname, file_glob='*.ipynb', skip_folder_re='^[_.]'): _write(f_in=f, disp=disp)
 
-# %% ../nbs/api/clean.ipynb 30
+# %% ../nbs/api/11_clean.ipynb 33
 def clean_jupyter(path, model, **kwargs):
     "Clean Jupyter `model` pre save to `path`"
     if not (model['type']=='notebook' and model['content']['nbformat']==4): return
     get_config.cache_clear() # Allow config changes without restarting Jupyter
     jupyter_hooks = get_config(path=path).jupyter_hooks
     if jupyter_hooks: _nbdev_clean(model['content'], path=path)
 
-# %% ../nbs/api/clean.ipynb 33
+# %% ../nbs/api/11_clean.ipynb 36
 _pre_save_hook_src = '''
 def nbdev_clean_jupyter(**kwargs):
     try: from nbdev.clean import clean_jupyter
@@ -157,7 +163,7 @@ def nbdev_clean_jupyter(**kwargs):
 c.ContentsManager.pre_save_hook = nbdev_clean_jupyter'''.strip()
 _pre_save_hook_re = re.compile(r'c\.(File)?ContentsManager\.pre_save_hook')
 
-# %% ../nbs/api/clean.ipynb 34
+# %% ../nbs/api/11_clean.ipynb 37
 def _add_jupyter_hooks(src, path):
     if _pre_save_hook_src in src: return
     mod = ast.parse(src)
@@ -175,12 +181,12 @@ def _add_jupyter_hooks(src, path):
     if src: src+='\n\n'
     return src+_pre_save_hook_src
 
-# %% ../nbs/api/clean.ipynb 38
+# %% ../nbs/api/11_clean.ipynb 41
 def _git_root(): 
     try: return Path(run('git rev-parse --show-toplevel'))
     except OSError: return None
 
-# %% ../nbs/api/clean.ipynb 41
+# %% ../nbs/api/11_clean.ipynb 44
 @call_parse
 def nbdev_install_hooks():
     "Install Jupyter and git hooks to automatically clean, trust, and fix merge conflicts in notebooks"

nbdev/cli.py

@@ -1,6 +1,6 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/api/cli.ipynb.
+# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/api/13_cli.ipynb.
 
-# %% ../nbs/api/cli.ipynb 2
+# %% ../nbs/api/13_cli.ipynb 2
 from __future__ import annotations
 import warnings
 
@@ -27,7 +27,7 @@
 # %% auto 0
 __all__ = ['nbdev_filter', 'extract_tgz', 'nbdev_new', 'chelp']
 
-# %% ../nbs/api/cli.ipynb 5
+# %% ../nbs/api/13_cli.ipynb 5
 @call_parse
 def nbdev_filter(
     nb_txt:str=None,  # Notebook text (uses stdin if not provided)
@@ -50,20 +50,20 @@ def nbdev_filter(
     if printit: print(res, flush=True)
     else: return res
 
-# %% ../nbs/api/cli.ipynb 8
+# %% ../nbs/api/13_cli.ipynb 8
 def extract_tgz(url, dest='.'):
     from fastcore.net import urlopen
     with urlopen(url) as u: tarfile.open(mode='r:gz', fileobj=u).extractall(dest)
 
-# %% ../nbs/api/cli.ipynb 9
+# %% ../nbs/api/13_cli.ipynb 9
 def _render_nb(fn, cfg):
     "Render templated values like `{{lib_name}}` in notebook at `fn` from `cfg`"
     txt = fn.read_text()
     txt = txt.replace('from your_lib.core', f'from {cfg.lib_path}.core') # for compatibility with old templates
     for k,v in cfg.d.items(): txt = txt.replace('{{'+k+'}}', v)
     fn.write_text(txt)
 
-# %% ../nbs/api/cli.ipynb 10
+# %% ../nbs/api/13_cli.ipynb 10
 def _update_repo_meta(cfg):
     "Enable gh pages and update the homepage and description in your GitHub repo."
     token=os.getenv('GITHUB_TOKEN')
@@ -74,7 +74,7 @@ def _update_repo_meta(cfg):
         except HTTPError:print(f"Could not update the description & URL on the repo: {cfg.user}/{cfg.repo} using $GITHUB_TOKEN.\n"
                   "Use a token with the correction permissions or perform these steps manually.")
 
-# %% ../nbs/api/cli.ipynb 11
+# %% ../nbs/api/13_cli.ipynb 11
 @call_parse
 @delegates(nbdev_create_config)
 def nbdev_new(**kwargs):
@@ -108,7 +108,7 @@ def nbdev_new(**kwargs):
     nbdev_export.__wrapped__()
     nbdev_readme.__wrapped__()
 
-# %% ../nbs/api/cli.ipynb 15
+# %% ../nbs/api/13_cli.ipynb 15
 @call_parse
 def chelp():
     "Show help for all console scripts"