Merge remote-tracking branch 'origin/main' into 238-linting-template

Author: fdiblen

.github/workflows/tests.yml

@@ -13,7 +13,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: ['ubuntu-latest', 'macos-latest']
+        os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
         python-version: ['3.6', '3.7', '3.8', '3.9']
     steps:
     - uses: actions/checkout@v2

.mlc-config.json

@@ -18,5 +18,6 @@
         }
     ],
     "replacementPatterns": [
-    ]
+    ],
+    "timeout": "20s"
 }

ADD_TO_EXISTING_PACKAGE.md

@@ -0,0 +1,29 @@
+# Add your existing code to directory generated by the NLeSC Python template
+
+The following steps requires that your existing code is in a GitHub repository.
+
+1. Follow the [instructions to create a new package](https://github.com/NLeSC/python-template#how-to-use) with the Python template, while answering the questions like you would for the existing package.
+
+2. Create a Git structure for the new directory: (replace  `<NEW_DIRECTORY>` with directory name you used during cookiecutter questionnaire)
+```shell
+$ cd <NEW_DIRECTORY>
+$ git init
+$ git add --all
+$ git commit -m "Added code generated by cookiecutter"
+$ git branch -M main
+```
+
+3. Import the existing repository from GitHub (Replace `<ORGANIZATION>` with your GitHub organization name , `<REPOSITORY>` with your GitHub repository name and `<BRANCH>` with your default branch for example `main` or `master`):
+```shell
+$ git remote add -f existing_code https://github.com/<ORGANIZATION>/<REPOSITORY>
+$ git fetch existing_code
+$ git merge existing_code/<BRANCH> --allow-unrelated-histories
+```
+
+4. The previous step will likely result in several merge conflicts. Solve the conflicts by editing the files.
+5. Once all conflicts have been resolved then add all the files to GitHub:
+```shell
+$ git add --all
+$ git commit -m "Merged existing code with code generated by cookiecutter"
+$ git push
+```

README.md

@@ -61,6 +61,8 @@ cookiecutter https://github.com/nlesc/python-template.git
 | package_name              | my_python_package | Name of the package. Avoid using spaces, dashes, or uppercase letters for the best experience across operating systems. |
 | project_name              | my-python-project | Name of the project that contains the package. Avoid using spaces or uppercase letters for the best experience across operating systems. |
 | package_short_description |              | The information that you enter here will end up in the README, documentation, license, and setup.cfg, so it may be a good idea to prepare something in advance. |
+| keyword1                  | keyword1          | A term that describes your package. |
+| keyword2                  | keyword2          | Another term that describes your package. |
 | version                   | 0.1.0             |   |
 | github_organization       | <my-github-organization> | GitHub organization that will contain this project's repository. This can also be your GitHub user name. |
 | license                   | Apache Software License 2.0 | The software license under which the code is made available.  |
@@ -107,6 +109,7 @@ my-python-project/
 ```
 
 For an explanation of what's there, read on in the [project_setup.md]({{cookiecutter.project_name}}/project_setup.md) file.
+There are also instructions on how to [apply the template to an existing Python package](ADD_TO_EXISTING_PACKAGE.md).
 
 ## Examples
 

cookiecutter.json

@@ -1,8 +1,10 @@
 {
   "package_name": "my_python_package",
   "project_name": "my-python-project",
-  "_copy_without_render": [".github/*"],
+  "_copy_without_render": [".github/workflows/*"],
   "package_short_description": "",
+  "keyword1": "keyword1",
+  "keyword2": "keyword2",
   "version": "0.1.0",
   "github_organization": "<my-github-organization>",
   "license": ["Apache Software License 2.0", "MIT license", "BSD license", "ISC license", "GNU General Public License v3 or later", "Not open source"],

tests/test_project.py

@@ -1,10 +1,15 @@
 import os
 import subprocess
+from pathlib import Path
+from shutil import which
 from sys import platform
 from typing import Sequence
 
 import pytest
 
+IS_WINDOWS = platform.startswith("win")
+IS_WINDOWS_CI = IS_WINDOWS and os.environ.get('CI', False)
+
 
 def test_project_folder(cookies):
     project = cookies.bake()
@@ -16,37 +21,46 @@ def test_project_folder(cookies):
 
 
 def run(args: Sequence[str], dirpath: os.PathLike) -> subprocess.CompletedProcess:
-    return subprocess.run(args=args,
-                          stdout=subprocess.PIPE,
-                          stderr=subprocess.PIPE,
-                          cwd=dirpath,
-                          encoding="utf-8")
+    completed_process = subprocess.run(args=args,
+                                       stdout=subprocess.PIPE,
+                                       stderr=subprocess.PIPE,
+                                       cwd=dirpath,
+                                       encoding="utf-8")
+    print(completed_process.stdout)
+    print(completed_process.stderr)
+    return completed_process
 
 
 @pytest.fixture
 def baked_with_development_dependencies(cookies):
     result = cookies.bake()
-    env_output = run(['python3', '-m', 'venv', 'env'], result.project)
-    assert env_output.returncode == 0
-    env_bin = 'env/Scripts/' if platform.startswith("win") else 'env/bin/'
-    latest_pip_output = run([f'{env_bin}pip3', 'install', '--upgrade', 'pip', 'setuptools'], result.project)
+    if IS_WINDOWS_CI:
+        # Creating virtualenv does not work on Windows CI,
+        # falling back to using current pip3 dir
+        pip = Path(which('pip3'))
+        bin_dir = str(pip.parent) + '\\'
+    else:
+        env_output = run(['python3', '-m', 'venv', 'env'], result.project)
+        assert env_output.returncode == 0
+        bin_dir = 'env/Scripts/' if IS_WINDOWS else 'env/bin/'
+    latest_pip_output = run([f'{bin_dir}pip3', 'install', '--upgrade', 'pip', 'setuptools'], result.project)
     assert latest_pip_output.returncode == 0
-    pip_output = run([f'{env_bin}pip3', 'install', '--editable', '.[dev]'], result.project)
+    pip_output = run([f'{bin_dir}pip3', 'install', '--editable', '.[dev]'], result.project)
     assert pip_output.returncode == 0
-    return result.project, env_bin
+    return result.project, bin_dir
 
 
 def test_pytest(baked_with_development_dependencies):
-    project_dir, env_bin = baked_with_development_dependencies
-    pytest_output = run([f'{env_bin}pytest'], project_dir)
+    project_dir, bin_dir = baked_with_development_dependencies
+    pytest_output = run([f'{bin_dir}pytest'], project_dir)
     assert pytest_output.returncode == 0
     assert '== 4 passed in' in pytest_output.stdout
     assert (project_dir / 'coverage.xml').exists()
     assert (project_dir / 'htmlcov/index.html').exists()
 
 
 def test_subpackage(baked_with_development_dependencies):
-    project_dir, env_bin = baked_with_development_dependencies
+    project_dir, bin_dir = baked_with_development_dependencies
     subpackage = (project_dir / 'my_python_package' / 'mysub')
     subpackage.mkdir()
     (subpackage / '__init__.py').write_text('FOO = "bar"', encoding="utf-8")
@@ -55,7 +69,19 @@ def test_subpackage(baked_with_development_dependencies):
     subsubpackage.mkdir()
     (subsubpackage / '__init__.py').write_text('FOO = "bar"', encoding="utf-8")
 
-    build_output = run([f'{env_bin}python3', 'setup.py', 'build'], project_dir)
+    if IS_WINDOWS_CI:
+        # On Windows CI python and pip executable are in different paths
+        bin_dir = ''
+    build_output = run([f'{bin_dir}python', 'setup.py', 'build'], project_dir)
     assert build_output.returncode == 0
     assert (project_dir / 'build' / 'lib' / 'my_python_package' / 'mysub' / '__init__.py').exists()
     assert (project_dir / 'build' / 'lib' / 'my_python_package' / 'mysub' / 'mysub2' / '__init__.py').exists()
+
+
+def test_generate_api_docs(baked_with_development_dependencies):
+    project_dir, bin_dir = baked_with_development_dependencies
+
+    build_output = run([f'{bin_dir}sphinx-build', '-b', 'html', 'docs', 'docs/_build/html'], project_dir)
+    assert build_output.returncode == 0
+    assert 'build succeeded' in build_output.stdout
+    assert (project_dir / 'docs' / '_build' / 'html' / 'index.html').exists()

{{cookiecutter.project_name}}/.github/next_steps/02_citation.md

@@ -0,0 +1,43 @@
+---
+title: 'Next step: Citation data'
+---
+
+It is likely that your `CITATION.cff` currently doesn't pass validation. The error messages you get from the [`cffconvert`]({{cookiecutter.repository}}/actions/workflows/cffconvert.yml) GitHub Action are unfortunately a bit cryptic, but doing the following helps:
+
+- [ ] Check if the `given-name` and `family-name` keys need updating. If your family name has a name particle like `von` or `van` or `de`, use the `name-particle` key; if your name has a suffix like `Sr` or `IV`, use `name-suffix`. For details, refer to the schema description: https://github.com/citation-file-format/citation-file-format
+- [ ] Update the value of the `orcid` key. If you do not have an orcid yet, you can get one here [https://orcid.org/](https://orcid.org/).
+- [ ] Add more authors if needed
+- [ ] Update `date-released` using the YYYY-MM-DD format.
+- [ ] Update the `doi` key with the conceptDOI for your repository (see [https://help.zenodo.org](https://help.zenodo.org/) for more information on what a conceptDOI is). If your project doesn't have a DOI yet, you can use the string `10.0000/FIXME` to pass validation.
+- [ ] Verify that the `keywords` array accurately describes your project.
+
+Once you do all the steps above, the `cffconvert` workflow will tell you what content it expected to see in `.zenodo.json`. Copy-paste from the GitHub Action log into a new file `.zenodo.json`. Afterwards, the `cffconvert` GitHub Action should be green.
+
+
+To help you keep the citation metadata up to date and synchronized, the [`cffconvert`]({{cookiecutter.repository}}/actions/workflows/cffconvert.yml) GitHub Action checks the following 6 aspects:
+
+1. Whether your repository includes a `CITATION.cff` file.
+
+    _By including this file, authors of the software can receive credit for the work they put in._
+
+1. Whether your `CITATION.cff` is valid YAML.
+
+    _Visit http://www.yamllint.com/ to see if the contents of your CITATION.cff are valid YAML._
+
+1. Whether your `CITATION.cff` adheres to the schema (as listed in the `CITATION.cff` file itself under key `cff-version`).
+
+    _The Citation File Format schema can be found [here](https://github.com/citation-file-format/citation-file-format), along with an explanation of all the keys. You're advised to use the latest available schema version._
+
+1. Whether your repository includes a `.zenodo.json` file.
+
+    _With this file, you can control what metadata should be associated with any future releases of your software on Zenodo: things like the author names, along with their affiliations and their ORCIDs, the license under which the software has been released, as well as the name of your software and a short description. If your repository doesn't have a .zenodo.json file, Zenodo will take a somewhat crude guess to assign these metadata._
+
+    _The `cffconvert` GitHub action will tell you what it expects to find in `.zenodo.json`, just copy and paste it to a new file named `.zenodo.json`. The suggested text ignores CITATION.cff's `version`, `commit`, and `date-released`. `cffconvert` considers these keys `suspect` in the sense that they are often out of date, and there is little purpose to telling Zenodo about these properties: Zenodo already knows._
+
+1. Whether `.zenodo.json` is valid JSON.
+
+    _Currently unimplemented, but you can check for yourself on [https://jsonlint.com/](https://jsonlint.com/)._ 
+
+1. Whether `CITATION.cff` and `.zenodo.json` contain equivalent data.
+
+    _This final check verifies that the two files are in sync. The check ignores CITATION.cff's `version`, `commit`, and `date-released`._

{{cookiecutter.project_name}}/.github/next_steps/02_zenodo_integration.md

@@ -0,0 +1,22 @@
+---
+title: 'Next step: Enable Zenodo integration'
+---
+
+By enabling Zenodo integration, your package will automatically get a DOI which can be used to cite your package. After enabling Zenodo integration for your GitHub repository, Zenodo will create a snapshot and archive each release you make on GitHub. Moreover, Zenodo will create a new DOI for each GitHub release of your code.
+
+To enable Zenodo integration:
+
+1. Go to http://zenodo.org and login with your GitHub account. When you are redirected to GitHub, *Authorize application* to give permission to Zenodo to access your account.
+1. Go to <https://zenodo.org/account/settings/github/> and enable Zenodo integration of your repository by clicking on `On` toggle button.
+2. Your package will get a DOI only after you make a release. Create a new release as described in [README.dev.md]({{cookiecutter.repository}}/blob/main/README.dev.md#33-github)
+3. At this point you should have a DOI. To find out the DOI generated by Zenodo:
+   1. Visit https://zenodo.org/deposit and click on your repository link
+   2. You will find the latest DOI in the right column in Versions box in **Cite all versions?** section
+   3. Copy the text of the link. For example `10.5281/zenodo.1310751`
+4. Update the badge in your repository
+   1. Edit README.md and replace the badge placeholder with the badge link you copied in previous step.
+   The badge placeholder is shown below.
+
+      `[![DOI](https://zenodo.org/badge/DOI/<replace-with-created-DOI>.svg)](https://doi.org/<replace-with-created-DOI>)`
+
+For FAQ about Zenodo please visit <https://help.zenodo.org/>.