Several big improvements (#55)

* Separate linting from testing pipeline. Run testing pipeline for all live versions of Python.

* Testing on osx.

* Use strategy matrix for os.

* setup-miniconda action.

* Run isort.

* Specify environment file for setup-miniconda.

* mamba for CI, specify shell.

* Fix environment name in CI yaml.

* Fix isort/black conflict.

* Remove subprocess.run check_output parameter for compatibility with Python 3.6

* Fix black issue. Add pre-commit.

* Update precommit package versions.

* Remove Windows CI, fix Mac + python 3.6 to use libc++ instead libstdc++.

* Apply CFLAGS to pytest, not the install... oops!

* Remove cpprun. It was never a good idea.

* Refactoring and building ARCHITECTURE.md

* Contributing and minor function renamings.

* Replace config module with settings dictionary.

* Add coverage to CI.

* XML coverage report.

* Missed coverage on the mac tests.

* Update README.

Author: tbenthompson

.coveragerc

@@ -0,0 +1,2 @@
+[run]
+source=cppimport

.github/workflows/lint_and_test.yml

@@ -0,0 +1,49 @@
+name: Test
+
+on: [push]
+
+jobs:
+  test:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: ["ubuntu-latest", "macos-latest"]
+        python-version: ["3.6", "3.7", "3.8", "3.9"]
+    name: Test (${{ matrix.python-version }}, ${{ matrix.os }})
+    runs-on:  ${{ matrix.os }}
+    defaults:
+      run:
+        shell: bash -l {0}
+    steps:
+    - uses: actions/checkout@v2
+    - uses: conda-incubator/setup-miniconda@v2
+      with:
+        mamba-version: "*"
+        channels: conda-forge
+        activate-environment: cppimport
+        environment-file: environment.yml
+        python-version: ${{ matrix.python-version }}
+    - name: Install cppimport
+      run: |
+        pip install --no-use-pep517 --no-deps --disable-pip-version-check -e .
+    - name: Lint with flake8
+      run: |
+        flake8 . 
+    - name: Check formatting with black
+      run: |
+        black --check . 
+    - name: Check import ordering with isort
+      run: |
+        isort --check . 
+    - name: Test
+      if: ${{ matrix.os == 'macos-latest' }}
+      run: |
+        CFLAGS='-stdlib=libc++' pytest --cov=./ --cov-report=xml
+    - name: Test
+      if: ${{ matrix.os != 'macos-latest' }}
+      run: |
+        pytest --cov=./ --cov-report=xml
+    - name: Upload coverage to Codecov
+      uses: codecov/codecov-action@v1
+      with:
+        fail_ci_if_error: true

.github/workflows/test.yml

@@ -3,10 +3,12 @@
 *.so
 *.cppimporthash
 .rendered.*
-.cpprunfiles
 __pycache__
 build
 cppimport.egg-info
 dist
 .cache
 .tox
+.mypy_cache/
+.coverage
+htmlcov

.gitignore

@@ -0,0 +1,15 @@
+repos:
+  - repo: https://github.com/psf/black
+    rev: 20.8b1
+    hooks:
+      - id: black
+        language_version: python3
+  - repo: https://gitlab.com/pycqa/flake8
+    rev: 3.8.4
+    hooks:
+      - id: flake8
+  - repo: https://github.com/pycqa/isort
+    rev: 5.7.0
+    hooks:
+      - id: isort
+        name: isort (python)

.pre-commit-config.yaml

@@ -0,0 +1,46 @@
+# Contributing 
+
+When contributing to this repository, feel free to add an issue or pull request! There really aren't any rules, but if you're mean, I'll be sad. I'm happy to collaborate on pull requests if you would like. There's no need to submit a perfect, finished product.
+
+To install in development mode and run the tests:
+```
+git clone [email protected]:tbenthompson/cppimport.git 
+cd cppimport
+conda env create
+conda activate cppimport
+pre-commit install
+pip install --no-use-pep517 --disable-pip-version-check -e .
+pytests
+```
+
+Helpful checklist for a pull request:
+
+* Run the tests! Check that the CI workflows are passing.
+* Update `README.md` and `CONTRIBUTING.md` with any relevant changes.
+
+# Architecture
+
+## Entrypoints:
+
+The main entrypoint for cppimport is the `cppimport.import_hook` module, which interfaces with the Python importing system to allow things like `import mycppfilename`. For a C++ file to be a valid import target, it needs to have the word `cppimport` in its first line. Without this first line constraint, it is possible for the importing system to cause imports in other Python packages to fail. Before adding the first-line constraint, the cppimport import_hook had the unfortunate consequence of breaking some scipy modules that had adjacent C and C++ files in the directory tree.
+
+There is an alternative, and more explicit interface provided by the `imp`, `imp_from_filepath` and `build` functions here.
+* `imp` does exactly what the import hook does except via a function so that instead of `import foomodule` we would do `foomodule = imp('foomodule')`.
+* `imp_from_filepath` is even more explicit, allowing the user to pass a C++ filepath rather than a modulename. For example, `foomodule = imp('../cppcodedir/foodmodule.cpp')`. This is rarely necessary but can be handy for debugging.
+* `build` is similar to `imp` except that the library is only built and not actually loaded as a Python module.
+
+`imp`, `imp_from_filepath` and `build` are in the `__init__.py` to separate external facing API from the guts of the package that live in internal submodules. 
+
+## What happens when we import a C++ module.
+
+1. First the `cppimport.find.find_module_cpppath` function is used to find a C++ file that matches the desired module name.
+2. Next, we determine if there's already an existing compiled extension that we can use. If there is, the `cppimport.importer.is_build_needed` function is used to determine if the extension is up to date with the current code. If the extension is up to date, we attempt to load it. If the extension is loaded successfully, we return the module and we're done! However, if for whichever reason, we can't load an existing extension, we need to build the extension, a process directed by `cppimport.importer.template_and_build`.
+3. The first step of building is to run the C++ file through the Mako templating system with the `cppimport.templating.run_templating` function. The main purpose of this is to allow users to embed configuration information into their C++ file. Without some sort of similar mechanism, there would be no way of passing information to build system because the `import modulename` statement can't carry information. The templating serves a secondary benefit in that simple code generation can be performed if needed. However, most users probably stick to a simple header or footer similar to the one demonstrated in the README. 
+4. Next, we use setuptools to build the C++ extension using `cppimport.build_module.build_module`. This function calls setuptools with the appropriate arguments to build the extension in place next to the C++ file in the directory tree.
+5. Next, we call `cppimport.checksum.checksum_save` to add a hash of the appended contents of all relevant source and header files. This checksum is appended to the end of the `.so` or `.dylib` file. This seems legal according to specifications and, in practice, causes no problems.
+6. Finally, the compiled and loaded extension module is returned to the user.
+
+## Useful links
+
+* PEP 302 that made this possible: https://www.python.org/dev/peps/pep-0302/ 
+* The gory details of Python importing: https://docs.python.org/3/reference/import.html

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2020 T. Ben Thompson
+Copyright (c) 2021 T. Ben Thompson
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

LICENSE

@@ -1,4 +1,3 @@
-include tox.ini
 include README.md
 include VERSION
 recursive-include tests *