Moved in funcs from faasm repo (#9)

* Moved in funcs from faasm repo

* Adding tasks for function files

* Build faasm func in Dockerfile

* gitignore

* Version

* retab gitmodules file

* Bump cpp version

* Add clang format

Author: Shillaker

.clang-format

@@ -0,0 +1,13 @@
+---
+BasedOnStyle: Mozilla
+
+---
+Language: Cpp
+
+# Avoid excessive linebreaks on function definitions
+AlwaysBreakAfterDefinitionReturnType: None
+AlwaysBreakAfterReturnType: None
+
+IndentWidth: 4
+DerivePointerAlignment: false
+---

.github/workflows/tests.yml

@@ -12,7 +12,7 @@ jobs:
     if: github.event.pull_request.draft == false
     runs-on: ubuntu-20.04
     container:
-      image: faasm/cpython:0.0.10
+      image: faasm/cpython:0.0.11
     defaults:
       run:
         working-directory: /code/python
@@ -27,6 +27,10 @@ jobs:
         run: pip3 install -r pyfaasm/test_requirements.txt
       - name: "Python formatting check"
         run: ./bin/check_python.sh
+      - name: "Check C/C++ formatting"
+        run: ./bin/run_clang_format.sh
+      - name: "Check no formatting changes"
+        run: git diff --exit-code
 
   build:
     if: github.event.pull_request.draft == false
@@ -36,7 +40,7 @@ jobs:
       REDIS_QUEUE_HOST: redis
       REDIS_STATE_HOST: redis
     container:
-      image: faasm/cpython:0.0.10
+      image: faasm/cpython:0.0.11
     defaults:
       run:
         working-directory: /code/python

.gitignore

@@ -1,4 +1,5 @@
 # Proj-specific
+build/
 cross_venv/
 
 # Byte-compiled / optimized / DLL files

.gitmodules

@@ -1,20 +1,23 @@
 [submodule "third-party/cpython"]
-	path = third-party/cpython
+    path = third-party/cpython
     url = https://github.com/faasm/cpython
     branch = faasm
 [submodule "third-party/numpy"]
-	path = third-party/numpy
+    path = third-party/numpy
     url = https://github.com/faasm/numpy
     branch = faasm
 [submodule "third-party/crossenv"]
-	path = third-party/crossenv
+    path = third-party/crossenv
     url = https://github.com/faasm/crossenv
     branch = faasm
 [submodule "third-party/horovod"]
-	path = third-party/horovod
+    path = third-party/horovod
     url = https://github.com/faasm/horovod
     branch = faasm
 [submodule "third-party/mxnet"]
-	path = third-party/mxnet
+    path = third-party/mxnet
     url = https://github.com/faasm/incubator-mxnet
     branch = faasm
+[submodule "third-party/cpp"]
+    path = third-party/cpp
+    url = https://github.com/faasm/cpp

Dockerfile

@@ -1,4 +1,4 @@
-FROM faasm/cpp-sysroot:0.0.18
+FROM faasm/cpp-sysroot:0.0.22
 ARG FAASM_PYTHON_VERSION
 
 RUN apt install -y \
@@ -38,6 +38,9 @@ RUN ./bin/crossenv_setup.sh
 # Install cross-compiled python packages
 RUN . ./cross_venv/bin/activate && inv libs.install
 
+# Build Faasm function
+RUN inv func
+
 # TODO - enable these once the MXNet/ Horovod work is completed
 # Build mxnet
 # RUN inv mxnet

README.md

@@ -9,21 +9,41 @@ interface](https://github.com/faasm/faasm/blob/master/docs/host_interface.md).
 
 ## Set-up and development
 
-We recommend using Docker for developing this repo. Start the container with:
+This repo is developed using the Faasm development environment, set up according
+to [the docs](https://github.com/faasm/faasm/blob/master/docs/development.md). 
 
-```
-./bin/cli.sh
-```
-
-To show the list of available tasks (inside the container):
+To set up your local environment, run the `python` CLI as per the Faasm docs, 
+then:
 
 ```bash
-inv -l
-```
+# Install a local dev version of the cpp tools
+pip3 uninstall faasmtools
+pip3 install -e third-party/cpp
+
+# Install the matching native python in your local env
+./bin/install_build_python.sh
+
+# Compile CPython to wasm
+inv cpython
+
+# Set up and activate cross-env
+./bin/crossenv_setup.sh
+
+# Activate cross-env
+. cross_venv/bin/activate
+
+# Build Python libraries
+inv libs.install
 
-The build uses the [Faasm
-toolchain](https://github.com/faasm/faasm-toolchain) to cross-compile both 
-CPython and C-extensions to WebAssembly.
+# Copy runtime files into place
+inv runtime
+
+# Build the Faasm function to wrap CPython
+inv func
+
+# Copy the actual Python functions into place
+inv func.uploadpy --local
+```
 
 ## Code changes
 
@@ -47,11 +67,11 @@ Several of the code changes to CPython and numpy were borrowed from
 This repo gets built as a container, `faasm/cpython`. If you want to release a
 new version, you can:
 
-- Update the version in `VERSION` and `.github`
+- Update the version in `VERSION` and `.github/workflows/tests.yml`
 - Commit to your branch
-- Check that the normal build works in CI
 - Run `inv git.tag`
-- Let the Github action do the rest
+- Check the release build has run
+- Create a pull request
 
 ## PyFaasm
 
@@ -113,26 +133,7 @@ Setuptools and distutils both interrogate the Python system environment during
 the build process. This makes it quite difficult to cross-compile libraries, so
 we use [crossenv](https://github.com/benfogle/crossenv).
 
-To set up crossenv:
-
-```
-./bin/crossenv_setup.sh
-```
-
-You can then activate with:
-
-```
-. cross_venv/bin/activate
-```
-
-From inside the virtual environment, you can inspect the set-up with:
-
-```
-python bin/sanity_check.py | less
-```
-
-This will display the environment used to install Python modules (including the
-relevant cross-compilation variables, e.g. `CC`, `CFLAGS` etc.).
+See the dev instructions above for set-up.
 
 ### Changing the crossenv environment
 
@@ -215,13 +216,8 @@ inv mxnet --clean
 inv mxnet.uninstall
 ```
 
-## Running in Faasm
-
-See the [Faasm python
-docs](https://github.com/faasm/faasm/blob/master/docs/python.md).
-
 ## BLAS and LAPACK
 
 Faasm's normal BLAS and LAPACK support using CLAPACK should be picked up by
 numpy due to the addition of the [site.cfg](../third-party/numpy/site.cfg).
-
+(note 19/03/21 this has been temporarily disabled).

VERSION

@@ -1 +1 @@
-0.0.10
+0.0.11

bin/cli.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+set -e
+
+function append_newline {
+    if [[ -z "$(tail -c 1 "$1")" ]]; then
+        :
+    else
+        echo >> "$1"
+    fi
+}
+
+if [ -z "$1" ]; then
+    TARGET_DIR="."
+else
+    TARGET_DIR=$1
+fi
+
+pushd ${TARGET_DIR} >> /dev/null
+
+# Find all source files using Git to automatically respect .gitignore
+FILES=$(git ls-files "*.h" "*.cpp" "*.c")
+
+# Run clang-format
+clang-format-10 -i ${FILES}
+
+# Check newlines
+for f in ${FILES}; do
+    append_newline $f
+done
+
+popd >> /dev/null