Merge pull request #11 from graphcore-research/packaging

Author: awf

.github/workflows/ci.yaml

@@ -1,4 +1,6 @@
-name: unit tests
+# Copyright (c) 2024 Graphcore Ltd.  All rights reserved.
+
+name: CI
 on:
   pull_request:
   push:
@@ -25,10 +27,18 @@ jobs:
         run: |
           python3 -m pip freeze
 
+      - name: Pre-commit all files
+        run: |
+          pre-commit run --all-files
+
       - name: Run unit tests
         run: |
           pytest .
 
+      - name: MyPy
+        run: |
+          mypy  --disallow-untyped-defs --enable-error-code redundant-expr src test
+
       - name: Ensure that docs build
         run: |
           cd docs && make html

.gitignore

@@ -1,3 +1,5 @@
+# Copyright (c) 2024 Graphcore Ltd.  All rights reserved.
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]
@@ -159,3 +161,4 @@ cython_debug/
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
 .vscode/settings.json
+.vscode/launch.json

.pre-commit-config.yaml

@@ -1,11 +1,28 @@
+# Copyright (c) 2024 Graphcore Ltd.  All rights reserved.
+
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.6.0
     hooks:
     -   id: check-yaml
     -   id: end-of-file-fixer
     -   id: trailing-whitespace
+
 -   repo: https://github.com/psf/black
     rev: 24.4.0
     hooks:
     - id: black
+
+-   repo: local
+    hooks:
+    - id: etc/check-copyright.sh
+      name: check copyright
+      entry: etc/check-copyright.sh
+      language: script
+      exclude: |
+            (?x)^(
+                docs/Makefile|
+                docs/make.bat|
+                .*\.md|
+                (.*/|)requirements.*\.txt
+            )$

.readthedocs.yaml

@@ -1,3 +1,5 @@
+# Copyright (c) 2024 Graphcore Ltd.  All rights reserved.
+
 version: 2
 
 build:

BUILDING.md

@@ -0,0 +1,11 @@
+## BUILDING
+
+```
+pip install -e .
+( cd docs && make html )
+```
+
+#### Pushing
+```
+sh etc/package.sh
+```

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2023 Andrew Fitzgibbon
+Copyright (c) 2023 Graphcore Ltd.  All rights reserved.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -16,56 +16,23 @@ See https://gfloat.readthedocs.io for documentation, or dive into the notebooks
 
 For example, here's a table from the [02-value-stats](docs/source/02-value-stats.ipynb) notebook:
 
-|name|B|P|E|T|lt1|gt1|rt16|maxFinite|minFinite|maxNormal|minNormal|minSubnormal|maxSubnormal|
-|--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |
-|ocp_e2m1|4|2|2|1|1|5|True|6|-6|6|1|0.5|0.5|
-|ocp_e2m3|6|4|2|3|7|23|True|7.5|-7.5|7.5|1|0.125|0.875|
-|ocp_e3m2|6|3|3|2|11|19|True|28|-28|28|0.25|0.0625|0.1875|
-|ocp_e4m3|8|4|4|3|55|70|True|448|-448|448|0.015625|1*2^-9|7/4*2^-7|
-|ocp_e5m2|8|3|5|2|59|63|True|57344|-57344|57344|1*2^-14|1*2^-16|3/2*2^-15|
-|p3109_p1|8|1|7|0|62|63|False|1*2^63|-1*2^63|1*2^63|1*2^-62|nan|nan|
-|p3109_p2|8|2|6|1|63|62|False|1*2^31|-1*2^31|1*2^31|1*2^-31|1*2^-32|1*2^-32|
-|p3109_p3|8|3|5|2|63|62|True|49152|-49152|49152|1*2^-15|1*2^-17|3/2*2^-16|
-|p3109_p4|8|4|4|3|63|62|True|224|-224|224|0.0078125|1*2^-10|7/4*2^-8|
-|p3109_p5|8|5|3|4|63|62|True|15|-15|15|0.125|0.0078125|15/8*2^-4|
-|p3109_p6|8|6|2|5|63|62|True|3.875|-3.875|3.875|0.5|0.015625|31/16*2^-2|
-|bfloat16|16|8|8|7|16255|16383|False|255/128*2^127|-255/128*2^127|255/128*2^127|1*2^-126|1*2^-133|127/64*2^-127|
-|ocp_int8|8|8|0|7|63|63|True|127/64*2^0|-2|nan|nan|0.015625|127/64*2^0|
-|ocp_e8m0|8|1|8|0|127|127|False|1*2^127|1*2^-127|1*2^127|1*2^-127|nan|nan|
+|name|B: Bits in the format|P: Precision in bits|E: Exponent field width in bits|0<x<1|1<x<Inf|Exact in float16?|maxFinite|minFinite|maxNormal|minNormal|minSubnormal|maxSubnormal|
+|--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |--- |
+|ocp_e2m1|4|2|2|1|5|True|6|-6|6|1|0.5|0.5|
+|ocp_e2m3|6|4|2|7|23|True|7.5|-7.5|7.5|1|0.125|0.875|
+|ocp_e3m2|6|3|3|11|19|True|28|-28|28|0.25|0.0625|0.1875|
+|ocp_e4m3|8|4|4|55|70|True|448|-448|448|0.015625|1*2^-9|7/4*2^-7|
+|ocp_e5m2|8|3|5|59|63|True|57344|-57344|57344|1*2^-14|1*2^-16|3/2*2^-15|
+|p3109_p1|8|1|7|62|63|False|1*2^63|-1*2^63|1*2^63|1*2^-62|nan|nan|
+|p3109_p2|8|2|6|63|62|False|1*2^31|-1*2^31|1*2^31|1*2^-31|1*2^-32|1*2^-32|
+|p3109_p3|8|3|5|63|62|True|49152|-49152|49152|1*2^-15|1*2^-17|3/2*2^-16|
+|p3109_p4|8|4|4|63|62|True|224|-224|224|0.0078125|1*2^-10|7/4*2^-8|
+|p3109_p5|8|5|3|63|62|True|15|-15|15|0.125|0.0078125|15/8*2^-4|
+|p3109_p6|8|6|2|63|62|True|3.875|-3.875|3.875|0.5|0.015625|31/16*2^-2|
+|bfloat16|16|8|8|16255|16383|False|255/128*2^127|-255/128*2^127|255/128*2^127|1*2^-126|1*2^-133|127/64*2^-127|
+|ocp_int8|8|8|0|63|63|True|127/64*2^0|-2|nan|nan|0.015625|127/64*2^0|
+|ocp_e8m0|8|1|8|127|127|False|1*2^127|1*2^-127|1*2^127|1*2^-127|nan|nan|
 
-Stats collected for each format are:
- - B: Bits in the format
- - P: Precision in bits
- - E: Exponent field width in bits
- - T: Trailing significand field width in bits
- - lt1: Number of values x such that `0 < x < 1`
- - gt1: Number of values x such that `1 < x < Inf`
- - rt16: True if all values are exactly representable in IEEE binary16
- - maxFinite: Largest finite value
- - minFinite: Smallest finite value
- - maxNormal: Largest finite normal value, NaN if all finite values are subnormal
- - minNormal: Smallest positive normal value, NaN if all finite values are subnormal
- - minSubnormal: Smallest positive subnormal value, NaN if no finite values are  - subnormal
- - maxSubnormal: Largest subnormal value, NaN if no finite values are subnormal
-
-
-
-## BUILDING
-
-```
-pip install -e .
-cd docs
-make html
-cd ..
-```
-
-#### Pushing
-```
-rm -rf dist
-pip install build twine
-python -m build
-echo __token__ | twine upload --repository pypi dist/* --verbose
-```
 
 #### Notes
 

docs/source/01-decode.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<!-- # Copyright (c) 2024 Graphcore Ltd. All rights reserved. -->\n",
     "\n",
-    "## GFloat Basics\n",
+    "# GFloat Basics\n",
     "\n",
     "This notebook shows the use of `decode_float` to explore properties of some float formats.\n"
    ]
@@ -29,7 +29,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### List all the values in a format\n",
+    "## List all the values in a format\n",
     "\n",
     "The first example shows how to list all values in a given format.\n",
     "We will choose the [OCP](https://www.opencompute.org/documents/ocp-8-bit-floating-point-specification-ofp8-revision-1-0-2023-12-01-pdf-1) E5M2 format.\n",
@@ -273,7 +273,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Additional format info: special values, min, max, dynamic range\n",
+    "## Additional format info: special values, min, max, dynamic range\n",
     "\n",
     "In addition, `FormatInfo` can tell us about other characteristics of each format.\n",
     "To reproduce some of the OCP spec's tables 1 and 2:"
@@ -316,14 +316,16 @@
     "    (\"Min subnormal number   \", lambda fi: fi.smallest_subnormal),\n",
     "    (\"Dynamic range (binades)\", lambda x: round(compute_dynamic_range(x))),\n",
     "):\n",
-    "    print(f\"{prop} {probe(format_info_ocp_e4m3):<20} {probe(format_info_ocp_e5m2):<20}  {probe(format_info_p3109(3))}\")"
+    "    print(\n",
+    "        f\"{prop} {probe(format_info_ocp_e4m3):<20} {probe(format_info_ocp_e5m2):<20}  {probe(format_info_p3109(3))}\"\n",
+    "    )"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### How do subnormals affect dynamic range?\n",
+    "## How do subnormals affect dynamic range?\n",
     "\n",
     "Most, if not all, low-precision formats include subnormal numbers, as they increase the number of values near zero, and increase dynamic range.\n",
     "A natural question is \"by how much?\".  To answer this, we can create a mythical new format, a copy of `e4m3`, but with `has_subnormals` set to true."