Updated Documentation and Added force_adjustment_voltage and failure_iteration_threshold Arguments for 1.1.6 Release (#128)

- Updated ReadTheDocs Documentation.
- Updated README.md.
- Added BiBTeX entry for the paper which can be used to cite the corresponding OSP paper.
- Added `force_adjustment_voltage` and `failure_iteration_threshold` Arguments to `memtorch.bh.Crossbar.write_conductance_matrix` and associated bindings.

Author: coreylammie

.github/workflows/build_release.yml

@@ -45,9 +45,9 @@ jobs:
       - name: Build wheels
         run: python -m cibuildwheel --output-dir wheelhouse
         env:
-          CIBW_BEFORE_BUILD_WINDOWS: pip3 install torch==1.9.0+cpu -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
-          CIBW_BEFORE_BUILD_MACOS: pip3 install torch==1.9.0 -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
-          CIBW_BEFORE_BUILD_LINUX: pip3 install torch==1.9.0+cpu -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
+          CIBW_BEFORE_BUILD_WINDOWS: pip3 install torch==1.10.0+cpu -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
+          CIBW_BEFORE_BUILD_MACOS: pip3 install torch==1.10.0 -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
+          CIBW_BEFORE_BUILD_LINUX: pip3 install torch==1.10.0+cpu -f https://download.pytorch.org/whl/torch_stable.html --ignore-installed
           CIBW_REPAIR_WHEEL_COMMAND: ""
           CIBW_BUILD: cp37-* cp38-* cp39-*
           CIBW_SKIP: "*-manylinux_i686 *-win32"

CHANGELOG.md

@@ -1,13 +1,18 @@
 ## Added
-
-1. Partial support for the `groups` argument for convolutional layers.
+1. The `random_crossbar_init` argument to memtorch.bh.Crossbar. If true, this is used to initialize crossbars to random device conductances in between 1/Ron and 1/Roff.
+2. `CUDA_device_idx` to `setup.py` to allow users to specify the `CUDA` device to use when installing `MemTorch` from source.
+3. Implementations of CUDA accelerated passive crossbar programming routines for the 2021 Data-Driven model.
+4. A BiBTeX entry, which can be used to cite the corresponding OSP paper.
 
 ## Fixed
-
-1. Patching procedure in `memtorch.mn.module.patch_model` and `memtorch.bh.nonideality.apply_nonidealities` to fix semantic error in `Tutorial.ipynb`.
-2. Import statement in `Exemplar_Simulations.ipynb`.
+1. In the getting started tutorial, Section 4.1 was a code cell. This has since been converted to a markdown cell.
+2. OOM errors encountered when modeling passive inference routines of crossbars.
 
 ## Enhanced
 
-1. Further modularized patching logic in `memtorch.bh.nonideality.NonIdeality` and `memtorch.mn.Module`.
-2. Modified default number of worker in `memtorch.utils` from 2 to 1.
+1. Templated quantize bindings and fixed semantic error in `memtorch.bh.nonideality.FiniteConductanceStates`.
+2. The memory consumption when modeling passive inference routines.
+3. The sparse factorization method used to solve sparse linear matrix systems.
+4. The `naive_program` routine for crossbar programming. The maximum number of crossbar programming iterations is now configurable.
+5. Updated ReadTheDocs documentation for `memtorch.bh.Crossbar`.
+6. Updated the version of `PyTorch` used to build Python wheels from `1.9.0` to `1.10.0`.

README.md

@@ -13,7 +13,9 @@
 [![codecov](https://codecov.io/gh/coreylammie/MemTorch/branch/master/graph/badge.svg)](https://codecov.io/gh/coreylammie/MemTorch)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
-MemTorch is a _Simulation Framework for Memristive Deep Learning Systems_, which integrates directly with the well-known PyTorch Machine Learning (ML) library. MemTorch is formally described in _MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems_, which is openly accessible [here](https://arxiv.org/abs/2004.10971).
+MemTorch is a _Simulation Framework for Memristive Deep Learning Systems_, which integrates directly with the well-known PyTorch Machine Learning (ML) library. MemTorch is formally described in _MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems_, which is openly accessible [here](https://arxiv.org/abs/2004.10971). 
+
+_MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems_ has been published as an Original Software Publication (OSP) in the *Neurocomputing* journal [here](https://doi.org/10.1016/j.neucom.2022.02.043). We kindly ask that the following [BibTeX entry](https://github.com/coreylammie/MemTorch/blob/master/citation.bib?raw=True) is used to cite MemTorch, if you use it in your work.
 
 ![Overview](https://github.com/coreylammie/MemTorch/blob/master/overview.svg)
 
@@ -81,14 +83,15 @@ _Be sure to merge the latest from 'upstream' before making a pull request_. This
 To cite _MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems_, use the following BibTex entry:
 
 ```
-@misc{lammie2020memtorch,
-  title={{MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems}},
-  author={Corey Lammie and Wei Xiang and Bernab\'e Linares-Barranco and Mostafa Rahimi Azghadi},
-  month=Apr.,
-  year={2020},
-  eprint={2004.10971},
-  archivePrefix={arXiv},
-  primaryClass={cs.ET}
+@Article{Lammie2022,
+  author   = {Corey Lammie and Wei Xiang and Bernabé Linares-Barranco and Mostafa Rahimi Azghadi},
+  title    = {{MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems}},
+  journal  = {Neurocomputing},
+  year     = {2022},
+  issn     = {0925-2312},
+  doi      = {https://doi.org/10.1016/j.neucom.2022.02.043},
+  keywords = {Memristors, RRAM, Non-Ideal Device Characteristics, Deep Learning, Simulation Framework},
+  url      = {https://www.sciencedirect.com/science/article/pii/S0925231222002053},
 }
 ```
 

citation.bib

@@ -0,0 +1,10 @@
+@Article{Lammie2022,
+  author   = {Corey Lammie and Wei Xiang and Bernabé Linares-Barranco and Mostafa Rahimi Azghadi},
+  title    = {{MemTorch: An Open-source Simulation Framework for Memristive Deep Learning Systems}},
+  journal  = {Neurocomputing},
+  year     = {2022},
+  issn     = {0925-2312},
+  doi      = {https://doi.org/10.1016/j.neucom.2022.02.043},
+  keywords = {Memristors, RRAM, Non-Ideal Device Characteristics, Deep Learning, Simulation Framework},
+  url      = {https://www.sciencedirect.com/science/article/pii/S0925231222002053},
+}

docs/conf.py

@@ -17,11 +17,11 @@
 # -- Project information -----------------------------------------------------
 
 project = "MemTorch"
-copyright = "2021, Corey Lammie"
+copyright = "2022, Corey Lammie"
 author = "Corey Lammie"
 
 # The full version, including alpha/beta/rc tags
-release = "1.1.5"
+release = "1.1.6"
 autodoc_inherit_docstrings = False
 
 # -- General configuration ---------------------------------------------------

docs/memtorch.bh.rst

@@ -30,6 +30,14 @@ Class used to model memristor crossbars and to manage modular crossbar tiles.
 .. note::
   **use_bindings** is enabled by default, to accelerate operation using C++/CUDA (if supported) bindings.
 
+.. warning::
+  As of version 1.1.6, the **write_conductance_matrix** method exhibits different behavior when **self.use_bindings** is True, **CUDA** operation is enabled, and the **Data_Driven2021** memristor model is used.
+
+  When **self.use_bindings** is True, **CUDA** operation is enabled, and the **Data_Driven2021** memristor model is used, the programming voltage is force adjusted by **force_adjustment_voltage** when a device becomes stuck.
+  For all others models, or when **CUDA** operation is not enabled or **self.use_bindings** is false, the conductance state of the device being modelled is adjusted using **force_adjustment** when it becomes stuck.
+
+  This behavior will made consistent across Python, C++, and CUDA bindings, in a future release.
+
 .. automodule:: memtorch.bh.crossbar.Crossbar
    :members:
    :undoc-members:

memtorch/bh/crossbar/Crossbar.py

@@ -210,6 +210,7 @@ def write_conductance_matrix(
             )
         else:
             raise Exception("Unsupported crossbar shape.")
+
         if self.tile_shape is not None:
             conductance_matrix, tiles_map = gen_tiles(
                 conductance_matrix,
@@ -232,7 +233,7 @@ def write_conductance_matrix(
         conductance_matrix = torch.max(
             torch.min(conductance_matrix.to(self.device), max), min
         )
-        if transistor or programming_routine is None:
+        if transistor:
             self.conductance_matrix = conductance_matrix
             self.max_abs_conductance = (
                 torch.abs(self.conductance_matrix).flatten().max()
@@ -265,6 +266,9 @@ def write_conductance_matrix(
                 )
                 self.update(from_devices=False)
             else:
+                assert (
+                    programming_routine is not None
+                ), "If memtorch_cuda_bindings.simulate_passive is not used, a programming routine must be provided."
                 if self.tile_shape is not None:
                     for i in range(0, self.devices.shape[0]):
                         for j in range(0, self.devices.shape[1]):

memtorch/bh/crossbar/Program.py

@@ -22,6 +22,7 @@ def naive_program(
     force_adjustment_rel_tol=1e-1,
     force_adjustment_pos_voltage_threshold=0,
     force_adjustment_neg_voltage_threshold=0,
+    failure_iteration_threshold=1000,
     simulate_neighbours=True,
 ):
     """Method to program (alter) the conductance of a given device within a crossbar.
@@ -54,6 +55,8 @@ def naive_program(
         Positive voltage level threshold (V) to enable force adjustment.
     force_adjustment_neg_voltage_threshold : float
         Negative voltage level threshold (V) to enable force adjustment.
+    failure_iteration_threshold : int
+        Failure iteration threshold.
     simulate_neighbours : bool
         Simulate neighbours (True).
 
@@ -142,7 +145,7 @@ def naive_program(
                     )
 
         iterations += 1
-        if iterations % 100 == 0 and time.time() > timeout:
+        if iterations >= failure_iteration_threshold and time.time() > timeout:
             warnings.warn("Failed to program device to rel_tol (%f)." % rel_tol)
             break