update README

Author: Jencir Lee

README.md

@@ -1,49 +1,76 @@
 
 # micrograd
+A tiny Autograd engine whose only dependency is NumPy. Implements backpropagation (reverse-mode autodiff) over a dynamically built DAG and a small neural networks library on top of it with a PyTorch-like API. Both are tiny.
 
-![awww](assets/puppy.jpg)
-
-A tiny Autograd engine (with a bite! :)). Implements backpropagation (reverse-mode autodiff) over a dynamically built DAG and a small neural networks library on top of it with a PyTorch-like API. Both are tiny, with about 100 and 50 lines of code respectively. The DAG only operates over scalar values, so e.g. we chop up each neuron into all of its individual tiny adds and multiplies. However, this is enough to build up entire deep neural nets doing binary classification, as the demo notebook shows. Potentially useful for educational purposes.
-
-### Installation
+This version is capable of working with matrices and higher-order tensors. For @karpathy's original scalar-based version, locate the code with tag `scalar`.
 
+## Installation
 ```bash
-pip install micrograd
+python3 -m venv venv
+. venv/bin/activate
+pip install .
 ```
 
-### Example usage
+## Get Started
+```python
+from micrograd import Value
+from numpy import array
+
+a = Value(array([[2, 3], [5, 4]]))
+b = Value(array([1, -1]))
+c = (a @ b).relu()
+print(c)      # Value(data=[0 1], grad=None)
+c.backward()
+print(c)      # Value(data=[0 1], grad=[1. 1.])
+print(a)      # Value(data=..., grad=[[0. 0.], [1. -1.]])
+print(b)      # Value(data=..., grad=[5. 4.])
+```
 
-Below is a slightly contrived example showing a number of possible supported operations:
+## Lazy evaluation
+When defining a tensor, one may just indicate `shape` and `name`, and later on provide the value.
 
 ```python
-from micrograd.engine import Value
-
-a = Value(-4.0)
-b = Value(2.0)
-c = a + b
-d = a * b + b**3
-c += c + 1
-c += 1 + c + (-a)
-d += d * 2 + (b + a).relu()
-d += 3 * d + (b - a).relu()
-e = c - d
-f = e**2
-g = f / 2.0
-g += 10.0 / f
-print(f'{g.data:.4f}') # prints 24.7041, the outcome of this forward pass
-g.backward()
-print(f'{a.grad:.4f}') # prints 138.8338, i.e. the numerical value of dg/da
-print(f'{b.grad:.4f}') # prints 645.5773, i.e. the numerical value of dg/db
+from micrograd import Value
+from numpy import array
+
+a = Value(shape=(2, 2), name='var1')
+b = Value(shape=(2,), name='var2')
+c = (a @ b).relu()
+c.forward(var1=array([[2, 3], [5, 4]]),
+          var2=array([1, -1]))
+c.backward()
 ```
 
-### Training a neural net
+The **essential pattern** is to call `forward()` once with the values for the varialbes, then `backward()` once for the mathematical derivatives.
+
+```python
+x.forward(var1=value1, var2=value2, ...)
+x.backward()
+```
 
+Each time the `forward()` is called (e.g. for minibatch evaluation), the lazily defined variables have to be fed values in the function signature. Otherwise, it will take all `nan` as value. The final result will likely be `nan` to signal missing values for some variables.
+
+## Efficient operator dependency topology computation
+The operator dependency topology computation is only calculated once then cached, supposing the topology is static once a variable is defined.
+
+## Supported operators
+* `__pow__`
+* `__matmul__`
+* `tensordot` for tensor contraction
+* `relu`
+* `log`
+* `log1p`
+* `arctanh`
+* `T` for transpose
+* `sum`
+* `mean`
+
+## Training a neural net
 The notebook `demo.ipynb` provides a full demo of training an 2-layer neural network (MLP) binary classifier. This is achieved by initializing a neural net from `micrograd.nn` module, implementing a simple svm "max-margin" binary classification loss and using SGD for optimization. As shown in the notebook, using a 2-layer neural net with two 16-node hidden layers we achieve the following decision boundary on the moon dataset:
 
 ![2d neuron](assets/moon_mlp.png)
 
-### Tracing / visualization
-
+## Tracing / visualization
 For added convenience, the notebook `trace_graph.ipynb` produces graphviz visualizations. E.g. this one below is of a simple 2D neuron, arrived at by calling `draw_dot` on the code below, and it shows both the data (left number in each node) and the gradient (right number in each node).
 
 ```python
@@ -56,14 +83,12 @@ dot = draw_dot(y)
 
 ![2d neuron](assets/gout.svg)
 
-### Running tests
-
+## Running tests
 To run the unit tests:
 
 ```bash
 python -m unittest tests/*.py
 ```
 
-### License
-
+## License
 MIT