1
1
# FastRNN and FastGRNN - FastCells
2
2
3
- This document aims to explain and elaborate on specific details of FastCells
4
- present as part of [ edgeml_tf/graph/rnn.py] ( .. /edgeml_tf/graph/rnn.py) . The
3
+ This document elaborates on the details of FastCells
4
+ present in [ tf/ edgeml_tf/graph/rnn.py] ( /tf /edgeml_tf/graph/rnn.py) . The
5
5
endpoint use case scripts with 3 phase training along with an example notebook
6
- are present in [ examples/tf/FastCells] ( .. /examples/tf/FastCells) . One can use the endpoint script to test
6
+ are present in [ examples/tf/FastCells] ( /examples/tf/FastCells ) . One can use the endpoint script to test
7
7
out the RNN architectures on any dataset while specifying budget constraints as
8
8
part of hyper-parameters in terms of sparsity and rank of weight matrices.
9
9
@@ -23,11 +23,11 @@ replace any of the standard RNN Cell in our architecture with FastCells.
23
23
One can see the plug and play nature at the endpoint script for FastCells, where the graph
24
24
building is very similar to LSTM/GRU in Tensorflow.
25
25
26
- Script: [ Endpoint Script] ( .. /examples/tf/FastCells/fastcell_example.py)
26
+ Script: [ Endpoint Script] ( /examples/tf/FastCells/fastcell_example.py )
27
27
28
- Example Notebook: [ iPython Notebook] ( .. /examples/tf/FastCells/fastcell_example.ipynb)
28
+ Example Notebook: [ iPython Notebook] ( /examples/tf/FastCells/fastcell_example.ipynb )
29
29
30
- Cells: [ FastRNNCell] ( ../ edgeml/graph/rnn.py#L206) and [ FastGRNNCell] ( .. /edgeml/graph/rnn.py#L31) .
30
+ Cells: [ FastRNNCell] ( /tf/ edgeml/graph/rnn.py#L206) and [ FastGRNNCell] ( /tf /edgeml/graph/rnn.py#L31) .
31
31
32
32
# 3 phase Fast Training
33
33
@@ -50,7 +50,7 @@ Sparsity is taken in as hyper-parameter during the 3 phase training into `fastTr
50
50
end spits out a sparse, low-rank model.
51
51
52
52
Further compression is achieved by byte Quantization and can be performed using ` quantizeFastModels.py `
53
- script which is part of [ examples/tf/FastCells] ( .. /examples/tf/FastCells) . This will give model size reduction of up to 4x if 8-bit
53
+ script which is part of [ examples/tf/FastCells] ( /examples/tf/FastCells ) . This will give model size reduction of up to 4x if 8-bit
54
54
integers are used. Lastly, to facilitate all integer arithmetic, including the non-linearities, one could
55
55
use ` quantTanh ` instead of ` tanh ` and ` quantSigm ` instead of ` sigmoid ` as the non-linearities in the RNN
56
56
Cells followed by byte quantization. These non-linearities can be set using the appropriate parameters in
0 commit comments