Skip to content

Commit ab714b9

Browse files
Johannes Ball?copybara-github
authored andcommitted
Further updates to documentation.
PiperOrigin-RevId: 258815306 Change-Id: I742e55713268f23b19eb9ca513aafcb3c3379230
1 parent 1123816 commit ab714b9

36 files changed

+123
-194
lines changed

README.md

Lines changed: 40 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# TensorFlow Compression
1+
# Introduction
22

33
This project contains data compression ops and layers for TensorFlow. The
44
project website is at
@@ -37,54 +37,44 @@ There's also an introduction to our `EntropyBottleneck` class
3737
and a description of the range coding operators
3838
[here](https://tensorflow.github.io/compression/docs/range_coding.html).
3939

40-
## Quick start
40+
## Installation
4141

42-
### Installing release 1.1 (stable)
43-
44-
Install TensorFlow 1.13 using one of the methods described in the
45-
[TensorFlow installation instructions](https://www.tensorflow.org/install).
46-
47-
Download the ZIP file for
48-
[release v1.1](https://github.com/tensorflow/compression/releases/tag/v1.1)
49-
and unpack it. Then include its root directory in your `PYTHONPATH`
50-
environment variable:
51-
52-
```bash
53-
cd <target directory>
54-
wget https://github.com/tensorflow/compression/archive/v1.1.zip
55-
unzip v1.1.zip
56-
export PYTHONPATH="$PWD/compression-1.1:$PYTHONPATH"
57-
```
58-
59-
To make sure the library imports succeed, try running the unit tests:
60-
61-
```bash
62-
cd compression-1.1
63-
for i in tensorflow_compression/python/*/*_test.py; do python $i; done
64-
```
65-
66-
### Installing release 1.2b2 (beta)
42+
***Note: Precompiled packages are currently only provided for Linux and Darwin
43+
(Mac OS). To use these packages on Windows, consider using a [TensorFlow
44+
Docker image](https://www.tensorflow.org/install/docker) and installing
45+
tensorflow-compression using pip inside the Docker container.***
6746

6847
Set up an environment in which you can install precompiled binary Python
6948
packages using the `pip` command. Refer to the
7049
[TensorFlow installation instructions](https://www.tensorflow.org/install/pip)
7150
for more information on how to set up such a Python environment.
7251

73-
Install TensorFlow 1.14 or above with or without GPU support:
52+
The current version of tensorflow-compression requires TensorFlow 1.14 or
53+
above. You can install TensorFlow from any source. To install it via `pip`, run
54+
the following command:
7455
```bash
75-
pip install tensorflow
56+
pip install tensorflow-gpu
7657
```
77-
or
58+
for GPU support, or
7859
```bash
79-
pip install tensorflow-gpu
60+
pip install tensorflow
8061
```
62+
for CPU-only.
8163

82-
Then, run the following command to install the binary PIP package:
64+
Then, run the following command to install the tensorflow-compression `pip`
65+
package:
8366
```bash
8467
pip install tensorflow-compression
8568
```
8669

87-
### Using the library
70+
To test that the installation works correctly, you can run the unit tests with
71+
```bash
72+
python -m tensorflow_compression.python.all_test
73+
```
74+
Once the command finishes, you should see a message ```OK (skipped=11)``` or
75+
similar in the last line.
76+
77+
## Usage
8878

8979
We recommend importing the library from your Python code as follows:
9080

@@ -95,31 +85,23 @@ import tensorflow_compression as tfc
9585

9686
### Using a pre-trained model to compress an image
9787

98-
***Note: you need to have a release >1.1 installed for pre-trained model
99-
support.***
100-
10188
In the
10289
[examples directory](https://github.com/tensorflow/compression/tree/master/examples),
10390
you'll find a python script `tfci.py`. Download the file and run:
104-
10591
```bash
10692
python tfci.py -h
10793
```
10894

10995
This will give you a list of options. Briefly, the command
110-
11196
```bash
11297
python tfci.py compress <model> <PNG file>
11398
```
114-
11599
will compress an image using a pre-trained model and write a file ending in
116100
`.tfci`. Execute `python tfci.py models` to give you a list of supported
117101
pre-trained models. The command
118-
119102
```bash
120103
python tfci.py decompress <TFCI file>
121104
```
122-
123105
will decompress a TFCI file and write a PNG file. By default, an output file
124106
will be named like the input file, only with the appropriate file extension
125107
appended (any existing extensions will not be removed).
@@ -135,15 +117,13 @@ contains an implementation of the image compression model described in:
135117
> https://arxiv.org/abs/1611.01704
136118
137119
To see a list of options, download the file `bls2017.py` and run:
138-
139120
```bash
140121
python bls2017.py -h
141122
```
142123

143124
To train the model, you need to supply it with a dataset of RGB training images.
144125
They should be provided in PNG format. Training can be as simple as the
145126
following command:
146-
147127
```bash
148128
python bls2017.py -v --train_glob="images/*.png" train
149129
```
@@ -163,34 +143,34 @@ enough (or larger). This is described in more detail in:
163143
If you wish, you can monitor progress with Tensorboard. To do this, create a
164144
Tensorboard instance in the background before starting the training, then point
165145
your web browser to [port 6006 on your machine](http://localhost:6006):
166-
167146
```bash
168147
tensorboard --logdir=. &
169148
```
170149

171150
When training has finished, the Python script can be used to compress and
172151
decompress images as follows. The same model checkpoint must be accessible to
173152
both commands.
174-
175153
```bash
176154
python bls2017.py [options] compress original.png compressed.bin
177155
python bls2017.py [options] decompress compressed.bin reconstruction.png
178156
```
179157

180-
## Building PIP packages
158+
## Building `pip` packages
181159

182-
This section describes steps to build PIP packages.
160+
This section describes the necessary steps to build your own `pip` packages of
161+
tensorflow-compression. This may be necessary to install it on platforms for
162+
which we don't provide precompiled binaries (currently only Linux and Darwin).
183163

184-
We use the Docker image `tensorflow/tensorflow:nightly-custom-op` for building
185-
PIP packages. Note that this is different from `tensorflow/tensorflow:devel`. To
186-
be compatible with the TensorFlow PIP package, the GCC version must match, but
187-
`tensorflow/tensorflow:devel` has a different GCC version installed.
164+
We use the Docker image `tensorflow/tensorflow:nightly-custom-op` for building
165+
`pip` packages. Note that this is different from `tensorflow/tensorflow:devel`.
166+
To be compatible with the TensorFlow `pip` package, the GCC version must match,
167+
but `tensorflow/tensorflow:devel` has a different GCC version installed.
188168

189-
Inside a Docker container from the image, the following steps need to be done.
169+
Inside a Docker container from the image, the following steps need to be taken.
190170

191-
1. Install TensorFlow PIP package.
192-
2. Clone `tensorflow-compression` repo from GitHub.
193-
3. Run `:build_pip_pkg` inside the cloned repo.
171+
1. Install TensorFlow `pip` package.
172+
2. Clone the `tensorflow/compression` repo from GitHub.
173+
3. Run `:build_pip_pkg` inside the cloned repo:
194174

195175
```bash
196176
sudo docker run -v /tmp/tensorflow_compression:/tmp/tensorflow_compression \
@@ -204,28 +184,22 @@ sudo docker run -v /tmp/tensorflow_compression:/tmp/tensorflow_compression \
204184
The wheel file is created inside `/tmp/tensorflow_compression`. Optimization
205185
flags can be passed via `--copt` to the `bazel run` command above.
206186

207-
### Testing PIP package
208-
209-
First install the built wheel file.
210-
187+
To test the created package, first install the resulting wheel file:
211188
```bash
212189
pip install /tmp/tensorflow_compression/tensorflow_compression-*.whl
213190
```
214191

215-
And run tests. (Do not run the tests in the workspace directory where
192+
Then run the unit tests (Do not run the tests in the workspace directory where
216193
`WORKSPACE` of `tensorflow_compression` repo lives. In that case, the Python
217-
interpreter attempts to import `tensorflow_compression` packages from the source
218-
tree rather than from the installed package system directory.)
219-
194+
interpreter would attempt to import `tensorflow_compression` packages from the
195+
source tree rather than from the installed package system directory):
220196
```bash
221197
pushd /tmp
222198
python -m tensorflow_compression.python.all_test
223199
popd
224200
```
225201

226-
When done, uninstall the PIP package. The package name is
227-
`tensorflow-compression` with a hyphen(-).
228-
202+
When done, you can uninstall the `pip` package again:
229203
```bash
230204
pip uninstall tensorflow-compression
231205
```

_config.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
theme: jekyll-theme-cayman
2-
title: TensorFlow/compression
1+
theme: jekyll-theme-hacker
2+
title: tensorflow-compression
33
show_downloads: false

docs/api_docs/python/index.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
* <a href="./tfc/SignalConv3D.md"><code>tfc.SignalConv3D</code></a>
1717
* <a href="./tfc/StaticParameterizer.md"><code>tfc.StaticParameterizer</code></a>
1818
* <a href="./tfc/SymmetricConditional.md"><code>tfc.SymmetricConditional</code></a>
19-
* <a href="./tfc/array_fingerprint.md"><code>tfc.array_fingerprint</code></a>
2019
* <a href="./tfc/irdft_matrix.md"><code>tfc.irdft_matrix</code></a>
2120
* <a href="./tfc/lower_bound.md"><code>tfc.lower_bound</code></a>
2221
* <a href="./tfc/pmf_to_quantized_cdf.md"><code>tfc.pmf_to_quantized_cdf</code></a>
@@ -49,7 +48,6 @@
4948
* <a href="./tfc/python/ops/padding_ops.md"><code>tfc.python.ops.padding_ops</code></a>
5049
* <a href="./tfc/same_padding_for_kernel.md"><code>tfc.python.ops.padding_ops.same_padding_for_kernel</code></a>
5150
* <a href="./tfc/python/ops/range_coding_ops.md"><code>tfc.python.ops.range_coding_ops</code></a>
52-
* <a href="./tfc/array_fingerprint.md"><code>tfc.python.ops.range_coding_ops.array_fingerprint</code></a>
5351
* <a href="./tfc/pmf_to_quantized_cdf.md"><code>tfc.python.ops.range_coding_ops.pmf_to_quantized_cdf</code></a>
5452
* <a href="./tfc/range_decode.md"><code>tfc.python.ops.range_coding_ops.range_decode</code></a>
5553
* <a href="./tfc/range_encode.md"><code>tfc.python.ops.range_coding_ops.range_encode</code></a>

docs/api_docs/python/tfc.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Data compression tools.
1111

1212

1313
<table class="tfo-github-link" align="left">
14-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/__init__.py>
14+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/__init__.py">
1515
<img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />
1616
View source on GitHub
1717
</a>
@@ -58,8 +58,6 @@ Data compression tools.
5858

5959
## Functions
6060

61-
[`array_fingerprint(...)`](./tfc/array_fingerprint.md): Produces fingerprint of the input data.
62-
6361
[`irdft_matrix(...)`](./tfc/irdft_matrix.md): Matrix for implementing kernel reparameterization with `tf.matmul`.
6462

6563
[`lower_bound(...)`](./tfc/lower_bound.md): Same as `tf.maximum`, but with helpful gradient for `inputs < bound`.

docs/api_docs/python/tfc/EntropyBottleneck.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Inherits From: [`EntropyModel`](../tfc/EntropyModel.md)
7070

7171

7272
<table class="tfo-github-link" align="left">
73-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>
73+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">
7474
<img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />
7575
View source on GitHub
7676
</a>
@@ -177,7 +177,7 @@ Read-only properties:
177177

178178
<h2 id="__init__"><code>__init__</code></h2>
179179

180-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
180+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
181181

182182
``` python
183183
__init__(
@@ -531,7 +531,7 @@ Output tensor(s).
531531

532532
<h3 id="build"><code>build</code></h3>
533533

534-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
534+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
535535

536536
``` python
537537
build(input_shape)
@@ -559,7 +559,7 @@ cumulative density functions used by the range coder.
559559

560560
<h3 id="compress"><code>compress</code></h3>
561561

562-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
562+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
563563

564564
``` python
565565
compress(inputs)
@@ -614,7 +614,7 @@ None or a tensor (or list of tensors,
614614

615615
<h3 id="compute_output_shape"><code>compute_output_shape</code></h3>
616616

617-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
617+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
618618

619619
``` python
620620
compute_output_shape(input_shape)
@@ -663,7 +663,7 @@ An integer count.
663663

664664
<h3 id="decompress"><code>decompress</code></h3>
665665

666-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
666+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
667667

668668
``` python
669669
decompress(

docs/api_docs/python/tfc/EntropyModel.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ Entropy model (base class).
6767

6868

6969
<table class="tfo-github-link" align="left">
70-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>
70+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">
7171
<img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />
7272
View source on GitHub
7373
</a>
@@ -123,7 +123,7 @@ Read-only properties:
123123

124124
<h2 id="__init__"><code>__init__</code></h2>
125125

126-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
126+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
127127

128128
``` python
129129
__init__(
@@ -483,7 +483,7 @@ This is typically used to create the weights of `Layer` subclasses.
483483

484484
<h3 id="compress"><code>compress</code></h3>
485485

486-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
486+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
487487

488488
``` python
489489
compress(inputs)
@@ -538,7 +538,7 @@ None or a tensor (or list of tensors,
538538

539539
<h3 id="compute_output_shape"><code>compute_output_shape</code></h3>
540540

541-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
541+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
542542

543543
``` python
544544
compute_output_shape(input_shape)
@@ -587,7 +587,7 @@ An integer count.
587587

588588
<h3 id="decompress"><code>decompress</code></h3>
589589

590-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py>View source</a>
590+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/entropy_models.py">View source</a>
591591

592592
``` python
593593
decompress(

docs/api_docs/python/tfc/GDN.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ Generalized divisive normalization layer.
6868

6969

7070
<table class="tfo-github-link" align="left">
71-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py>
71+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py">
7272
<img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" />
7373
View source on GitHub
7474
</a>
@@ -152,7 +152,7 @@ Read-only properties:
152152

153153
<h2 id="__init__"><code>__init__</code></h2>
154154

155-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py>View source</a>
155+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py">View source</a>
156156

157157
``` python
158158
__init__(
@@ -509,7 +509,7 @@ Output tensor(s).
509509

510510
<h3 id="build"><code>build</code></h3>
511511

512-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py>View source</a>
512+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py">View source</a>
513513

514514
``` python
515515
build(input_shape)
@@ -557,7 +557,7 @@ None or a tensor (or list of tensors,
557557

558558
<h3 id="compute_output_shape"><code>compute_output_shape</code></h3>
559559

560-
<a target="_blank" href=https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py>View source</a>
560+
<a target="_blank" href="https://github.com/tensorflow/compression/tree/master/tensorflow_compression/python/layers/gdn.py">View source</a>
561561

562562
``` python
563563
compute_output_shape(input_shape)

0 commit comments

Comments
 (0)