Skip to content

Commit c2ae0e1

Browse files
Johannes Ballécopybara-github
authored andcommitted
Updates instructions for building custom pip packages.
PiperOrigin-RevId: 467328687 Change-Id: I8dff10d0a625c88ed7051b88a16ec8b3e1c99b0b
1 parent 08bdedf commit c2ae0e1

File tree

1 file changed

+30
-20
lines changed

1 file changed

+30
-20
lines changed

README.md

Lines changed: 30 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -246,32 +246,45 @@ This section describes the necessary steps to build your own pip packages of
246246
TensorFlow Compression. This may be necessary to install it on platforms for
247247
which we don't provide precompiled binaries (currently only Linux and Darwin).
248248

249-
You can use the custom-op Docker images (e.g.
250-
`tensorflow/tensorflow:nightly-custom-op-ubuntu16`) for building pip packages
251-
for Linux. Note that this is different from `tensorflow/tensorflow:devel`. To be
252-
compatible with the TensorFlow pip package, the GCC version must match, but
253-
`tensorflow/tensorflow:devel` has a different GCC version installed. For more
254-
information, refer to the [custom-op
255-
instructions](https://github.com/tensorflow/custom-op).
249+
To be compatible with the official TensorFlow pip package, the TFC pip package
250+
must be linked against a matching version of the C libraries. For this reason,
251+
to build the official Linux pip packages, we use [these Docker
252+
images](https://hub.docker.com/r/tensorflow/build) and use the same toolchain
253+
that TensorFlow uses.
256254

257-
Inside a Docker container from the image, the following steps need to be taken.
255+
Inside the Docker container, the following steps need to be taken:
258256

259257
1. Clone the `tensorflow/compression` repo from GitHub.
260-
2. Run `:build_pip_pkg` inside the cloned repo.
258+
2. Install Python dependencies.
259+
3. Run `:build_pip_pkg` inside the cloned repo.
261260

262261
For example:
263262

264263
```bash
265-
sudo docker run -v /tmp/tensorflow_compression:/tmp/tensorflow_compression \
266-
tensorflow/tensorflow:nightly-custom-op-ubuntu16 bash -c \
267-
"git clone https://github.com/tensorflow/compression.git
268-
/tensorflow_compression &&
264+
sudo docker run -i --rm -v /tmp/tensorflow_compression:/tmp/tensorflow_compression \
265+
tensorflow/build:latest-python3.10 bash -c \
266+
"git clone https://github.com/tensorflow/compression.git /tensorflow_compression &&
269267
cd /tensorflow_compression &&
270-
bazel run -c opt --copt=-mavx :build_pip_pkg"
268+
python -m pip install -U pip setuptools wheel &&
269+
python -m pip install -r requirements.txt &&
270+
bazel build -c opt --copt=-mavx [email protected]_manylinux2014-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain :build_pip_pkg
271+
python build_pip_pkg.py bazel-bin/build_pip_pkg.runfiles/tensorflow_compression /tmp/tensorflow_compression custom_version"
271272
```
272273

273-
The wheel file is created inside `/tmp/tensorflow_compression`. Optimization
274-
flags can be passed via `--copt` to the `bazel run` command above.
274+
For Darwin, the Docker image and specifying the toolchain is not necessary. We
275+
just build the package like this (note that you may want to create a clean
276+
Python virtual environment to do this):
277+
278+
```bash
279+
git clone https://github.com/tensorflow/compression.git /tensorflow_compression
280+
cd /tensorflow_compression
281+
python -m pip install -U pip setuptools wheel
282+
python -m pip install -r requirements.txt
283+
bazel build -c opt --copt=-mavx --macos_minimum_os=10.14 :build_pip_pkg
284+
python build_pip_pkg.py bazel-bin/build_pip_pkg.runfiles/tensorflow_compression /tmp/tensorflow_compression custom_version"
285+
```
286+
287+
In both cases, the wheel file is created inside `/tmp/tensorflow_compression`.
275288
276289
To test the created package, first install the resulting wheel file:
277290
@@ -296,9 +309,6 @@ When done, you can uninstall the pip package again:
296309
pip uninstall tensorflow-compression
297310
```
298311
299-
To build packages for Darwin (and potentially other platforms), you can follow
300-
the same steps, but the Docker image should not be necessary.
301-
302312
## Evaluation
303313
304314
We provide evaluation results for several image compression methods in terms of
@@ -314,7 +324,7 @@ If you use this library for research purposes, please cite:
314324
author = "Ballé, Johannes and Hwang, Sung Jin and Agustsson, Eirikur",
315325
title = "{T}ensor{F}low {C}ompression: Learned Data Compression",
316326
url = "http://github.com/tensorflow/compression",
317-
version = "2.9.1",
327+
version = "2.9.2",
318328
year = "2022",
319329
}
320330
```

0 commit comments

Comments
 (0)