|
| 1 | +# Computer vision with TensorFlow |
| 2 | + |
| 3 | +TensorFlow provides a number of computer vision (CV) and image classification |
| 4 | +tools. This document introduces some of these tools and provides an overview of |
| 5 | +resources to help you get started with common CV tasks. |
| 6 | + |
| 7 | +## Vision libraries and tools |
| 8 | + |
| 9 | +TensorFlow provides CV tools through the higher-level Keras libraries and the |
| 10 | +lower-level `tf.image` module. For most use cases, the Keras libraries |
| 11 | +will be more convenient than the built-in TensorFlow alternatives. |
| 12 | +But if the Keras options don't fit your use case, or you want lower-level |
| 13 | +control over image preprocessing, you might need the lower-level TensorFlow |
| 14 | +tools. |
| 15 | + |
| 16 | +### KerasCV |
| 17 | + |
| 18 | +If you're just getting started with a CV project, and you're not sure which |
| 19 | +libraries and tools you'll need, [KerasCV](https://keras.io/keras_cv/) is a good |
| 20 | +place to start. KerasCV is a library of modular CV components built on Keras |
| 21 | +Core. KerasCV includes models, layers, metrics, callbacks, and other tools that |
| 22 | +extend the high-level Keras API for CV tasks. The KerasCV APIs can help with |
| 23 | +data augmentation, classification, object detection, segmentation, |
| 24 | +image generation, and other common CV workflows. You can use KerasCV to quickly |
| 25 | +assemble production-grade, state-of-the-art training and inference pipelines. |
| 26 | + |
| 27 | +### Keras utilities |
| 28 | + |
| 29 | +`tf.keras.utils` provides several high-level image preprocessing utilities. For |
| 30 | +example, `tf.keras.utils.image_dataset_from_directory` generates a |
| 31 | +`tf.data.Dataset` from a directory of images on disk. |
| 32 | + |
| 33 | +### `tf.image` |
| 34 | + |
| 35 | +If KerasCV doesn't fit your use case, you can use `tf.image` and `tf.data` to |
| 36 | +write your own data augmentation pipelines or layers. |
| 37 | + |
| 38 | +The `tf.image` module contains various functions for image processing, such as |
| 39 | +`tf.image.flip_left_right`, `tf.image.rgb_to_grayscale`, |
| 40 | +`tf.image.adjust_brightness`, `tf.image.central_crop`, and |
| 41 | +`tf.image.stateless_random*`. |
| 42 | + |
| 43 | +The `tf.data` API enables you to build complex input pipelines from simple, |
| 44 | +reusable pieces. |
| 45 | + |
| 46 | +### TensorFlow Datasets |
| 47 | + |
| 48 | +[TensorFlow Datasets](https://www.tensorflow.org/datasets) is a collection of |
| 49 | +datasets ready to use with TensorFlow. Many of the datasets (for example, |
| 50 | +[MNIST](https://www.tensorflow.org/datasets/catalog/mnist), |
| 51 | +[Fashion-MNIST](https://www.tensorflow.org/datasets/catalog/fashion_mnist), and |
| 52 | +[TF Flowers](https://www.tensorflow.org/datasets/catalog/tf_flowers)) can be |
| 53 | +used to develop and test computer vision algorithms. |
| 54 | + |
| 55 | +## Where to start |
| 56 | + |
| 57 | +The following resources will help you get up and running with TensorFlow and |
| 58 | +Keras CV tools. |
| 59 | + |
| 60 | +* [KerasCV](https://keras.io/keras_cv/): Documentation and resources for |
| 61 | + KerasCV. |
| 62 | +* [KerasCV developer guides](https://keras.io/guides/keras_cv/): Guides to |
| 63 | + performing common CV tasks using KerasCV. If you're new to KerasCV, |
| 64 | + [Classification with KerasCV](https://keras.io/guides/keras_cv/classification_with_keras_cv/) |
| 65 | + is a good place to start. |
| 66 | +* [TensorFlow tutorials](https://www.tensorflow.org/tutorials): The core |
| 67 | + TensorFlow documentation (this guide) includes a number of CV and image |
| 68 | + processing tutorials. |
| 69 | + * [Basic classification: Classify images of clothing](https://www.tensorflow.org/tutorials/keras/classification): |
| 70 | + Train a neural network model to classify images of clothing, like sneakers |
| 71 | + and shirts. |
| 72 | + * [Load and preprocess images](https://www.tensorflow.org/tutorials/load_data/images): |
| 73 | + Load and preprocess an image dataset in three ways: |
| 74 | + |
| 75 | + 1. Use high-level Keras preprocessing utilities to read a directory of |
| 76 | + images on disk. |
| 77 | + 2. Write your own input pipeline from scratch |
| 78 | + [using `tf.data`](https://www.tensorflow.org/guide/data). |
| 79 | + 3. Download a dataset from the large |
| 80 | + [catalog](https://www.tensorflow.org/datasets/catalog/overview) |
| 81 | + available in |
| 82 | + [TensorFlow Datasets](https://www.tensorflow.org/datasets). |
| 83 | + |
| 84 | + * [Load video data](https://www.tensorflow.org/tutorials/load_data/video): |
| 85 | + Load and preprocess AVI video data using the |
| 86 | + [UCF101 human action dataset](https://www.tensorflow.org/datasets/catalog/ucf101). |
| 87 | + * [Convolutional Neural Network (CNN)](https://www.tensorflow.org/tutorials/images/cnn): |
| 88 | + Train a simple [Convolutional Neural Network](https://developers.google.com/machine-learning/glossary/#convolutional_neural_network) |
| 89 | + (CNN) to classify |
| 90 | + [CIFAR images](https://www.cs.toronto.edu/~kriz/cifar.html) |
| 91 | + using the |
| 92 | + [Keras API](https://www.tensorflow.org/guide/keras/overview). |
| 93 | + * [Image classification](https://www.tensorflow.org/tutorials/images/classification): |
| 94 | + Classify images of flowers using a `tf.keras.Sequential` model and load data |
| 95 | + using `tf.keras.utils.image_dataset_from_directory`. |
| 96 | + * [Transfer learning and fine-tuning](https://www.tensorflow.org/tutorials/images/transfer_learning): |
| 97 | + Classify images of cats and dogs by using transfer learning from a |
| 98 | + pre-trained network. |
| 99 | + * [Data augmentation](https://www.tensorflow.org/tutorials/images/data_augmentation): |
| 100 | + Increase the diversity of your training set by applying random (but |
| 101 | + realistic) transformations, such as image rotation. |
| 102 | + * [Image segmentation](https://www.tensorflow.org/tutorials/images/segmentation): |
| 103 | + Perform image segmentation, using a modified |
| 104 | + [U-Net](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/){: .external}. |
| 105 | + * [Video classification with a 3D convolutional neural network](https://www.tensorflow.org/tutorials/video/video_classification): |
| 106 | + Train a 3D convolutional neural network (CNN) for video classification using |
| 107 | + the [UCF101](https://www.crcv.ucf.edu/data/UCF101.php){: .external} action |
| 108 | + recognition dataset. |
| 109 | + * [Transfer learning for video classification with MoViNet](https://www.tensorflow.org/tutorials/video/transfer_learning_with_movinet): |
| 110 | + Use a pre-trained MoViNet model and the |
| 111 | + [UCF101 dataset](https://www.crcv.ucf.edu/data/UCF101.php){: .external} to |
| 112 | + classify videos for an action recognition task. |
| 113 | + |
0 commit comments