|
| 1 | +OpenImageIO's Architecture |
| 2 | +========================== |
| 3 | + |
| 4 | +```mermaid |
| 5 | +block-beta |
| 6 | + columns 3 |
| 7 | + Apps |
| 8 | + style Apps fill:#0000,stroke:#0000 |
| 9 | + cmd["oiiotool, iinfo, maketx, ..."]:2 |
| 10 | +
|
| 11 | + Classes["Image and caching\nclasses"] |
| 12 | + style Classes fill:#0000,stroke:#0000 |
| 13 | +
|
| 14 | + block:IB |
| 15 | + style IB fill:#0000,stroke:#0000 |
| 16 | + columns 1 |
| 17 | + ImageBufAlgo ImageBuf |
| 18 | + end |
| 19 | + block:ICTA |
| 20 | + style ICTA fill:#0000,stroke:#0000 |
| 21 | + columns 1 |
| 22 | + TextureSystem ImageCache |
| 23 | + end |
| 24 | +
|
| 25 | + file["File reading/writing\nclasses"]:1 |
| 26 | + style file fill:#0000,stroke:#0000 |
| 27 | + ImageInput ImageOutput |
| 28 | + |
| 29 | + plugins["File format\nimplementations"] |
| 30 | + style plugins fill:#0000,stroke:#0000; |
| 31 | +
|
| 32 | + block:formats:2 |
| 33 | + style formats fill:#0000,stroke:#0000 |
| 34 | + columns auto |
| 35 | + TIFF OpenEXR JPEG DPX etc["..."] |
| 36 | + style etc fill:#0000,stroke:#0000 |
| 37 | + end |
| 38 | +``` |
| 39 | + |
| 40 | +## Core level / file abstraction: ImageInput and ImageOutput |
| 41 | + |
| 42 | +At the heart of OpenImageIO are the `ImageInput` and `ImageOutput` classes, |
| 43 | +which are the abstract interfaces for reading and writing image files using a |
| 44 | +simple file-format-agnostic API. These classes provide a file-oriented |
| 45 | +abstraction, with operations such as opening an image file, reading the |
| 46 | +resolution and other metadata about the image in the file, reading or writing |
| 47 | +scanlines or tiles to/from caller-provided memory, and closing the file. |
| 48 | + |
| 49 | +All the details of the individual file formats and how the data is arranged or |
| 50 | +stored in the file are hidden by these classes. Furthermore, when reading or |
| 51 | +writing scanlines or tiles, the caller-side memory can use any of several |
| 52 | +different common data types (sometimes called "data formats" or "formats" in |
| 53 | +OIIO) such as `float`, `half`, 8 or 16 bit integers, etc. The `ImageInput` and |
| 54 | +`ImageOutput` classes will automatically convert the data to or from the |
| 55 | +format stored in the file. |
| 56 | + |
| 57 | +This level of abstraction imposes some conceptual constraints on the |
| 58 | +presentation of the image data, for the sake of simplification, so that every |
| 59 | +application doesn't need to consider every possible combination of choices |
| 60 | +that all the different file formats might make. For example, on the |
| 61 | +"application side" of the `ImageInput` and `ImageOutput` classes, all images |
| 62 | +have pixels that are 8- 16- or 32-bit int (signed or unsigned), 16- 32- or |
| 63 | +64-bit float. If the pixel data in the file uses another data type, or a |
| 64 | +number of bits per channel that is not 8, 16, or 32, those details are hidden |
| 65 | +from the application. There are many other such simplifications. |
| 66 | + |
| 67 | +## Low level: File format implementations |
| 68 | + |
| 69 | +If the `ImageInput` and `ImageOutput` present the *interface* for reading and |
| 70 | +writing image files, the implementations of each individual file format are |
| 71 | +implemented in the file format plugins. (We call them "plugins" because it's |
| 72 | +possible to organize them as an extensible list of dynamically loaded modules, |
| 73 | +but all the common file formats have their plugins compiled into the |
| 74 | +OpenImageIO library itself, so are not dynamically loaded and not generally |
| 75 | +experienced as "plugins" in the usual sense.) |
| 76 | + |
| 77 | +Each file format plugin consists of a concrete subclass of `ImageInput`, and |
| 78 | +usually `ImageOutput`. Most format plugins are capable of both reading and |
| 79 | +writing the file format, but some of them only include the reading portion. |
| 80 | + |
| 81 | +## Higher level / Image abstraction: ImageBuf and ImageBufAlgo |
| 82 | + |
| 83 | +For many applications, you just want to manipulate whole images, without |
| 84 | +worrying about the details of opening and closing files, reading and writing |
| 85 | +of individual scanlines or tiles, or how and where the pixel data is stored. |
| 86 | +The `ImageBuf` class provides such an interface. It is a higher-level |
| 87 | +abstraction that encapsulates an entire image. |
| 88 | + |
| 89 | +For an ImageBuf that is meant to read an image file, it is simply told the |
| 90 | +name of the file, and it will handle all the details of how and when to open, |
| 91 | +close, and read the file, including handling all the combinations of tiles, |
| 92 | +scanlines, and so on. Similarly, an ImageBuf can be written to an image file |
| 93 | +on disk simply by providing the filename and telling it to `write()`, without |
| 94 | +needing to specify any scanline-by-scanline instructions. The internal |
| 95 | +implementation of ImageBuf uses ImageInput and ImageOutput to read and write |
| 96 | +the image data from and to files, and so can read or write any file format for |
| 97 | +which there is an ImageInput or ImageOutput plugin available. |
| 98 | + |
| 99 | +ImageBuf typically owns the memory holding its pixel data and is responsible |
| 100 | +for allocating and freeing it. Thus, file reads and writes are to/from this |
| 101 | +internal memory, as opposed to ImageInput and ImageOutput, which must be |
| 102 | +supplied with memory by the caller as the source or destination of the pixel |
| 103 | +data. (That said, there is also a way to make an ImageBuf "wrap" memory owned |
| 104 | +by the application rather than allocate internally.) |
| 105 | + |
| 106 | +In addition to restricting the pixel data types to a limited set, ImageBuf |
| 107 | +further imposes the simplification that all color channels are stored in the |
| 108 | +same data type. This is a simplification that is not present in the ImageInput |
| 109 | +and ImageOutput classes, which can handle files where different channels are |
| 110 | +stored in different data types. For such a file, the ImageBuf will "promote" |
| 111 | +all the channels of an image to a single same data type (usually the "widest" |
| 112 | +or "most precise" used by any of its channels). |
| 113 | + |
| 114 | +If `ImageBuf` is the in-memory representation of an entire image, then |
| 115 | +`ImageBufAlgo` is a collection of algorithms that operate on `ImageBuf` |
| 116 | +objects. These algorithms include simple operations like copying, resizing, |
| 117 | +and compositing images, as well as more complex operations like color |
| 118 | +conversions, resizing, filtering, etc. |
| 119 | + |
| 120 | +## Image caching: TextureSystem and ImageCache |
| 121 | + |
| 122 | +There are situations where ImageBuf is still not the right abstraction, |
| 123 | +and you have so many images -- potentially tens of thousands of images |
| 124 | +totalling multiple terabytes of pixel data -- that they can't possibly |
| 125 | +be held in memory at once. This is actually a typical case for a |
| 126 | +high-quality film renderer, where the renderer needs to be able to |
| 127 | +access a texture data set of that size range. |
| 128 | + |
| 129 | +In such cases, you want to handle both the pixel memory and the set of |
| 130 | +open files as a *pool* centrally managed. This is done by the |
| 131 | +`ImageCache` class. In short, it manages two central caches: |
| 132 | + |
| 133 | +- A pool of image files currently held open, with a limit on the number of |
| 134 | + files that can be open at once (typically hundreds to thousands). When the |
| 135 | + limit is reached, a file that has not been accessed recently is closed to |
| 136 | + make room for a new file to be opened. |
| 137 | + |
| 138 | +- A pool of pixel memory, organized into image *tiles*, with a limit on the |
| 139 | + total amount of tile memory to be used (typically a small number of GB). |
| 140 | + Accesses to pixel values check if the tile is in the cache, in which case no |
| 141 | + read from disk is necessary. If pixel access requires a tile not currently |
| 142 | + in the cache and the tile memory limit is reached, a tile that has not been |
| 143 | + accessed recently is evicted from the cache to make room for the new tile. |
| 144 | + |
| 145 | +The TextureSystem is a level of abstraction above the ImageCache that further |
| 146 | +adds the ability to do texture lookups with texture filtering, MIP-mapping, |
| 147 | +and other features typical of texture mapping in a renderer. |
| 148 | + |
| 149 | +## Command line applications |
| 150 | + |
| 151 | +OpenImageIO has several command line utilities that use various combinations |
| 152 | +of ImageBuf, ImageInput, and ImageOutput. |
| 153 | + |
| 154 | +`oiiotool` is the most important of these (and technically can do everything |
| 155 | +that the other tools can do, and more). It is a general-purpose image |
| 156 | +manipulation tool that can read and write images, apply various image |
| 157 | +operations, and write the results to disk. |
| 158 | + |
| 159 | +Other command line tools include `iinfo` (which prints out information about |
| 160 | +an image file), `iconvert` (which converts between different file formats), |
| 161 | +and `maketx` (which generates tiled mipmaps in an efficient arrangement for |
| 162 | +texture mapping in a renderer). |
| 163 | + |
| 164 | +## Language Bindings |
| 165 | + |
| 166 | +The main APIs, and the underlying implementation, are in C++ (currently |
| 167 | +using C++17 features). |
| 168 | + |
| 169 | +A Python binding is provided, currently using the pybind11 library to |
| 170 | +generate the Python bindings for each C++ class or function. |
| 171 | + |
| 172 | +There is an effort underway to eventually provide a Rust binding. |
| 173 | + |
| 174 | +## Helper classes |
| 175 | + |
| 176 | +Several image-related auxiliary classes are used by the public APIs of the |
| 177 | +image- and file-oriented classes described above. The most important ones are: |
| 178 | + |
| 179 | +- `ImageSpec` stores the specification for an image: its resolution, pixel data |
| 180 | + type, number of channels (as well as their names and individual data types), |
| 181 | + pixel and display windows, and all other metadata from the image file. |
| 182 | +- `ParamValueList` is used to store a list of of name/value pairs (each |
| 183 | + individual name+value is a `ParamValue`), such as arbitrary image metadata. |
| 184 | +- `ROI` describes a region of interest: the x, y, z, and channel range of |
| 185 | + pixels on which to perform some operation. |
| 186 | + |
| 187 | + Additionally, there are many small utility classes that are not about images |
| 188 | + per se, but also are used by the public interfaces of our main image classes. |
| 189 | + A few of them that you will see commonly used are: |
| 190 | + |
| 191 | + - `string_view` is a non-owning reference to a string (of any of several |
| 192 | + types, or partial substrings thereof), akin to `std::string_view` in C++17. |
| 193 | + - `span` is a non-owning reference to a contiguous sequence of objects, akin |
| 194 | + to `std::span` in C++20. |
| 195 | +- `ustring` is a string class where the characters are stored in an internal |
| 196 | + shared pool of strings, so all ustrings that have the same character |
| 197 | + sequence point to the same character memory. In addition to saving memory, |
| 198 | + this makes string assignment and equality comparison as inexpensive as |
| 199 | + integer operations. |
0 commit comments