-
Notifications
You must be signed in to change notification settings - Fork 685
Commit cef2094
authored
## Changes
* Add dim order to the list of tensor metadata that can be ingested by compute shaders
* Do not persistently store derivative metadata (i.e. padded sizes, padded numel, unsqueezed strides, etc.) as members of vTensor; instead store these in `uniform_data_` and use `uniform_data_` as the source of truth
## Motivation
> Add dim order to the list of tensor metadata that can be ingested by compute shaders
Knowing the dim order is necessary to convert between a linear buffer index to N-dimensional tensor index using a tensor's strides. Technically, the dim order can be inferred from the strides by performing an index sort on the strides array; however to prevent compute shaders from having to do this operation frequently, it is more efficient to pass in the dim order directly to the compute shader.
Currently, ET-VK compute shaders make strong assumptions about the dim order of buffer backed tensors so as to avoid having to dynamically generate the dim order from the strides array. However, these assumptions are not enforced and it is more correct to just account for the dim order rather than make assumptions. This will be addressed in the next diff.
> Do not persistently store derivative metadata (i.e. padded sizes, padded numel, unsqueezed strides, etc.) as members of vTensor; instead store these in `uniform_data_` and use `uniform_data_` as the source of truth
I realized that the purpose of these "derived metadata" is to simply convert default tensor metadata such sizes, strides, etc. to a form where they can be used in a compute shader. There is no need to store these derived metadata persistently, since they are pretty much only useful in the final `ivec4` form they exist as inside `UniformData`. So to simplify `vTensor` and to reduce the size of the class, I elected to remove these superfluous data members.
## Performance Impact
* Potential memory footprint improvement from reducing the size of `vTensor`.
Differential Revision: [D76393427](https://our.internmc.facebook.com/intern/diff/D76393427/)
1 parent 8895573 commit cef2094Copy full SHA for cef2094
File tree
Expand file treeCollapse file tree
6 files changed
+397
-212
lines changedFilter options
- backends/vulkan
- runtime
- api/containers
- graph
- vk_api
- test
Expand file treeCollapse file tree
6 files changed
+397
-212
lines changed
0 commit comments