|
1 | 1 | # Analyzer |
2 | 2 |
|
3 | | -The Analyzer is a class library that can be used to analyze the content of Unity data files such |
4 | | -as AssetBundles and SerializedFiles. It iterates through all the serialized objects and uses the |
5 | | -TypeTree to extract information about these objects (e.g. name, size, etc.) |
6 | | - |
7 | | -The most common use of this library is through the [analyze](../UnityDataTool/README.md#analyzeanalyse) |
8 | | -command of the UnityDataTool. This uses the Analyze library to generate a SQLite database. |
9 | | - |
10 | | -Once generated, a tool such as the [DB Browser for SQLite](https://sqlitebrowser.org/), or the command line `sqlite3` tool, can be used to look at the content of the database. |
11 | | - |
12 | | -# Example usage |
13 | | - |
14 | | -See [this topic](../Documentation/analyze-examples.md) for examples of how to use the SQLite output of the UnityDataTool Analyze command. |
15 | | - |
16 | | -# DataBase Reference |
17 | | - |
18 | | -The database provides different views. The views join multiple tables together and often it is not necessary to write your own SQL queries to find the information you want, especially when you are using a visual SQLite tool. |
19 | | - |
20 | | -This section gives an overview of the main views. |
21 | | - |
22 | | -## object_view |
23 | | - |
24 | | -This is the main view where the information about all the objects in the AssetBundles is available. |
25 | | -Its columns are: |
26 | | -* id: a unique id without any meaning outside of the database |
27 | | -* object_id: the Unity object id (unique inside its SerializedFile but not necessarily acros all |
28 | | - AssetBundles) |
29 | | -* asset_bundle: the name of the AssetBundle containing the object (will be null if the source file |
30 | | - was a SerializedFile and not an AssetBundle) |
31 | | -* serialized_file: the name of the SerializedFile containing the object |
32 | | -* type: the type of the object |
33 | | -* name: the name of the object, if it had one |
34 | | -* game_object: the id of the GameObject containing this object, if any (mostly for Components) |
35 | | -* size: the size of the object in bytes (e.g. 3343772) |
36 | | -* pretty_size: the size in an easier to read format (e.g. 3.2 MB) |
37 | | - |
38 | | -## view_breakdown_by_type |
39 | | - |
40 | | -This view lists the total number and size of the objects, aggregated by type. |
41 | | - |
42 | | -## view_potential_duplicates |
43 | | - |
44 | | -This view lists the objects that are possibly included more than once in the AssetBundles. This can |
45 | | -happen when an asset is referenced from multiple AssetBundles but is not assigned to one. In this |
46 | | -case, Unity will include the asset in all the AssetBundles with a reference to it. The |
47 | | -view_potential_duplicates provides the number of instances and the total size of the potentially |
48 | | -duplicated assets. It also lists all the AssetBundles where the asset was found. |
49 | | - |
50 | | -If the skipReferences option is used, there will be a lot of false positives in that view. Otherwise, |
51 | | -it should be very accurate because CRCs are used to determine if objects are identical. |
52 | | - |
53 | | -## asset_view (AssetBundleProcessor) |
54 | | - |
55 | | -This view lists all the assets that have been explicitly assigned to AssetBundles. The dependencies |
56 | | -that were automatically added by Unity at build time won't appear in this view. The columns are the |
57 | | -same as those in the *object_view* with the addition of the *asset_name* that contains the filename |
58 | | -of the asset. |
59 | | - |
60 | | -## asset_dependencies_view (AssetBundleProcessor) |
61 | | - |
62 | | -This view lists the dependencies of all the assets. You can filter by id or asset_name to get all |
63 | | -the dependencies of an asset. Conversely, filtering by dep_id will return all the assets that |
64 | | -depend on this object. This can be useful to figure out why an asset was included in a build. |
65 | | - |
66 | | -## animation_view (AnimationClipProcessor) |
67 | | - |
68 | | -This provides additional information about AnimationClips. The columns are the same as those in |
69 | | -the *object_view*, with the addition of: |
70 | | -* legacy: 1 if it's a legacy animation, 0 otherwise |
71 | | -* events: the number of events |
72 | | - |
73 | | -## audio_clip_view (AudioClipProcessor) |
74 | | - |
75 | | -This provides additional information about AudioClips. The columns are the same as those in |
76 | | -the *object_view*, with the addition of: |
77 | | -* bits_per_sample: number of bits per sample |
78 | | -* frequency: sampling frequency |
79 | | -* channels: number of channels |
80 | | -* load_type: either *Compressed in Memory*, *Decompress on Load* or *Streaming* |
81 | | -* format: compression format |
82 | | - |
83 | | -## mesh_view (MeshProcessor) |
84 | | - |
85 | | -This provides additional information about Meshes. The columns are the same as those in |
86 | | -the *object_view*, with the addition of: |
87 | | -* sub_meshes: the number of sub-meshes |
88 | | -* blend_shapes: the number of blend shapes |
89 | | -* bones: the number of bones |
90 | | -* indices: the number of vertex indices |
91 | | -* vertices: the number of vertices |
92 | | -* compression: 1 if compressed, 0 otherwise |
93 | | -* rw_enabled: 1 if the mesh has the *R/W Enabled* option, 0 otherwise |
94 | | -* vertex_size: number of bytes used by each vertex |
95 | | -* channels: name and type of the vertex channels |
96 | | - |
97 | | -## texture_view (Texture2DProcessor) |
98 | | - |
99 | | -This provides additional information about Texture2Ds. The columns are the same as those in |
100 | | -the *object_view*, with the addition of: |
101 | | -* width/height: texture resolution |
102 | | -* format: compression format |
103 | | -* mip_count: number of mipmaps |
104 | | -* rw_enabled: 1 if the mesh has the *R/W Enabled* option, 0 otherwise |
105 | | - |
106 | | -## shader_view (ShaderProcessor) |
107 | | - |
108 | | -This provides additional information about Shaders. The columns are the same as those in |
109 | | -the *object_view*, with the addition of: |
110 | | -* decompressed_size: the approximate size in bytes that this shader will need at runtime when |
111 | | - loaded |
112 | | -* sub_shaders: the number of sub-shaders |
113 | | -* sub_programs: the number of sub-programs (usually one per shader variant, stage and pass) |
114 | | -* unique_programs: the number of unique program (variants with identical programs will share the |
115 | | - same program in memory) |
116 | | -* keywords: list of all the keywords affecting the shader |
117 | | - |
118 | | -## shader_subprogram_view (ShaderProcessor) |
119 | | - |
120 | | -This view lists all the shader sub-programs and has the same columns as the *shader_view* with the |
121 | | -addition of: |
122 | | -* api: the API of the shader (e.g. DX11, Metal, GLES, etc.) |
123 | | -* pass: the pass number of the sub-program |
124 | | -* pass_name: the pass name, if available |
125 | | -* hw_tier: the hardware tier of the sub-program (as defined in the Graphics settings) |
126 | | -* shader_type: the type of shader (e.g. vertex, fragment, etc.) |
127 | | -* sub_program: the subprogram index for this pass and shader type |
128 | | -* keywords: the shader keywords specific to this sub-program |
129 | | - |
130 | | -## shader_keyword_ratios |
131 | | - |
132 | | -This view can help to determine which shader keywords are causing a large number of variants. To |
133 | | -understand how it works, let's define a "program" as a unique combination of shader, subshader, |
134 | | -hardware tier, pass number, API (DX, Metal, etc.), and shader type (vertex, fragment, etc). |
135 | | - |
136 | | -Each row of the view corresponds to a combination of one program and one of its keywords. The |
137 | | -columns are: |
138 | | - |
139 | | -* shader_id: the shader id |
140 | | -* name: the shader name |
141 | | -* sub_shader: the sub-shader number |
142 | | -* hw_tier: the hardware tier of the sub-program (as defined in the Graphics settings) |
143 | | -* pass: the pass number of the sub-program |
144 | | -* api: the API of the shader (e.g. DX11, Metal, GLES, etc.) |
145 | | -* pass_name: the pass name, if available |
146 | | -* shader_type: the type of shader (e.g. vertex, fragment, etc.) |
147 | | -* total_variants: total number of variants for this program. |
148 | | -* keyword: one of the program's keywords |
149 | | -* variants: number of variants including this keyword. |
150 | | -* ratio: variants/total_variants |
151 | | - |
152 | | -The ratio can be used to determine how a keyword affects the number of variants. When it is equal |
153 | | -to 0.5, it means that it is in half of the variants. Basically, that means that it is not stripped |
154 | | -at all because each of the program's variants has a version with and without that keyword. |
155 | | -Therefore, keywords with a ratio close to 0.5 are good targets for stripping. When the ratio is |
156 | | -close to 0 or 1, it means that the keyword is in almost none or almost all of the variants and |
157 | | -stripping it won't make a big difference. |
158 | | - |
159 | | -## view_breakdowns_shaders (ShaderProcessor) |
160 | | - |
161 | | -This view lists all the shaders aggregated by name. The *instances* column indicates how many time |
162 | | -the shader was found in the data files. It also provides the total size per shader and the list of |
163 | | -AssetBundles in which they were found. |
164 | | - |
165 | | -# Advanced |
166 | | - |
167 | | -## Using the library |
168 | | - |
169 | | -The [AnalyzerTool](./AnalyzerTool.cs) class is the API entry point. The main method is called |
170 | | -Analyze. It is currently hard coded to write using the [SQLiteWriter](./SQLite/SQLiteWriter.cs), |
171 | | -but this approach could be extended to add support for other outputs. |
172 | | - |
173 | | -Calling this method will recursively process the files matching the search pattern in the provided |
174 | | -path. It will add a row in the 'objects' table for each serialized object. This table contain basic |
175 | | -information such as the size and the name of the object (if it has one). |
176 | | - |
177 | | -## Extending the Library |
178 | | - |
179 | | -The extracted information is forwarded to an object implementing the [IWriter](./IWriter.cs) |
180 | | -interface. The library provides the [SQLiteWriter](./SQLite/SQLiteWriter.cs) implementation that |
181 | | -writes the data into a SQLite database. |
182 | | - |
183 | | -The core properties that apply to all Unity Objects are extracted into the `objects` table. |
184 | | -However much of the most useful Analyze functionality comes by virtue of the type-specific information that is extracted for |
185 | | -important types like Meshes, Shaders, Texture2D and AnimationClips. For example, when a Mesh object is encountered in a Serialized |
186 | | -File, then rows are added to both the `objects` table and the `meshes` table. The meshes table contains columns that only apply to Mesh objects, for example the number of vertices, indices, bones, and channels. The `mesh_view` is a view that joins the `objects` table with the `meshes` table, so that you can see all the properties of a Mesh object in one place. |
187 | | - |
188 | | -Each supported Unity object type follows the same pattern: |
189 | | -* A Handler class in the SQLite/Handlers, e.g. [MeshHandler.cs](./SQLite/Handler/MeshHandler.cs). |
190 | | -* The registration of the handler in the m_Handlers dictionary in [SerializedFileSQLiteWriter.cs](./SQLite/Writers/SerializedFileSQLiteWriter.cs). |
191 | | -* SQL statements defining extra tables and views associated with the type, e.g. [Mesh.sql](./SQLite/Resources/Mesh.sql). |
192 | | -* A Reader class that uses RandomAccessReader to read properties from the serialized object, e.g. [Mesh.cs](./SerializedObjects/Mesh.cs). |
193 | | - |
194 | | -It would be possible to extend the Analyze library to add additional columns for the existing types, or by following the same pattern to add additional types. The [dump](../UnityDataTool/README.md#dump) feature of UnityDataTool is a useful way to see the property names and other details of the serialization for a type. Based on that information, code in the Reader class can use the RandomAccessReader to retrieve those properties to bring them into the SQLite database. |
195 | | - |
196 | | -## Supporting Other File Formats |
197 | | - |
198 | | -Another direction of possible extension is to support analyzing additional file formats, beyond Unity SerializedFiles. |
199 | | - |
200 | | -This the approach taken to analyze Addressables Build Layout files, which are JSON files using the format defined in [BuildLayout.cs](./SQLite/Parsers/Models/BuildLayout.cs). |
201 | | - |
202 | | -Support for another file format could be added by deriving an additional class from SQLiteWriter and implementing a class derived from ISQLiteFileParser. Then follow the existing code structure convention to add new Commands (derived from AbstractCommand) and Resource .sql files to establish additional tables in the database. |
203 | | - |
204 | | -An example of another file format that could be useful to support, as the tool evolves, are the yaml [.manifest files](https://docs.unity3d.com/Manual/assetbundles-file-format.html), generated by BuildPipeline.BuildAssetBundles(). |
| 3 | +See [Documentation/analyzer.md](../Documentation/analyzer.md) |
0 commit comments