You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: Enhance performance tuning documentation for odoo-data-flow
This commit updates the `performance_tuning.md` guide to reflect new features and provide clearer guidance for optimizing data imports.
Key changes include:
- Corrected the `params` key for `--groupby` from `split` to `groupby` for consistency and clarity.
- Added comprehensive details on the `--size` parameter, including its CLI option, `params` key, default value, and its role in preventing server timeouts and managing transactions.
- Refined recommendations for mapper performance, advising on exporting Odoo data for in-memory mapping instead of live `mapper.relation` calls.
- Significantly expanded the section on "Importing Related or Computed Fields," clarifying the "cascading updates" problem and providing a detailed solution using the `--ignore` option, along with updated code examples.
These updates ensure the documentation accurately reflects the tool's capabilities and helps users achieve optimal import performance and reliability.
Copy file name to clipboardExpand all lines: docs/guides/performance_tuning.md
+13-9Lines changed: 13 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -48,7 +48,7 @@ This will add the `--worker=4` flag to the command in your generated `load.sh` s
48
48
The `--groupby` option is a powerful feature designed to solve the "race condition" problem that occurs during high-performance, multi-worker imports.
49
49
50
50
-**CLI Option**: `--groupby`
51
-
-**`params` Key**: `'split'` (Note: the internal key is`split`)
51
+
-**`params` Key**: `'groupby'` (Note: use `groupby`, not`split`)
52
52
-**Default**: `None`
53
53
54
54
### The Problem: A Race Condition
@@ -116,7 +116,7 @@ import_params = {
116
116
'model': 'res.partner',
117
117
'worker': 4,
118
118
# This is the crucial part
119
-
'split': 'parent_id/id', # The internal key is 'split'
119
+
'groupby': 'parent_id/id', # The internal key is 'groupby'
120
120
}
121
121
```
122
122
@@ -128,6 +128,10 @@ The `--size` option is one of the most critical parameters for controlling the p
128
128
129
129
To understand why this is so important, think of it like going through a checkout at a grocery store.
130
130
131
+
-**CLI Option**: `--size`
132
+
-**`params` Key**: `'size'`
133
+
-**Default**: `1000` (or the default set in the application's configuration)
134
+
131
135
### The Default Odoo Behavior: One Big Basket
132
136
133
137
When you use Odoo's standard import wizard, it's like putting all of your items (every single row in your file) into **one giant shopping basket**. This "all-or-nothing" approach has two major problems:
@@ -215,7 +219,7 @@ The choice of mappers can impact performance.
215
219
216
220
-**Slow Mappers**: The `mapper.relation` function should be used with caution. For **every single row**, it performs a live search request to the Odoo database, which can be very slow for large datasets.
217
221
218
-
**Recommendation**: If you need to map values based on data in Odoo, it is much more performant to first export the mapping from Odoo into a Python dictionary and then use the much faster `mapper.map_val` to do the translation in memory.
222
+
**Recommendation**: If you need to map values based on data in Odoo, it is much more performant to first export the necessary mapping data from Odoo (e.g., using `odoo-data-flow export`) into a Python dictionary or a separate CSV file, and then use the much faster `mapper.map_val`or other in-memory lookups to do the translation.
219
223
220
224
---
221
225
@@ -225,14 +229,14 @@ A common but very slow practice is to import values into related or computed fie
225
229
226
230
### The Problem: Cascading Updates
227
231
228
-
Consider an example where you are importing a list of contacts and setting their `parent_id` (parent company).
232
+
Consider an example where you are importing a list of contacts and setting their `parent_id` (parent company). If Odoo needs to update the `child_ids` field on the parent for every child, this becomes inefficient.
229
233
230
234
```python
231
235
# SLOW - DO NOT DO THIS
232
236
my_mapping = {
233
237
'id': mapper.m2o_map('child_', 'Ref'),
234
238
'name': mapper.val('Name'),
235
-
# This next line causes the performance issue
239
+
# This next line, if imported directly, causes the performance issue
'parent_id/id': mapper.m2o_map('parent_', 'ParentRef'),# Define the mapping
257
261
}
258
262
259
-
# The params tell the client to IGNORE the parent_id/id field
263
+
# The params tell the client to IGNORE the parent_id/id field during import
260
264
import_params = {
261
265
'model': 'res.partner',
262
-
'ignore': 'parent_id/id', # The field to ignore
266
+
'ignore': 'parent_id/id', # The field to ignore for direct import
263
267
}
264
268
265
269
processor.process(
@@ -271,4 +275,4 @@ processor.process(
271
275
272
276
This will generate a `load.sh` script with the `--ignore=parent_id/id` flag. The import client will then skip this column, avoiding the cascading updates entirely. Odoo's internal logic will still correctly establish the relationship based on the other direction of the field, but far more efficiently.
273
277
274
-
**Recommendation**: For performance, **always**use `--ignore` for related fields that have an inverse relation (like `parent_id` and `child_ids`). Only import the "forward" direction of the relationship.
278
+
**Recommendation**: For performance, **always**consider using `--ignore` for related fields that have an inverse relation (like `parent_id` and `child_ids`). Only import the "forward" direction of the relationship where necessary, letting Odoo manage the inverse.
0 commit comments