Skip to content

Commit 9e0dbc5

Browse files
committed
feat: Enhance performance tuning documentation for odoo-data-flow
This commit updates the `performance_tuning.md` guide to reflect new features and provide clearer guidance for optimizing data imports. Key changes include: - Corrected the `params` key for `--groupby` from `split` to `groupby` for consistency and clarity. - Added comprehensive details on the `--size` parameter, including its CLI option, `params` key, default value, and its role in preventing server timeouts and managing transactions. - Refined recommendations for mapper performance, advising on exporting Odoo data for in-memory mapping instead of live `mapper.relation` calls. - Significantly expanded the section on "Importing Related or Computed Fields," clarifying the "cascading updates" problem and providing a detailed solution using the `--ignore` option, along with updated code examples. These updates ensure the documentation accurately reflects the tool's capabilities and helps users achieve optimal import performance and reliability.
1 parent ff93021 commit 9e0dbc5

File tree

1 file changed

+13
-9
lines changed

1 file changed

+13
-9
lines changed

docs/guides/performance_tuning.md

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ This will add the `--worker=4` flag to the command in your generated `load.sh` s
4848
The `--groupby` option is a powerful feature designed to solve the "race condition" problem that occurs during high-performance, multi-worker imports.
4949

5050
- **CLI Option**: `--groupby`
51-
- **`params` Key**: `'split'` (Note: the internal key is `split`)
51+
- **`params` Key**: `'groupby'` (Note: use `groupby`, not `split`)
5252
- **Default**: `None`
5353

5454
### The Problem: A Race Condition
@@ -116,7 +116,7 @@ import_params = {
116116
'model': 'res.partner',
117117
'worker': 4,
118118
# This is the crucial part
119-
'split': 'parent_id/id', # The internal key is 'split'
119+
'groupby': 'parent_id/id', # The internal key is 'groupby'
120120
}
121121
```
122122

@@ -128,6 +128,10 @@ The `--size` option is one of the most critical parameters for controlling the p
128128

129129
To understand why this is so important, think of it like going through a checkout at a grocery store.
130130

131+
- **CLI Option**: `--size`
132+
- **`params` Key**: `'size'`
133+
- **Default**: `1000` (or the default set in the application's configuration)
134+
131135
### The Default Odoo Behavior: One Big Basket
132136

133137
When you use Odoo's standard import wizard, it's like putting all of your items (every single row in your file) into **one giant shopping basket**. This "all-or-nothing" approach has two major problems:
@@ -215,7 +219,7 @@ The choice of mappers can impact performance.
215219

216220
- **Slow Mappers**: The `mapper.relation` function should be used with caution. For **every single row**, it performs a live search request to the Odoo database, which can be very slow for large datasets.
217221

218-
**Recommendation**: If you need to map values based on data in Odoo, it is much more performant to first export the mapping from Odoo into a Python dictionary and then use the much faster `mapper.map_val` to do the translation in memory.
222+
**Recommendation**: If you need to map values based on data in Odoo, it is much more performant to first export the necessary mapping data from Odoo (e.g., using `odoo-data-flow export`) into a Python dictionary or a separate CSV file, and then use the much faster `mapper.map_val` or other in-memory lookups to do the translation.
219223

220224
---
221225

@@ -225,14 +229,14 @@ A common but very slow practice is to import values into related or computed fie
225229

226230
### The Problem: Cascading Updates
227231

228-
Consider an example where you are importing a list of contacts and setting their `parent_id` (parent company).
232+
Consider an example where you are importing a list of contacts and setting their `parent_id` (parent company). If Odoo needs to update the `child_ids` field on the parent for every child, this becomes inefficient.
229233

230234
```python
231235
# SLOW - DO NOT DO THIS
232236
my_mapping = {
233237
'id': mapper.m2o_map('child_', 'Ref'),
234238
'name': mapper.val('Name'),
235-
# This next line causes the performance issue
239+
# This next line, if imported directly, causes the performance issue
236240
'parent_id/id': mapper.m2o_map('parent_', 'ParentRef'),
237241
}
238242
```
@@ -253,13 +257,13 @@ The correct way to handle this is to prevent the import client from writing to t
253257
my_mapping = {
254258
'id': mapper.m2o_map('child_', 'Ref'),
255259
'name': mapper.val('Name'),
256-
'parent_id/id': mapper.m2o_map('parent_', 'ParentRef'),
260+
'parent_id/id': mapper.m2o_map('parent_', 'ParentRef'), # Define the mapping
257261
}
258262

259-
# The params tell the client to IGNORE the parent_id/id field
263+
# The params tell the client to IGNORE the parent_id/id field during import
260264
import_params = {
261265
'model': 'res.partner',
262-
'ignore': 'parent_id/id', # The field to ignore
266+
'ignore': 'parent_id/id', # The field to ignore for direct import
263267
}
264268

265269
processor.process(
@@ -271,4 +275,4 @@ processor.process(
271275

272276
This will generate a `load.sh` script with the `--ignore=parent_id/id` flag. The import client will then skip this column, avoiding the cascading updates entirely. Odoo's internal logic will still correctly establish the relationship based on the other direction of the field, but far more efficiently.
273277

274-
**Recommendation**: For performance, **always** use `--ignore` for related fields that have an inverse relation (like `parent_id` and `child_ids`). Only import the "forward" direction of the relationship.
278+
**Recommendation**: For performance, **always** consider using `--ignore` for related fields that have an inverse relation (like `parent_id` and `child_ids`). Only import the "forward" direction of the relationship where necessary, letting Odoo manage the inverse.

0 commit comments

Comments
 (0)