forked from tfrancoi/odoo_csv_import
-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy path.idea-export-fix
More file actions
201 lines (187 loc) · 43.5 KB
/
.idea-export-fix
File metadata and controls
201 lines (187 loc) · 43.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
Improve export strategy / errors
I am doing an export on a real remote cloud database it fails with a not so user friendly error.
I need to understand why it fails. The error message needs to be improved as well.
Is there an automatic recovery / fallback if such error occurs?
Here is the console output:
sh res_partner_export-test.sh
[19:20:32] INFO Starting export for model 'res.partner'... exporter.py:46
INFO Connecting to Odoo server at odoo-test.sps-international.com... conf_lib.py:51
[19:20:33] INFO Hybrid export mode activated. Using 'read' with XML ID enrichment. export_threaded.py:519
INFO Searching for records to export in model 'res.partner' with domain: export_threaded.py:565
[('type', '!=', 'private')]
[19:20:34] INFO Found 37000 records to export. Splitting into batches of 4000. export_threaded.py:575
[21:21:04] ERROR Export for batch 0 failed permanently: Response ended prematurely export_threaded.py:232
╭────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────────────╮
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/models.py:820 in generate │
│ │
│ 817 │ │ │ # Special case for urllib3. │
│ 818 │ │ │ if hasattr(self.raw, "stream"): │
│ 819 │ │ │ │ try: │
│ ❱ 820 │ │ │ │ │ yield from self.raw.stream(chunk_size, decode_content=True) │
│ 821 │ │ │ │ except ProtocolError as e: │
│ 822 │ │ │ │ │ raise ChunkedEncodingError(e) │
│ 823 │ │ │ │ except DecodeError as e: │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/urllib3/response.py:1088 in stream │
│ │
│ 1085 │ │ │ 'content-encoding' header. │
│ 1086 │ │ """ │
│ 1087 │ │ if self.chunked and self.supports_chunked_reads(): │
│ ❱ 1088 │ │ │ yield from self.read_chunked(amt, decode_content=decode_content) │
│ 1089 │ │ else: │
│ 1090 │ │ │ while not is_fp_closed(self._fp) or len(self._decoded_buffer) > 0: │
│ 1091 │ │ │ │ data = self.read(amt=amt, decode_content=decode_content) │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/urllib3/response.py:1248 in read_chunked │
│ │
│ 1245 │ │ │ │ amt = None │
│ 1246 │ │ │ │
│ 1247 │ │ │ while True: │
│ ❱ 1248 │ │ │ │ self._update_chunk_length() │
│ 1249 │ │ │ │ if self.chunk_left == 0: │
│ 1250 │ │ │ │ │ break │
│ 1251 │ │ │ │ chunk = self._handle_chunk(amt) │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/urllib3/response.py:1178 in _update_chunk_length │
│ │
│ 1175 │ │ │ │ raise InvalidChunkLength(self, line) from None │
│ 1176 │ │ │ else: │
│ 1177 │ │ │ │ # Truncated at start of next chunk │
│ ❱ 1178 │ │ │ │ raise ProtocolError("Response ended prematurely") from None │
│ 1179 │ │
│ 1180 │ def _handle_chunk(self, amt: int | None) -> bytes: │
│ 1181 │ │ returned_chunk = None │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ProtocolError: Response ended prematurely
During handling of the above exception, another exception occurred:
╭────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────────────╮
│ /home/bosd/git/odoo-data-flow/src/odoo_data_flow/export_threaded.py:193 in _execute_batch │
│ │
│ 190 │ │ │ │
│ 191 │ │ │ raw_data = cast( │
│ 192 │ │ │ │ list[dict[str, Any]], │
│ ❱ 193 │ │ │ │ self.model.read(ids_to_export, list(read_fields)), │
│ 194 │ │ │ ) │
│ 195 │ │ │ if not raw_data: │
│ 196 │ │ │ │ return [] │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/odoolib/main.py:311 in proxy │
│ │
│ 308 │ │ │ """ │
│ 309 │ │ │ self.connection.check_login(False) │
│ 310 │ │ │ self.__logger.debug(args) │
│ ❱ 311 │ │ │ result = self.connection.get_service('object').execute_kw( │
│ 312 │ │ │ │ │ │ │ │ │ │ │ │ │ self.connection.database, │
│ 313 │ │ │ │ │ │ │ │ │ │ │ │ │ self.connection.user_id, │
│ 314 │ │ │ │ │ │ │ │ │ │ │ │ │ self.connection.password, │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/odoolib/main.py:184 in proxy │
│ │
│ 181 │ │ │ :param args: A list of values for the method │
│ 182 │ │ │ """ │
│ 183 │ │ │ self.__logger.debug('args: %r', args) │
│ ❱ 184 │ │ │ result = self.connector.send(self.service_name, method, *args) │
│ 185 │ │ │ self.__logger.debug('result: %r', result) │
│ 186 │ │ │ return result │
│ 187 │ │ return proxy │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/odoolib/main.py:159 in send │
│ │
│ 156 │ │ self.url = 'https://%s:%d/jsonrpc' % (hostname, port) │
│ 157 │ │
│ 158 │ def send(self, service_name, method, *args): │
│ ❱ 159 │ │ return json_rpc(self.url, "call", {"service": service_name, "method": method, │
│ "args": args}) │
│ 160 │
│ 161 class Service(object): │
│ 162 │ """ │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/odoolib/main.py:115 in json_rpc │
│ │
│ 112 │ │ "params": params, │
│ 113 │ │ "id": random.randint(0, 1000000000), │
│ 114 │ } │
│ ❱ 115 │ result_req = requests.post(url, data=json.dumps(data), headers={ │
│ 116 │ │ "Content-Type":"application/json", │
│ 117 │ }) │
│ 118 │ result = result_req.json() │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/api.py:115 in post │
│ │
│ 112 │ :rtype: requests.Response │
│ 113 │ """ │
│ 114 │ │
│ ❱ 115 │ return request("post", url, data=data, json=json, **kwargs) │
│ 116 │
│ 117 │
│ 118 def put(url, data=None, **kwargs): │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/api.py:59 in request │
│ │
│ 56 │ # avoid leaving sockets open which can trigger a ResourceWarning in some │
│ 57 │ # cases, and look like a memory leak in others. │
│ 58 │ with sessions.Session() as session: │
│ ❱ 59 │ │ return session.request(method=method, url=url, **kwargs) │
│ 60 │
│ 61 │
│ 62 def get(url, params=None, **kwargs): │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/sessions.py:589 in request │
│ │
│ 586 │ │ │ "allow_redirects": allow_redirects, │
│ 587 │ │ } │
│ 588 │ │ send_kwargs.update(settings) │
│ ❱ 589 │ │ resp = self.send(prep, **send_kwargs) │
│ 590 │ │ │
│ 591 │ │ return resp │
│ 592 │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/sessions.py:746 in send │
│ │
│ 743 │ │ │ │ pass │
│ 744 │ │ │
│ 745 │ │ if not stream: │
│ ❱ 746 │ │ │ r.content │
│ 747 │ │ │
│ 748 │ │ return r │
│ 749 │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/models.py:902 in content │
│ │
│ 899 │ │ │ if self.status_code == 0 or self.raw is None: │
│ 900 │ │ │ │ self._content = None │
│ 901 │ │ │ else: │
│ ❱ 902 │ │ │ │ self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b"" │
│ 903 │ │ │
│ 904 │ │ self._content_consumed = True │
│ 905 │ │ # don't need to release the connection; that's been handled by urllib3 │
│ │
│ /home/bosd/doodba/sps_12_18/.venv/lib64/python3.13/site-packages/requests/models.py:822 in generate │
│ │
│ 819 │ │ │ │ try: │
│ 820 │ │ │ │ │ yield from self.raw.stream(chunk_size, decode_content=True) │
│ 821 │ │ │ │ except ProtocolError as e: │
│ ❱ 822 │ │ │ │ │ raise ChunkedEncodingError(e) │
│ 823 │ │ │ │ except DecodeError as e: │
│ 824 │ │ │ │ │ raise ContentDecodingError(e) │
│ 825 │ │ │ │ except ReadTimeoutError as e: │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ChunkedEncodingError: Response ended prematurely
Finally there is a:
ChunkedEncodingError: Response ended prematurely
⠙ Exporting res.partner... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78% • 29000 of 37000 records • 0:00:40
[22:23:53] INFO Writing 29000 records to origin/res_partner_testing.csv... export_threaded.py:462
[22:23:54] INFO Export complete. export_threaded.py:464
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Export Complete ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Successfully exported 29000 records to origin/res_partner_testing.csv │
╰─────────────────────────────────────────────────────────────────────────────────────
Which is confusing.
The export ended pre-maturily.
But the panel says Export complete!
tarting Pass 2: Updating parent records...
Connecting to Odoo database 'sps-group-sps-cleaning' on sps-cleaning.sps-group.opa.as:443...
Connection successful.
Found 38316 records in /home/bosd/doodba/sps_12_18/data/res_partner_transformed.csv.
Processing record 26931/38316 | Updated: 18324 | Skipped: 8606 | Errors: 0
An unexpected error occurred during file processing: module 'odoorpc.error' has no attribute 'ConnectorError'
When long time no response from server, abort and scale down batches.