Error uploading some images #42
Description
Hello, I'm having this problem while uploading some images. It happens like once every 2.5k images in my crude estimations.
Setting the loglevel flag to DEBUG gets me this
expand log output
21:37:06 INFO User: omitted@gmail.com client.py:64
INFO Language: en_US client.py:65
DEBUG Starting new HTTPS connection (1): android.googleapis.com:443 connectionpool.py:1049
DEBUG https://android.googleapis.com:443 "POST /auth HTTP/1.1" 200 None connectionpool.py:544
DEBUG Starting new HTTPS connection (1): photosdata-pa.googleapis.com:443 connectionpool.py:1049
DEBUG https://photosdata-pa.googleapis.com:443 "POST /6136726531001561323/5124265722730810727 HTTP/1.1" 200 None connectionpool.py:544
DEBUG Could not decode a buffer for field (['1']) as a message:
Encountered error decoding field 1884:
Decoded length for field 1884 goes over end: 1401 > 21 length_delim.py:591
DEBUG Could not decode a buffer for field (['1']) as a message:
Encountered error decoding field 6:
Field 6 has mismatched wiretypes. Previous: 1 Now: 7 length_delim.py:591
DEBUG Could not decode a buffer for field (['3']) as a message:
Encountered error decoding field 6:
Got unknown wire type: 7 length_delim.py:591
Files processed: 1/1 0:00:00 Errors: 0
{
'/root/omitted.jpg': {
'8': [6302029498285109574, 3778876670251254632],
'10': 8661359182826980421,
'15': 54,
'12': [1647395700,
The file does get uploaded even with these errors. the problem comes when the toll starts to add images to albums (--album "AUTO") and reaches that image. then the whole tool crashes
expand log output
Adding items to album cosplay omitted: ⠋ 0/84 0:00:00
Traceback (most recent call last):
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 59, in encode_string
value = six.ensure_text(value)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/six.py", line 960, in ensure_text
raise TypeError("not expecting type '%s'" % type(s))
TypeError: not expecting type '<class 'dict'>'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/.local/bin/gpmc", line 8, in <module>
sys.exit(main())
^^^^^^
File "/root/.local/lib/python3.12/site-packages/gpmc/cli.py", line 53, in main
output = client.upload(
^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/gpmc/client.py", line 312, in upload
self._handle_album_creation(results, album_name, show_progress)
File "/root/.local/lib/python3.12/site-packages/gpmc/client.py", line 195, in _handle_album_creation
self.add_to_album(media_keys, album_name_from_path, show_progress=show_progress)
File "/root/.local/lib/python3.12/site-packages/gpmc/client.py", line 567, in add_to_album
current_album_key = self.api.create_album(album_name=current_album_name, media_keys=batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/gpmc/api.py", line 425, in create_album
serialized_data = encode_message(proto_body, message_types.CREATE_ALBUM) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/api.py", line 152, in encode_message
blackboxprotobuf.lib.types.length_delim.encode_message(
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 165, in encode_message
field_number, outputs = _encode_message_field(
^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 275, in _encode_message_field
six.reraise(*sys.exc_info())
File "/root/.local/lib/python3.12/site-packages/six.py", line 724, in reraise
raise value
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 269, in _encode_message_field
outputs.append(tag + field_encoder(repeated))
^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 241, in <lambda>
field_encoder = lambda data: encode_lendelim_message(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 634, in encode_lendelim_message
message_out = encode_message(
^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 165, in encode_message
field_number, outputs = _encode_message_field(
^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 275, in _encode_message_field
six.reraise(*sys.exc_info())
File "/root/.local/lib/python3.12/site-packages/six.py", line 724, in reraise
raise value
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 271, in _encode_message_field
outputs.append(tag + field_encoder(value))
^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 241, in <lambda>
field_encoder = lambda data: encode_lendelim_message(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 634, in encode_lendelim_message
message_out = encode_message(
^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 165, in encode_message
field_number, outputs = _encode_message_field(
^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 275, in _encode_message_field
six.reraise(*sys.exc_info())
File "/root/.local/lib/python3.12/site-packages/six.py", line 724, in reraise
raise value
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 271, in _encode_message_field
outputs.append(tag + field_encoder(value))
^^^^^^^^^^^^^^^^^^^^
File "/root/.local/lib/python3.12/site-packages/blackboxprotobuf/lib/types/length_delim.py", line 61, in encode_string
six.raise_from(
File "<string>", line 3, in raise_from
blackboxprotobuf.lib.exceptions.EncoderException: Encountered error encoding field 4->1->1: Error encoding string to message: {'8': 7376699859854766406, '7': 102, '6': [1497650511, 1634612579], '12': 5424100279491983169, '5': 1500475987, '11': 3633639940936643128}
And well since the tool only starts adding to albums AFTER it uploads all the files... I would have to rerun the tool again and make it rescan hashes of thousands or tens of thousands of files so it can add the them to their appropriate albums.
Not to mention it's not easy to know which image is the problem unless you set LOGLEVEL to DEBUG.