Merge pull request #363 from reef-technologies/list_files_with_no_responses_but_follow_up

Additional tests for listing files / versions – edge case scenario

Author: ppolewicz

CHANGELOG.md

@@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Infrastructure
+* Additional tests for listing files/versions
+
 ## [1.18.0] - 2022-09-20
 
 ### Added

doc/source/advanced.rst

@@ -45,7 +45,7 @@ Streaming interface
 
 Some object creation methods start writing data before reading the whole input (iterator). This can be used to write objects that do not have fully known contents without writing them first locally, so that they could be copied. Such usage pattern can be relevant to small devices which stream data to B2 from an external NAS, where caching large files such as media files or virtual machine images is not an option.
 
-Please see :ref:`advanced method support table <advanced_methods_support_table>` to see where streaming interface is supported. 
+Please see :ref:`advanced method support table <advanced_methods_support_table>` to see where streaming interface is supported.
 
 Continuation
 ============
@@ -184,7 +184,7 @@ Change the middle of the remote file
 For more information see :meth:`b2sdk.v2.Bucket.create_file`.
 
 
-Synthetize a file from local and remote parts
+Synthesize a file from local and remote parts
 =============================================
 
 This is useful for expert usage patterns such as:

test/unit/bucket/test_bucket.py

@@ -2173,3 +2173,72 @@ def test_file_info_3(self):
     def test_file_info_4(self):
         download_version = self.bucket.get_file_info_by_name('test.txt%253Ffoo%253Dbar')
         assert download_version.file_name == 'test.txt%253Ffoo%253Dbar'
+
+
+# Listing where every other response returns no entries and pointer to the next file
+
+
+class EmptyListBucketSimulator(BucketSimulator):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # Whenever we receive a list request, if it's the first time
+        # for this particular ``start_file_name``, we'll return
+        # an empty response pointing to the same file.
+        self.last_queried_file = None
+
+    def _should_return_empty(self, file_name: str) -> bool:
+        # Note that every other request is empty – the logic is as follows:
+        #   1st request – unknown start name – empty response
+        #   2nd request – known start name – normal response with a proper next filename
+        #   3rd request – unknown start name (as it's the next filename from the previous request) – empty response
+        #   4th request – known start name
+        # etc. This works especially well when using limiter of number of files fetched set to 1.
+        should_return_empty = self.last_queried_file != file_name
+        self.last_queried_file = file_name
+        return should_return_empty
+
+    def list_file_versions(
+        self,
+        account_auth_token,
+        start_file_name=None,
+        start_file_id=None,
+        max_file_count=None,  # noqa
+        prefix=None,
+    ):
+        if self._should_return_empty(start_file_name):
+            return dict(files=[], nextFileName=start_file_name, nextFileId=start_file_id)
+        return super().list_file_versions(
+            account_auth_token,
+            start_file_name,
+            start_file_id,
+            1,  # Forcing only a single file per response.
+            prefix,
+        )
+
+    def list_file_names(
+        self,
+        account_auth_token,
+        start_file_name=None,
+        max_file_count=None,  # noqa
+        prefix=None,
+    ):
+        if self._should_return_empty(start_file_name):
+            return dict(files=[], nextFileName=start_file_name)
+        return super().list_file_names(
+            account_auth_token,
+            start_file_name,
+            1,  # Forcing only a single file per response.
+            prefix,
+        )
+
+
+class EmptyListSimulator(RawSimulator):
+    BUCKET_SIMULATOR_CLASS = EmptyListBucketSimulator
+
+
+class TestEmptyListVersions(TestListVersions):
+    RAW_SIMULATOR_CLASS = EmptyListSimulator
+
+
+class TestEmptyLs(TestLs):
+    RAW_SIMULATOR_CLASS = EmptyListSimulator