Pagination and recursive calls (#20)

* Add ApiResourceSet class

* Add 'get_recursive'

* Add pycodestyle

* Fix code style

* Rename tests

* Update tests

* Update docs

* Remove deprecated methods

* Update changelog

* Update changelog

* Update CHANGELOG.md

Author: trizz

.gitignore

@@ -9,3 +9,4 @@ dist/*
 .coverage.*
 .pytest_cache/*
 /htmlcov/
+/dev

CHANGELOG.md

@@ -5,7 +5,20 @@ All notable changes to `exonet-api-python` will be documented in this file.
 Updates should follow the [Keep a CHANGELOG](http://keepachangelog.com/) principles.
 
 ## [Unreleased]
-[Compare 2.0.0 - Unreleased](https://github.com/exonet/exonet-api-python/compare/2.0.0...master)
+[Compare 3.0.0 - Unreleased](https://github.com/exonet/exonet-api-python/compare/3.0.0...master)
+
+## [3.0.0](https://github.com/exonet/exonet-api-python/releases/tag/3.0.0) - 2020-09-11
+[Compare 2.1.0 - 3.0.0](https://github.com/exonet/exonet-api-python/compare/2.1.0...3.0.0)
+### Breaking
+- When multiple resources are returned from the API, an instance of `ApiResourceSet` is returned instead of a list. This class is traversable so unless the code does specific `list` things or type checks, no changes are necessary.
+
+### Added
+- Add the `total()` method to resource sets to get the total number of resources (and not only the number of resources in the current resource set).
+- Add `next_page`, `previous_page`, `first_page` and `last_page` methods to the `ApiResourceSet` for easy loading of paginated resource sets.
+- Add a `get_recursive` method to the `RequestBuilder` to get the resource set including recursively the resource sets from the following pages.
+
+### Removed
+- The `store` method for creating `POST` requests. (Deprecated since 2.0.0)
 
 ## [2.1.0](https://github.com/exonet/exonet-api-python/releases/tag/2.1.0) - 2019-11-19
 [Compare 2.0.0 - 2.1.0](https://github.com/exonet/exonet-api-python/compare/2.0.0...2.1.0)
@@ -18,7 +31,7 @@ Updates should follow the [Keep a CHANGELOG](http://keepachangelog.com/) princip
 - The `Api` prefix has been added from the following classes for consistency:
   - `Resource` --> `ApiResource`
   - `ResourceIdentifier` --> `ApiResourceIdentifier`
-  
+
 ### Added
 - Support for `PATCH` and `DELETE` requests.
 

docs/calls.md

@@ -27,6 +27,19 @@ After setting the options you can call the `get()` method to retrieve the resour
 certificates = certificates_request.get()
 ```
 
+It is also possible to get all resource sets recursively. The package will check the URL defined in `links.next` and as
+long as the value is not `null` it will make an additional request and merge the results:
+
+```python
+certificates = certificates_request.get_recursive()
+```
+
+Please note that the `get_recursive` method respects pagination and filters. So the following example will get all
+non-expired certificates, starting from page two in batches of ten:
+```python
+certificates = certificates_request.filter('expired', False).page(2).size.get_recursive()
+```
+
 ## Getting a single resource by ID
 If you want to get a specific resource by its ID, you can pass it as an argument to the `get` method:
 ```python

docs/responses.md

@@ -1,6 +1,37 @@
 # Working with API Responses
 There are two types of API responses upon a successful request. If a single resource is requested then a [`ApiResource`](resources.md) instance is
-returned, if multiple resources are requested then an `list` of `ApiResource`'s is returned.
+returned, if multiple resources are requested then an `ApiResourceSet` is returned.
+
+## The `ApiResourceSet` class
+When the API returns multiple resources, for example when getting an overview page, an instance of the `ApiResourceSet` class
+is returned. The instance of this class contains the requested resources. Traverse each individual resource by using a
+`for`-loop on the instance:
+```python
+certificates = client.resource('certificates').get()
+for certificate in certificates:
+    # Each item is an instance of an ApiResource.
+    print(certificate.id())
+```
+
+The get the number of items in a resource set, you can use one of the following methods:
+```php
+len(certificates); // Returns the number of resources in the current resource set.
+certificates.total() // Returns the total number of resources in the resource set, ignoring pagination.
+```
+
+If `len != total` you can get the next/previous/first/last page by calling one of the pagination methods:
+```python
+# Get the next resource set:
+certificates.next_page()
+# Get the previous resource set:
+certificates.previous_page()
+# Get the first resource set:
+certificates.first_page()
+# Get the last resource set:
+certificates.last_page()
+```
+
+Each of this methods will return `None` if not available.
 
 ## The [`ApiResource`](resources.md) class
 Each resource returned by the API is transformed to an [`ApiResource`](resources.md) instance. This makes it possible to have easy access
@@ -21,24 +52,6 @@ print(
 )
 ```
 
-## Multiple resources
-When the API returns multiple resources, for example when getting an overview page, a list is returned.
-This list contains the requested resources. Iterate over the list to handle the resources:
-
-```python
-
-# Get all certificates
-certificates = client.resource('certificates').size(10).get()
-
-for certificate in certificates:
-    print(
-        '- {domain} Expires at {expire_date}'.format(
-            domain=certificate.attribute('domain'),
-            expire_date=certificate.attribute('expires_at')
-        )
-    )
-```
-
 ---
 
 [Back to the index](index.md)

examples/dns_zone_details.py

@@ -9,9 +9,10 @@
 client.authenticator.set_token(sys.argv[1])
 
 '''
-Get a single dns_zone resource. Because depending on who is authorized, the dns_zone IDs change, all dns_zones are
-retrieved with a limit of 1. From this result, the first DNS zone is used. In a real world scenario you would call
-something like `zone = client.resource('dns_zones').get('VX09kwR3KxNo')` to get a single DNS zone by it's ID.
+Get a single dns_zone resource. Because depending on who is authorized, the dns_zone IDs change, all
+dns_zones are retrieved with a limit of 1. From this result, the first DNS zone is used. In a real
+world scenario you would call something like
+`zone = client.resource('dns_zones').get('VX09kwR3KxNo')` to get a single DNS zone by it's ID.
 '''
 zones = client.resource('dns_zones').size(1).get()
 

examples/ticket_details.py

@@ -9,9 +9,10 @@
 client.authenticator.set_token(sys.argv[1])
 
 '''
-Get a single ticket resource. Because depending on who is authorized, the ticket IDs change, all tickets are
-retrieved with a limit of 1. From this result, the first ticket is used. In a real world scenario you would call
-something like `ticket = client.resource('tickets').get('VX09kwR3KxNo')` to get a single ticket by it's ID.
+Get a single ticket resource. Because depending on who is authorized, the ticket IDs change, all
+tickets are retrieved with a limit of 1. From this result, the first ticket is used. In a real world
+scenario you would call something like `ticket = client.resource('tickets').get('VX09kwR3KxNo')` to
+get a single ticket by it's ID.
 '''
 tickets = client.resource('tickets').size(1).get()
 

exonetapi/RequestBuilder.py

@@ -1,20 +1,22 @@
 """
 Build requests to send to the API.
 """
+import json
 import warnings
 
 import requests
 
-from .result import Parser
+from exonetapi.structures import ApiResourceSet
 from exonetapi.exceptions.ValidationException import ValidationException
+from .result import Parser
 
 
 class RequestBuilder(object):
     """Create and make requests to the API.
     """
 
-    def __init__(self, resource, client=None):
-        if not resource.startswith('/'):
+    def __init__(self, resource=None, client=None):
+        if resource is not None and not resource.startswith('/'):
             resource = '/' + resource
 
         self.__resource = resource
@@ -98,9 +100,8 @@ def get(self, identifier=None):
 
         return Parser(response.content).parse()
 
-    def store(self, resource):
-        warnings.warn("store() is deprecated; use post().", DeprecationWarning)
-        self.post(resource)
+    def get_recursive(self):
+        return self.__get_recursive()
 
     def post(self, resource):
         """Make a POST request to the API with the provided resource as data.
@@ -176,7 +177,10 @@ def __build_url(self, identifier=None):
 
         :return: A URL.
         """
-        url = self.__client.get_host() + self.__resource
+        url = self.__client.get_host()
+
+        if self.__resource is not None:
+            url += self.__resource
 
         if identifier:
             url += '/' + identifier
@@ -211,3 +215,32 @@ def __make_call(self, method, url, json_data=None, params=None):
         response.raise_for_status()
 
         return response
+
+    def __get_recursive(self, data=None, url=None):
+        """
+        Get the URL and call this method recursivly as long as there is an URL in the 'next' field
+        of the 'links' data.
+
+        :param data: The ApiResourceSet to append the resources to.
+        :param url: The URL to call.
+        :return: The ApiResourceSet containing all requested resources.
+        """
+        response = self.__make_call(
+            'GET',
+            url or self.__build_url(),
+            params=self.__query_params if not url else None
+        )
+
+        content = Parser(response.content).parse()
+
+        if data is None:
+            data = ApiResourceSet()
+            data.set_meta(content.meta().copy())
+
+        data.add_resource(content.resources())
+
+        next_link = content.links().get('next')
+        if next_link is not None:
+            return self.__get_recursive(data, next_link)
+
+        return data

exonetapi/result/Parser.py

@@ -3,7 +3,7 @@
 """
 import json
 from exonetapi.create_resource import create_resource
-from exonetapi.structures import ApiResourceIdentifier
+from exonetapi.structures import ApiResourceIdentifier, ApiResourceSet
 from exonetapi.structures.Relationship import Relationship
 
 
@@ -14,17 +14,23 @@ class Parser:
 
     def __init__(self, data):
         self.__data = data
-        self.__json_data = json.loads(self.__data).get('data')
+        self.__json = json.loads(self.__data)
+        self.__json_data = self.__json.get('data')
 
     def parse(self):
         """Parse JSON string into a ApiResource or a list of Resources.
 
-        :return list|ApiResource: List with ApiResources or a single ApiResource.
+        :return ApiResourceSet|ApiResource: An ApiResourceSet or a single ApiResource.
         """
         if type(self.__json_data) is list:
-            resources = []
+            resources = ApiResourceSet()
+            resources \
+                .set_meta(self.__json.get('meta')) \
+                .set_links(self.__json.get('links'))
+
             for resource_data in self.__json_data:
-                resources.append(self.make_resource(resource_data))
+                resource = self.make_resource(resource_data)
+                resources.add_resource(resource)
 
             return resources
         else:

exonetapi/structures/ApiResource.py

@@ -104,10 +104,6 @@ def to_json_resource_identifier(self):
     def patch(self):
         return exonetapi.RequestBuilder(self.type()).patch(self)
 
-    def store(self):
-        warnings.warn("store() is deprecated; use post().", DeprecationWarning)
-        return self.post()
-
     def post(self):
         return exonetapi.RequestBuilder(self.type()).post(self)
 

exonetapi/structures/ApiResourceIdentifier.py

@@ -9,6 +9,7 @@
 class ApiResourceIdentifier(object):
     """Basic ApiResource identifier.
     """
+
     def __init__(self, type, id=None):
         """Initialize the resource.
         :param type: The type of the resource.
@@ -71,7 +72,7 @@ def get_relationship(self, name):
         :param name: The name of the relation to get.
         :return: The defined relation or None
         """
-        if not name in self.__relationships.keys():
+        if name not in self.__relationships.keys():
             self.__relationships[name] = Relationship(name, self.type(), self.id())
 
         return self.__relationships[name]