Retry on pysignalr's ConnectionError (#196)

* Retry on pysignalr ConnectionError

* Changelog

* Bump

Author: droserasprout

CHANGELOG.md

@@ -4,25 +4,26 @@ Please use [this](https://docs.gitlab.com/ee/development/changelog.html) documen
 
 ## [unreleased]
 
-## Added
+### Added
 
 * tzkt: Added optional `delegate_address` and `delegate_alias` fields to `OperationData`.
 
-## Fixed
+### Fixed
 
-* tzkt: Fixed incorrect parsing of `OperationData.amount` field.
+* tzkt: Fixed crash due to unprocessed pysignalr exception.
+* tzkt: Fixed parsing `OperationData.amount` field.
 
 ## 4.0.1 - 2021-12-30
 
-## Fixed
+### Fixed
 
 * codegen: Fixed generating storage typeclasses with `Union` fields.
 * codegen: Fixed preprocessing contract JSONSchema.
 * index: Fixed processing reindexing reason saved in the database.
 * tzkt: Fixed processing operations with default entrypoint and empty parameter.
 * tzkt: Fixed crash while recursively applying bigmap diffs to the storage.
 
-## Performance
+### Performance
 
 * tzkt: Increased speed of applying bigmap diffs to operation storage.
 

Makefile

@@ -1,5 +1,5 @@
 .ONESHELL:
-.PHONY: test
+.PHONY: test build
 .DEFAULT_GOAL: all
 
 DEV=1

poetry.lock

@@ -32,7 +32,7 @@ datamodel-code-generator = "^0.11.1"
 fcache = "^0.4.7"
 pydantic = "^1.8.1"
 pyhumps = "^3.0.2"
-pysignalr = "^0.1.0"
+pysignalr = "^0.1.1"
 pytezos = {version = "^3.2.4", optional = true}
 python-dotenv = "^0.18.0"
 "ruamel.yaml" = "^0.17.2"

pyproject.toml

@@ -1,5 +1,6 @@
 import asyncio
 import logging
+import sys
 from asyncio import Event
 from asyncio import create_task
 from asyncio import gather
@@ -24,6 +25,7 @@
 
 from aiohttp import ClientResponseError
 from pysignalr.client import SignalRClient
+from pysignalr.exceptions import ConnectionError as WebsocketConnectionError
 from pysignalr.messages import CompletionMessage  # type: ignore
 from pysignalr.transport.websocket import DEFAULT_MAX_SIZE
 
@@ -594,7 +596,19 @@ def _get_ws_client(self) -> SignalRClient:
 
     async def run(self) -> None:
         self._logger.info('Establishing realtime connection')
-        tasks = [create_task(self._get_ws_client().run())]
+        ws = self._get_ws_client()
+
+        async def _wrapper():
+            retry_sleep = self._http_config.retry_sleep
+            for _ in range(self._http_config.retry_count or sys.maxsize):
+                try:
+                    await ws.run()
+                except WebsocketConnectionError as e:
+                    self._logger.error('Websocket connection error: %s', e)
+                    await asyncio.sleep(retry_sleep)
+                    retry_sleep *= self._http_config.retry_multiplier
+
+        tasks = [create_task(_wrapper())]
 
         if self._watchdog:
             tasks.append(create_task(self._watchdog.run()))