Add error handling and backoff when making requests to Quick Base

[wkirby007]

Current Summary: We don't have any error handling around making requests to Quick Base which exposes us to blips in the SSL handshake which are apparently a known issue.

Adding backoff to the request code should improve the situation.

---

Hi.

We've just started using this integration into Quick Base from Stitchdata. Within in QB, we have multiple applications, but we only want the data from one of them. We setup a user token with access to the singular app and have used that app reference in the integration.

I can see from the logs that the tap receives all the apps, but only structures the streams for those that relate to the app we are interested in. However, we get error reports as it shows the below in the logs, but it actually appears to relate to the other apps in the QB realm which the user token will not have access too?

HTTPSConnectionPool(host='pulse.quickbase.com', port=443): Max retries exceeded with url: /db/bmpvry5t7?options=num-100&query=%7B2.AF.1532359647000%7D&act=API_DoQuery&includeRids=1&clist=23.2.24.6.16.48.55.31.25.54.12.38.32.18.3.21.14.1.26.44.42.46.43.41.45.36.13.33.47.49.5.4.20&slist=2&fmt=structured&realmhost=&usertoken=....................................... (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))

[emlama]

Hi Kirby,

We do have a known issue where our API will sometimes have an SSLHandshake error and you'll typically have to wait a few seconds and then retry. Is there any backoff for the API calls here?

Best,

[wkirby007]

Thanks for the reply. We are currently backing off to PostgreSQL. The experience we have with the above error however coincides with not all our tables being created and populated? We see a single table of data only in this instance but the stream would suggest, as does the QV application itself, that more tables should be processed.

…

Sent from my iPhone

On 8 Nov 2018, at 03:30, Matthew Saforrian ***@***.***> wrote: Hi Kirby, We do have a known issue where our API will sometimes have an SSLHandshake error and you'll typically have to wait a few seconds and then retry. Is there any backoff for the API calls here? Best, — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[emlama]

I also encountered this issue using the Stitch platform. I did observe that the Quick Base connector seems to be having an issue with how it's iterating through all of the records. I'm seeing it as trying to load 544,500 records but only have 4,448 records in that table. I suspect that it's not incrementing the counter for how many records to retrieve.

I wonder if this SSLhandshake is being caused by too many fetches.

Anyhow, where can I see the retry logic for requests being made into Quick Base? That would be a good place to start.

[wkirby007]

Thanks again for the response. I've downloaded the logs and attached if that is what you are referring to? Please let me know if there is something else I can capture to assist. Thanks Will

…

On 8 Nov 2018, at 15:45, Matthew Saforrian ***@***.***> wrote: I also encountered this issue using the Stitch platform. I did observe that the Quick Base connector seems to be having an issue with how it's iterating through all of the records. I'm seeing it as trying to load 544,500 records but only have 4,448 records in that table. I suspect that it's not incrementing the counter for how many records to retrieve. I wonder if this SSLhandshake is being caused by too many fetches. Anyhow, where can I see the retry logic for requests being made into Quick Base? That would be a good place to start. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMQS_XYRj-nKVBROSjekLwvqHKGfD7Caks5utFG0gaJpZM4YQwVK>.

[timvisher]

@msaforrian At the moment it doesn't look like we're doing any error handling or backoff when we make requests to Quick Base. Adding those should be fairly easy.

We've had good success using the backoff module for this purpose. Here's a decently complex example of doing so.

Would either of you want to take a stab at utilizing it to make this error handling better?

[WaverlyK]

Wondering if there is any update on this?