Issue setting up incremental sync on custom connector #36877
Unanswered
SwanSodja
asked this question in
Connector Questions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi folks,
TL;DR: is it possible to recast a string version of a timestamp as the string version of the corresponding date using jinja expressions allowed in order to properly configure my query parameter?
Here's the details. Any help would be much appreciated.
I'm attempting to set up a custom connector using the polygon api, and I've got it working for the most part. The part I'm struggling with is, having set up incremental sync to work with the cursor key (this is in epoch time as %ms), how to inject it into the appropriate query parameter as %Y-%m-%d.
here's an example of the format the api requires for url:
https://api.polygon.io/v2/aggs/ticker/AAPL/range/1/day/2023-01-09/2023-01-09?adjusted=true&sort=desc&limit=120&apiKey={{ config['api_key'] }}
here's an excerpt of my yaml where I think I'm doing something wrong:
and here's where I the result of injecting both the stream_partition value and the incremental sync into the json payload (I did this for troubleshooting:
Beta Was this translation helpful? Give feedback.
All reactions