manifest syle connector #46349
Unanswered
guykatz
asked this question in
Connector Questions
Replies: 1 comment
-
is this not the place to ask questions? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all
my source data sits in muiltiple remote http web servers.
each server has a manifest file (predefined name) in a well known address I can connect to.
inside this manifest there are many lines, each line is a http address of a source data file I need to transfer to the datawarehouse.
for example:
in remote web server 1.1.1.1 I have a manifest style file accesible via 1.1.1.1/manifest.mf
the content of this file can be:
i do not control this manifest file creation. lines are added and removed from it periodicly and i need to get the new ones only and trasfer them.
I assume that even though this type of source is not rare, its still rare enough not to exist as a connector in airbyte?
in case it does not exist, I assume i need to build a custom connector?
thanks for the help./
Beta Was this translation helpful? Give feedback.
All reactions