How to programmatically add source connector to UI in OSS version?

[LuisSaybe]

I am running airbyte locally and looking to write a bash script which will install and run airbyte along with my custom sources.
I'm not able to find a way to add a connector source programatically.
If I have the remote docker image and the minimum needed information for the custom source, it seems there should be a way to install airbyte with a customer connector from a bash script.

The documentation seems to indicate the only way to add a custom source locally is by adding the connector manually in the UI.

Is this really the case? It seems like I am missing something.

[vbalalian]

Hi @LuisSaybe, did you ever get an answer on this?

[LuisSaybe]

Yes, I did actually, I joined the office hours which they advertise on their slack channel and asked this questions and one of the devs pointed me in the right direction.

You can read here about the differences in the Airbyte API and the Airbyte Configuration API.

You'll note the Airbyte API does not have any references to the functionality I was looking for in my original post but apparently the Airbyte Configuration API does include this. I have not tested it end-to-end yet because I've been quite busy but the Airbyte Configuration API is the place where you would want to look for this.

The create source endpoint is finally available in the airbyte API, so it should be possible now to programmatically create a source, destination and connection programmatically.

[vbalalian]

It seems like any mention or documentation of this has been removed, but I'm continuing to research this. Apparently, there is an endpoint you can post to, "/api/v1/source_definitions/create_custom" to add a custom connector programatically; I'm still figuring it out though. Mentioned in Airbyte - Slack here: "https://airbytehq.slack.com/archives/C02RRUG9CP5/p1679988794764499?thread_ts=1678963208.296829&cid=C02RRUG9CP5"

[LuisSaybe]

I haven't tested it end-to-end but I think the configuration API would be the best option. It doesn't seem there are any command line nor HTTP APIs other than the configuration API

[LuisSaybe]

They finally added an endpoint to add custom source via the API, see this endpoint

[vbalalian]

I'm fairly certain that doesn't change anything; you could already "create" a source from a custom connector using the definition id with that api call. The issue is, you need to be able to add the custom connector definition to the airbyte instance before you can use it to "create" a source, and the api doesn't have that functionality. As you mentioned previously, the configuration API does allow this, albeit without sufficient documentation. You can check out this script to see how I was able to programmatically add a custom source connector and then "create" a source based on it. I had to use both APIs to do so.