External table load from local script data

[AlmightyOatmeal]

Since Netezza doesn't support inserting multiple value sets per insert statement and doesn't have a native bulk inserting method, is it possible to do an external table load with data present in the local script but without having write access to a volume where the Netezza instance can read from locally?

The Netezza JDBC driver can detect a bulk insert and it converts it to an external table load but it would be nice to find a way without using JDBC to accomplish this.

[joebeeson]

We've used named pipes in Linux environments to achieve "external" loads without writing to disk.

…

On Tue, Oct 11, 2022, 2:47 PM Jamie Ivanov ***@***.***> wrote: Since Netezza doesn't support inserting multiple value sets per insert statement and doesn't have a native bulk inserting method, is it possible to do an external table load with data present in the local script but without having write access to a volume where the Netezza instance can read from locally? The Netezza JDBC driver can detect a bulk insert and it converts it to an external table load but it would be nice to find a way without using JDBC to accomplish this. — Reply to this email directly, view it on GitHub <#65>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGD5WUCIKC5NCRJFSX3ZDWCXG7VANCNFSM6AAAAAARCU6U5A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AlmightyOatmeal]

We've used named pipes in Linux environments to achieve "external" loads without writing to disk.
…

That involves host access and not through a remote connection such as this library (which is what this request pertains to)

[joebeeson]

Yeah we've done it on remote connections, thanks.

…

On Thu, Oct 13, 2022, 9:42 PM Jamie Ivanov ***@***.***> wrote: We've used named pipes in Linux environments to achieve "external" loads without writing to disk. … <#m_4007661057599205059_> That involves host access and not through a remote connection such as this library (which is what this request pertains to) — Reply to this email directly, view it on GitHub <#65 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGD5VTFUBK4TSO4L2VWR3WDDJDLANCNFSM6AAAAAARCU6U5A> . You are receiving this because you commented.Message ID: ***@***.***>

[AlmightyOatmeal]

Yeah we've done it on remote connections, thanks.
…

Can you elaborate?

[joebeeson]

It's possible, remote side, to create a named pipe that doesn't consume disk, but provides a source for external table loads from the client, that looks exactly like a file to the driver. We've found it to be markedly faster than other means, but this was using the *DBC drivers.

…

On Fri, Oct 14, 2022, 11:52 AM Jamie Ivanov ***@***.***> wrote: Yeah we've done it on remote connections, thanks. … <#m_5850529653358950978_> Can you elaborate? — Reply to this email directly, view it on GitHub <#65 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGD5W7SBQYSPFQ5VPNS23WDGMVVANCNFSM6AAAAAARCU6U5A> . You are receiving this because you commented.Message ID: ***@***.***>

[joebeeson]

To elaborate though this was also Linux only, unsure of what it would require on other platforms.

…

On Fri, Oct 14, 2022, 11:54 AM Joe Beeson ***@***.***> wrote: It's possible, remote side, to create a named pipe that doesn't consume disk, but provides a source for external table loads from the client, that looks exactly like a file to the driver. We've found it to be markedly faster than other means, but this was using the *DBC drivers. On Fri, Oct 14, 2022, 11:52 AM Jamie Ivanov ***@***.***> wrote: > Yeah we've done it on remote connections, thanks. > … <#m_-6117590594304313865_m_5850529653358950978_> > > Can you elaborate? > > — > Reply to this email directly, view it on GitHub > <#65 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAGD5W7SBQYSPFQ5VPNS23WDGMVVANCNFSM6AAAAAARCU6U5A> > . > You are receiving this because you commented.Message ID: > ***@***.***> >