Performance of different Data Access Methods (OBDC/JBDC vs Sockets)

[parthkarnawat]

Previously, SASPy's data access methods, such as sd2df with method='DISK', utilized JDBC and ODBC objects for data transfer. However, this approach has recently been changed. Could someone help me understand the reasoning behind this change? Also, does the new method offer better performance compared to the older one?

[tomweber-sas]

Hey @parthkarnawat I'm not sure what you're referring to, could you point me at the information you've found? SASPy never used jdbc nor odbc for any of these data transfer methods. The various methods (csv, disk, mem) were different implementations early on. CSV used proc import/proc export to read/write csv files. However, these procs don't handle all forms of data the best and the DISK/MEM methods used generated data step code to transfer and convert data. The MEM originally took rows and appended them to a data frame as data was being retrieved, while DISK wrote it to a file to then read in with pandas. However, about 4 years ago I reworked that (disk and mem are the same now, and the default) where I could stream the data directly to pandas instead of writing to disk or trying to append rows. So, that is the fastest, and default when using sd2df. df2sd streams the data to SAS with a generated data step reading the stream directly to create a SAS Data Set in a similar way.
So, I'm not sure what you're referring to about something recently changing with these, or where jdbc/odbc cam from in regard to this.
Thanks,
Tom