support for asyncio (PLT-31454)

[jpassaro]

The Snowflake connector for python seems to be implemented essentially as API calls over HTTP. Using aiohttp, companion subclasses to SnowflakeConnector, SnowflakeCursor, SnowflakeRestful etc, could be created that implement the key methods as asynchronous coroutines. Then asyncio tools could be used to run Snowflake connection routines alongside other I/O-centric or API-driven tasks.

Has this been considered? Is it a viable addition to the Snowflake Connector? If so I'm happy to contribute, I'd love to hear any requirements you folks might have in mind. Or on the other hand, is it more appropriate as a fork, or as a separate project altogether in the style of aiobotocore?

[smtakeda]

Thanks for suggestion. Yes, we considered aiohttp as well as any async feature in 3.4+. There are two concerns: 1) since the driver needs to support both 2.7+ and 3.4+, it may need to have a branch for 3.4+, 2) it is not clear how to handle OCSP check in aiohttp.

OCSP check is a requirement from our security team, though the standard Python's SSL library doesn't support it (and even most other clients don't care about the certificate revocation status!). We use a monkey patch on the top of https://github.com/requests/requests to intercept SSL handshake and add the OCSP checks.

So definitely all contribution would be pleased along with the above concerns addressed.

[szelenka]

Checking if there has been any progress with asyncio support for this package, the last update seems to be over 2 years ago..

[smtakeda]

sorry we didn't have enough bandwidth to work on this. A plan is add async support after dropping python2.

[krrg]

Python 2 will stop being supported by the Python Foundation on Jan 1, 2020. Will Snowflake also drop support at that time? Just curious if any timeline has been established on when Snowflake will EOL Python 2 support.

If it helps anyone interested, as a temporary workaround, I've been running Snowflake queries inside a concurrent.futures.ThreadPoolExecutor using what basically amounts to await asyncio.get_event_loop().run_in_executor(...).

This has been just fine for my purposes (internal reporting) and allows me to still use the other neat asyncio features while not blocking the main thread.

[smtakeda]

For Python 2 support, feel free to discuss in #107.
We have not decided when we are going to drop Python 2 support yet.
Good new is the tread of Python 2 download is down:
https://pypistats.org/packages/snowflake-connector-python

We'll keep eyes on the usage metrics on our end to determine the timing of Python2 drop.

[smtakeda]

We want this for Snowflake:
https://github.com/MagicStack/asyncpg
https://github.com/aio-libs/aiomysql

[farvour]

What's the progress on aio support? Anything that uses it, such as sqlalchemy-aio cannot take advantage of this dialect+driver without it...

[smtakeda]

Not much progress. In the planning meeting.

[mvoitko]

Thanks for suggestion. Yes, we considered aiohttp as well as any async feature in 3.4+. There are two concerns: 1) since the driver needs to support both 2.7+ and 3.4+, it may need to have a branch for 3.4+, 2) it is not clear how to handle OCSP check in aiohttp.

Python 2.7 will be deprecated since Jan 1 2020. ANy progress on async SnowFlake connector?

[islobodch]

Commenting here to re-emphasise the issue. We are going to use asyncio in ongoing projects involving Python, and not having support for async is a bit frustrating. Are there any news regarding this? Thanks.

[jimfang]

It is very important for snowflake to support the async connector. Otherwise, we have to workaround. It is a big bottleneck for performance optimization. Thanks.

[samstiyer]

Hey Everyone, just wanted to bump this feature request. With FastAPI becoming more and more utilized in the python ecosystem, support for an Asyncio continues to become more important. Thanks!