Insight into "auto-commit mode" error

[robertness]

I am trying to write an Airflow hook for connecting to Athena, and I'm getting this error about 'Athena JDBC connection is only supported for auto-commit mode.' Can you offer any insight into this error?

`class AthenaHook(S3Hook):
"""
Hook to work with Athena and associated data in S3 buckets.
"""

def __init__(self, s3_staging_dir, aws_conn_id):
    self.region_name = None
    self.conn = None
    self.s3_staging_dir = s3_staging_dir
    super(AthenaHook, self).__init__(aws_conn_id)

def connect(self,**kwargs):
    """
    Connect to Athena using JDBC. Credentials handled by S3hook
    """

    credentials = map(str, self.get_credentials())
    self.conn = jdbc_connect(
        s3_staging_dir=self.s3_staging_dir,
        access_key=credentials[0],
        secret_key=credentials[1],
        region_name=self.region_name
    )

def dataframe(self, query):
    """
    Pull the results of a query into a data frame

    Args:
        query(str): A SQL query to be run in Athena.

    Returns:
        (pd.DataFrame): A dataframe containing the results of the query
    """
    return pd.read_sql(query, self.conn)

class AthenaPlugin(AirflowPlugin):
name = "athena_plugin"
hooks = [AthenaHook]
`
Here is the traceback:

In [15]: hook.dataframe('SELECT * from posterior_db.strain_posterior_local LIMIT 10')

[2017-11-08 14:19:27,349] {cursor.py:193} ERROR - Failed to execute query.
Traceback (most recent call last):
File "/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pyathenajdbc/cursor.py", line 172, in execute
result_set = self._statement.executeQuery(query)
java.sql.SQLExceptionPyRaisable: java.sql.SQLException: Failed to run query

DatabaseError Traceback (most recent call last)
in ()
----> 1 hook.dataframe('SELECT * from posterior_db.strain_posterior_local LIMIT 10')

/home/rness/projects/ztl/zflow/plugins/athena_plugin.pyc in dataframe(self, query)
55 (pd.DataFrame): A dataframe containing the results of the query
56 """
---> 57 return pd.read_sql(query, self.conn)
58
59

/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pandas/io/sql.pyc in read_sql(sql, con, index_col, coerce_float, params, parse_dates, columns, chunksize)
397 sql, index_col=index_col, params=params,
398 coerce_float=coerce_float, parse_dates=parse_dates,
--> 399 chunksize=chunksize)
400
401 try:

/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pandas/io/sql.pyc in read_query(self, sql, index_col, coerce_float, params, parse_dates, chunksize)
1434
1435 args = _convert_params(sql, params)
-> 1436 cursor = self.execute(*args)
1437 columns = [col_desc[0] for col_desc in cursor.description]
1438

/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pandas/io/sql.pyc in execute(self, *args, **kwargs)
1407 ex = DatabaseError("Execution failed on sql: %s\n%s\nunable"
1408 " to rollback" % (args[0], exc))
-> 1409 raise_with_traceback(ex)
1410
1411 ex = DatabaseError(

/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pandas/io/sql.pyc in execute(self, *args, **kwargs)
1403 except Exception as exc:
1404 try:
-> 1405 self.con.rollback()
1406 except Exception: # pragma: no cover
1407 ex = DatabaseError("Execution failed on sql: %s\n%s\nunable"

/home/rness/projects/ztl/venv/lib/python2.7/site-packages/pyathenajdbc/connection.pyc in rollback(self)
148
149 def rollback(self):
--> 150 raise NotSupportedError('Athena JDBC connection is only supported for auto-commit mode.')

DatabaseError: Execution failed on sql: SELECT * from posterior_db.strain_posterior_local LIMIT 10
java.sql.SQLException: Failed to run query
unable to rollback

[laughingman7743]

Does the error appear in the history of the Athena Web console?
And if you know the query ID, you can also use Boto3 to see details, so please check it.
http://boto3.readthedocs.io/en/latest/reference/services/athena.html#Athena.Client.get_query_execution

[Parth2017]

I am also getting same error. When I am executing pd.Read_sql statement it is showing results in athena but in python I am getting below error:

NotSupportedError: Athena JDBC connection is only supported for auto-commit mode.

Below is the code

df_700 = pd.read_sql('SELECT * FROM jqerosh.alestbun_nioi_1110 limit 10', conn, chunksize=10)

Please help

[laughingman7743]

Does the same error occur in the following libraries?
https://github.com/laughingman7743/PyAthena
I would like to know the stack trace of the exception.

[Parth2017]

yes the same error occur using pyathenajdbc library. This is the stack trace of the exception: Failed to execute query. Traceback (most recent call last): File "C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py", line 181, in execute query_execution = self._query_execution() File "C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py", line 145, in _query_execution client = self._athena_client() File "C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py", line 140, in _athena_client field.setAccessible(True) jpype._jexception.java.lang.NullPointerExceptionPyRaisable: java.lang.NullPointerException

…

---------------------------------------------------------------------------java.lang.NullPointerExceptionPyRaisable Traceback (most recent call last)C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py in execute(self, operation, parameters) 180 self._update_count = self._statement.getUpdatecount()--> 181 query_execution = self._query_execution() 182 C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py in _query_execution(self) 144 def _query_execution(self):--> 145 client = self._athena_client() 146 request = jpype.JPackage('com.amazonaws.athena.jdbc.shaded.' + C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py in _athena_client(self) 139 if field.name == 'athenaServiceClient':--> 140 field.setAccessible(True) 141 return field.get(self._connection).getClient() java.lang.NullPointerExceptionPyRaisable: java.lang.NullPointerException The above exception was the direct cause of the following exception: DatabaseError Traceback (most recent call last)C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in execute(self, *args, **kwargs) 1403 else:-> 1404 cur.execute(*args) 1405 return cur C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\util.py in _wrapper(*args, **kwargs) 33 jpype.attachThreadToJVM()---> 34 return wrapped(*args, **kwargs) 35 return _wrapper C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\util.py in _wrapper(*args, **kwargs) 23 with _lock:---> 24 return wrapped(*args, **kwargs) 25 return _wrapper C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py in execute(self, operation, parameters) 194 _logger.exception('Failed to execute query.')--> 195 raise_from(DatabaseError(unwrap_exception(e)), e) 196 C:\Users\parth\Anaconda3\lib\site-packages\future\utils\__init__.py in raise_from(exc, cause) 397 execstr = "raise __python_future_raise_from_exc from __python_future_raise_from_cause"--> 398 exec(execstr, myglobals, mylocals) 399 C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\cursor.py in <module>() DatabaseError: java.lang.NullPointerException During handling of the above exception, another exception occurred: NotSupportedError Traceback (most recent call last)C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in execute(self, *args, **kwargs) 1407 try:-> 1408 self.con.rollback() 1409 except Exception: # pragma: no cover C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\connection.py in rollback(self) 157 def rollback(self):--> 158 raise NotSupportedError('Athena JDBC connection is only supported for auto-commit mode.') NotSupportedError: Athena JDBC connection is only supported for auto-commit mode. During handling of the above exception, another exception occurred: DatabaseError Traceback (most recent call last)<ipython-input-81-5dd1e2afb546> in <module>() 1 query = 'SELECT * FROM josh.buxton_imei_1110 limit 10'----> 2 df_700 = pd.read_sql(query, conn, chunksize=10) 3 4 df_700C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in read_sql(sql, con, index_col, coerce_float, params, parse_dates, columns, chunksize) 398 sql, index_col=index_col, params=params, 399 coerce_float=coerce_float, parse_dates=parse_dates,--> 400 chunksize=chunksize) 401 402 try: C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in read_query(self, sql, index_col, coerce_float, params, parse_dates, chunksize) 1437 1438 args = _convert_params(sql, params)-> 1439 cursor = self.execute(*args) 1440 columns = [col_desc[0] for col_desc in cursor.description] 1441 C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in execute(self, *args, **kwargs) 1410 ex = DatabaseError("Execution failed on sql: %s\n%s\nunable" 1411 " to rollback" % (args[0], exc))-> 1412 raise_with_traceback(ex) 1413 1414 ex = DatabaseError( C:\Users\parth\Anaconda3\lib\site-packages\pandas\compat\__init__.py in raise_with_traceback(exc, traceback) 342 if traceback == Ellipsis: 343 _, _, traceback = sys.exc_info()--> 344 raise exc.with_traceback(traceback) 345 else: 346 # this version of raise is a syntax error in Python 3 C:\Users\parth\Anaconda3\lib\site-packages\pandas\io\sql.py in execute(self, *args, **kwargs) 1406 except Exception as exc: 1407 try:-> 1408 self.con.rollback() 1409 except Exception: # pragma: no cover 1410 ex = DatabaseError("Execution failed on sql: %s\n%s\nunable" C:\Users\parth\Anaconda3\lib\site-packages\pyathenajdbc\connection.py in rollback(self) 156 157 def rollback(self):--> 158 raise NotSupportedError('Athena JDBC connection is only supported for auto-commit mode.') DatabaseError: Execution failed on sql: SELECT * FROM josh.buxton_imei_1110 limit 10 java.lang.NullPointerException unable to rollback

On Tue, Jun 26, 2018 at 6:25 PM, laughingman7743 ***@***.***> wrote: Does the same error occur in the following libraries? https://github.com/laughingman7743/PyAthena I would like to know the stack trace of the exception. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ajr51LUMw2gXP9-bWgmk9z0pvpVX5SSzks5uArTHgaJpZM4QXJVu> .

[laughingman7743]

Will it work out if you use read_sql_query?
pyathena-dev/PyAthena#30 (comment)

[Parth2017]

Yes I tried with both functions. Still getting same error.

[laughingman7743]

I checked with the following simple code, but there was no problem.

from pyathenajdbc import connect
import pandas as pd
conn = connect(s3_staging_dir='s3://BUCKET/path/to/',
               region_name='us-west-2')
df = pd.read_sql("SELECT * FROM one_row_complex", conn)
print(df.describe())

The definition of the one_row_complex table is as follows.
https://github.com/laughingman7743/PyAthenaJDBC/blob/master/tests/sql/create_table.sql#L14

If possible, please tell me the query and table definition where the error occurred.

[yuxx1006]

Same problem here. It seems a problem of pd.read_sql() or read_sql_query. The table definition is same as the example of one_row_complex.

[daniel1608]

I get the same error with this small example.

from pyathenajdbc import connect
import pandas as pd

conn = connect(access_key='KEY',
               secret_key='KEY"',
               s3_staging_dir='DIR',
               region_name='eu-central-1')
df = pd.read_sql_query("SELECT TRUE", conn)

[laughingman7743]

Perhaps it depends on the version of pandas?
It seemed that there would be no problem if the following libraries were installed in a virtual env environment.

botocore==1.12.0
docutils==0.14
future==0.16.0
jmespath==0.9.3
JPype1==0.6.3
numpy==1.15.1
pandas==0.23.4
PyAthenaJDBC==1.3.3
python-dateutil==2.7.3
pytz==2018.5
six==1.11.0
urllib3==1.23

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pandas = "*"
pyathenajdbc = "*"

[dev-packages]

[requires]
python_version = "3.6"

Pipfile.lock:

{
    "_meta": {
        "hash": {
            "sha256": "48cd71ffb13391fb4aeac611ad388feefe346734c0d754d69f5024fef1a43ec0"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "botocore": {
            "hashes": [
                "sha256:354dd98082dc1ca8aa127f5aff0e28404363f94a8c13c9a9ddbd24b9a70eb259",
                "sha256:81b0b15f5cfe16bce7280010daaee4c88c046aa0eef0b65432cf4832a0d8c5e8"
            ],
            "version": "==1.12.0"
        },
        "docutils": {
            "hashes": [
                "sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6",
                "sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274",
                "sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6"
            ],
            "version": "==0.14"
        },
        "future": {
            "hashes": [
                "sha256:e39ced1ab767b5936646cedba8bcce582398233d6a627067d4c6a454c90cfedb"
            ],
            "version": "==0.16.0"
        },
        "jmespath": {
            "hashes": [
                "sha256:6a81d4c9aa62caf061cb517b4d9ad1dd300374cd4706997aff9cd6aedd61fc64",
                "sha256:f11b4461f425740a1d908e9a3f7365c3d2e569f6ca68a2ff8bc5bcd9676edd63"
            ],
            "version": "==0.9.3"
        },
        "jpype1": {
            "hashes": [
                "sha256:6841523631874a731e1f94e1b1f130686ad3772030eaa3b6946256eeb1d10dd1"
            ],
            "version": "==0.6.3"
        },
        "numpy": {
            "hashes": [
                "sha256:1c362ad12dd09a43b348bb28dd2295dd9cdf77f41f0f45965e04ba97f525b864",
                "sha256:2156a06bd407918df4ac0122df6497a9c137432118f585e5b17d543e593d1587",
                "sha256:24e4149c38489b51fc774b1e1faa9103e82f73344d7a00ba66f6845ab4769f3f",
                "sha256:340ec1697d9bb3a9c464028af7a54245298502e91178bddb4c37626d36e197b7",
                "sha256:35db8d419345caa4eeaa65cd63f34a15208acd87530a30f0bc25fc84f55c8c80",
                "sha256:361370e9b7f5e44c41eee29f2bb5cb3b755abb4b038bce6d6cbe08db7ff9cb74",
                "sha256:36e8dcd1813ca92ce7e4299120cee6c03adad33d89b54862c1b1a100443ac399",
                "sha256:378378973546ecc1dfaf9e24c160d683dd04df871ecd2dcc86ce658ca20f92c0",
                "sha256:419e6faee16097124ee627ed31572c7e80a1070efa25260b78097cca240e219a",
                "sha256:4287104c24e6a09b9b418761a1e7b1bbde65105f110690ca46a23600a3c606b8",
                "sha256:549f3e9778b148a47f4fb4682955ed88057eb627c9fe5467f33507c536deda9d",
                "sha256:5e359e9c531075220785603e5966eef20ccae9b3b6b8a06fdfb66c084361ce92",
                "sha256:5ee7f3dbbdba0da75dec7e94bd7a2b10fe57a83e1b38e678200a6ad8e7b14fdc",
                "sha256:62d55e96ec7b117d3d5e618c15efcf769e70a6effaee5842857b64fb4883887a",
                "sha256:719b6789acb2bc86ea9b33a701d7c43dc2fc56d95107fd3c5b0a8230164d4dfb",
                "sha256:7a70f2b60d48828cba94a54a8776b61a9c2657a803d47f5785f8062e3a9c7c55",
                "sha256:7b9e37f194f8bcdca8e9e6af92e2cbad79e360542effc2dd6b98d63955d8d8a3",
                "sha256:83b8fc18261b70f45bece2d392537c93dc81eb6c539a16c9ac994c47fc79f09a",
                "sha256:9473ad28375710ab18378e72b59422399b27e957e9339c413bf00793b4b12df0",
                "sha256:95b085b253080e5d09f7826f5e27dce067bae813a132023a77b739614a29de6e",
                "sha256:98b86c62c08c2e5dc98a9c856d4a95329d11b1c6058cb9b5191d5ea6891acd09",
                "sha256:a3bd01d6d3ed3d7c06d7f9979ba5d68281f15383fafd53b81aa44b9191047cf8",
                "sha256:c81a6afc1d2531a9ada50b58f8c36197f8418ef3d0611d4c1d7af93fdcda764f",
                "sha256:ce75ed495a746e3e78cfa22a77096b3bff2eda995616cb7a542047f233091268",
                "sha256:dae8618c0bcbfcf6cf91350f8abcdd84158323711566a8c5892b5c7f832af76f",
                "sha256:df0b02c6705c5d1c25cc35c7b5d6b6f9b3b30833f9d178843397ae55ecc2eebb",
                "sha256:e3660744cda0d94b90141cdd0db9308b958a372cfeee8d7188fdf5ad9108ea82",
                "sha256:f2362d0ca3e16c37782c1054d7972b8ad2729169567e3f0f4e5dd3cdf85f188e"
            ],
            "version": "==1.15.1"
        },
        "pandas": {
            "hashes": [
                "sha256:11975fad9edbdb55f1a560d96f91830e83e29bed6ad5ebf506abda09818eaf60",
                "sha256:12e13d127ca1b585dd6f6840d3fe3fa6e46c36a6afe2dbc5cb0b57032c902e31",
                "sha256:1c87fcb201e1e06f66e23a61a5fea9eeebfe7204a66d99df24600e3f05168051",
                "sha256:242e9900de758e137304ad4b5663c2eff0d798c2c3b891250bd0bd97144579da",
                "sha256:26c903d0ae1542890cb9abadb4adcb18f356b14c2df46e4ff657ae640e3ac9e7",
                "sha256:2e1e88f9d3e5f107b65b59cd29f141995597b035d17cc5537e58142038942e1a",
                "sha256:31b7a48b344c14691a8e92765d4023f88902ba3e96e2e4d0364d3453cdfd50db",
                "sha256:4fd07a932b4352f8a8973761ab4e84f965bf81cc750fb38e04f01088ab901cb8",
                "sha256:5b24ca47acf69222e82530e89111dd9d14f9b970ab2cd3a1c2c78f0c4fbba4f4",
                "sha256:647b3b916cc8f6aeba240c8171be3ab799c3c1b2ea179a3be0bd2712c4237553",
                "sha256:66b060946046ca27c0e03e9bec9bba3e0b918bafff84c425ca2cc2e157ce121e",
                "sha256:6efa9fa6e1434141df8872d0fa4226fc301b17aacf37429193f9d70b426ea28f",
                "sha256:be4715c9d8367e51dbe6bc6d05e205b1ae234f0dc5465931014aa1c4af44c1ba",
                "sha256:bea90da782d8e945fccfc958585210d23de374fa9294a9481ed2abcef637ebfc",
                "sha256:d318d77ab96f66a59e792a481e2701fba879e1a453aefeebdb17444fe204d1ed",
                "sha256:d785fc08d6f4207437e900ffead930a61e634c5e4f980ba6d3dc03c9581748c7",
                "sha256:de9559287c4fe8da56e8c3878d2374abc19d1ba2b807bfa7553e912a8e5ba87c",
                "sha256:f4f98b190bb918ac0bc0e3dd2ab74ff3573da9f43106f6dba6385406912ec00f",
                "sha256:f71f1a7e2d03758f6e957896ed696254e2bc83110ddbc6942018f1a232dd9dad",
                "sha256:fb944c8f0b0ab5c1f7846c686bc4cdf8cde7224655c12edcd59d5212cd57bec0"
            ],
            "index": "pypi",
            "version": "==0.23.4"
        },
        "pyathenajdbc": {
            "hashes": [
                "sha256:11be804192b6b09174dee46aa2316669b8c7a068bec79873ff1334c6e00ce108",
                "sha256:d236c146a91b9390e7ccd9e29e1395fde103e20e1feadabf6013a997461d7b98"
            ],
            "index": "pypi",
            "version": "==1.3.3"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:1adb80e7a782c12e52ef9a8182bebeb73f1d7e24e374397af06fb4956c8dc5c0",
                "sha256:e27001de32f627c22380a688bcc43ce83504a7bc5da472209b4c70f02829f0b8"
            ],
            "version": "==2.7.3"
        },
        "pytz": {
            "hashes": [
                "sha256:a061aa0a9e06881eb8b3b2b43f05b9439d6583c206d0a6c340ff72a7b6669053",
                "sha256:ffb9ef1de172603304d9d2819af6f5ece76f2e85ec10692a524dd876e72bf277"
            ],
            "version": "==2018.5"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:a68ac5e15e76e7e5dd2b8f94007233e01effe3e50e8daddf69acfd81cb686baf",
                "sha256:b5725a0bd4ba422ab0e66e89e030c806576753ea3ee08554382c14e685d117b5"
            ],
            "version": "==1.23"
        }
    },
    "develop": {}
}

Script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from pyathenajdbc import connect
import pandas as pd


def main():
    conn = connect(s3_staging_dir='s3://BUCKET/path/to/',
                   region_name='us-west-2')
    df = pd.read_sql_query("SELECT TRUE", conn)
    print(df.describe())


if __name__ == '__main__':
    main()

[daniel1608]

Ok, this is strange.
I created a virtual environment with Python 3.6.5 and all packages with the version you said. Nevertheless, I get the same error. It doesn't matter if I use pd.read_sql or pd.read_sql_query.

Here is the error.

log4j:WARN No appenders could be found for logger (com.amazonaws.athena.jdbc.AthenaDriver).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Failed to execute query.
Traceback (most recent call last):
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 181, in execute
    query_execution = self._query_execution()
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 145, in _query_execution
    client = self._athena_client()
  File "MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 140, in _athena_client
    field.setAccessible(True)
jpype._jexception.java.lang.NullPointerExceptionPyRaisable: java.lang.NullPointerException
Traceback (most recent call last):
  File "MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 181, in execute
    query_execution = self._query_execution()
  File "MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 145, in _query_execution
    client = self._athena_client()
  File "MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 140, in _athena_client
    field.setAccessible(True)
jpype._jexception.NullPointerExceptionPyRaisable: java.lang.NullPointerException

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 1378, in execute
    cur.execute(*args)
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/util.py", line 34, in _wrapper
    return wrapped(*args, **kwargs)
  File "/MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/util.py", line 24, in _wrapper
    return wrapped(*args, **kwargs)
  File "/MYVENV/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 195, in execute
    raise_from(DatabaseError(unwrap_exception(e)), e)
  File "MYVENV/lib/python3.6/site-packages/future/utils/__init__.py", line 398, in raise_from
    exec(execstr, myglobals, mylocals)
  File "<string>", line 1, in <module>
pyathenajdbc.error.DatabaseError: java.lang.NullPointerException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 1382, in execute
    self.con.rollback()
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/connection.py", line 158, in rollback
    raise NotSupportedError('Athena JDBC connection is only supported for auto-commit mode.')
pyathenajdbc.error.NotSupportedError: Athena JDBC connection is only supported for auto-commit mode.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 26, in <module>
    main()
  File "test.py", line 21, in main
    df = pd.read_sql("SELECT TRUE", conn)
  File "/MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 381, in read_sql
    chunksize=chunksize)
  File "MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 1413, in read_query
    cursor = self.execute(*args)
  File "MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 1386, in execute
    raise_with_traceback(ex)
  File "MYVENV/lib/python3.6/site-packages/pandas/compat/__init__.py", line 404, in raise_with_traceback
    raise exc.with_traceback(traceback)
  File "MYVENV/lib/python3.6/site-packages/pandas/io/sql.py", line 1382, in execute
    self.con.rollback()
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/connection.py", line 158, in rollback
    raise NotSupportedError('Athena JDBC connection is only supported for auto-commit mode.')
pandas.io.sql.DatabaseError: Execution failed on sql: SELECT TRUE
java.lang.NullPointerException
unable to rollback

A few days ago, everything was fine. Unfortunately, I can't tell if I changed anything that has and impact on that matter.

[laughingman7743]

Traceback (most recent call last):
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 181, in execute
    query_execution = self._query_execution()
  File "MYVENV/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 145, in _query_execution
    client = self._athena_client()
  File "MYVENV/myenv/lib/python3.6/site-packages/pyathenajdbc/cursor.py", line 140, in _athena_client
    field.setAccessible(True)
jpype._jexception.java.lang.NullPointerExceptionPyRaisable: java.lang.NullPointerException

A NullPointerException occurs at the place where query execution information is retrieved. 🤔

[daniel1608]

I just posted the output. Unfortunately, I don't understand in detail what's going on there. But I'll try everything that might solve the problem.

[laughingman7743]

Sorry, I do not know the cause. There is no problem in my environment.
Since an error has occurred while retriving query execution information, perhaps the role authority of the user executing the query may be different.
If you do not have to be JDBC, please try using PyAthena (https://github.com/laughingman7743/PyAthena).

[devinbostIL]

Which JDBC driver version for Athena are you all using?