Airflow Scheduler Getting fail #31090
Description
Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
Airflow Scheduler is getting failed randomly and after checking the server I have got this issue. We had Reset and initialize the Database but still same error. Can anyone help us out in this error?
[2023-05-05 12:23:18,597] {__init__.py:56} WARNING - Failed operation _get_task_meta_for. Retrying 2 more times.
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.ProtocolViolation: invalid string in message
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/celery/backends/database/__init__.py", line 51, in _inner
return fun(*args, **kwargs)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/celery/backends/database/__init__.py", line 153, in _get_task_meta_for
task = list(session.query(self.task_cls).filter(self.task_cls.task_id == task_id))
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3389, in __iter__
return self._execute_and_instances(context)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3414, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 982, in execute
return meth(self, multiparams, params)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
distilled_params,
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
e, statement, parameters, cursor, context
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "/home/ubuntu/anaconda3/envs/alphavu/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (psycopg2.errors.ProtocolViolation) invalid string in message
[SQL: SELECT celery_taskmeta.id AS celery_taskmeta_id, celery_taskmeta.task_id AS celery_taskmeta_task_id, celery_taskmeta.status AS celery_taskmeta_status, celery_taskmeta.result AS celery_taskmeta_result, celery_taskmeta.date_done AS celery_taskmeta_date_done, celery_taskmeta.traceback AS celery_taskmeta_traceback
FROM celery_taskmeta
WHERE celery_taskmeta.task_id = %(task_id_1)s]
[parameters: {'task_id_1': '2652a5ec-f13f-455b-9aab-0e98698b75c6'}]
(Background on this error at: http://sqlalche.me/e/4xp6)
What you think should happen instead
No response
How to reproduce
We had tried resolving it through resetting and initializing database but still could not found anything.
Operating System
Ubuntu 15.04
Versions of Apache Airflow Providers
No response
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct