Skip to content

ingestion ometa client 不能处理text/plain格式的返回值 #24969

@GuoMaoFei

Description

@GuoMaoFei

Affected module
Ingest metadata
Describe the bug
当连接器链接的service中,登录任务返回的是text/plain格式的数据,client将会返回null与连接器链接失败

To Reproduce

Screenshots or steps to reproduce
def _one_request(self, method: str, url: URL, opts: dict, retry: int):
"""
Perform one request, possibly raising RetryException in the case
the response is 429. Otherwise, if error text contain "code" string,
then it decodes to json object and returns APIError.
Returns the body json in the 200 status.
"""
retry_codes = self._retry_codes
limit_codes = self._limit_codes
try:
resp = self._session.request(method, url, **opts)
resp.raise_for_status()

        if resp.text != "":
            try:
                return resp.json()
            except JSONDecodeError as json_decode_error:
                logger.error(
                    f"Json decoding error while returning response {resp} in json format - {json_decode_error}."
                    f"The Response still returned to be handled by client..."
                )
                return resp
            except Exception as exc:
                logger.debug(traceback.format_exc())
                logger.warning(
                    f"Unexpected error while returning response {resp} in json format - {exc}"
                )

    except HTTPError as http_error:
        # retry if we hit Rate Limit
        if resp.status_code in retry_codes and retry > 0:
            raise RetryException() from http_error
        if resp.status_code in limit_codes:
            raise LimitsException() from http_error
        if "code" in resp.text:
            error = resp.json()
            if "code" in error:
                raise APIError(error, http_error) from http_error
        else:
            raise
    except requests.ConnectionError as conn:
        # Trying to solve https://github.com/psf/requests/issues/4664
        try:
            return self._session.request(method, url, **opts).json()
        except Exception as exc:
            logger.debug(traceback.format_exc())
            logger.warning(
                f"Unexpected error while retrying after a connection error - {exc}"
            )
            raise conn
    except Exception as exc:
        logger.debug(traceback.format_exc())
        logger.warning(
            f"Unexpected error calling [{url}] with method [{method}]: {exc}"
        )

    return None

这个代码导致了返回空,因为resp.json()解析失败,报错不为json_decode_error

Expected behavior
报错的时候将resp返回,或者添加对txt/plain非json返回值格式的判断

Version:

  • OS: [e.g. iOS]
  • Python version: 3.11
  • OpenMetadata version: [e.g. 1.10.12]
  • OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
链接apache nifi 2.6.0的时候出现这个问题

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions