DependencyConflict: requested: "psycopg2 >= 2.7.3.1" but found: "None"

[eugeniyost]

After upgrade opentelemetry-instrumentation-psycopg2 from 0.19b0 to 0.23b2 there was an message DependencyConflict: requested: "psycopg2 >= 2.7.3.1" but found: "None", my project dependencies do not have psycopg2, there is psycopg2-binary.

Describe your environment

• python 3.9.6
• django 3.2.5
• uwsgi 2.0.19.1
• psycopg2-binary 2.9.1
• opentelemetry-instrumentation-psycopg2 0.23b2

Steps to reproduce
install psycopg2-binary to project dependencies and opentelemetry-instrumentation-psycopg2 0.23b2, run django server using uwsgi

What is the expected behavior?
no DependencyConflict message

What is the actual behavior?
message DependencyConflict: requested: "psycopg2 >= 2.7.3.1" but found: "None"

[Dipenbhatt03]

Did you get any solution, i am trying to setup this whole distributed tracing and stuck at instrumenting psycopg2_binary.

[owais]

psycopg project recommends using psycopg2 in production instead of psycopg binary so we preferred to use that as the dependency.

https://www.psycopg.org/docs/install.html#psycopg-vs-psycopg-binary

For production use you are advised to use the source distribution.

If there are use cases where one might want to use the binary version, may be we can add both deps and update our dep resolution system treat such cases as OR so that presence of either library resolves the requirement.

[justmobilize]

We also use psycopg-binary due to how our app is built and deployed. We are trying to figure out if we need to fork or come up with another option.

Although harder for you to maintain, instead of updating your dep resolution system, you could duplicate opentelemetry-instrumentation-psycopg2 and make opentelemetry-instrumentation-psycopg2-binary (or at least for a short term solution)

[justmobilize]

Or something like this:

def get_dependency_conflicts(
    deps: Collection[str],
) -> Optional[DependencyConflict]:
    error = None
    for dep in deps:
        dep_options = dep.split("|")
        for dep_option in dep_options:
            try:
                error = None
                get_distribution(dep_option)
                break
            except VersionConflict as exc:
                error = DependencyConflict(dep, exc.dist)
            except DistributionNotFound:
                error = DependencyConflict(dep)
            except RequirementParseError as exc:
                logger.warning(
                    'error parsing dependency, reporting as a conflict: "%s" - %s',
                    dep,
                    exc,
                )
                error = DependencyConflict(dep)
        if error:
            break
    return error

[justmobilize]

@owais can I help open a pull request for one of these options?

[owais]

That would be great but please summarize your solution here first. My only concern is if we'll ever need to support the AND case in addition to OR and if we should account for that today.

[justmobilize]

@owais isn't AND already supported with:

_instruments = ("package1=1.0","package2=1.0")

in which case:

_instruments = ("package1=1.0","package2=1.0|package2-binary=1.0")

Would work with the above option.

If I'm missing something (which I may be since I'm not as familiar with all the instrumentation yet). Please let me know and I'll write up a solution

[phillipuniverse]

Here is an extremely disgusting workaround that I'm using in the meantime. Caveat - this only really works because I'm doing manual "auto-instrumentation" which you're not supposed to do. But it does work!

import opentelemetry

orig_get_dependency_conflicts = opentelemetry.instrumentation.dependencies.get_dependency_conflicts

def psycopg2_or_psycopg2_binary_dependency_conficts(
    deps: typing.Collection[str],
) -> typing.Optional[DependencyConflict]:
    if 'psycopg2>=2.7.3.1' in deps:
        conflict = orig_get_dependency_conflicts(deps)
        if conflict and not conflict.found:
            return orig_get_dependency_conflicts(['psycopg2-binary>=2.7.3.1'])
    return orig_get_dependency_conflicts(deps)

opentelemetry.instrumentation.dependencies.get_dependency_conflicts = psycopg2_or_psycopg2_binary_dependency_conficts
# This import kicks off all of the autoinstrumentation of OpenTelemetry
import opentelemetry.instrumentation.auto_instrumentation.sitecustomize as autotrace

[owais]

@justmobilize "package2=1.0|package2-binary=1.0" < this is not something supported by pip out of the box right? So you are proposing a syntax to denote OR? That could work but I feel if one string/element always represented a single package, it'd be so much easier to work with. How about this:

_instruments = Collection[Collection[Package...]...]

Inner tuples would be AND'ed while outer ones would be OR'ed. For example:

_instruments = (('a', 'b',), ('c',))

would mean the instrumentation will be applied either when both a and b are found or when c is found. So psycopg use case would look like

_instruments = (('psycopg2'), ('pyscopg-binary'),)

[justmobilize]

@owais I can do that. Just the guidance I needed. Should I follow SOP and join the Slack channel and find a OpenTelemetry buddy?

[owais]

@justmobilize no, you don't have to. You can just send the PR and discuss everything on Github. You're welcome to join the slack though :)