[Feature]: Enhance observability for a developer

[damirkrstanovic]

Problem Statement

Let’s say you are a developer using cognee to build your own solution, like we are doing. Due to the complexity of the solution, we find that it can get difficult to debug a solution using cognee, especially related to the performance of the system.

When we implemented the solution, we wanted to understand why does a specific part of the pipeline or a query take so much time and where this time is spent. We are interested not only in query latency, but quality of the output.

It is not obvious how to get this information from cognee - there are options, but they either seem clunky or require integration with additional systems (like langfuse).

What we wanted to understand in detail:

1. When I type cognee.cognify(), what actually happens? Same for cognee.search("<whatever>").
2. Which queries are issued to the graph database and how much time they take - it can get non-obvious.
3. Which inputs are sent to LLMs, which outputs are received back.
4. How much time does it take to talk to graph database or the LLM.

In general - the problem is how to trace the execution of a particular query (we were more interested in queries than in pipelines).

Proposed Solution

Some ideas:

1. Have a better debug mode, where you would be able to see exactly what a certain query did and be able to trace the inputs and outputs, including especially graph queries and LLM inputs and outputs. Perhaps some sort of "profiling" mode instead of debug mode.
2. Have standard metric pipeline for shipping observability data via open telemetry or even just have basic prometheus metrics out of the box would be helpful. We can instrument our own code, but we cannot get insights into cognee internals.

Alternatives Considered

Which options are available currently in cognee and what we tried:

• Using the debugger - trace the execution, get what you need. This can get tiresome and is not viable for tracing the thing later.
• Using debug logs - you can get really in-depth data using debug logs, but it is difficult to get the debug level right and logs are positively overwhelming. For example, we never had any reason to look at relational database performance, but debug logs are full of stuff like:

    executing functools.partial(<built-in method close of sqlite3.Cursor object at 0x11998b4c0>)
    operation functools.partial(<built-in method close of sqlite3.Cursor object at 0x11998b4c0>) completed
    executing functools.partial(<built-in method commit of sqlite3.Connection object at 0x1195655d0>)
    operation functools.partial(<built-in method commit of sqlite3.Connection object at 0x1195655d0>) completed
    executing functools.partial(<built-in method rollback of sqlite3.Connection object at 0x1195655d0>)
    operation functools.partial(<built-in method rollback of sqlite3.Connection object at 0x1195655d0>) completed
    executing functools.partial(<built-in method close of sqlite3.Connection object at 0x1195655d0>)
    operation functools.partial(<built-in method close of sqlite3.Connection object at 0x1195655d0>) completed
    executing <function Connection.stop.<locals>.close_and_stop at 0x11965e3e0>
    operation <function Connection.stop.<locals>.close_and_stop at 0x11965e3e0> completed

Debug logs also contain secret data, like LLM provider API keys that need to be purged before giving them to someone else. Very often, debug logs do not have time and it is difficult to find out how much time certain operation took.

It is not viable to use debug logs in production - this feedback is written primarily from the PoV of a developer developing software, but also it may be important to get some debug data for issue reproduction.

We could find LLM inputs and outputs, but not exact graph queries and not how much time it took to perform LLM call or graph query. For example, we wanted to understand which models to use for balancing performance and cost.

• Using cognee integrations with providers such as langfuse - https://docs.cognee.ai/integrations/langfuse-integration. We didn’t try this one, as we are not using langfuse currently.

Use Case

Our use case is using cognee as a library for the memory layer in our product. During development it is important to get performance data and exact inputs and outputs of certain layers - most important for us is the query layer.
For example, we noted that our graph queries are slow, but without the exact graph query it is not possible to tweak the database (add indices and what not) to improve performance.

Implementation Ideas

No response

Additional Context

No response

Pre-submission Checklist

• I have searched existing issues to ensure this feature hasn't been requested already
• I have provided a clear problem statement and proposed solution
• I have described my specific use case

[github-actions]

Hello @damirkrstanovic, thank you for your interest in our work!

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.