Evaluate Apache AGE vs Neo4J for graph db.

[dhirenmathur]

Reevaluate graph database choice: Neo4j vs Apache AGE

This issue aims to discuss whether we should stick with Neo4j or switch to Apache AGE.

Current Situation

1. Initially used SQLite wrapper as a graph database
   • Worked reliably, easy to debug, no learning curve
   • Limitation: Not distributed
2. Switched to Neo4j for distribution capabilities
   • Experiencing random connection errors in logs

Project Requirements

• Track relationships
• Fetch immediate neighbors recursively
• Distributed system
• No expectation of significantly more complex graph use cases

Options to Consider

1. Continue with Neo4j
   Pros:

   • Already implemented, fully featured
   • Mature graph database with extensive ecosystem
     Cons:
   • Ongoing connection issues
   • Unfamiliarity

2. Migrate to Apache AGE
   Pros:

   • Postgres-based (leverages familiar SQL environment)
   • Supports both Cypher and SQL queries
   • Allows integration of existing Postgres data with graph data
     Cons:
   • Requires migration
   • Less mature than Neo4j as a graph db (but backed by Apache)

Questions to Address

• Can we resolve Neo4j connection issues with reasonable effort?
• Is Apache AGE capable of handling our current and foreseeable needs?
• How much would we benefit from AGE's SQL integration capabilities?
• What's the estimated effort and risk for migrating to AGE?
• I don't want to migrate yet another database, can we use our existing Postgres installation with AGE? That would be a big factor.

Next Steps

• Investigate root cause of Neo4j connection issues
• Set up a test environment for Apache AGE and explore migration from Neo4j
• Evaluate performance of both systems with our specific use cases
• Assess team familiarity with Postgres and potential learning curve for AGE

Please share your thoughts, experiences, and concerns regarding these options. Consider how our existing Postgres knowledge might influence this decision.