NoSQL database interface

[rdboyes]

I have to work with NoSQL databases pretty often (specifically Firestore), and I would love to have a better julia interface to do so. I don't know how much of TidierDB can be translated over since the data storage model is so different from the database backends that it currently supports.

If I can make something like this, should it be part of TidierDB (since it still is a database)? Or should it be its own package?

[kdpsingh]

The main thing that TidierDB does is generate the SQL query - it doesnt actually handle much of the backend stuff other than handling slight variations in the SQL generation.

Do NoSQL database queries look anything like SQL queries? From quick googling it looks like NoSQL queries are JavaScript converted into web API calls?

[drizk1]

I'm unfamiliar with NoSQL, so i used GPT4 to spin up quick comparison in syntax below. I think (assuming this is accurate) retrofitting the existing TidierDB macros to work for NoSQL might be tricky. TidierDB uses interchangeable parsers to generate the different sqls, but there is still some generation that happens in macro for the parts that are similar across them all. I don't think it would be impossible

SELECT b.title, b.publication_year
FROM books b
JOIN authors a ON b.author_id = a.id
WHERE a.name = 'John Doe' AND b.publication_year > 2000
ORDER BY b.publication_year DESC;

vs

db.books.aggregate([
  { $lookup: {
      from: "authors",
      localField: "author_id",
      foreignField: "_id",
      as: "author_info"
    }
  },
  { $match: {
      "author_info.name": "John Doe",
      "publication_year": { $gt: 2000 }
    }
  },
  { $sort: { "publication_year": -1 } },
  { $project: { "title": 1, "publication_year": 1, "author_info.name": 1 } }
]);

[rdboyes]

Took a closer look through the TidierDB code - I'm thinking that integrating the two will be difficult. Firebase essentially stores "JSON objects" rather than tables like a relational database would, so I'm thinking the package will be closely linked with the new TidierIteration package. I'm going to start experimenting with some stuff on a personal repo, and I can transfer it over if/when it becomes useful

[drizk1]

@rdboyes This a random thought 11 mo later.. I was peeking at Firestone and mongodb.. both allow sql queries.

So in theory, if we set up a connection via http or odbc or anything really, TidierDB could query them with sql.