Road to v2

[czosel]

With our current architecture, the features we can implement in TT-mobile are quite limited by what Click-TT offers (because every request maps 1:1 to a request to Click-TT. In order to implement new, interesting ways to combine data, we want to build our own, complete database. The following pictures illustrate the idea:

Architecture overview compared:

source: Miro

Flowchart:

source: Hackmd

ERD:

source: Hackmd

Some considerations that lead to this design:

• The API should strike a good balance between performance and up-to-date results. We achieve this by first returning the content of our own database (very fast) and simultaneously scraping Click-TT and pushing updates (if we find any) through a GraphQL subscription (via Websocket) to the client (see https://www.graphile.org/postgraphile/subscriptions/). The fact that GraphQL supports subscriptions is the reason that I think we should choose it over a REST API for our own client (we could still offer an additional REST API for external users in the future if there is demand).
• GraphQL APIs can be tough to implement, especially in a performant way. We just use PostGraphile, because we don't have any requirements related to data visibility or mutation permissions. PostGraphile should offer great performance and enough extensibility to cover our use-cases - and we don't have to write any resolvers!
• We need a proper ORM to manage the database, since we'll have a few models and relationships. I'd suggest Prisma, because it seems to offer all we need with a nice API, and also supports DB migrations.
• The ERD should reflect Click-TT's data model - sticking to that probably helps keeping the import code simple.

Steps:

• Set up Postgres DB, Prisma ORM, configure data model, write scraping results to the database (WIP PR: WIP: add postgres DB, prisma ORM #155)
• Set up GraphQL API based on PostGraphile (read-only)
• Trigger Scraper when specific GraphQL resolvers are called (via WrapResolversPlugin)
• Wire up client with new API by integrating a GraphQL client (maybe Apollo, or Relay?)

It would be awesome if TT-mobile 2.0 would be a community effort, so if you're interested to contribute, please reach out!
@tfluehmann