Server re-build with latest AdonisJS framework & Typescript

[cino]

I apologize for the size of this PR but as we discussed earlier on Discord it's impossible to upgrade the dependencies without heavily restructuring the project. For anyone wanting to have a look at the code please open this in an editor or make use of github.dev by pressing the . (dot) on your keyboard.

So for now, this is a WIP Pull request so anyone can contribute and see where we are at the moment.

There are a few rules I've committed to while doing this "v2":

• No breaking changes in the public interface
• Everything in Typescript
• Everything tested

Other improvements:

• All routes should be pointed at controllers, no direct view response / callbacks from the route file
• Never using Env() to retrieve environment variables outside the config files, the config files should be retrieving the environment variables and providing these to the application

Routes:

Dashboard (Conditional, only if enabled)

• Login
• Logout
• Forgot password
• Reset password
• Account page
• Export services/workspaces
• Transfer services/workspaces (importing above export)
• Delete account
• Import endpoint (different from transfer)

API:

• Login @cino
• Signup @cino
• "Me" endpoint
• Service
  • Create
  • Reorder
  • Edit
  • Delete
  • List
  • List own services
  • Icon
• Recipes
  • Search
  • Popular
  • Download :recipe
  • Update recipe
• Workspaces
  • Edit
  • Delete
  • Create
  • List
  • Reorder WIP: Add ability to reorder Workspaces using drag and drop ferdium-app#497
• Static: features endpoint
• Static: services endpoint
• Static: news endpoint
• Static: payment plans endpoint
• Static announcements endpoint

GitHub

• Add linter/tests in Github action

General:

• Copy docker building from the original branch into this one
• ??

This also closes #70

[mcmxcdev]

Hey @cino, I had some available time and pushed 4 new commits to the server overhaul.

All dependencies are up-to-date now, no more CVEs are reported in prod code, no TS/ESLint issues and made sure that all adonis commands work as expected.

I think this migration is moving to be in a great state, but ofc most of the routes have to be rewritten still.

Let's see, maybe some other Ferdium contributors want to jump in and help as well, this is probably one of the biggest tech debt we have to solve.

[SpecialAro]

Hi all!

I've been trying to learn a bit of AdonisJS so I can push this PR forward. So far I was able to update the following endpoints:

• Signup
• Login
• Me (GET and PUT)

Tested all 3 with Ferdium and they were all looking fine. Let me know what you think (pushing bellow).

Important security note: I'm using the API token guard of adonis auth for handling API authentication, which is different from the JWT we used to have (now it is not supported natively by adonis). I've set an expiration of 7 days, but let me know what you think about this. For this to work I had to add a new migration to introduce two more columns in the Tokens table.

Tomorrow, hopefully, I'll start with the Service or Workspace endpoints x)

@cino you were tagged on the Login and Signup endpoints - if you could review my work I would really apreciate it :D Thank you!

[SpecialAro]

Hey again!

I just refactored the code to use JWT (so that we don't have to change anything on the Ferdium Client to accomodate that change - that is something we can do in the future, if needed).

I've also finished porting all the API endpoints!! 😄 But I'll be glad if someone could help me test them so that we can validate what I've done. The majority of them are working properly (tested with Ferdium Client), but some specific ones I havent tested (check the TODOS in the files)

@ferdium/reviewers I'm pinging everyone in hope that someone can review my code, given that this substantialy pushes this PR forward.

@cino, sorry to tag you again, but I'm trying to test the web endpoints and I don't seem to be able to past the initial page (no access to login, or dashboad). All I get is:

... or if I click on the button to go to dashboard:

[mcmxcdev]

Why did you add these new fields? How are they going to be backwards compatible?

[SpecialAro]

That is a great question. I'm not entirely sure if it is ok to add them like so or if we need to provide a @default value. I've never used Adonis before so I'm not sure how to handle it.

If you test the auth jwt or even the api you will noticed that this 2 fields were added in the Adonis upgrade.

Generally, we have a bigger problem than this... I've tried to test the db in production and it fails to even initiate migrations (even without this new migration I added). We will probably need to code a script to migrate old databases to the new database schema manually

[mcmxcdev]

This is the same hardcoded id for every user?

[SpecialAro]

Copy-pasted from the previous code.... It was strange to me as well xD

[mcmxcdev]

We might want to drop these fields since Ferdium is fully OSS, but probably better to split it off in a PR that would happen after this one.

[SpecialAro]

I think this is here to circunvent the fact that we are OSS. Forcing features that depend on those fields to be set to true on the Ferdium-App

[mcmxcdev]

Same here

[SpecialAro]

Yep, don't know why xD

[mcmxcdev]

Does this code even make sense? It is querying for a service but then the result of the call is not assigned to any variable?

[SpecialAro]

This is inside a while block. so it will run while it is > 0 (don't like it either but lets make improvements after we merge this to codebase)

[mcmxcdev]

Feel free to disable that lint rule directly in eslint config, I always do that for all projects where I use eslint-plugin-unicorn, guess I forgot here when I implemented it.

[mcmxcdev]

Probably more descriptive as "Sucessfully deleted service: xyz"

[mcmxcdev]

Is there a less verbose way than specifying the JWT keys like this?

[SpecialAro]

I don't think there is... this was automatically created by running the install commands on adonis-jwt 😞

[mcmxcdev]

https://github.com/maxgalbu/adonis5-jwt hasn't had any commits for a year now. which means we will pull in some outdated dependencies eventually through them.

But I guess there is no alternative.

[SpecialAro]

yep... I dont think we do... The only alternative would be to refactor both Ferdium-App and Ferdium-Server afterwards - which would result in Breaking Changes on both sides for users that haven't been updating the client (they would be forced to update unless we keep a v1 API that works with jwt middleware).

[mcmxcdev]

"Created new recipe with id abc and name xyz"

[SpecialAro]

Been doing some more work in this. I think we are close to finish.

Things that are missing:

• In the data.edge file it seems like @each(service in services) and the same logic for workspaces is not working (services and workspaces' info are not shown). I would gladly accept help on this point!
• The import and export data is currently failing (this is also a issue in the current server, so we should try to fix it in this new version) - the exported data is not being parsed correctly (too many JSON.parse and JSON.stringify)
• I think all endpoints are working (API at least). But it would be great if someone other than me could test them out, please.
• We really need to be careful on how we are going to migrate the database in production. It seems that it doesn't recognized the already applied migrations... I dont think I did anything wrong because node ace migrate:run should work but it fails. This implies that we are probably going to need a script that runs a migration "manually" to update from adonisjs 4 to 5. Just made a script that solves this migration 😄
• Tests are still missing. But maybe we can add them after we merge this to codebase, wdyt? - I'm not saying they are not important, but we should really try to push this forward as soon as possible (it has been stalled from long enough)
• Lint all the files. Typescript is failing in plenty of the files even though the js code is working just fine. We should try to fix it before merging - or if we can't, this could also be done after.

These are my takes. Please let me know what you guys think 😄

[mcmxcdev]

ferdium/ferdium-recipes@cdafac1 actually uses newer versions than this since recently

[SpecialAro]

damn xD I'll fix it before merging

[mcmxcdev]

I usually take the TS error message and put it there like // @ts-expect-error foo to make it more self-explanatory what the problem is.

[mcmxcdev]

Don't we have some Recipe to use for this instead of any?

[SpecialAro]

Yep, but it isn't working (types are not correct) - and I'm not in the "mood" of fixing it.... I just want to close this PR as soon as possible to work on improvements later (mainly lint, types and tests). Don't know what you think about that?

[mcmxcdev]

Amazing work! 🎉

I noticed a couple of improvement possiblities:

• pnpm reformat-files produces 10 files diff when run with latest commit which it shouldn't
  • make sure to disable the lint rules which conflict with prettier formatting
• pnpm start always errored with "missing env value HOST", although I had HOST defined inside .env
• pnpm test gave me failed tests for various files
  • add the pnpm test step to the CI so we avoid test failures in the future

[SpecialAro]

Amazing work! 🎉

I noticed a couple of improvement possiblities:

• pnpm reformat-files produces 10 files diff when run with latest commit which it shouldn't

  • make sure to disable the lint rules which conflict with prettier formatting

• pnpm start always errored with "missing env value HOST", although I had HOST defined inside .env

• pnpm test gave me failed tests for various files

  • add the pnpm test step to the CI so we avoid test failures in the future

Thank you for your comment and review @mcmxcdev. Indeed I haven't focused too much on tests up until now.

I've pushed some commits that fix the majority of the fails but I still have to fix the tests with the transfer endpoint because it seems that the endpoint data is being seen as body json (if you look at the controller) and not so much as file - so those tests are not up to date. I'm commenting them out for now if you agree (later we can improve tests)