Conversation
…js into chore/improve-tests
AuHau
left a comment
There was a problem hiding this comment.
I would maybe add a note to README about how tu run the tests locally with the codex-factory. Eq. before running tests spin up Codex cluster with npx codex-factory start --detach latest.
Also, I would be bit cautious about using latest. That runs the latest nim-codex master commit, which might not be really something you want. First of all the stability of the tests might be compromised (eq. you run the tests once and they pass, then you rerun them later on again without any change in codex-js and suddenly they might start failing). Second of all, IMHO it is good idea to target specific version of nim-codex that the library supports and run the tests against that. Then of course upon every new nim-codex release this will need to be updated, but again that is a good thing. You can automatize it with Github Actions, where you can get automatic PR that will update the supported version and thanks to that you will immediately know if there are some problems from the CI tests. I have this setup for example for the Python's bindings library (example).
For this there is IMHO great feature in Codex Factory - External Codex version configuration. Where you can put the version of the nim-codex that the library supports into package.json: engines.codex property which will then get automatically picked up by Codex Factory (you then run only npx codex-factory start --detach without version). And then you can have the automatic PR modifying this property.
| - run: npm ci | ||
|
|
||
| - name: Start codex-factory | ||
| run: npx codex-factory start latest & |
There was a problem hiding this comment.
You can run this with --detach and then you don't need the & and following "Wait for SP" step, because it will automatically "exit" when the cluster is ready (eq. all the nodes are up and running) ;-)
This PR enhances test quality by leveraging
codex-factoryfor integration testing.cc @AuHau