-
Notifications
You must be signed in to change notification settings - Fork 2k
ci: enable new CI for vision package #3968
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ghost
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi team,
I'm currently reviewing this pull request and will provide my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed:
This pull request, authored by briandorsey, aims to enable a new CI configuration for the vision package. The changes are primarily focused on CI configuration and don't modify the tests themselves.
Here's a breakdown of the changes:
- Added
visionpackage to CI configuration:- The
visionpackage was added to the list of packages included in the Node.js development (nodejs-dev.jsonc) CI configuration (line 201). This ensures that thevisionpackage's tests are included in the development CI pipeline. - Conversely, the
visionpackage was removed from the Node.js production (nodejs-prod.jsonc) CI configuration (line 111). This suggests that the production CI pipeline will not include tests for thevisionpackage. This might be intentional, but it warrants a closer look.
- The
- Created
ci-setup.jsonfile: A new file,vision/ci-setup.json, was created (it's currently empty). This file likely serves as a configuration file for the new CI setup for thevisionpackage. It's worth investigating why this file is empty at this stage.
Potential Issues to Consider:
- Empty
ci-setup.json: The purpose of the emptyvision/ci-setup.jsonfile needs clarification. Is this a placeholder, or is there a reason for it being empty? - Production CI Exclusion: The removal of
visionfrom the production CI configuration should be reviewed to ensure it's intentional and doesn't introduce any risks.
I'll provide a more detailed review shortly, but this summary should give everyone a good starting point.
And now, a little haiku to brighten your day:
Code flows like a stream,
Tests run, a green light appears,
Vision takes its flight.
ghost
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This pull request enables a new CI for the vision package by adding a ci-setup.json file and updating the CI configuration files. The changes are minimal and straightforward. However, the addition of an empty ci-setup.json file raises questions about its purpose and potential future content. Additionally, there are opportunities to improve clarity and consistency in the configuration files.
I have referenced the following elements from the provided style guide:
- File naming:
ci-setup.jsonfollows the kebab-case convention used in other configuration files likenodejs-dev.jsoncandnodejs-prod.jsonc. - Configuration files: The use of JSON or JSONC for configuration files is consistent with the project's existing practice.
c24f089 to
957b956
Compare
957b956 to
d9db94d
Compare
0394a61 to
58571e5
Compare
vision/system-test/detect.test.js
Outdated
| // something changed in the output and this text is no longer in the output | ||
| // however, the API call itself still works, commenting this out so | ||
| // it's not breaking the tests. | ||
| // assert.match(output, /Google/); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test checks the output of the call node detect.js logos resources/google.png, which runs detectLogos, and writes results to console that this then checks (execSync returns stdout from the command).
The problem here is that one of the failing test executions shows that the output is just "Logos:\n". Which I can replicate on my local machine:
$ node detect.js logos resources/google.png
Logos:
$
The test will always return the "Logos:" string, but then loops around the results to return information. Since we're capturing stdout we can't easily get a count of the loops, but we could confirm that there is a description, rather than the contents of the description (description: appears in the output if there is a match).
Checking the other skipped tests, I am able to get consistent successes with the Landmarks detection, but I cannot get the Logo detection to work.
Looking further, it looks like the test data used in this test differs from the other repos (this repo uses "google.png", a transparent PNG, other repos use "logos.png", an upscaled logo on white background). But the existing resource "logos.png" in this repo, which uses the old logo, is detected correctly.
I would suggest changing the two logos tests to check that the literal string "description:" is in the output rather than the contents, and change the checked file to files[2].name (which is logos.png; while different to other repos logos.png, appears to succeed the logo identification test).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this detailed investigation. Followed this and the suggestions in next commit.
vision/system-test/detect.test.js
Outdated
| // something changed in the output and this text is no longer in the output | ||
| // however, the API call itself still works, commenting this out so | ||
| // it's not breaking the tests. | ||
| // assert.match(output, /Google/); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // something changed in the output and this text is no longer in the output | |
| // however, the API call itself still works, commenting this out so | |
| // it's not breaking the tests. | |
| // assert.match(output, /Google/); | |
| // confirm there is output with a description, but not necessarily an exact value match | |
| assert.match(output, /description:/); |
vision/system-test/detect.test.js
Outdated
| // something changed in the output and this text is no longer in the output | ||
| // however, the API call itself still works, commenting this out so | ||
| // it's not breaking the tests. | ||
| // assert.match(output, /Palace of Fine Arts/i); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given below megacomment, change the check to confirm there is output, rather than the exact output.
| // something changed in the output and this text is no longer in the output | |
| // however, the API call itself still works, commenting this out so | |
| // it's not breaking the tests. | |
| // assert.match(output, /Palace of Fine Arts/i); | |
| // FLAKY: confirm there is output, if not an exact match on output | |
| assert.match(output, /description:/i); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
vision/system-test/detect.test.js
Outdated
| // something changed in the output and this text is no longer in the output | ||
| // however, the API call itself still works, commenting this out so | ||
| // it's not breaking the tests. | ||
| // assert.match(output, /Palace of Fine Arts/i); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| const stdout = execSync('node quickstart.js'); | ||
| assert.match(stdout, /Labels:/); | ||
| assert.match(stdout, /cat/); | ||
| assert.match(stdout.toLowerCase(), /cat/i); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
don't need both .toLowerCase() and /i (presuming that's a case-insensitive regex i)
| assert.match(stdout.toLowerCase(), /cat/i); | |
| assert.match(stdout, /cat/i); |
Description
Batch CI related config change. Adding ci-setup.json to support running these tests under the new CI runner. No changes to tests themselves.