Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Greptile SummaryThis PR completes the migration from Key observations:
Confidence Score: 4/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["python -m nose2\n(python_new_invoke_test)"] --> B["nose2 loads plugins from unittest.cfg"]
B --> C["nose2.plugins.attrib\n(AttributeSelector)"]
B --> D["nose2_attrib_generators\n(AttributeGeneratorFilter)\nalwaysOn=True"]
B --> E["nose2.plugins.collect\nnose2.plugins.printhooks"]
D --> F["handleArgs()\n→ _patch_generator_plugin()"]
F --> G["Find Generators plugin\nin session.plugins"]
G --> H["Monkey-patch\nGenerators._testsFromGeneratorFunc"]
H --> I{"-A attribute filter?"}
I -- "No filter" --> J["Call original\n_testsFromGeneratorFunc"]
I -- "Filter present" --> K["_matches_attrib_filter(obj)\nchecks func attributes"]
K -- "Match" --> J
K -- "No match" --> L["return [] ← skip generator,\nno import/execution"]
J --> M["Generator function called\n→ yields test cases\n(via nose2 params/cartesian_params)"]
C --> N["Per-test method:\nAttributeSelector filters\nby -A expression"]
N --> O["Run test method\n(with setUp/tearDown)"]
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional Comments (1)
-
dali/test/python/nose_utils.py, line 129-130 (link)style: creating a new
TestCaseinstance for each assertion call is unconventionalConsider creating a module-level instance once:
Then define
tcat module level (after imports) and reuse it in bothassert_raisesandassert_warnsNote: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
25 files reviewed, 1 comment
There was a problem hiding this comment.
Additional Comments (2)
-
dali/test/python/test_functional_api.py, line 22 (link)logic: This file still imports
nosedirectly and usesnose.tools.eq_on line 188. Since nose is being removed as a dependency, this will cause an import error. -
dali/test/python/test_functional_api.py, line 188 (link)logic: Replace
nose.tools.eq_with standard assertion or useassert_equalsfromnose_utilsNote: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
28 files reviewed, 2 comments
There was a problem hiding this comment.
Additional Comments (1)
-
dali/test/python/nose_utils.py, line 68 (link)syntax: missing
selfparameter - instance methods must haveselfas first parameter
29 files reviewed, 1 comment
|
!build |
|
CI MESSAGE: [40627092]: BUILD STARTED |
|
CI MESSAGE: [40627092]: BUILD FAILED |
|
!build |
|
CI MESSAGE: [40628704]: BUILD STARTED |
There was a problem hiding this comment.
Additional Comments (1)
-
dali/test/python/nose_utils.py, line 83-85 (link)style: Consider using the module-level
_test_caseinstance instead of creating a newempty_case()for each assertion:This is more efficient and aligns with how
assert_raisesandassert_warnsare implemented.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
29 files reviewed, 1 comment
|
Do we need to touch |
|
CI MESSAGE: [40628704]: BUILD FAILED |
mzient
left a comment
mzient
left a comment
There was a problem hiding this comment.
The way the tests are first discovered and then run will most certainly prevent parallel external source from working (see test job 247483858) was the primary reason for not pursuing full transition to nose2 to date.
Perhaps you can move the troublesome tests to separate files to guarantee proper execution order.
Also, some of the tests used to generate vast amounts of data in the "discovery" stage, because yielding in nose didn't cause data accumulation. Please make sure that this is not the case any more - the tests might work when run one by one but case out-of-memory condition when multiple test files are run.
There was a problem hiding this comment.
Additional Comments (1)
-
dali/test/python/nose_utils.py, line 74 (link)syntax: Missing
selfparameter in method definition
33 files reviewed, 1 comment
Greptile's behavior is changing!From now on, if a review finishes with no comments, we will not post an additional "statistics" comment to confirm that our review found nothing to comment on. However, you can confirm that we reviewed your changes in the status check section. This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR". |
Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
| pass | ||
|
|
||
|
|
||
| # Module-level TestCase instance for assertions |
There was a problem hiding this comment.
with_setup no longer exported — silent breakage risk
The old nose_utils.py exported with_setup via from nose import SkipTest, with_setup. The new file no longer provides or exports with_setup. All call-sites in the PR have been migrated, but any out-of-tree test file or future code that still does from nose_utils import with_setup will get an ImportError at import time.
If a backward-compatible shim is desirable (the PR description mentions "a backward compatible decorator is added"), adding even a simple one would reduce breakage risk:
def with_setup(setup=None, teardown=None):
"""Backward-compatible shim – prefer class-level setUp/tearDown instead."""
import warnings, functools
warnings.warn("with_setup is deprecated; use setUp/tearDown in a test class", DeprecationWarning)
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
if setup:
setup()
try:
return func(*args, **kwargs)
finally:
if teardown:
teardown()
return wrapper
return decorator
There was a problem hiding this comment.
The point is to move away from nose2 for good without providing backward compatibility.
|
!build |
|
CI MESSAGE: [45742567]: BUILD STARTED |
|
CI MESSAGE: [45742567]: BUILD FAILED |
|
|
||
| test_pytorch() { | ||
| ${python_invoke_test} --attr '!slow,pytorch' test_dali_variable_batch_size.py | ||
| ${python_new_invoke_test} --attr '!slow,pytorch' test_dali_variable_batch_size.py |
There was a problem hiding this comment.
This line was not fully converted from the old nose invocation style to nose2. It has two problems:
-
--attris not a nose2 flag — nose2 uses-Afor attribute filtering. With nose2,--attris an unknown argument and will likely either error out or silently skip the attribute filtering entirely, causing all tests (including slow/non-pytorch ones) to run without filtering. -
.pyextension in the module path — nose2 takes Python module names (dot-separated), not file names. All other migrated calls in this script already use${test_script%.py}or bare module names (e.g., line 33).
| ${python_new_invoke_test} --attr '!slow,pytorch' test_dali_variable_batch_size.py | |
| ${python_new_invoke_test} -A '!slow,pytorch' test_dali_variable_batch_size |
|
!build |
|
CI MESSAGE: [45743750]: BUILD STARTED |
|
CI MESSAGE: [45743750]: BUILD FAILED |
| expected_out, | ||
| ) | ||
|
|
||
| def test_numba_func_with_cond(self): |
There was a problem hiding this comment.
Missing @attr("sanitizer_skip") decorator
test_numba_func_with_cond is missing the @attr("sanitizer_skip") decorator that was present on the original standalone function. Without this attribute, the test will not be excluded when running with sanitizers enabled (via the -A '!sanitizer_skip' filter used in CI), causing potential test failures or false positives in sanitizer runs.
| def test_numba_func_with_cond(self): | |
| @attr("sanitizer_skip") | |
| def test_numba_func_with_cond(self): |
| expected_out=[np.full((10, 10, 10), 255, dtype=np.uint8)], | ||
| enable_conditionals=True, | ||
| ) | ||
| def test_numba_func_with_cond_do_not_convert(self): |
There was a problem hiding this comment.
Missing @attr("sanitizer_skip") decorator
test_numba_func_with_cond_do_not_convert is missing the @attr("sanitizer_skip") decorator that was present on the original standalone function. Without this attribute, the test will not be excluded when running with sanitizers enabled (via the -A '!sanitizer_skip' filter used in CI), causing potential test failures or false positives in sanitizer runs.
| def test_numba_func_with_cond_do_not_convert(self): | |
| @attr("sanitizer_skip") | |
| def test_numba_func_with_cond_do_not_convert(self): |
| def _build_attribs_list(self, attrib_plugin): | ||
| """Build the attribs list from the attrib plugin's -A configuration. | ||
|
|
||
| This replicates the logic from AttributeSelector.moduleLoadedSuite | ||
| for -A filters only (not -E eval filters). | ||
| """ | ||
| attribs = [] | ||
|
|
||
| # Handle -A (attribute) filters | ||
| for attr in attrib_plugin.attribs: | ||
| attr_group = [] | ||
| for attrib in attr.strip().split(","): | ||
| if not attrib: | ||
| continue | ||
| items = attrib.split("=", 1) | ||
| if len(items) > 1: | ||
| # "name=value" | ||
| key, value = items | ||
| else: | ||
| key = items[0] | ||
| if key[0] == "!": | ||
| # "!name" | ||
| key = key[1:] | ||
| value = False | ||
| else: | ||
| # "name" | ||
| value = True | ||
| attr_group.append((key, value)) | ||
| attribs.append(attr_group) | ||
|
|
||
| return attribs | ||
|
|
||
| def _matches_attrib_filter(self, test_func, attrib_plugin): | ||
| """Check if test_func matches the attribute filter from attrib plugin.""" | ||
| if not attrib_plugin: | ||
| return True | ||
|
|
There was a problem hiding this comment.
Duplicated internal attribute-parsing logic from nose2
_build_attribs_list manually replicates the parsing logic inside AttributeSelector.moduleLoadedSuite (including the comma-split, =-split, and !-negation handling). If nose2 changes how it parses -A expressions (e.g. to support quoting, ranges, or OR-groups within a single -A value), this copy will silently diverge and produce incorrect filter decisions for generator functions only.
Consider whether you can obtain the parsed attribs structure directly from the attrib_plugin instance after argument processing, rather than re-parsing from the raw string. For example, attrib_plugin.attribs may already be processed by the time handleArgs fires. If not, at minimum adding a comment explaining that this must be kept in sync with nose2's AttributeSelector internals would prevent silent future breakage.
|
!build |
|
CI MESSAGE: [45822574]: BUILD STARTED |
|
CI MESSAGE: [45743750]: BUILD PASSED |
|
CI MESSAGE: [45822574]: BUILD FAILED |
|
CI MESSAGE: [45822574]: BUILD PASSED |
There was a problem hiding this comment.
Nitpick: there could only two classes, one for CPU and one for GPU tests and they'd all contain multiple test functions
Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
|
!build |
| run_fn, | ||
| out_types, | ||
| in_types, | ||
| outs_ndim, |
Check notice
Code scanning / CodeQL
Unnecessary lambda Note test
| [3], | ||
| rot_image_setup, | ||
| None, | ||
| lambda x: np.rot90(x), |
Check notice
Code scanning / CodeQL
Unnecessary lambda Note test
|
CI MESSAGE: [45987009]: BUILD STARTED |
|
CI MESSAGE: [45987009]: BUILD FAILED |
|
CI MESSAGE: [45987009]: BUILD PASSED |
5 participants
Use nose2 as the only testing framework. Drop nose.
Category: Other
Description:
Remove the WARs used to keep nose alive.
nose2 supports the yield-style test discovery by default @attr has a different filtering syntax (
-A) and just checks for presence of truthy test_foo.attribute_name. A decorator uses this mechanism for backward compatibility.nose2 splits with_setup(setup, teardown) into two separate decorators, a backward compatible decorator is added.
nottest sets special attribute.
SkipTest from unittest is recommended to be used directly (with the same functionality).
Test scripts are adjusted with minimal changes to run through nose2. Followup cleanup can be used for renaming.
Replace unsupported -m regex by attributes
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A