New Rule: `test-case-property-name`

[michaelfaith]

Proposal

I'd like to propose a new rule that require's the use of a name property in test cases that aren't simple strings. I'm thinking the default behavior would be for it to require it, only if the test case object has options and / or settings defined. If the object just has 'code`, then don't flag it. And then perhaps there's also an option to require it always?

Motivation

The default Ruletester output is usually sufficient when the test cases only consist of the code. But if the test case includes options and / or settings, the code for that test case is likely shared with other test cases (that just have different options / settings applied). In such cases, the output is still just the code. So, if you're trying to track down a failure, and you only have the code from the failure to work with, it can be hard to find the culprit test case.

Here's an example of where we adopted this pattern in eslint-plugin-package-json

Before	After

Thoughts?

[bmish]

Sounds like a good idea. I think it should be an optional (not recommended) rule because it's more opinionated and potentially burdensome for those who don't believe it's worth the effort to come up with test names.

Maybe a more generic name like test-case-property-name.

We could limit it to object test cases by default. Why? It's easier to add another property when it's already an object, object test cases are more complex, and it's more helpful for differentiating object test cases, especially when the code is identical but other options/settings differ.

Perhaps the rule could accept an object of options (to allow room for expansion):

• ignoreStringTests (default: true)

[michaelfaith]

it's more helpful for differentiating object test cases, especially when the code is identical but other options/settings differ

This is the key. If a test case is just a string, then you're unlikely to have another code-only test case that has the same string. So there's a natural distinguishing factor built in. Similarly, if it's an object and has only the code (or code and output and errors, basically anything except options and settings) then again, the code, on its own should be unique enough. It's when the object has options and / or settings that it's more likely to have the same code as some other test case, and less likely to be unique from just the code. ignoreStringTests wouldn't really capture that nuance? But maybe that makes things too complicated, and just sticking to all objects is more easy to reason about?

[bmish]

Brainstorming more, sounds like it could be useful to have a "required mode" given the possible scenarios:

required / when / mode enum option:

• "objects": require for all object-form cases
• "objects-with-config": require only when the object case has any non-code properties that can affect the test (leading to the issue with ambiguous test runner output)
• "always" / "all": require for all cases (strings must be converted to object)

[michaelfaith]

I like that.

I was thinking about it a bit more this morning about another direction that was a bit more of a "light touch", zeroing in more on the uniqueness problem: only report test case objects that have the same code as some other test case (string or object) in the same invalid/valid group, and neither has a name. But that might be too specific? I like your suggestion.