Let's change the default for RSpec/PredicateMatcher

[marcandre]

RSpec/PredicateMatcher tells me to prefer the first form to the second one:

    context 'given some result' do
      let(:error) { 'Oops, a few things blew up and nothing worked' }

      it 'check with nil?' do
        expect(error.nil?).to be(true)
      end

      it 'check with be_nil' do
        expect(error).to be_nil
      end
    end

This is gravely mistaken. It's longer, it's imo less clear, but much more importantly a failure (like in this example) gives uninformative info:

  1) some example given some result check with nil?
     Failure/Error: expect(error.nil?).to be(true)
     
       expected true
            got false

  2) some example given some result check with be_nil
     Failure/Error: expect(error).to be_nil
     
       expected: nil
            got: "Oops, a few things blew up and nothing worked"

Notice how the second failure is much more informative, showing the unexpected error and all. That is the reason to always use precise matchers.

I don't mind if we keep the functionality for the (imo always bad) style, I don't care if we propose be_nil or eq(nil), but the default should change.

Let's minimize the reasons to curse at RuboCop.

[pirj]

We don't have a formal checklist for changing the defaults, but if I had to come up with it, it would be:

• make sure to only change defaults on major version updates
• check settings for that cop in
  • https://github.com/eliotsykes/real-world-rails
  • https://github.com/jeromedalbert/real-world-ruby-apps
• check with the style guide https://rspec.rubystyle.guide/ (https://rspec.rubystyle.guide/#predicate-matchers)

It would be really nice to harmonize the defaults with the style guide and to harmonize the style guide with what is really commonly used in those open-source apps. It looks like a task on its own, but before we've done that, we can follow that "simple" checklist.

Would you like to grep through .rubocop.yml settings for PredicateMatcher of real-world-rails/real-world-ruby-apps? Part of them doesn't use RuboCop or RSpec, unfortunately, but some can be used to make a judgememnt on this topic.

[pirj]

Found a mistake in the example, "also good" part:

Strict: true, EnforcedStyle: inflected (default)

# bad
expect(foo.something?).to be_truthy

# good
expect(foo).to be_something

# also good - It checks "true" strictly.
expect(foo).to be(true)

# ^^^ should be
expect(foo.something?).to be(true)

As far as I understand, to follow your proposal we would have to change Strict from true to false, but it seems to accept be_truthy/be_falsey then. Looks very unintuitive. We probably need to rethink the approach unless I'm missing something.

[marcandre]

FWIW, I think be_truthy/be_falsey should be forbidden in a different cop. I can think of no circumstance where they should be used.

[pirj]

be_truthy/be_falsey should be forbidden in a different cop

Completely agree.

Splitting cops ☝️

be_truthy/be_falsey ... can think of no circumstance where they should be used.

If we are talking about predicates - sure, but there are certain cases when it's better to be less strict about expectations.

In general, we recommend you use the loosest matcher that still specifies the behavior you care about.

• Effective Testing with RSpec 3

The need can be implied by a weird interface or even some weirdness in the Ruby itself:

0.zero?
# => true
0.nonzero?
#=> nil
1.zero?
#=> false
1.nonzero?
#=> 1

https://metaredux.com/posts/2019/06/11/weird-ruby-zeroing-in-on-a-couple-of-numeric-predicates.html

[marcandre]

The need can be implied by a weird interface or even some weirdness in the Ruby itself:

0.zero?
# => true
0.nonzero?
#=> nil
1.zero?
#=> false
1.nonzero?
#=> 1

The "weirdness" of nonzero? is misconceived. Probably simply because of the "?". It would also a horrible use of be_truthy (that it returns the receiver and not just a truthy value is paramount in (a <=> b).nonzero? || (c <=> d) for example) or be_falsy (it never returns false).

there are certain cases when it's better to be less strict about expectations.

Legitimate ones? I'll still need to see one be convinced 😸

[pirj]

Hold on for a second. What are the PredicateMatcher configuration where it tells the prefer expect(error.nil?).to be(true) over expect(error).to be_nil? Is it the default setting?

With the default, which is Strict: true, EnforcedStyle: inflected (default) I don't get any offences in your code.

With EnforcedStyle: explicit there's "Prefer using nil? over be_nil matcher" (no matter what Strict is set to.
With EnforcedStyle: inflected Strict: false there's "Prefer using be_nil matcher over nil?".
It's strange, I would expect it to raise an offence with Strict: true as well. Might be a bug.

So are you talking about this bug, or do you propose to retire/deprecate the explicit option altogether? This sounds different from this ticket's title.

Regarding be_falsey/be_truthy there are quite some examples in RSpec's code itself [be_falsey, be_truthy]. Are you up to make those specs better as an example of your point?

[marcandre]

The reason these are in rspec is because these matchers used to be called be_false and be_true which was very confusing as most people thought they meant be(false) and be(true)
When these were (rightfully) renamed, they did the easy thing: rspec/rspec-core@eb5bb45a09

I don't think I needed a PR as an example of my point, but here it is nevertheless: rspec/rspec-core#2736

[pirj]

I'm sorry I didn't check properly initially, and it's good that we've researched this.
So, from my point of view:

1. The code example expect(foo).to be(true) is incorrect
2. For the code in the description, the cop with default settings should have raised "Prefer using be_nil matcher over nil?" but it didn't
3. We may consider splitting the cop into one that takes care of predicates, and another that keeps an eye on be_falsey/be_falsy/be_truthy (check related Deprecate expect { }.not_to raise_error(SpecificErrorClass) rspec/rspec-expectations#244, Determine why (and patch) build fails on rbx-head rspec/rspec-expectations#176 and Properly sub-class Exception so YARD can detect ExpectationNotMetError. rspec/rspec-expectations#693) - it would be nice to clean this all up with a massive sweep

Do you think there is anything else actionable? Are you happy with the current default settings?

PS I deeply respect your persistence and strong opinions.

[marcandre]

1. The code example `expect(foo).to be(true)` is incorrect

Yes, I'm sure expect(foo.something?) was meant.

2. For the code in the description, the cop with default settings should have raised "Prefer using be_nil matcher over nil?" but it didn't

Should prefer foo).to be_nil over foo.nil?).to be(true), without a doubt.

3. We may consider splitting the cop into one that takes care of predicates, and another that keeps an eye on `be_falsey`/`be_falsy`/`be_truthy` (check related rspec/rspec-expectations#244, rspec/rspec-expectations#176 and rspec/rspec-expectations#693) - it would be nice to clean this all up with a massive sweep

Absolutely. I can imagine many people wanting to avoid be_truthy but being ok with predicate?).to be(true)

Do you think there is anything else actionable? Are you happy with the current default settings?

Besides enforcing expect(predicate?).to be(true) => to be_predicate, I'm sure there are many other uses of be(true) that are "incorrect" in my book. The vast majority of specs with be(true) can be improved; something the API is at fault (if one defines foo instead of foo?), or a better matcher can be used.

There are very few legitimate uses of be(true) and be(false) I can think of:

expect { foo }.to change { bar? }.to be(true)
expect(foo?(args)).to be(true)

subject(:enabled?)
it { is_expected.to be(true) }

Note that even foo?(arg) should probably be turned into a matcher. We have a lot of:

expect(described_class.match_path?('c{at,ub}s', 'cubs')).to be(true)
# should be
subject { described_class }
is_expected to.match_path('c{at,ub}s', 'cubs')

Edit: Actually I just realized that automagical predicate matchers accept arguments. So be_match_path would work, if a bit odd,

Other builtin matchers

I didn't check if there's a cop for this, but I see a lot of cases where builtin matchers are not used.
eg.:

expect(word_regexp.is_a?(::Regexp)).to be(true)
# should be
expect(word_regexp).to be_a(::Regexp)

Arrays are a typical example:

expect(ary.size).to be(3)
# should be
expect(ary).to have(3).items

What's worse is that then the elements are tested one after the other:

expect(ary.size).to be(3)
expect(ary[0]).to some_matcher
expect(ary[2]).to some_other_matcher
# should be
expect(ary).to match_exactly [
  some_matcher,
  anything,
  some_other_matcher,
] # Note: size checked implicitly

When that fails, we have the whole array we can compare.

Yes, I put the trailing comma out of spite, that's another default I disagree with 😆

PS I deeply respect your persistence and strong opinions.

Thank, I hope I don't come out too strong to others.

[pirj]

I'm working on a pull request (not pushed yet) to address the issues of the PredicateMatcher.

1. fixed

For the code in the description, the cop with default settings should have raised "Prefer using be_nil matcher over nil?" but it didn't.

Should prefer foo).to be_nil over foo.nil?).to be(true), without a doubt.

It turns out that the default Strict: true, EnforcedStyle: inflected (default) means that a expect(foo).to be_something style is generally preferred, but "strict" checks, e.g. be(true), eq(true) etc are tolerated:

# also good - It checks "true" strictly.
expect(foo.something?).to be(true)

This goes against https://rspec.rubystyle.guide/#predicate-matchers that states "strict" usages as a bad example:

# bad
expect(subject.published?).to be true

To add even more controversy, quoting Effective Testing with RSpec again:

In general, we recommend you use the loosest matcher that still specifies the behaviour you care about.

So from my understanding, "strict" is worse than be_falsey/be_truthy, as it's excessively specific when applied to predicate matchers, e.g. if they happen to return nil which can be considered as "not defined, but presumably false".

To be fair, be_truthy is mentioned in this guideline https://rspec.rubystyle.guide/#be-matcher as one of the allowed replacements for be matcher without arguments passed.
not_to be_nil arguably sounds better, but it misses the case when the SUT is false. And it's less generic than be_truthy.

I guess the justified predicate-look-alike methods like nonzero? are scarce (unjustified example).

3. I'm going to leave the above problem with "strict" ☝️ as-is, probably providing different defaults for separated cops basing on real-world-ruby/real-world-rails usage statistics.
   Since RuboCop 1.0 is about to happen, we may bump our major version too, and shamelessly set the new defaults.

To be continued...