Show number of assertions in output.

[utkarshkukreti]

I searched around, but couldn't find a way to do it. Something like:

1 example, 0 failures, 3 assertions

In this question, @dchelimsky suggested to open a feature request, so here I am 😺

Edit: @xaviershay's summary of comments below as of Jan 2017:

• Proposed API: Show number of assertions in output. #4
• External gem that claims to do this (I haven't verified): https://github.com/bootstraponline/rspec_expectation_count

[ethnt]

👍 It would be good information to have, even if it is best practice to have only one assertion per test (see Better Specs).

[utkarshkukreti]

@eturk, interesting link.

If I were to follow that guideline, how should I rewrite this block?

it "should parse chained monads/dyads" do
  eval1("1 + 2 + 3 4 + 4 5").should eq [10, 12]
  eval1("1 - 2 * 4 + 4 5").should eq [-15, -17]
  eval1("1 - 2 * 4 + 4 5").should eq [-15, -17]
  eval1("1 - 1 - 1 2 3 + 3 4 5 - 6 7 8").should eq [-2, -1, 0]
end

(eval1 is just a helper to parse, transform, and evaluate the string).

[dchelimsky]

One expectation per example is a useful guideline, not a moral imperative. @utkarshkukreti the risk you run with your example is that if the first statement fails and you get it to pass, you might find that the 2nd is now failing. Then you get that passing and the third one might be failing, etc.

Having each statement in separate examples gives you clearer insight into what the underlying problem is.

You could rewrite your example with a custom matcher. Something like

context "parsing chained modads/dyads" do
  it { should parse("1 + 2 + 3 4 + 4 5").as([10, 12])}
  it { should parse("1 - 2 * 4 + 4 5").as([-15, -17])}
  # etc
end

[vanstee]

Is this issue resolved?

[ethnt]

Would it be best to wait and see if anyone else wants to give their opinion? I'm for it, even if it is a best practice to write one assertion per example.

Shall I begin writing an implementation?

[myronmarston]

I'm moderately in favor of the idea, so sure....@eturk, feel free to take a stab at this :).

[ethnt]

The output will look something like:

1 example (1 assertion), 0 failures

Is this going to be an optional configuration option or on by default?

[soulcutter]

How about 1 example, 0 failures, 3 assertions ? Or at least 1 example, 0 failures (3 assertions). It seems a bit weird to break up the output in the middle.

I feel like this should be an optional configuration option that is off by default. Maybe I'm being over-conservative, but I feel that changes to output format should be introduced gradually.

[ethnt]

1 example, 0 failures, 3 assertions looks good.

I agree that it should be optional. How does the --assertions flag sound?

[dchelimsky]

It's called rspec-expectations for a reason: assertion != expectation. Please use "expectation/s" instead of "assertion/s".

[myronmarston]

Is this going to be an optional configuration option or on by default?

I like making RSpec configurable, but with a feature like this that gives the user more info in the output, I have a hard time coming up with a reason why a user would want it off, or coming up with a way this could be a breaking change for anyone. So I lean towards not making it a config option. We don't want the complexity of a config option for each bit of output the formatter spit out.

@soulcutter -- what problems do you think it could cause someone by adding this bit of output to the formatters?

That said, there is one thing that probably should be configurable: what exception classes are considered to be expectations. We don't want rspec-core to be coupled to rspec-expectations in this regard, and we should make this feature usable by folks who use other libraries (e.g. minitest's assertions or wrong). Something like:

RSpec.configure do |c|
  c.expectation_failure_errors = [SomeErrorClass, SomeOtherErrorClass]
end

rspec-expectations can use this API if RSpec.respond_to?(:configure) so that it auto-configures this correctly.

Actually, I guess this config option would provide a simple way to disable this behavior: if you assigned the config option to an empty array (or nil, if we want to support that), then it would effectively turn it off as rspec-core would have no way to distinguish exception types.

[myronmarston]

Actually, ignore most of what I just wrote. I just realized that this feature is to list the number of expectations, not simply to distinguish between expectation failures and other types of errors. (Sorry, I was thinking about this all wrong!)

So the difficulty here is that rspec-expectations should be the place that keeps track of the # of expectations, but we don't to couple rspec to it. So...maybe the config should be something like this?

RSpec.configure do |c|
  c.fetch_expectation_count_from do
    RSpec::Expectations.expectation_count
  end
end

(I'm not happy with the name, but you get the idea: it's a block that's called to fetch the number of expectations when the formatter is printing the numbers. rspec-expectations can still self-configure this when loaded).

[soulcutter]

@myronmarston I am hard pressed to think of what changing the output would break. I think you are right, and I should have paid attention to my instinct that I was being over-conservative.

As far as the block syntax goes, maybe it would make more sense to register objects which are called for summation? e.g.

RSpec.configure do |c|
  # this would be a default, so you wouldn't really do this, but for other libs or something... 
  c.expectation_counters << RSpec::Expectations.public_method(:expectation_count) 
end

[ethnt]

Isn't RSpec::Expectations included along with RSpec::Core when you install rspec? If that's the case, you wouldn't need to call RSpec::Expecations.public_ethod(:expectation_count) explicitly, correct? Then it would make sense to just do c.expectation_counts, true.