Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
c69b584
Add blank query object and remove all junit formatter implementations
luke-hill Sep 5, 2025
72b0f16
Comment out the rerun formatter instead
luke-hill Sep 5, 2025
23f7452
C+P for existing query
luke-hill Sep 5, 2025
4616d2e
WIP - create a simultaneous new run formatter to try get the new enve…
luke-hill Sep 5, 2025
875d40d
Remove trace
luke-hill Sep 5, 2025
3157de5
Alphabeticise the iVars in query
luke-hill Sep 5, 2025
958abdb
Note to self on first two items that are required
luke-hill Sep 5, 2025
ba1a767
Store test case started by id in query
luke-hill Sep 5, 2025
c6b0f4e
Add in implementation for next query method
luke-hill Sep 5, 2025
fdd9c45
Comment WIP about getting the iVar from query
luke-hill Sep 5, 2025
1a3700b
Add third implementor and add notes for implemented ones
luke-hill Sep 5, 2025
5b3abf4
Early terminate in #update as only one call is ever valid
luke-hill Sep 5, 2025
f4a53da
Add readerss
luke-hill Sep 5, 2025
2615ff7
WIP - more guesswork
luke-hill Sep 6, 2025
10466da
WIP: Implement new repository / query interface
luke-hill Nov 24, 2025
7daf0f2
Staging point. We now have 3 complex finders and 1 simple finder inte…
luke-hill Nov 24, 2025
19ccbff
Reached a checkpoint where we have 2 tests (running for minimal and e…
luke-hill Nov 24, 2025
2c02397
Extend query to run against 4 sample ndjson files with expected answers
luke-hill Nov 24, 2025
6456f44
More tidy and added a 5th example for findAllPickles
luke-hill Nov 24, 2025
ec538b8
Implement two more sub-update methods for generic update for hooks;
luke-hill Nov 24, 2025
7e79823
Complete initial iteration for update in repository
luke-hill Nov 24, 2025
a3c7254
Tidy up to use ruby writers;
luke-hill Nov 24, 2025
583d342
Have ChatGPT write a much more optimal iterator
luke-hill Nov 25, 2025
832a76b
Use the new test iterator
luke-hill Nov 25, 2025
de47515
Final tidy for specs;
luke-hill Nov 25, 2025
c66f47b
Update remaining items for the TODO markers for the enumerable items …
luke-hill Nov 25, 2025
1659c70
Mark the 2*TODO items as ready for sign-off
luke-hill Nov 25, 2025
5e2323b
Re-order #update according to Java impl
luke-hill Nov 25, 2025
c8b3842
Add in TODO markers for count and findAll methods
luke-hill Nov 25, 2025
30f817f
1 all method complete
luke-hill Nov 25, 2025
64d20a6
Complete 3 more findAll methods ported from java
luke-hill Nov 25, 2025
92f629d
Add in 2 new queries to test the implementation
luke-hill Nov 25, 2025
a052e70
Add in assertion files for findAllTestSteps and findAllTestCases for …
luke-hill Nov 25, 2025
623caf9
Fix rubocop
luke-hill Nov 25, 2025
710fb26
Remove redundant line
luke-hill Nov 25, 2025
ddce394
Split event handlers from initializer in new implementation of rerun …
luke-hill Nov 25, 2025
592d5b6
CR: Make plurality caches store by default as a hash. Which has each …
luke-hill Nov 25, 2025
4365b91
Clarify all items should be good to go now
luke-hill Nov 25, 2025
fc2ae13
Reduce LOC and memoizations by using initializer for state
luke-hill Nov 25, 2025
d819b69
findAllTestStepStarted needs to reduce the multi-dimensional construc…
luke-hill Nov 25, 2025
60eb849
Add in 2 new queries and then complete some re-ordering
luke-hill Nov 25, 2025
d0c35ae
Add in 2 findAll methods for hooks based on repository;
luke-hill Nov 25, 2025
fbf9c09
Add in one more find_by one more find_all and one new count_ queries.
luke-hill Nov 25, 2025
7649e52
Fix up findAllTestCaseStarted
luke-hill Nov 26, 2025
21b1783
Add in 2 new queries for findTestCaseBy and findPickleBy
luke-hill Nov 26, 2025
f4d33b6
Add in supporting files to support 2 new queries
luke-hill Nov 26, 2025
d59a08c
Add in the new query findAllTestCaseFinished
luke-hill Nov 26, 2025
8bf7bd1
Replace outdated erroneous hooks file
luke-hill Nov 26, 2025
dfc11d5
Replace outdated erroneous hooks test data files
luke-hill Nov 26, 2025
53279c9
Fix up completion metrics
luke-hill Nov 26, 2025
f3d8422
Add update gherkin document and sub updates update feature and gherki…
luke-hill Nov 26, 2025
fd0b888
Fix alphabeticising by placing started before finished before each pa…
luke-hill Nov 26, 2025
47816eb
Add in upate_steps which requires step_by_id store
luke-hill Nov 26, 2025
3f4ac03
Add update_step_definition as well as step_definition_by_id hash stor…
luke-hill Nov 26, 2025
bcc3696
Add / copy top level doc
luke-hill Nov 26, 2025
2503c3d
Classify initial state for new target - migrate majority of By finders
luke-hill Nov 26, 2025
c91944b
Updated current status quo 3 fully complete 1 partially complete
luke-hill Nov 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions lib/cucumber/formatter/message_builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
require 'cucumber/formatter/query/test_case_started_by_test_case'
require 'cucumber/formatter/query/test_run_started'

require 'cucumber/query'

module Cucumber
module Formatter
class MessageBuilder
Expand Down Expand Up @@ -39,6 +41,8 @@ def initialize(config)
@current_test_run_started_id = nil
@current_test_case_started_id = nil
@current_test_step_id = nil

@query = Cucumber::Query.new
end

def output_message
Expand Down
59 changes: 59 additions & 0 deletions lib/cucumber/formatter/new_rerun.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# frozen_string_literal: true

require 'cucumber/formatter/io'
require 'cucumber/query'
require 'cucumber/formatter/message_builder'

module Cucumber
module Formatter
class NewRerun
include Formatter::Io

def initialize(config)
@io = ensure_io(config.out_stream, config.error_stream)
@config = config
@failures = {}
config.on_event :test_case_finished, &method(:on_test_case_finished)
config.on_event :test_run_finished, &method(:on_test_run_finished)
end

def on_test_case_finished(event)
test_case, result = *event.attributes
if @config.strict.strict?(:flaky)
next if result.ok?(strict: @config.strict)

add_to_failures(test_case)
else
unless @latest_failed_test_case.nil?
if @latest_failed_test_case != test_case
add_to_failures(@latest_failed_test_case)
@latest_failed_test_case = nil
elsif result.ok?(strict: @config.strict)
@latest_failed_test_case = nil
end
end
@latest_failed_test_case = test_case unless result.ok?(strict: @config.strict)
end
end

def on_test_run_finished(_event)
add_to_failures(@latest_failed_test_case) unless @latest_failed_test_case.nil?
next if @failures.empty?

@io.print file_failures.join("\n")
end

private

def file_failures
@failures.map { |file, lines| [file, lines].join(':') }
end

def add_to_failures(test_case)
location = test_case.location
@failures[location.file] ||= []
@failures[location.file] << location.lines.max unless @failures[location.file].include?(location.lines.max)
end
end
end
end
139 changes: 139 additions & 0 deletions lib/cucumber/query.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# frozen_string_literal: true

require 'cucumber/repository'

# Given one Cucumber Message, find another.
#
# Queries can be made while the test run is incomplete - and this will naturally return incomplete results
# see <a href="https://github.com/cucumber/messages?tab=readme-ov-file#message-overview">Cucumber Messages - Message Overview</a>
#
module Cucumber
class Query
attr_reader :repository
private :repository

def initialize(repository)
@repository = repository
end

# TODO: count methods (1/2) Complete
# Missing: countMostSevereTestStepResultStatus

# TODO: findAll methods (11/12) Complete
# Missing: findAllUndefinedParameterTypes

# TODO: find****By methods (3/25) Complete
# Missing: findAttachmentsBy (2 variants)
# Missing: findHookBy (3 variants)
# Missing: findMeta (1 variant) - This strictly speaking isn't a findBy but is located within them
# Missing: findMostSevereTestStepResultBy (2 variants)
# Missing: findLocationOf (1 variant) - This strictly speaking isn't a findBy but is located within them
# Partially Complete (3/5): findPickleBy (5 variants)
# Missing: findPickleStepBy (1 variant)
# Missing: findSuggestionsBy (2 variants)
# Missing: findStepBy (1 variant)
# Missing: findStepDefinitionsBy (1 variant)
# Missing: findUnambiguousStepDefinitionBy (1 variant)
# Fully Complete (4/4): findTestCaseBy (4 variants)
# Missing: findTestCaseDurationBy (2 variant)
# Fully Complete (3/3): findTestCaseStartedBy (3 variants)
# Fully Complete (1/1): findTestCaseFinishedBy (1 variant)
# Missing: findTestRunHookFinishedBy (1 variant)
# Missing: findTestRunHookStartedBy (1 variant)
# Missing: findTestRunDuration (1 variant) - This strictly speaking isn't a findBy but is located within them
# Missing: findTestRunFinished (1 variant) - This strictly speaking isn't a findBy but is located within them
# Missing: findTestRunStarted (1 variant) - This strictly speaking isn't a findBy but is located within them
# Missing: findTestStepBy (2 variants)
# Missing: findTestStepsStartedBy (2 variants)
# Missing: findTestStepsFinishedBy (2 variants)
# Missing: findTestStepFinishedAndTestStepBy (1 variant)
# Missing: findLineageBy (9 variants!)

def count_test_cases_started
find_all_test_case_started.length
end

def find_all_pickles
repository.pickle_by_id.values
end

def find_all_pickle_steps
repository.pickle_step_by_id.values
end

def find_all_step_definitions
repository.step_definition_by_id.values
end

# This finds all test cases that have started, but not yet finished
# AS WELL AS (AND)
# This finds all test cases that have started AND have finished, but that will NOT be retried
def find_all_test_case_started
repository.test_case_started_by_id.values.select do |test_case_started|
test_case_finished = find_test_case_finished_by(test_case_started)
test_case_finished.nil? || !test_case_finished.will_be_retried
end
end

# This finds all test cases that have finished AND will not be retried
def find_all_test_case_finished
repository.test_case_finished_by_test_case_started_id.values.reject do |test_case_finished|
test_case_finished.will_be_retried
end
end

def find_all_test_cases
repository.test_case_by_id.values
end

def find_all_test_run_hook_started
repository.test_run_hook_started_by_id.values
end

def find_all_test_run_hook_finished
repository.test_run_hook_finished_by_test_run_hook_started_id.values
end

def find_all_test_step_started
repository.test_steps_started_by_test_case_started_id.values.flatten
end

def find_all_test_step_finished
repository.test_steps_finished_by_test_case_started_id.values.flatten
end

def find_all_test_steps
repository.test_step_by_id.values
end

# This method will be called with 1 of these 3 messages
# [TestCaseStarted || TestCaseFinished || TestStepStarted]
def find_pickle_by(element)
test_case = find_test_case_by(element)
raise 'Expected to find TestCase from TestCaseStarted' unless test_case

repository.pickle_by_id[test_case.pickle_id]
end

# This method will be called with 1 of these 4 messages
# [TestCaseStarted || TestCaseFinished || TestStepStarted || TestStepFinished]
def find_test_case_by(element)
test_case_started = element.respond_to?(:test_case_started_id) ? find_test_case_started_by(element) : element
raise 'Expected to find TestCaseStarted by TestStepStarted' unless test_case_started

repository.test_case_by_id[test_case_started.test_case_id]
end

# This method will be called with 1 of these 3 messages
# [TestCaseFinished || TestStepStarted || TestStepFinished]
def find_test_case_started_by(element)
repository.test_case_started_by_id[element.test_case_started_id]
end

# This method will be called with only 1 message
# [TestCaseStarted]
def find_test_case_finished_by(test_case_started)
repository.test_case_finished_by_test_case_started_id[test_case_started.id]
end
end
end
127 changes: 127 additions & 0 deletions lib/cucumber/repository.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# frozen_string_literal: true

module Cucumber
# In memory repository i.e. a thread based link to cucumber-query
class Repository
attr_accessor :meta, :test_run_started, :test_run_finished
attr_reader :pickle_by_id, :pickle_step_by_id,
:step_by_id, :step_definition_by_id,
:test_case_by_id, :test_case_started_by_id, :test_case_finished_by_test_case_started_id,
:test_run_hook_started_by_id, :test_run_hook_finished_by_test_run_hook_started_id,
:test_step_by_id, :test_steps_started_by_test_case_started_id, :test_steps_finished_by_test_case_started_id

def initialize
@pickle_by_id = {}
@pickle_step_by_id = {}
@step_by_id = {}
@step_definition_by_id = {}
@test_case_by_id = {}
@test_case_started_by_id = {}
@test_case_finished_by_test_case_started_id = {}
@test_run_hook_started_by_id = {}
@test_run_hook_finished_by_test_run_hook_started_id = {}
@test_step_by_id = {}
@test_steps_started_by_test_case_started_id = Hash.new { |hash, key| hash[key] = [] }
@test_steps_finished_by_test_case_started_id = Hash.new { |hash, key| hash[key] = [] }
end

def update(envelope)
return self.meta = envelope.meta if envelope.meta
return self.test_run_started = envelope.test_run_started if envelope.test_run_started
return self.test_run_finished = envelope.test_run_finished if envelope.test_run_finished
return update_gherkin_document(envelope.gherkin_document) if envelope.gherkin_document
return update_pickle(envelope.pickle) if envelope.pickle
return update_test_run_hook_started(envelope.test_run_hook_started) if envelope.test_run_hook_started
return update_test_run_hook_finished(envelope.test_run_hook_finished) if envelope.test_run_hook_finished
return update_test_case_started(envelope.test_case_started) if envelope.test_case_started
return update_test_case_finished(envelope.test_case_finished) if envelope.test_case_finished
return update_test_step_started(envelope.test_step_started) if envelope.test_step_started
return update_test_step_finished(envelope.test_step_finished) if envelope.test_step_finished
return update_test_case(envelope.test_case) if envelope.test_case

nil
end

private

def update_feature(feature)
# Java impl:
# feature.getChildren()
# .forEach(featureChild -> {
# featureChild.getBackground().ifPresent(background -> updateSteps(background.getSteps()));
# featureChild.getScenario().ifPresent(this::updateScenario);
# featureChild.getRule().ifPresent(rule -> rule.getChildren().forEach(ruleChild -> {
# ruleChild.getBackground().ifPresent(background -> updateSteps(background.getSteps()));
# ruleChild.getScenario().ifPresent(this::updateScenario);
# }));
# });

feature.children.each do |feature_child|
update_steps(feature_child.background.steps) if feature_child.background
update_scenario(feature_child.scenario) if feature_child.scenario
if feature_child.rule
feature_child.rule.children.each do |rule_child|
update_steps(rule_child.background.steps) if rule_child.background
update_scenario(rule_child.scenario) if rule_child.scenario
end
end
end
end

def update_gherkin_document(gherkin_document)
# Java impl:
# private void updateGherkinDocument(GherkinDocument document) {
# lineageById.putAll(Lineages.of(document));
# document.getFeature().ifPresent(this::updateFeature);
# }
# TODO: Update lineage??
update_feature(gherkin_document.feature) if gherkin_document.feature
end

def update_pickle(pickle)
pickle_by_id[pickle.id] = pickle
pickle.steps.each { |pickle_step| pickle_step_by_id[pickle_step.id] = pickle_step }
end

def update_scenario(scenario)
update_steps(scenario.steps)
end

def update_steps(steps)
steps.each { |step| step_by_id[step.id] = step }
end

def update_step_definition(step_definition)
step_definition_by_id[step_definition.id] = step_definition
end

def update_test_case(test_case)
test_case_by_id[test_case.id] = test_case
test_case.test_steps.each { |test_step| test_step_by_id[test_step.id] = test_step }
end

def update_test_case_started(test_case_started)
test_case_started_by_id[test_case_started.id] = test_case_started
end

def update_test_case_finished(test_case_finished)
test_case_finished_by_test_case_started_id[test_case_finished.test_case_started_id] = test_case_finished
end

def update_test_run_hook_started(test_run_hook_started)
test_run_hook_started_by_id[test_run_hook_started.id] = test_run_hook_started
end

def update_test_run_hook_finished(test_run_hook_finished)
test_run_hook_finished_by_test_run_hook_started_id[test_run_hook_finished.test_run_hook_started_id] = test_run_hook_finished
end

def update_test_step_started(test_step_started)
test_steps_started_by_test_case_started_id[test_step_started.test_case_started_id] << test_step_started
end

def update_test_step_finished(test_step_finished)
test_steps_finished_by_test_case_started_id[test_step_finished.test_case_started_id] << test_step_finished
end
end
end
5 changes: 3 additions & 2 deletions spec/cucumber/formatter/rerun_spec.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

require 'cucumber/formatter/rerun'
require 'cucumber/formatter/new_rerun'
require 'cucumber/core'
require 'cucumber/core/gherkin/writer'
require 'support/standard_step_actions'
Expand All @@ -9,7 +9,7 @@

module Cucumber
module Formatter
describe Rerun do
describe NewRerun do
include Cucumber::Core::Gherkin::Writer
include Cucumber::Core

Expand All @@ -34,6 +34,7 @@ module Formatter
end
end
end

described_class.new(config)
execute [gherkin], [StandardStepActions.new], config.event_bus
config.event_bus.test_run_finished
Expand Down
Loading
Loading