Provide a way to programmatically remove formatters

[agis]

Currently there's #add_formatter for registering a new formatter. But there's no counterpart to remove one. This would be useful in skroutz/rspecq#6 or other custom runners maybe. The rationale is that we want to run a dry-run with a JSON formatter to grab the example IDs, and right after (in the same process) issue the regular run.

I'd like to do something like this:

def files_to_example_ids(files)
  formatter = RSpec::Core::Formatters::JsonFormatter.new(StringIO.new)
  RSpec.configuration.add_formatter(formatter)
  RSpec.configuration.dry_run = true

  opts = RSpec::Core::ConfigurationOptions.new(files)
  RSpec::Core::Runner.new(opts).run($stderr, $stdout)

  # collect individual examples
  JSON.parse(out.string)["examples"].map { |e| e["id"] }
ensure
  RSpec.configuration.dry_run = false

  # NOTE: this doesn't exist
  RSpec.configuration.remove_formatter(formatter)
end

Does that sound reasonable? Or perhaps there's already a way to do this that slipped my eyes? Thanks in advance!

[agis]

Ah, a bit too hasty. It seems that #clear_examples does the trick.

[pirj]

What do you collect from this dry run exactly? Just the example ids?
RSpec.world has a all_examples method. It's marked as private, but I guess for such a closely integrated tool as rspecq, it's quite ok to rely in the internals that deeply.

[agis]

@pirj Yep, I want to obtain the example ids of a subset of the files. skroutz/rspecq#36 is what I came up with. Do you think it's better to do this via Rspec.world.all_examples?

[JonRowe]

The RSpec globals are intended for use by RSpec itself in a single pass, but you can create your own versions of configuration, world etc and use those when you need something more complex. In theory you should be able to duplicate a config and add the formatter / dry run for this purpose.

[pirj]

@agis If all_examples is providing the information you need, that you can feed to RSpec later on, sure.

Just a word of precaution specifically about splitting by examples. Examples may have stateful setup. The simplest example is:

before(:all) { @admin = create(:admin) }
before(:each) { sign_in @admin }

Sometimes, this setup is so heavy that it takes the majority of the spec run time.
If you split the examples that need this setup across workers, those workers will each have to run this setup.
let_it_be/let_once are used in the wild, I've seen several major projects using them, primarily to speed tests up.

The same applies to splitting by example groups, as they similarly can have shared setup.

So in a generic case, I'd stick to splitting by spec files.

[agis]

Thanks for the heads up @pirj. In the generic case, we do schedule whole spec files. We only split very slow files to individual examples (see documentation).