Discussion: Use of data providers in test classes

[mk-mxp]

Using data providers is a convenient way to reduce code and increase coherence inside the test classes. Having the test case data separated from the test code itself follows best practices and is encouraged in professional use.

But in Exercism it raises a bunch of problems:

• In the test runner, data provider based tests must be handled separately
  • Nested test suites issued for data provider tests only
  • Code reflection based line number calculation fails with input variables being on separate lines
  • Test values are only included in students test feedback when tests are designed accordingly
  • See No debug output and test inputs in online editor for dataProvider php-test-runner#116 for some issues
• PHPUnit converts values to a different format (not like PHP code) to emit them in test names
• For beginners, even when working with the CLI, the test data is not intuitively connected to the test code
• It is much more inconvenient to connect UUIDs of practice exercises to test values in a data provider
• In the test generator, data providers might add more complexity and so may not be used

So I would prefer to convert existing data provider based tests to flat test methods and not add new exercises with data providers.

Any opinions about that?

[tomasnorre]

As there are limited tests, even thought they are more or less always looking the same. I'm OK with not using dataproviders, if that improves user experience in Online Editor.

I would always reach for data providers personally in a private project, with descriptive keys.

But I think user experience is key here. We will not need to tweak tests that often.