WIP: NUnit on-ramp minibook #745

SeanKilleen · 2023-03-11T02:51:20Z

Going for it. No time like the present. I imagine this will be in progress for a little while 😅

Thoughts & feedback always welcome via comments as I go.

Supports #706.

Remaining Work I don't want to Forget about

Do a pass for "we" and strong opinions and make sure the core team shares them.
Run the long-form text through something like Hemingway / Grammarly. Make sure it's clear.
Check for and address TODOs within /book

SeanKilleen · 2023-03-11T04:59:14Z

Leaving off here for a little bit. I left off on this challenge: I'd created some projects in the Snippets solution with the intent of building a small supermarket checkout system to demonstrate the evolution into state control, mocking, etc.

The challenge is: I want to show how these things evolve, but snippets can only pull from a file's current state. So I can't really do something like the Santa Sleigh blog post I did, because it's not in its own repo where I can reference the commits and trying to do so would be messy.

So my choices are: write the code right in the markdown, or structure the snippets project so that there a lot of duplicate classes and structure so that I can pick the appropriate point-in-time snippet for the docs. This is further complicated because I can't just append V1, V2 etc. because that would show up in any captured docs snippets that had the classname.

For now I'm leaning toward writing the code in markdown to get it to a good place (copying from VS), and then extracting into the snippets project once it feels more settled, to at least avoid some of the confusion.

OsirisTerje

Great structure and setup. Love this!

OsirisTerje · 2023-03-11T09:30:30Z

docs/articles/book/concepts/TestTradeoffs.md

+
+Different types of tests have different trade-offs in their usage.
+
+Typically, automated tests are thought of as a pyramid or a funnel.


This might deserve a separate heading "The Test Triangle/Pyramid", and a figure would also be nice.

Definitely, that's in the plan 👍

OsirisTerje · 2023-03-11T09:33:36Z

docs/articles/book/concepts/TestingConcepts.md

+
+## SUT / CUT (Situation/Class Under Test)
+
+You might hear the term `SUT` or `CUT` or see these variables in tests you come across. Typically they mean the situation or class under test.


I am pretty sure the acronym stands for "System Under Test"

I don't think that it needs to, and I've commonly taught it as situation because it helps students get in the mindset of thinking what they're actually testing, in which case system may be entirely too broad or confusing.

Well, https://glossary.istqb.org/en_US/search?term=SUT and also https://acronyms.thefreedictionary.com/SUT. I don't think it is wrong to use situation, but I would add System to the list there. It makes it inline with what I think is a more common use of the acronym

OsirisTerje · 2023-03-11T09:38:42Z

docs/articles/book/concepts/TestingConcepts.md

+
+Like many things, this is a balance that a team should attempt to find and actively maintain. Some parts of tests can be extracted into common methods; other abstractions might make tests too hard to understand. Be willing to re-evaluate these choices as projects evolve.
+
+## Test Coverage


I agree with this, but would add some specifics: Test Coverage should only include coverage of the production code, and not include the test code. If you do, e.g. then removing redundant test code will actually seem to reduce code coverage. The other also applies. Adding test code increases code coverage. This is part of what makes code coverage numbers very unreliable. That said, concentrate on covering complex code. It can be good here to mention the CRAP index. https://blog.ndepend.com/crap-metric-thing-tells-risk-code/

I like this because it surfaces something that is a natural part of how I think but wouldn't be for others. I'll consider adding a section on "what do we test?" Noting that we want to test production code and want to avoid our tests mimicking production code with a lot of logic that itself can fail.

OsirisTerje · 2023-03-11T09:41:16Z

docs/articles/book/concepts/TestingConcepts.md

@@ -0,0 +1,88 @@
+# Testing Concepts


I think it would be wise here to include Shift-Left as a core testing principle. That is what you often want to do when refactoring systems. I would also stress that there is "war" between top level testing and low level testing. Even if you shift left, then you should NOT abandon your integration tests, just make sure that the bottom of your pyramid is rock solid. Imho the concept of test levels is even better than the triangle/pyramid. E.g. https://learn.microsoft.com/en-us/devops/develop/shift-left-make-testing-fast-reliable

By introducing shift-left, it is also easy to explain why testing these days is actually moving into the editors, and why Live Unit Testing has appeared. The Shift-left thinking explains a lot of why testing have been moving forward as it has.

Absolutely. Love it. For me that's built into thinking about "smaller loops" but it deserves its own callout.

OsirisTerje · 2023-03-11T09:46:35Z

docs/articles/book/concepts/TypesOfTests.md

+
+These are typically meant as the "lowest level" of automated tests. They aim to test a specific, isolated class. Any dependencies that class has on other classes would be faked in unit tests, so that they can execute quickly and with a clear understanding of how the class will behave. There will be more on this later when we discuss [test doubles and mocks](TODO).
+
+## Integration Tests


I would define integration tests as tests that go outside the component itself and touches external components, e.g. databases, and in particular these days - external APIs. That means that complex unit tests, that cover more than just a few classes fall out of this, and I normally have called them Component Tests, tests that also might use integration test frameworks (like the asp net core one), but restrict the testing to the component itself, mocking everything outside the component.
The definition of a component may of course vary, but I often teach that "it all depends" and you might look at a VS solution as a component if the parts are just that, parts that make up a whole, OR you might consider each dll/csproj as its own component. The latter I normally teach can be hard to do.

Btw, the article https://learn.microsoft.com/en-us/devops/develop/shift-left-make-testing-fast-reliable introduce the "levels", and I have used that much more than just the naming. So you might have integration tests separated into different levels, depending upon how much they actually touch. Component tests would then just be a lower level of an integration test. This makes a lot of sense when you start to integrate these into CI/CD build systems, and have to decide which tests are to be run where.

OsirisTerje · 2023-03-11T10:02:49Z

docs/articles/book/concepts/TheWhy.md

+
+So, the question becomes -- who will benefit from NUnit's particular _flavor_ of test framework?
+
+* NUnit benefits in particular from its longevity -- our first NuGet package was published in 2011, 13 years prior to this article being written. In that time, the library has amassed 200+ million downloads and has seen a ton of support from the community. To borrow the Farmers Insurance ad campaign, "We know a thing or two because we've seen a thing or two."


NUnit dates back to 1998 (I think), but picked up speed around 2002 with NUnit 2.2. That means actually 25 years of experience :-)

OsirisTerje · 2023-03-11T10:05:11Z

docs/articles/book/concepts/TheWhy.md

+So, the question becomes -- who will benefit from NUnit's particular _flavor_ of test framework?
+
+* NUnit benefits in particular from its longevity -- our first NuGet package was published in 2011, 13 years prior to this article being written. In that time, the library has amassed 200+ million downloads and has seen a ton of support from the community. To borrow the Farmers Insurance ad campaign, "We know a thing or two because we've seen a thing or two."
+* We think that beginners and newcomers benefit from the _explicitness_ of NUnit and the way it approaches tests with attributes such as `[Test]`, `[SetUp]`, and `[TearDown]`. In some cases, xUnit relies on C# conventions in ways that might not be familiar.


Very much agree. Also the "broadness", like the ability to add messages to Asserts (not in XUnit), the easy syntax for data-driven tests (TestCase, TestCaseSource), and extensibility - something that is not so much known, see a later section for more info :-)

OsirisTerje · 2023-03-11T10:09:50Z

docs/articles/book/getting-started/index.md

+
+* `Microsoft.NET.Test.Sdk` brings the test platform along and allows tests to be run using the `dotnet test` command. It's a good idea to add this to any .NET automated test project.
+* `NUnit` brings in the core NUnit library.
+* `NUnit3TestAdapter` surfaces NUnit3 tests for the Visual Studio / `dotnet test` runner. It essentially allows NUnit tests to be discovered.


It essentially allows NUnit tests to be discovered.

It essentially allows NUnit tests to be executed.

(Discovery may or may not be happening with the adapter, so its main focus is to handle the execution)

Good catch, thanks 👍

docs/snippets/book/GettingStarted.Tests/UnitTest1.cs

OsirisTerje · 2023-03-11T10:14:36Z

docs/snippets/book/getting-started/Converter.Tests/TemperatureConverterTests.cs

+        var result = sut.ConvertCToF(37);
+
+        // Assert
+        Assert.That(result, Is.EqualTo(98.6));


Since this is a decimal, the result may or may not match. A good time to use the Within

Assert.That(result, Is.EqualTo(98.6).Within(0.1);

Yes, technically. I think that gets into docs territory or another example scenario though. As is the test passes and AFAIK will always pass. Too many concepts to introduce for a "first use".

Though I do think it speaks to the need for a "common pitfalls and remedies" article that maybe briefly delves into summarizing some of the analyzer checks and additional things to watch for.

Sure, when doubles are used in particular one should be aware that the EqualTo may not match, decimals I think are better, but not quite sure. It all depends. Perhaps have a pointer to a Tricks and Traps section for that?

And just for fun:

SeanKilleen added 21 commits March 10, 2023 21:32

bring over contents so far

48f368a

Add projects to solution

f4bcd6f

smile emoji

6e66936

Add book to TOC

26ff70b

Last sentence + Fadi credit

477b025

add some terms to dictionary

590364b

some ignore words

7e63439

update cSpell file version to remove warning

36102a5

some markdown fixes

d86c724

para on commercial products

a3e4665

note on other assertion libs

33086ba

Note formatting

a02b219

table formatting

f79b085

minor wording

7d1d984

finish a sentence

d3a4ef8

Note that this doesn't replace the docs.

bd9825f

outline including some blank placeholder docs

c1a9909

minor updates

811ed62

Add GettingStarted project for the upcoming general samples

638ad53

update packages

a18e314

Add a classlib project for getting started

d429c60

some content & next thoughts

ba23d44

OsirisTerje reviewed Mar 11, 2023

View reviewed changes

SeanKilleen mentioned this pull request Nov 7, 2023

Should we document Modifiers, e.g. Using #823

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: NUnit on-ramp minibook #745

WIP: NUnit on-ramp minibook #745

SeanKilleen commented Mar 11, 2023 •

edited

Loading

SeanKilleen commented Mar 11, 2023

OsirisTerje left a comment •

edited

Loading

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

SeanKilleen Mar 11, 2023

OsirisTerje Mar 11, 2023

OsirisTerje Mar 11, 2023


		Different types of tests have different trade-offs in their usage.

		Typically, automated tests are thought of as a pyramid or a funnel.


		## SUT / CUT (Situation/Class Under Test)

		You might hear the term `SUT` or `CUT` or see these variables in tests you come across. Typically they mean the situation or class under test.


		Like many things, this is a balance that a team should attempt to find and actively maintain. Some parts of tests can be extracted into common methods; other abstractions might make tests too hard to understand. Be willing to re-evaluate these choices as projects evolve.

		## Test Coverage


		These are typically meant as the "lowest level" of automated tests. They aim to test a specific, isolated class. Any dependencies that class has on other classes would be faked in unit tests, so that they can execute quickly and with a clear understanding of how the class will behave. There will be more on this later when we discuss [test doubles and mocks](TODO).

		## Integration Tests


		So, the question becomes -- who will benefit from NUnit's particular _flavor_ of test framework?

		* NUnit benefits in particular from its longevity -- our first NuGet package was published in 2011, 13 years prior to this article being written. In that time, the library has amassed 200+ million downloads and has seen a ton of support from the community. To borrow the Farmers Insurance ad campaign, "We know a thing or two because we've seen a thing or two."

WIP: NUnit on-ramp minibook #745

Are you sure you want to change the base?

WIP: NUnit on-ramp minibook #745

Conversation

SeanKilleen commented Mar 11, 2023 • edited Loading

Remaining Work I don't want to Forget about

SeanKilleen commented Mar 11, 2023

OsirisTerje left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SeanKilleen commented Mar 11, 2023 •

edited

Loading

OsirisTerje left a comment •

edited

Loading