Now, I'm going to preempt the obvious: Tests should ALWAYS be independent. If you require a specific state to be true prior to the meat of the test running, you should be mocking/stubbing it out and asserting before executing the test.
Which is my MO when writing unit tests. When writing integration tests – those tests that specifically exercise the UI – it's borderline impossible to use mocks and stubs, since the state I'm expecting at the beginning of the test is a state that can only exist in the UI.
Now, let me give an example of what I'm going through right now – and I'm going to be intentionally vague (non-disclosure and all that).
Let's say, for sake of argument, that I have a website. I want to test the ability to submit a form on this website, so I write a test to do just that. My test goes through these steps:
Log into the site
Enter info in the form (pretend the form is right there after logging in)
Report result (assert success)
The preexisting condition we're expecting? That the user exists. Now, I can use hard-coded emails just fine. In the interest of my example though, let's pretend that management has told me, "we don't care about being able to submit the form as an existing user – we know that. Tell me if we can submit as a brand new user."
So, I've also got another test that creates a user:
Go to website
Enter vitals (randomly generated, name, phone, etc.)
Attempt to log in with that user (and the one-time-pass they're given) and assert success.
Now, it's a requirement that this second test succeeds prior to the first one running. I want to add a dependency – if the first test fails, I don't want the second test to run at all. If the test doesn't run, it'll show up as inconclusive in my results – which is exactly what I want for metrics.
One piece of advice people have given me is "just create a new user at the beginning of the submit form test." Which sounds all well and good – but that's just adding extra time to the process. These tests are automated, they're scheduled nightly, and the whole suite (I built most of it, very proud of it) takes about three hours to run. Adding 5 minutes willy-nilly is not my preference 🙂
I know MbUnit has the DependsOn attribute. I also know Windows.System.Markup has one that can be used as an NUnit attribute – I haven't had any success with it however.
What I do know NUnit has is the Assume.That assertion type – which seems like it's the best fit for me. The problem is I'm not sure how to craft the syntax such that:
Assume.That(this other test passed)
I know I can get a test result with TestContext, however from what I've read I can only interact with the current context, and that the results of other tests just don't exist in this context.
Another option I have is placing a boolean in test setup – something like, bool userCreated;, then, if the user creation test passes, set that true, then call Assume.That(userCreated); – the issue with that is that I'll end up with a lot of overhead, as I have many instances where dependencies come in handy.
Anyone have any advice for me? I'm feeling a bit defeated by this – it sucks when your questions aren't googleable 🙁
by ProfWhite via /r/csharp