Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts

Nov 13, 2014

“Smart Unit Tests” Are “Pinning Tests”

You’ve probably seen all the various announcements from Microsoft today, with the most notable being the release of Visual Studio 2015 Preview and the jaw dropping move to open source the .NET Core Framework. Fantastic stuff! Though I won’t go into it in detail here.

What I do want to cover are the “Smart Unit Tests” released with the VS2015 Preview, based on the Microsoft Pex research project. The messaging around it has created more than a little confusion. In fact, here’s what one of my colleagues on Twitter sent my way:

During the MVP Summit I spent some time with the people who built this feature to better understand what the intent behind these so called “smart” unit tests. Is it simply so we can tell our managers that we can hit that magic 100% unit test coverage mark now, meet our KPI’s, claim our annual bonus for hitting an arbitrary metric, and still be able to produce rubbish quality applications or is there some other value to it?

It turns out that the purpose of the Smart Unit Test feature is to help people in their refactoring efforts. It’s good practice before you go and refactor code to ensure you have a healthy level of test coverage in place so that you can be confident that any changes you make to code don’t break the behaviour of the application. On existing, legacy code without tests, this is often a very expensive and difficult process, and as a result many refactoring efforts occur either without any test coverage at all or simply never happen because they’re too scary.

This is where “Smart Unit Tests” come into play. They help you do the “smart” thing and create unit tests that help you pin down the current behaviour of the code, before you do any refactoring.  These type of tests are known, unsurprisingly, as pinning tests.

Here’s a good description of a pinning test (source), emphasis added:

A “pinning test” is not a good test. It doesn’t try to be. It’s just the simplest, fastest-running test you can write that will allow you to refactor your code. It’s often an end-to-end test, but it could also look at your log files, monkey-patch a core library function, or do something similarly ridiculous.

The key here is that the pinning test lets you do your dozens-of-tiny-refactorings loop really quickly. As the code improves, you add high-quality unit tests. Once the tests are good enough, you get rid of the pinning test. Lather, rinse, repeat.

Hopefully that helps clear up some confusion. Oh, as a final note, Brian Harry mentioned that the Smart Unit Tests name is likely to change. Let’s hope so! :-)

Mar 4, 2012

How To Unit Test Async Methods with MSTest, XUnit and VS11 Beta

MSTest finally got some love with the Visual Studio 11 Beta and one of those changes was to enable tests to run asynchronously using the async and await keywords.

This is required if you want to write tests against any async methods (especially with WinRT!) but can also be used anywhere else you need to perform asynchronous operations.

Here’s a silly sample test to show you how it’s done

[TestMethod]
public async Task LoadGoogleHomePageAsync()
{
    var client = new System.Net.Http.HttpClient();
    var page = await client.GetStringAsync("www.google.com");
    Microsoft.VisualStudio.TestTools.UnitTesting.StringAssert.Contains(page, "google");
}

XUnit also supports this option as shown here:

[Xunit.Fact]
public async Task XUnitAsyncTestMethod()
{
    var c = new System.Net.Http.HttpClient();
    var result = await c.GetStringAsync("http://www.google.com");
    Xunit.Assert.Contains("google", result);
}

Be aware that if you have a testsettings file specified in the Unit Test Explorer that async tests will not work.  This applies to the beta only.  Apart from that everything works as expected.

Nov 11, 2011

Underscores in Test Names are a Pain to Type, Right? Not Anymore!

It might be a large assumption given some of the customers I deal with, but I’m going to assume you write unit tests.  I’m also going to assume that when you write tests your test names have underscores separating all the words and making your test names human readable. Something like This_is_a_really_long_test_name_but_thats_ok_because_its_easy_to_read_when_I_use_underscores.

The only (minor) problem I have with this approach is that underscore isn’t the easiest key to type.  I’d rather just hit the space bar and have Visual Studio change that space to an underscore for me – after all, that’s exactly what computers are good for :-)

Here’s the good news, there’s a great little ReSharper macro that has been developed that does just that – it’s available from https://github.com/joaroyen/ReSharperExtensions and all you need to do is create an R# live template that wires the macro up to your test name field in a live template.  For example:

image

Now when you use the live template to create a test you can simply type the test name with spaces and they’ll be converted into underscores for you automagically.  Fanstastic!

Kudos to Joar Oyen for his great work, and feel free to check out his blog post for how this works under the hood.

Sep 2, 2011

Using Parallel Task Library to Unit Test Threading Issues

I was doing some work recently on a demo application where data was being pulled in from multiple locations and being added to a collection that was also being iterated over in the same method.  Because this data was arriving on multiple threads (i.e. async network call backs for example) I’d occasionally see the usual “collection was modified” error messages indicating that another thread had altered the collection while the first was iterating over it.  Obvious threading bug, #FacePlam applied.

While it can be complex at times to find these kinds of errors, in this case it was fairly easy to diagnose and fix, so following good bug fix practices I took the standard approach of writing a test to prove the bug exists, fixing the code and then running the test again to prove it’s fixed.

Now, it should be noted that testing threading issues in a deterministic way is nigh on impossible, and there is no guarantee that a unit test for threading issues will genuinely prove the code bug free, however the approach taken here was good enough to throw the threading exception each and every time I ran the test and also the throw the exception on the build server.

Here’s the code:

[TestMethod]
public void ThreadingFun()
{
    InitializeControllerAndGroup();

    Task[] tasks = new Task[10]
                            {
                                Task.Factory.StartNew(() => MakeMove(1)),
                                Task.Factory.StartNew(() => MakeMove(2)),
                                Task.Factory.StartNew(() => MakeMove(1)),
                                Task.Factory.StartNew(() => MakeMove(2)),
                                Task.Factory.StartNew(() => MakeMove(1)),
                                Task.Factory.StartNew(() => MakeMove(2)),
                                Task.Factory.StartNew(() => MakeMove(1)),
                                Task.Factory.StartNew(() => MakeMove(2)),
                                Task.Factory.StartNew(() => MakeMove(1)),
                                Task.Factory.StartNew(() => MakeMove(2)),
                            };
    Task.WaitAll(tasks);
}
Ignore the first line, that’s just where the collection is being initialised.  Also ignore the fact that there’s no Assert statements in this code.  The test passes if we have no threading exceptions thrown and fails if we have one.

The important thing here is to see how easy it is to fire off a lot of threads in a single, easy to read unit test without all the usual threading plumbing code that would litter something like this.

The way it works is that we define a set of tasks via the Task Parallel Library (part of .NET 4.0) each of which calls the code where we have our threading problem.  When Task.Factory.StartNew() is called the Task Parallel Library (TPL) immediately creates a new thread and calls the method returning control to our code along with a Task object so would can check the state of the task or cancel it if so desired.  In this case we don’t care and immediately start another thread as soon as possible.

We then use the Task.WaitAll statement to wait until all the Tasks we defined are completed so that the test doesn’t complete prematurely.  Too easy.

Note that we could also just as easily have used Parallel.Invoke for this.  The same test using Parallel Invoke would be something like this:

[TestMethod]
public void ParallelInvoke()
{
    InitializeControllerAndGroup();

    Parallel.Invoke(
        () => MakeMove(1),
        () => MakeMove(2),
        () => MakeMove(1),
        () => MakeMove(2),
        () => MakeMove(1),
        () => MakeMove(2),
        () => MakeMove(1),
        () => MakeMove(2),
        () => MakeMove(1),
        () => MakeMove(2)
    );
}
I personally prefer the first approach because I like the more explicit control over the thread creation, though it’s obviously noisier than the Parallel.Invoke version.  Note that with Parallel.Invoke you hand over control to the TPL and it figures out how many threads it will use to run the actions you define based on the number of cores available on the machine.

Regardless of the method you choose you can take advantage of the TPL to help you unit test your multithreaded code and make your application more resilient.

Nov 8, 2010

How to Use CodedUI Tests, Watin and MTM Together

A customer I’m working with has placed a heavy investment in Watin testing over the years and with a recent move to TFS2010 they also wanted to take advantage of the new Microsoft Test Manager (MTM) feature and the ability to associate automated tests to test cases in MTM.  Here’s a quick how-to for those of you wanting to do the same thing.

Create A Test Case

First up, let’s create a test case:

image

Pretty simple – open browser and check the url in the address bar.  What I have also done with this test is use parameters to supply the data allowing testers to decide what values they want to test with.  This is better than the devs doing this, plus it makes for a much nicer UI for data driven tests than excel or CSV files do.

Create The CodedUI Watin Test

Next we need to create the automation for this test.

Add a Coded UI test class to a test project but don’t go create any tests via the wizard that appears.  Just press cancel and then go to your test code and add a normal Watin test as per usual.  Oh, if you’re a person who normally deletes all that TestContext stuff, then you’ll need to leave it in place this time around – don’t delete it.

Here’s an example:

using Microsoft.VisualStudio.TestTools.UITesting;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using WatiN.Core;

namespace CodedUITesting
{
[CodedUITest]
public class CodedUITest1
{
...
[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.TestCase",
"http://tfs2008-vm:8080/tfs/TemplateTrials;Scrum v5",
"101", DataAccessMethod.Sequential)]
public void RunWatinTestParameterized()
{
var url = TestContext.DataRow["url"].ToString();
var result = TestContext.DataRow["result"].ToString();
using (IE ie = new IE(url))
{
Assert.AreEqual(result, ie.Url);
}
}
}

The things to note in this test are that the data source attribute is used to pull data from the TFS Test Case work item – specifically the URL of the team project is included, as it the test case id. 

In the test itself we pull data from the parameters in the test case using the TestContext.DataRow[“parameter'NameHere”].ToString() calls, and the rest is just normal Watin code.

If you have any issues with missing references make sure you reference Watin and Interop.ShDocVw and that both get copied to the output folder as part of the build.

Attach the Automation

The only thing we need to do once we have this test is to link it to the test case as it’s automation method.  Open the test case in Visual Studio, navigate to the Associated Automation tab and click the […] button to select the method.  You should see something like this once it’s done:

image

Run The Test

Assuming you have a lab environment with test agents installed you should then be able to trigger a new build and when it’s complete start an automated test run for that build and see everything working as you would expect.

Here’s a result from a run on my local TFS server:

image

I hope this helps!

Jul 6, 2010

Cleaner Mocking with NSubstitute

Rhino Mocks is a great mocking framework, being seen as one of the best ones around and rightly so, however it’s syntax at times is a little verbose and clunky.

A few of the guys from the Sydney Alt.Net group who work together at Cochlear got a little fed up with the syntax Rhino uses for creating and managing mocks and figured there must be a better way.  A different approach that will give them much cleaner code.  As a result they’ve gone and created a project to produce a clean API for mocking and released it as an open sourced project called NSubstitute.

For example, consider the following two code samples:

[Fact]
public void Rhino_MethodWasCalled()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
var keeper = new ZooKeeper {AssignedMonkey = monkey};
monkey.Stub(m => m.CurrentFleaCount()).Return(0);

keeper.CleanMonkey();

monkey.AssertWasCalled(m => m.CurrentFleaCount());
monkey.AssertWasNotCalled(m => m.Clean());
}

versus the equivalent test in NSubstitute

[Fact]
public void NSubstitute_MethodWasCalled()
{
var monkey = Substitute.For<IMonkey>();
var keeper = new ZooKeeper { AssignedMonkey = monkey };
monkey.CurrentFleaCount().Returns(0);

keeper.CleanMonkey();

monkey.Received().CurrentFleaCount();
monkey.DidNotReceive().Clean();
}

In particular, not the way mocks are created, the way a method is stubbed and how the asserts are performed.  The NSubstitute version has less code to create the mock and doesn’t require lambda’s for either the method stub or the asserts making it much more readable.  I’m finding it much nicer to use than Rhino’s syntax and for me less typing and better readability is always a good thing :-)

There’s a lot of other things that the guys have done to make mocking cleaner and I’d strongly recommend you get yourself over to the NSubstitute project site on GitHub, check out the other features and have a play.  There’s still a few rough edges and features to be implemented but the majority of usage scenarios are covered so it’s very much usable for day to day work right now. By the way, Dave Tchepak announces NSubstitute and shows a few more usage examples on his blog, which you should also go and have a look at.

Congratulations to both Dave Tchepak (@davetchepak) and Anthony Egerton (@guywithbeard) for getting this thing together.  Well done, and I hope this project really takes off :-)

Jun 10, 2010

DDD Sydney – July 17

DDD Sydney logoPut Saturday July 17th in your diaries people – it’s when DDD Sydney is on.  The community run, zero cost day where you can get your fill of technical content on a whole range of Microsoft developer related topics.

I’m lucky enough to have landed a speaking spot and so I’ll be doing a session on mocking frameworks and their various features which should (hopefully) be some fun, but apart from that there are another 14 great sessions on during the day across 3 tracks so there should be something there that takes your fancy, plus you’ll also have the opportunity for some hands on time with Windows Phone 7 development in the labs as well.

So, go ahead, check out the agenda and register before it’s all booked out. I’ll see you there!

Apr 6, 2010

How to Add Assert.Throws() to MSTest

One of the frustrations with MSTest is that it is falling behind the times for unit testing when compared to other testing frameworks.

I find it a real shame this this is happening and that MSTest didn’t gain an Assert.Throws method at any point along the way and that we still have to stick the ExpectedException attribute on our test methods, not to mention the fact that the expected message in the attribute still doesn’t get checked properly.

Now you may be asking “What’s wrong with just using the attribute on the test method?” Simple – consider what happens if the exception your looking for gets thrown by a line other than the one you were expecting it to be thrown on? You will get a false positive.  A test that passes even though it should be breaking.  Nasty stuff that.

I think the Assert.Throws() method from xUnit is so much better because it’s very explicit.  When you use it you know exactly what line the exception should be thrown on and you can also check that the message provided by the exception matches your expectations.

So, why not just use the xUnit asserts thent?  Well, we can.  We can actually keep the MSTest runner and use all the xUnit Asserts instead of the MSTest ones.  Not only do we gain the Assert.Throws() method but we also get xUnits Assert.Equals() method, which can compare arrays and lists, and more.

Here’s how…

  1. Add a reference to xUnit to your test project
  2. Add “using Assert=Xunit.Assert;” to your code
  3. Use the Assert.Throws method as you would in an xUnit test.

Pretty tough, huh?  Here’s an example of it in action:

using System;
using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Assert=Xunit.Assert;

namespace MyTests
{
[TestClass]
public class SomeSillyTests
{
[TestMethod]
public void ShouldThrowNullReferenceException()
{
string myNullString = null;
int length;
Assert.Throws(() => length = myNullString.Length);
}

[TestMethod]
public void ShouldCompareListEquality()
{
List<string> List1 = new List<string>() { "a", "b", "c" };
List<string> List2 = new List<string>() { "a", "b", "c" };
Assert.Equal(List1, List2);
}
}
}

Enjoy your new found Assert freedoms! :-)  P.S. The same techniques can be applied to using the NUnit or MBUnit Asserts or your own Assertion library of choice.

Feb 10, 2010

Australian ALM Conference

ImSpeakingButton1_389x287

I guess the picture on the left says it all, but just in case you have images turned off I’m speaking at the Australian ALM Conference coming up this April.

I’ll have two sessions that I’ll be foisting on an unsuspecting public, the first of which will be on Unit Testing in Visual Studio 2010 and the second of which will be on Lessons Learned from Agile Implementations.

I’m looking forward to the conference given it will be occurring in conjunction with the launch of Visual Studio 2010, that there are a number of international speakers coming along and that it will be held on the shores of Sydney Harbour at Luna Park.  If you haven’t already booked then take a few minutes and go bug your boss to see if they will spring for a ticket for you.  It will be well worthwhile, and may just change the way you approach software development.

Nov 28, 2009

Unit Testing Code Contracts with xUnit

Code contracts are great and I really recommend you use them, but how do they fit into a TDD/BDD flow of development?

When you do test first development you really want to write your unit test and then use the code contracts to satisfy those tests and hit some of those boundary conditions.  You could also use Pex, but Pex is an “after the code is written” tool and really only fits in as a “did I cover all the edges” tool.

The latest versions of code contracts have removed the ability to catch the exception thrown by the contracts when assertions are off, so you can just do a plan Assert.Throws() check.  instead you need to rely on xUnit’s ability to catch an assert failure in a method call and wrap that into it’s own internal exception class – the Xunit.Sdk.TraceAssertException.

First – make sure you’ve got run time checking of contracts set to throw Assert failures in the project properties:

image

Then you can just do code like the following:

public class ContractTestClass
{
[Fact]
public void ShouldPassRequires()
{
Assert.DoesNotThrow(() => AClass.PositiveInt(1));
}

[Fact]
public void ShouldPassEnsures()
{
Assert.DoesNotThrow(() => AClass.AnotherPositiveInt(3));
}

[Fact]
public void ShouldFailRequires()
{
Assert.Throws<TraceAssertException>(() => AClass.PositiveInt(-1));
}

[Fact]
public void ShouldFailEnsures()
{
Assert.Throws<TraceAssertException>(() => AClass.AnotherPositiveInt(-2));
}
}

public class AClass
{
public static void PositiveInt(int i)
{
Contract.Requires(i >= 0);
}

public static int AnotherPositiveInt(int i)
{
Contract.Ensures(Contract.Result<int>() >= 0);
return i;
}
}

Nov 27, 2009

It’s Amazing The Difference a Little Practice Makes

A few days ago I wrote about code katas and the poor effort I thought the first one was for me and how eye opening that was.  Well that effort has spurred me on to improve myself and I’ve stuck with it over the last few days by going through the kata a number of times on train trips and late at night trying to improve the time it takes and how efficiently I can get to done and now I’m able to run through the kata in a matter of minutes, as you can see here.

As you can also see, I still make plenty of mistakes in my typing, I think I make better use ReSharper and I tend to type out variable names long hand instead of relying on auto completion to help me out.  To that end I’ve still got a ways to go to get it right but even so it’s a big improvement on the first run.  And if you’re wondering, I’ve put the video here not to boast, but to act as a baseline against which to measure my improvement over time and also so I can get feedback from you for further improvement.  So, what else can I do to improve?  What other mistakes am I making?  I’d like to know.

Nov 16, 2009

Raising Events With Rhino Mocks AAA Syntax

There’s various posts on the web showing how to raise events in Rhino Mocks but they typically show you how to do it using the Record/Replay syntax, which I personally find quite awkward.

I was just helping out someone today and showing them how to do it using the Arrange, Act, Assert (AAA) syntax in Rhino Mocks 3.5 and I thought you might be interested in know how to do this as well.

Let’s start with a basic scenario… let’s assume that in our class under test we wanting to check that when we call a method on another object, that it will raise a CancelEvent and we can listen to that event and take action as appropriate.  I like to test behaviour in my classes, so in my test I want to be sure that my class under test correctly subscribes to the event and that it will set the cancel flag correctly..

Here’s some code.
public interface IProcessor
{
 string AMethodThatRaisesAnEvent(int value);
 event EventHandler<CancelEventArgs> AboutToProcess;
}

public class EventListener
{
    IProcessor processor;
    public EventListener(IProcessor processor)
    {
        this.processor = processor;
        this.processor.AboutToProcess += HandleTheEvent;
    }

    public string MakeItHappen()
    {
        return processor.AMethodThatRaisesAnEvent(0);
    }

    public void HandleTheEvent(object sender, CancelEventArgs args)
    {
        if (sender is IProcessor)
        {
            args.Cancel = true;
        }
        return;
    }
}
So as we can see, the constructor takes in the processor object and starts listening for the AboutToProcess event.  When it fires it checks the senders type and and sets the Cancel flag.  It’s a silly thing to do in reality since normally you would check property values on the sender and decide wether to cancel or not, but for the purposes of the post, it will do.

Now let’s write our behaviour test.
[Test]
public void TheEventListenerShouldCauseProcessingToBeCancelled()
{
    var processor = MockRepository.GenerateStub<IProcessor>();
    var args = new CancelEventArgs();
    var listener = new EventListener(processor);
    processor.Stub(p => p.AMethodThatRaisesAnEvent(0)).IgnoreArguments()
        .Do(new Func<int , string>(value =>
                {
                    processor.Raise(x => x.AnEvent += null, processor, args);
                    Assert.IsTrue(args.Cancel);
                    return string.Empty;
                }));
    listener.MakeItHappen();
}
So what’s happening here?

In the first few lines we arrange the objects we want for the test, then we get into the juicy bit.

It may look a little scary but if you break it down into it’s components, it’s actually prettty easy to grok.  First we set up a stub for when the AMethodThatRaisesAnEvent() is called.  We ignore any argument values that may be passed into it and we then supply a method implementation using Rhino’s Do() method.

Inside Do we supply a new function that takes an int as a parameter and returns a string – the same signature as the AMethodThatRaisesAnEvent() method that we are stubbing.  Inside it we Raise() the event we want to fire and have an Assert that will check if the Cancel flag has been set to true.  Note that we also need to supply a return value otherwise the code won’t compile.

Look a little closer at the Raise() method.  You’ll see that you don’t just say what event to raise, you actually supply a lambda that subscribes to the event you want to raise.  It looks a little wierd but it’s a workaround for the language limitations.  The second two parameters are the event sender and eventargs parameters.

Finally we make a call on our class under tests MakeItHappen() method.  There’s no asserts aftrer that because we are only wanting to check the event handling behaviour, not the return value from the mock object.  Also, if you set a breakpoint inside the Do method, you’ll see that the event raising happens after the MakeItHappen() call, even though it appears before it in the source.

And there you have it – raising events from mock objects using Rhino Mocks.

P.S. Some of you will have noticed that I’m using Stubs instead of Mocks.  This is because I’m not asserting anything on the fake object itself and I’m simply stubbing out code.  Please don’t get too hung up on this.  There’s a whole world of pointless arguments about what to call fake objects; stubs, mocks, fakes, whatever.  I certainly don’t care and the code works just the same if you use GenerateMock and processor.Expect() so do what you’re comfortable with and ignore the storm in a teacup argument over what to call them.

Jan 16, 2009

Decoupling Your Code, By Example

Let’s just say, for arguments sake, that you’ve run across a piece of code that is tightly coupled, i.e. there are lots of hard links between classes, it’s hard to test, changes are difficult, the code breaks when it shouldn’t, etc.  Let’s also say, for arguments sake, that you want to improve the design of that code and make it somewhat more maintainable so that you don’t spend the rest of your natural life trying to keep the code from completely falling apart.

How would you do that?

In this screen cast I’ll walk through a sample application, show where the tight coupling occurs and how to remove it.  In the process I’ll also be cover a bit of unit testing vs integration testing, mock frameworks, and inversion of control containers (Unity in this case).

Hopefully you get some value from it.

Download the WMV (93Mb, 44mins)

Double click for full screen

References:

For those interested in the Visual Studio add-ons they saw, I am using:

Finally, the video is hosted at http://silverlight.live.com.

Oct 20, 2008

Unit Testing WCF Services

I've seen a lot of people test WCF services using integration tests.  You know, where you create a small program as a test harness that creates a client for a web service, calls the service and checks what happened.  Similarly when you want to test your client side code is using the service properly then you have to have your client talking to something, so you need to have a test service running and that can be a pain at times, especially if the service is hosted somewhere in the cloud.

Testing a WCF Service

Testing WCF services themselves is actually quite straightforward.  Since a WCF service library is really just a normal class library it means that WCF services can can actually be called and tested using NUnit, MSTest or your favourite xUnit framework without needing a proper WCF client at all as I'll show you here:

Let's start by adding a WCF Service to a service library:

image

When we do, we get two new files added to our project - MyService.cs and IMyService.cs.

MyService.cs is just a simple skeleton as shown here:

namespace WcfServiceLibrary1
{
public class MyService : IMyService
{
public void DoWork()
{
}
}
}

The interface likewise is also very simple:

namespace WcfServiceLibrary1
{
[ServiceContract]
public interface IMyService
{
[OperationContract]
void DoWork();
}
}

Nothing to it.  So let's implement something simple so that we at least have something to test :-)  How about we change DoWork() to return an int with the current hour of the day and update the interface definition to match.  Something like this:

        public int DoWork()
{
return DateTime.Now.Hour;
}

Now, being good programmers, we want to make sure our incredibly complex method works as expected so let's write a test for it.  Remembering that the MyService class is just a normal class we don't actually need to do anything via WCF at all.  Just test it like so:

        [TestMethod()]
public void DoWorkTest()
{
MyService target = new MyService();
int expected = DateTime.Now.Hour;
Assert.AreEqual(expected, target.DoWork());
}

I know the example is a little too simplistic, but you should get the idea.


Testing a WCF Client


So now that we have a working service how do call it?  And how do we test our client?  Now when we add a WCF service reference in our client project Visual Studio will generate the code for a WCF service client - it's this service client that our application will use, so is there any value in testing the client generated by Visual Studio itself?  Probably not.  So the question then becomes: how do we test that the WCF service client is being used correctly by our application code.


Let's start an example by adding a service reference to our WCF Service we created above, similar to what's shown here.  This is what we'll use in our client application.


image


Visual Studio goes ahead and adds the reference to the solution and behind the scenes creates a MyServiceClient class that we can then call from our application as shown in the following code:

        public string CallMyServiceClient()
{
int result;
MyServiceClient client = new MyServiceClient();
result = client.DoWork();
return result.ToString();
}

Code like the above is something you will see in many places around the web, however we have just made life a lot harder for ourselves than we need to.  Can you see the problem?  If I want to test the CallMyServiceClient method I have to instantiate a WCF client and call it, which also means I also have to have a WCF service running and ensure all my WCF end points are configured correctly for the test environment.  It's OK when it's a simple like this, but in a large application with many services and people involved this sort of thing can get out of hand very quickly.


Further, if I wanted some way to see what happens to my code should the service throw an exception I'd have some difficulties so my error handling code (or lack thereof) will usually go largely untested.


Thankfully there are a few simple changes we can make to make this testability problem go away.


First, have a look at the declaration of the MyServiceClient and you'll see that Visual Studio has generated something like this:

    [System.Diagnostics.DebuggerStepThroughAttribute()]
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
public partial class MyServiceClient :
System.ServiceModel.ClientBase<WCFClientLibrary1.MyServiceReference.IMyService>
, WCFClientLibrary1.MyServiceReference.IMyService {

public MyServiceClient() {
}

The interesting thing to note here is that MyServiceClient is implementing the IMyService interface.  So if we wanted we could remove the concrete class reference and use the interface like so:

        public string CallMyServiceClient()
{
int result;
IMyService client = new MyServiceClient();
result = client.DoWork();
return result.ToString();
}

But this doesn't help much since we still have that instantiation of the MyServiceClient in our method to deal with.  We need to get that out of there.  To do that we'll use Dependency Injection - a technique where instead of the class creating the dependencies it needs, we pass into the class everything else that the class has a dependency on - in this case the MyServiceClient class.  Here's how we could do it:

    public class MyClass
{
private IMyService client;

public MyClass(IMyService wcfClient)
{
client = wcfClient;
}

public string CallMyServiceClient()
{
int result = client.DoWork();
return result.ToString();
}

So now when we create an instance of MyClass we pass in an instance of the MyServiceClient class so that MyClass doesn't have to create a reference to MyServiceClient.  We could do this either through our own application code higher up the call stack or through the use of an Inversion of Control container such as Unity, Castle Windsor or any of the other choices out there.


Now that we've introduced dependency injection we have also given ourselves a way to test MyClass without actually hitting the WCF service client at all.


What we need to do is to create a fake version of the MyServiceClient class that we can use as a substitute for the real WCF service client and pass that to MyClass instead.  We could, if we chose, create this fake class ourselves, but mocking frameworks exist to make our life easier in the regard and come with a bunch of extra features that let us do things such as setting return values when calls are made, without us having to write that ourselves.


Here's a test we might use with our client using the Rhino Mocks framework:

        [TestMethod()]
public void ClientTest()
{
IMyService mock = MockRepository.GenerateMock<IMyService>();
mock.Expect(t => t.DoWork()).Return(10);

MyClass classUnderTest = new MyClass(mock);

Assert.AreEqual("10", classUnderTest.CallMyServiceClient());
mock.VerifyAllExpectations();
}

Let's have a look at this test in a little more detail.  What are we doing here?



  1. We're creating a fake version of the IMyService class called "mock".
  2. We then tell the mock object to return the value 10 when its DoWork method is called.
  3. Next we create a MyClass instance and pass the fake class through to it in place of a real WCF service client.
  4. Then we call our classUnderTest and assert that the method returned the expected value
  5. And on the last line we also check that the DoWork() method was called on our mock object (i.e. we didn't just return from MyClass without calling the IMyService method)

This is quite useful as we now have a way to test our application code is making the appropriate calls to the WCF client without needing to actually spin up a full WCF client & service nor do we have to do any testing across the wire.  This example is also simple enough to extend so that when the DoWork() method is called we can throw a WCF exception instead to test how well our class handles failures.


I hope this helps you improve your testing with WCF in the future.  Good luck!

Aug 11, 2008

What is a Unit Test?

I've talked about unit testing a number of times in the past, however I often do so with the assumption that we have a shared understanding of what a unit test actually is.

A unit test is simply a test that ensures that a single class does what it is meant to without needing to cross classes boundaries in order to execute, with the exception of using mocks and stubs.  This is often represented using the ACID acronym, commonly talked used when referring to database transactions.:

  • Atomic
  • Consistent
  • Isolated
  • Durable

A test that verifies multiple pieces of functionality is not atomic.

A test that changes results based on external factors (such as time of day) is not consistent.

A test that hits a database, file system, network or even interacts with other business classes is not isolated.  Neither are tests that need to be run in a specific order.

A test that fails if a config files isn't set correctly, or if specific data isn't available or breaks when other classes in the application change (not including the class under test) is not durable.

It's a pretty simple way of thinking of things.

And just to be really clear a test written using a unit testing framework (NUnit, MSTest, xUnit, etc) isn't automatically a unit test either.

 

So does that mean we should only write tests to meet this criteria.  Not at all! Just don't call them "unit tests".  They are integration tests. And they are absolutely required when testing your application.  Unit testing only one part of the overall testing picture.

As a note, when I write tests I typically store unit tests and integration tests in separate assemblies to make it clear what type of tests are stored in each.  And when I do TDD I will quite happily use integration tests as a part of the TDD process - but only after I have already done as much as I can with unit tests.

Jun 17, 2008

Merging NUnit build results into TFS 2008 Team Build Logs - First Attempts

Paul Stovell was talking about TFS on the internal Readify mailing list recently and was bemoaning the fact that you can't get NUnit test results into the TFS Team Build log in Visual Studio 2008.

Well, I decided to have a crack at it today and see how hard it really was and I managed to get something going.

So here’s where we start.  A completed build with no test results showing.  Pretty normal situation.

clip_image002

Now I wanted to get something in there so I ran some unit tests (from a completely different solution as it turns out) using NUnit and saved the results to an XML file.  Then I wrote an XSLT to transform the nunit xml output into a test result file Visual Studio 2008 would load.

That took some work, and the XSLT is pretty ugly, but eventually I got it to the point where VS loaded the trx file and viewed the results. I then used MSTest to publish the results to the build log (using the /publish options) but even that was a bit problematic as it exposed some issues with GUIDs in the XSLT that I had to deal with, but in the end I got it working, albeit manually at this stage.

It does mean that the build server will need MSTest installed on it, but at least all the development team can have a bit more flexibility in their testing tools, and don't have to migrate all their Nunit tests to MSTest just to get them in the build log.

The process is pretty simple

  1. Run Nunit-console and produce an XML log file
  2. Convert the Nunit XML output to an MSTest test results file (with a .trx extension)
  3. Use MSTest /publish to push merge the trx file with an existing build

So here’s the result, and surprisingly enough it looks just like a normal TFS team build log:

clip_image004

And if I click the build results?  Yep – they load up as expected as well.

clip_image006

And clicking through to an individual test gives me something like this:

clip_image008

And because it's a published build, the test results appear in the data warehouse as well, available for reporting as seen here:

image

At this point this is all just proving a point, and it’s still very much a manual process.  Also I’ve only tried loading test results where all tests are passing but it's working and I think it's a good start for doing things properly.  Assuming there's enough interest I'll try and turn this into something that can be used in a build process - maybe write it as an MSBuild custom task or something like that.

Let me know if you'd like to see me put some more effort into this or if you've got any feedback.

May 23, 2008

MSDN Revamped

Looks like MSDN just got a face lift (according to Soma it's to make it more useful).  It's too early to tell on that front, but what I can say is that Microsoft should have figured out multilingual testing processes by now.  Seriously, this shouldn't happen:

clip_image002

Who wants to bet that the testing team is based in China...

May 7, 2008

Breaking Dependencies using Dependency Injection

Let's assume you've done some reading recently and heard talk of this Loosely Coupled Architecture thing and how it's meant to help keep an application maintainable, make it more readily changeable and will assist in the creation of unit tests .   You know that it's the right way to go and you can see all the benefits of it. You even think "I should do this on that application at work", but every time you look at the code at work and wonder what space monkeys were hired to write the code in the first place and you think about how tightly bound everything is it all starts to look too hard, so you give up and get back to working with things as they because the problem seems insurmountable.

Now a loosely coupled architecture is at it's heart just another way of saying object oriented design.  I'm sure you'll remember from when you learnt all that Object Oriented Programming stuff that there are some basic principle for OO Programming, namely:

Abstraction, Encapsulation, Polymorphism and Inheritance

And there are also some OO Design principles that sit alongside the OOP principles:

Encapsulate what changes

Favour composition over inheritance

Program to interfaces not implementations

Depend on abstractions, not concrete classes (dependency inversion)

A class should have only one reason to change (single responsibility)

Your work code and the "this problem is too big" feeling doesn't have to be the case.  It's just an indication that you've got legacy code and that it isn't well designed.  The good news is that by following some basic steps you can refactor a legacy, tightly-bound application into one with a loosely coupled architecture, though it will take some time (think many months, not weeks).

There are two basic methods for doing this.  The first is Extract & Override (see Roy Osherove's blog entry) and the other is Dependency Injection.  This is what I'll focus on here.

First, let's have a look at some code from a simple sales application:

    public class Sale
{
private List<SaleLine> saleLines = new List<SaleLine>();
private decimal totalTax;
private decimal totalValue;
...
public bool Add(string inputLine)
{
SaleLine saleLine;

saleLine = InputParser.ProcessInput(inputLine);
if (saleLine == null)
return false;
saleLines.Add(saleLine);
...

Here we see that a sale has multiple sale lines associated with it.  Pretty normal, and something you'll see in many code bases the world over, but we're breaking some of the OOD rules - we're programming to implementations not interfaces and we're using a static method (on the InputParser) which means we're tightly coupled between the Sale object and the SaleLine object.


We're also doing something else that's a no-no.  The Add method is coordinating a whole lot of work to convert an input string into a SaleLine object before adding it to the saleLines collection, whichmeans the class does more than one thing. 


Let's have a quick look at the ProcessInput method on the InputParser class as well

        public static SaleLine ProcessInput(string input)
{
SaleLine saleLine;
...
// create the sale line
saleLine = new SaleLine(quantity, name, price);
return saleLine;
}

What we have here is another tight coupling.  We're creating saleline objects from within the method (the new SaleLine() call) which means we have a tight dependency between the saleline class and the InputParser class.  If we wanted to change the SaleLine to some other class, we'd have some refactoring to do.


 


So let's make some changes...


Lets start by creating interfaces for all of our concrete classes, for example:

    public class Sale : ISale
{
private List<ISaleLine> saleLines = new List<ISaleLine>();

Now, if our definition of what a sale line is changes we don't have to make any changes in the Sale object.  Nice.


Let's also change the input parser so it doesn't create Sale objects directly.  We should do that through the use of a factory pattern implementation, such as the following:

    public class SaleLineFactory : ISaleLineFactory
{
public ISaleLine GetNewSaleLine(int lineQuantity, string name, decimal unitPrice)
{
return new SaleLine(lineQuantity, name, unitPrice);
}
}

And in our InputParser we change the code as follows:

            saleLine = saleLineFactory.GetNewSaleLine(quantity, productName, price);

Good. So now we're not actually creating the SaleLine object directly in our code.  This is a good thing as it breaks the tight coupling between the classes, but doesn't it just move the dependency from the SaleLine class to the SaleLineFactory class?


It would if we just added saleLineFactory = new SaleLineFactory() to the InputParser, but not if we pass an instance of the saleLineFactory class to the InputParser.  This is the Dependency Injection pattern at work.


Now, while we're working in the InputParser class we also have the opportunity to fix another tight coupling.  That between the Sale class and the InputParser.  Let's change the input parser so that we're no longer a static class, but rather a normal class (in a real world situation this could be complex) and let's pass to the ProcessInput method the Sale we want the new line to be added to.  We can then change the sale object so that it's Add method just adds ISaleLine objects instead of being passed a string and then calling to the parser.  Something like this:

    public class Sale : ISale
{
private List<ISaleLine> saleLines = new List<ISaleLine>();
...
public bool Add(ISaleLine saleLine)
{
if (saleLine == null)
return false;
saleLines.Add(saleLine);

public class InputParser : IInputParser
{
private ISaleLineFactory saleLineFactory;
public InputParser(ISaleLineFactory saleLineFactory)
{
this.saleLineFactory = saleLineFactory;
}

public void ProcessInput(string input, ISale sale)
{
ISaleLine saleLine;
...
saleLine = saleLineFactory.GetNewSaleLine(quantity, name, price);
sale.Add(saleLine);
return;
}

Much cleaner.  We now have no hard dependencies between our classes and we are programming to interfaces not concrete classes. 


But we're not done yet.  We still have to create a saleLineFactory class, an InputParser class, a Sale object, etc for us to be able to do any real work.  Where/when/how do we do all of this?


This is where an Inversion of Control (IoC) container can come in handy.  We could just create these objects manually at application startup, but using an IoC container is much easier.  An IoC container lets you map interfaces to classes, i.e. to say that InterfaceX is implemented by ClassY and the container will resolve these dependencies for you.  Most IoC containers let you do a whole lot more as well (Aspects for example).  For this example we'll use the Castle Windsor container to control our object creation for us:

        static void Main(string[] args)
{
IWindsorContainer container = new WindsorContainer(new XmlInterpreter());

IInputControl inputter = container.Resolve<IInputControl>();
IInputParser parser = container.Resolve<IInputParser>();

ISale sale;
string input;

sale = new Sale();
input = inputter.GetInput();
while (!inputter.EOF())
{
parser.ProcessInput(input, sale);
input = inputter.GetInput();
}

And here's the configuration file:

    <configSections>
<
section name="castle" type="Castle.Windsor.Configuration.AppDomain.CastleSectionHandler, Castle.Windsor" />
</
configSections>

<
castle>
<
components>
<
component id="prompt.input" service="Sales.IInputControl, Sales"
type="Sales.PromptInputControl, Sales" />
<component id="input.parser" service="Sales.IInputParser, Sales"
type="Sales.InputParser, Sales" />
<
component id="salelinefactory" service="Sales.ISaleLineFactory, Sales"
type="Sales.SaleLineFactory, Sales" />
</
components>
</
castle>

Now the astute among you will immediately ask why I didn't put the Sale or SaleLine objects into the container?  The answer is fairly simple - they're not good candidates for objects to place in the container.  You want to put things in the container that are service classes, not data classes.  The definition of a sale and saleline will be fairly static, plus the use of the ISaleLineFactory lets us control which ISaleLine implementation we want to use. it's also why the factory is one of the classes the IoC container is aware of - a factory class is a service that the application uses.


You might also ask, why am I using sale = new Sale() in my sample code?  Well, I could have written an ISaleFactory and used that instead (and in a real world system it's what I'd recommend) but this is a sample app, so I beg your forgiveness :-)


Finally, the really astute amongst you will notice that I never call Resolve for the ISaleLineFactory interface and I don't provide parameters for the IInputParser class when it's resolved.  So how does it get into the input parser?  Well, the cool thing about the Windsor container is that it will automatically scan the constructors for classes it's aware of, and if that constructor has a dependency (i.e. the ISaleLineFactory interface) then the container will automatically instantiate an object for that interface and pass it through.


This makes managing dependencies and instantiation of classes so much simpler, and in large applications this is invaluable.


 


Now there is one other side effect to these changes.  By having loosely coupled objects writing unit tests is a lot easier now, and they are genuine unit tests, in other words they test one thing and one thing only.  Previously I would have had an input parser test as follows:

        [Test]
public void ParseACorrectLine()
{
saleLine = InputParser.ProcessInput("1 book at 12.49");
Assert.IsNotNull(saleLine);
Assert.AreEqual("book",saleLine.ProductName);
Assert.AreEqual(1, saleLine.Quantity);
Assert.AreEqual(12.49m, saleLine.Price);
}

 


But this is effectively an integration test.  It's testing that the parser and the saleline classes are both working and working together properly.  Not an ideal situation.


But with a loosely coupled design I can now use mocking and write a test as follows:

        [TestMethod]
public void ParseACorrectLine()
{
mocksRepository = new MockRepository();
saleLineFactory = mocksRepository.DynamicMock<ISaleLineFactory>();
sale = mocksRepository.DynamicMock<ISale>();
parser = new InputParser(display, saleLineFactory);
Expect.Call(saleLineFactory.GetNewSaleLine(1,"book",12.49m)).Return(new SaleLine(1,"book",12.49m));
mocksRepository.ReplayAll();
parser.ProcessInput("1 book at 12.49", sale);
mocksRepository.VerifyAll();
}

Now my test is checking that the interactions between the InputParser and the other classes are correct, not that the other classes are functioning correctly as well (which is what the first test did).  Note that the return of a new SaleLine object is just there to provide a return value for the call to the saleline factory.  Not having it would cause the test to fail.


 


Hopefully this is enough to get you started on improving that crappy code you have to work with every day and making it a more loosely coupled application, and as always I'd love some feedback :-)

Jan 20, 2008

An Introduction to Test Driven Development

As part of my recent Readify Developer Network presentation on Agile Development for .NET Developers I did a quick demo on test driven development and unit testing.  I've made a 10-minute video for the TDD section of the presentation and put it on YouTube so you can look at it any time you like.  Enjoy :-)

 

Please note that the slides from the presentation and a higher resolution video will be available from the RDN site shortly.

Dec 12, 2007

Assert.Equals and MSTest

If you are writing unit tests using MSTest then be aware of the Assert.Equals method as shown here:

Assert.Equals(string1, string2);

Instead of using the .Equals() method you should actually be using

Assert.AreEqual(string1, string2);

- OR -

StringAssert.Equals(string1, string2);

The problem is that Assert.Equals is just the inherited Object.Equals method. It does nothing but compare object equality (not value equality) and return a true or false value instead of breaking the unit test if the items are not equal.  Because the true/false return value is ignored nothing untoward will happen and the unit test will continue through giving us a false sense of security that our unit test is passing.

This is a common mistake for those of us who are used to NUnit testing as the original line is a correct assert under NUnit.

The good news is that for those using VS2008, you don’t have to worry as much since using Assert.Equals will fail any unit test it's used in:

clip7