Showing posts with label mocking. Show all posts
Showing posts with label mocking. Show all posts

Aug 6, 2010

Mocking Comparison – Part 12: The UnMockables

The frameworks we’ve been looking at in previous parts only work well when they can override the properties of the classes you wish to mock, or when you use interfaces. So what happens when you need to deal with a test that is time dependant, or if you want to verify data being written to the console or you have to deal with SharePoint or other similar products that are a dogs vomit of static classes, sealed types and leaky abstractions that make testing with them almost impossible using normal techniques?

The answer is to move away from regular mocking frameworks and use tools like TypeMock Isolator, Microsoft Moles or Telerik Just Mock (as a note, I don’t have a copy of JustMock so I’m going to leave it out of this series)

DateTime.Now

So let’s say you’ve got a class that references DateTime.Now and changes behaviour based on the time of day. How do you test behaviour correctly and more importantly,how do you do this reliably?  You can’t really mess with the system clock.  The trick is to intercept the call to DateTime.Now and provide your own method that gets called instead, and this is what TypeMock and Moles both allow you to do.

TypeMock

Here’s the basic code for mocking out DateTime.Now with TypeMock

readonly DateTime dateToUse = new DateTime(2010, 07, 17, 8, 0, 0);

[Fact]
public void Isolate_current_time()
{
Isolate.WhenCalled(() => DateTime.Now).WillReturn(dateToUse);
Assert.Equal(dateToUse, DateTime.Now);
}

Pretty simple stuff.

If you wanted to provide a method implementation rather the just returning a value you would replace the .WillReturn() call with .DoInstead()

So let’s see what this looks like in a test.  Once again, this is somewhat testing the mock which you shouldn’t do in the real world, but it does show you what the syntax is like, which is the goal after all.

Oh, as a reminder the relevant code from the class under test is as follows:

public void CleanMonkey()
{
// <snip!>
if (AssignedMonkey.IsAwake(DateTime.Now))
AssignedMonkey.Clean();
}

So in our test we are going to set the mock to only return true if the time passed to the IsAwake method matches our specific date time.  In other words, we should see that DateTime.Now is returning the value we specify, not the current time.

[Fact]
public void Isolate_monkey_should_be_awake()
{
Isolate.WhenCalled(() => DateTime.Now).WillReturn(dateToUse);

var monkey = Substitute.For<IMonkey>();
monkey.CurrentFleaCount().Returns(20);
monkey.IsAwake(Arg.Is(dateToUse)).Returns(true);

var keeper = new ZooKeeper();
keeper.AssignedMonkey = monkey;

keeper.CleanMonkey();

monkey.Received().Clean();
}

And as expected, this test works.  Now the keen eyed amongst you will see that I’m actually mixing TypeMock Isolator and NSubstitute in the one test.  I’m using Isolator to mock out the static DateTime.Now method and NSubstitute to create the mock monkey object.

Also, in terms of running, the tests there’s a few little changes to the tests and things take longer because Isolator is injecting itself in your code.  The other thing to note is that DateTime is part of the Base Class Library and TypeMock doesn’t support all of the methods in the BCL, just certain methods so if you want to mock out a method Isolator doesn’t support you’re on you own.

Mircosoft Moles

Moles is a framework that is usually obtained with the Pex download from Microsoft Research, but it’s also available in a standalone form.

Moles is still very much a product with some rough edges.  I’ve had moles randomly stop working with the only fix being an un/reinstall at times.

Now the good thing about Moles is that it can replace the method call for anything you want.  No limitations at all, however the way it does it is to inspect the assembly you want to mock and then it generates a separate buddy library that provides a way to mock out or stub the method calls you are interested in.  It’s fine when working with things like mscorlib or other assemblies that don’t really change, but if you’re applying it to your own libraries and they change method signatures regularly then you will need to regenerate the moles assemblies each time which is a pain.

The following code is using the Moles.Xunit extension to run the tests you can’t just run the tests via TestDriven.Net like you can with Isolator – you need to call the tests via the Moles TestRunner.  I use a batch file as follows to help with this:

cd c:\MyCode\bin\debug
"C:\Program Files (x86)\Microsoft Moles\bin\moles.runner.x86.exe" Monkeys.Moles.Tests.dll /runner:c:\path\xunit-1.5\xunit.console.x86.exe /x86
pause

You’ll also need to update assemblyinfo.cs to include attributes that tells Moles which methods you are using like so:

[assembly: MoledType(typeof(System.DateTime))]
[assembly: MoledType(typeof(System.Console))]

And once that’s in place finally you can write a test as follows:

[Fact]
[Moled]
public void Replace_current_time()
{
MDateTime.NowGet = () => dateToUse;
Assert.Equal(dateToUse, DateTime.Now);
}

[Fact]
[Moled]
public void Monkey_should_be_awake_for_cleaning_at_eight_am()
{
MDateTime.NowGet = () => dateToUse;
var monkey = Substitute.For<IMonkey>();
monkey.CurrentFleaCount().Returns(20);
monkey.IsAwake(Arg.Is(dateToUse)).Returns(true);

var keeper = new ZooKeeper();
keeper.AssignedMonkey = monkey;

keeper.CleanMonkey();

monkey.Received().Clean();
}

So here you may notice that we have an extra attribute on our test method to indicate that this test method uses Moles.  In addition we provide a lambda to MDateTime.NowGet instead of DateTime.Now.  This is the method in the buddy assembly that gets called when DateTime.Now is referenced.  Also, the method naming for Moled types gets a bit funny.  Methods are named based on the method you are mocking and then either a list of types based on the method being called or a Get/Set for properties.  It works, but it’s a little clunky.

Conclusion

So, my preference here from a syntax viewpoint is the Isolator syntax by far, though the fact that it can’t get to everything in the BCL (such as Console.WriteLine) and that it is a commercial product takes the shine off a little.  On the flip side I like the power of Moles but it’s very clunky to work with, breaks easily and is a pain to use when the target assembly changes a lot and the price and power doesn’t offset this problem.

That’s All Folks

And that, my friends, is that for this series.

I hope you’ve enjoyed it and have a good idea of what the various frameworks are capable of.  If you haven’t already guessed, my new favourite framework is NSubstitute.  If you haven’t already, go give it a try and see what you think and give feedback to the guys that wrote it on the NSubstitute mailing list.

Happy testing!

 

Other posts in this series:

Aug 5, 2010

Mocking Comparison – Part 11: Multiple Interfaces

Continuing with our comparison of Rhino Mocks, Moq and NSubstitute we have a look at a little used feature in mocking being the ability to generate mocks that implement multiple interfaces.

Why would you do this though? Well, that’s a good question.  Simple example would be when your class under test expects and object to implement interface X and also implement IDisposable.  It doesn’t happen often, but when it does it’s nice to know the facility is there.

What you’ll see in all the examples is that the mock natively implements a main interface and that to do any interactions with the methods of the second interface requires casting of the mock to that interface.

For the purposes of the code we’re going to pretend that the monkeys of our little zoo are self managing, can act as ZooKeepers and can thus look after themselves.  It’s silly, but it shows the syntax.

Rhino Mocks

The thing to note here is that we can’t use GenerateStub here.  We have to use GenerateMock, which then means we don’t get automatically backed properties, so we have to set them up ourselves as well.

[Fact]
public void Rhino_multiple_interfaces()
{
var monkey = MockRepository.GenerateMock<IMonkey, IZooKeeper>();
monkey.Stub(m => m.Name).PropertyBehavior();
((IZooKeeper)monkey).Stub(k=> k.AssignedMonkey).PropertyBehavior();

monkey.Name = "Spike";
((IZooKeeper)monkey).AssignedMonkey = monkey;

Assert.Equal("Spike", ((IZooKeeper)monkey).AssignedMonkey.Name);

Assert.IsAssignableFrom<IMonkey>(monkey);
Assert.IsAssignableFrom<IZooKeeper>(monkey);
}

You can also see that we have to cast monkey to IZooKeeper every time we want to do something on the IZooKeeper interface.  Annoying, but that’s the way it goes.

Moq

The code here is a little different in that we create the mock the normal way, and then add a new interface to it after it’s already created using the .As<T>() method.

Also, when we set up the property behaviour on the IZooKeeper interface we have to go through some ugly casting and the use of Mock.Get() because of the way Moq separates the Mock and mocked object instances.  Blech.

[Fact]
public void Moq_multiple_interfaces()
{
var monkey = new Mock<IMonkey>();
monkey.As<IZooKeeper>();
monkey.SetupProperty(m => m.Name);
Mock.Get((IZooKeeper)monkey.Object).SetupProperty(k => k.AssignedMonkey);

monkey.Object.Name = "Spike";
((IZooKeeper)monkey.Object).AssignedMonkey = monkey.Object;

Assert.Equal("Spike", ((IZooKeeper)monkey.Object).AssignedMonkey.Name);

Assert.IsAssignableFrom<IMonkey>(monkey.Object);
Assert.IsAssignableFrom<IZooKeeper>(monkey.Object);
}

NSubstitute

This code is by far the smallest because we don’t need to set up the property behaviours.  It’s uses the same approach as Rhino to create the mock itself, and still has the issues of needing to cast to the IZooKeeper interface, but apart from that it’s once again nice and clean code.

[Fact]
public void Nsubstitute_multiple_interfaces()
{
var monkey = Substitute.For<IMonkey, IZooKeeper>();

monkey.Name = "Spike";
((IZooKeeper)monkey).AssignedMonkey = monkey;

Assert.Equal("Spike", ((IZooKeeper)monkey).AssignedMonkey.Name);

Assert.IsAssignableFrom<IMonkey>(monkey);
Assert.IsAssignableFrom<IZooKeeper>(monkey);
}

 

The choice of syntax here is again, quite easy to make.  NSubstitute takes it out.

This also represents a conclusion to the posts focusing on Rhino Mocks, Moq and NSubstitute in the mocking comparison series.  I’m not quite done though as I want to show some usage scenarios where those frameworks don’t work and where tools like Microsoft Moles and TypeMock fit in, so stay tuned – we’re not quite done yet! :-)

 

Other posts in this series:

Aug 4, 2010

Mocking Comparison – Part 10: Events

So far in our comparison we’ve been looking at mock objects as if they were much like any other object, but what happens when we want our mocks to either raise or subscribe to events?

If you’re testing how your class under test reacts when it receives an event, or want to know if it raises an event with correct values then you really need your mock framework to be able to support this.

Subscribing To Events

To get a mock object to subscribe to an event is pretty easy.  Just do it like you normally would, and then if you want to assert anything about how the event was raised, simply assert that the subscription method was called as expected.

Rhino Mocks

[Fact]
public void Rhino_event_subscriber()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
var keeper = new ZooKeeper();

keeper.OnBananaReady += monkey.BananaReady;

keeper.FeedMonkeys();
monkey.AssertWasCalled(m => m.BananaReady(
Arg<object>.Is.Equal(keeper)
,Arg<BananaEventArgs>.Matches(b => b.IsRipe
)));
}

As you can see, we subscribe to the event normally, with the event being raised by the FeedMonkeys() call.

We then check that the event was called correctly and that the IsRipe flag was set correctly.

Moq

[Fact]
public void Moq_event_subscriber()
{
var monkey = new Mock<IMonkey>();
var keeper = new ZooKeeper();

keeper.OnBananaReady += monkey.Object.BananaReady;

keeper.FeedMonkeys();
monkey.Verify(m => m.BananaReady(keeper,
It.Is<BananaEventArgs>(b => b.IsRipe)));
}

The syntax is much the same as for Rhino Mocks apart from the fact that the .Object. syntax again makes things feel clunky.  On the positive side of things, the better constraint syntax in Moq makes the verify call far less noisy.

NSubstitute

[Fact]
public void Nsubstitute_event_subscriber()
{
var monkey = Substitute.For<IMonkey>();
var keeper = new ZooKeeper();

keeper.OnBananaReady += monkey.BananaReady;

keeper.FeedMonkeys();
monkey.Received().BananaReady(keeper,
Arg.Is<BananaEventArgs>(e => e.IsRipe));
}

The NSubstitute version is looks much the same as the Moq syntax, just without the .Object. stuff.

Raising Events

Raising an event is a pretty simply process.  We simply get our class under test to subscribe to an event on our mock object and then ask our mock to raise an event.

Rhino Mocks

[Fact]
public void Rhino_raising_an_event()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
var tourist = new Tourist();

tourist.SeeAMonkey(monkey);

monkey.Raise(m => m.Dance += null, monkey, new EventArgs());
Assert.Equal(1, tourist.PhotosTaken);
}

The line to pay attention to here is the second last one.  The monkey.Raise() call.  What you notice in here is that in order to raise an event we need to pass in an expression that registers an empty event listener.  This is so Rhino can pick up the signature of the event you wish to raise.

Once that’s done we just provide arguments for the sender and event args that we wish to use for the event.

It’s a little weird looking but it works.

Moq

[Fact]
public void Moq_raising_an_event()
{
var monkey = new Mock<IMonkey>();
var tourist = new Tourist();

tourist.SeeAMonkey(monkey.Object);

monkey.Raise(m => m.Dance += null, new EventArgs());
Assert.Equal(1, tourist.PhotosTaken);
}

The Moq code is much the same as the Rhino code, with the only difference being that by default the sender is the object raising the event and we don’t need to specify it.

NSubstitute

[Fact]
public void Nsubstitute_raising_an_event()
{
var monkey = Substitute.For<IMonkey>();
var tourist = new Tourist();

tourist.SeeAMonkey(monkey);

monkey.Dance += Raise.Event(new EventArgs());
Assert.Equal(1, tourist.PhotosTaken);
}

Here you see the code is much simpler.  We still need to register a pseudo event listener when we want to raise the argument, but at the same time, it’s a little more obvious what’s going on.

All three frameworks suffer from needing to use event subscriptions to figure out what event to fire, but that’s a result of limitations in the way C# works rather than a poor design decisions.  With that in mind the verdict on which syntax to choose goes to NSubstitute.  Since event subscription is much the same across all 3 frameworks there’s not much to differentiate them, however event raising in NSubstitute is cleaner and more expressive than the syntax of both Rhino and Moq, making my choice rather easy. 

 

Other posts in this series:

Jul 30, 2010

Mocking Comparison – Part 9: Functions

Let’s say you’ve got a mock object and you want a method on your mock to do something with a particular method that goes beyond just returning a specified value.  What do you do?  Why, you use a function of course!  When you stub a method call on a mock you simply tell the mock object to call a method or use a lambda to calculate the return value rather than just returning a specified value.

Let’s have a look at how Rhino Mocks, Moq and NSubstitute do this.

What we’re going to do is supply some simple methods that our mock object will use when we call certain methods so that we can dynamically alter the return value from the CurrentFleaCount method and we’ll then assert that the value has changed.

Rhino Mocks

[Fact]
public void Rhino_functions_and_callbacks()
{
var fleaCount = 25;
var monkey = MockRepository.GenerateMock<IMonkey>();

monkey.Stub(m => m.CurrentFleaCount()).Do(new Func<int>(() => fleaCount));
monkey.Stub(m => m.TryAddFleas(1)).IgnoreArguments()
.Do(new Func<int, bool>(fleas =>
{
fleaCount = fleas;
return true;
}));

Assert.Equal(25, monkey.CurrentFleaCount());

monkey.TryAddFleas(10);
Assert.Equal(10, monkey.CurrentFleaCount());
}

Since this is a reasonable chunk of code, let’s just look at the key parts.  When we stub the CurrentFleaCount method call we tell it to use a function that takes no arguments, and use the value of the fleaCount variable (defined in the test) to figure out what returns.

You may ask why didn’t we just do a .Return(fleaCount) like we normally would? Simple answer: we want to get the value to change as the test progresses.  By using a Func<T> means that the return value is calculated each time we call the stubbed method, rather than just having a single fixed value returned for every call.

Similarly in the stubbed TryAddFleas() method, we are providing a function that takes one parameter and returns a boolean.  Our stub method simply changes the value of fleaCount variable so that the next CurrentFleaCount() call will return a different value, and this is what the Asserts are checking for.

Hopefully that makes sense.  Form a day to day usage viewpoint it’s not something you do that often but when you need the functionality it’s really useful.

Moq

[Fact]
public void Moq_functions_and_callbacks()
{
var fleaCount = 25;
var monkey = new Mock<IMonkey>();

monkey.Setup(m => m.CurrentFleaCount()).Returns(() => fleaCount);
monkey.Setup(m => m.TryAddFleas(It.IsAny<int>()))
.Returns<int>(fleas =>
{
fleaCount = fleas;
return true;
});

Assert.Equal(25, monkey.Object.CurrentFleaCount());

monkey.Object.TryAddFleas(10);
Assert.Equal(10, monkey.Object.CurrentFleaCount());
}

The Moq syntax is much the same as the Rhino syntax, as you can see.  It’s probably a little nicer in that you use the same Returns() method to either return fixed values or call a function, unlike Rhino Mocks where you have to use the .Do() method.

It also doesn’t require the creation of new Func<T> objects – you just supply the method body you wish to use, which means less code.

NSubstitute

[Fact]
public void Nsubstitute_functions_and_callbacks()
{
var fleaCount = 25;
var monkey = Substitute.For<IMonkey>();

monkey.CurrentFleaCount().Returns(args => fleaCount);
monkey.TryAddFleas(Arg.Any<int>())
.Returns(args =>
{
fleaCount = (int)args[0];
return true;
});

Assert.Equal(25, monkey.CurrentFleaCount());

monkey.TryAddFleas(10);
Assert.Equal(10, monkey.CurrentFleaCount());
}

The NSubstitute code is a little different to the others in that instead of having parameter lists to deal with you are passed a single array containing all the parameters and it’s up to you to pull out the ones you wish to use and cast them as required by your test.

This has it’s advantages in that you avoid code littered with arguments you never use, but it has a downside in that you need to cast every parameter you do use.  That said, just like Moq, you get to use the same Returns() method for either returning a fixed value or supplying a method and you don’t need to create new Func<T> objects so the code isn’t noisy.  And better than Moq is it’s overall cleaner syntax, as we have seen in many of the previous posts.

My preference? NSubstitute.  The casting of parameters is a little annoying, but it doesn’t have Moq .Object. tax and this is one of the few times you’ll actually see a lambda in a test using NSubstitute.

 

Other posts in this series:

Jul 29, 2010

Mocking Comparison – Part 8: Recursive Mocks

Recursive mocks (or nested mocks as some people call them) are a feature of the mocking frameworks where they will automatically create mock objects for items they should return if you reference a property or method of that object when setting up expectations.  The advantages of doing this are that we have less “arrange” code in our test classes and can focus our efforts more on our act and assert code.

Maybe that explanation of recursive mocks doesn’t clarify things much for you so let’s just look at some code instead and see how Rhino Mocks, Moq and NSubstitute help us out.

Rhino Mocks

[Fact]
public void Rhino_recursive_mocks()
{
var monkey = MockRepository.GenerateStub<IMonkey>();
monkey.Name = "Spike";
monkey.Stub(m => m.Keeper().AssignedMonkey).Return(monkey);
Assert.NotNull(monkey.Keeper());
Assert.Equal("Spike", monkey.Keeper().AssignedMonkey.Name);
}

If you have a look at the code you can see that we create a stub object for the the IMonkey interface.  Within that we set up an expectation that when we check which monkey the keeper is assigned to that it should be the monkey object we just created (i.e. we’re setting up a circular object reference).

We don’t need to create a specific IZooKeeper stub object here as Rhino will do that for us and if you step through the test in debug mode you can see that we have a fake object created for us:

image

Moq

The code for Moq is much the same apart from the fact that we have to set up the property behaviour for the monkey’s name and remember if we’re dealing with the mock or the object it’s returning.

[Fact]
public void Moq_recursive_mocks()
{
var monkey = new Mock<IMonkey>();
monkey.Setup(m => m.Keeper().AssignedMonkey).Returns(monkey.Object);
monkey.SetupProperty(m => m.Name);
monkey.Object.Name = "Spike";
Assert.NotNull(monkey.Object.Keeper());
Assert.Equal("Spike", monkey.Object.Keeper().AssignedMonkey.Name);
}

Interestingly, if you look at the code in the final assert closely you’ll see that the mock/object distinction is lost when we start using the recursive mocks.  Instead of using Keeper().Object.AssignedMonkey we just have Keeper().AssignedMonkey.  It’s not a big deal, but it seems to break the paradigm that Moq has used elsewhere of separating the Mock from the object itself.

NSubstitute

And finally the NSubstitute approach

[Fact]
public void Nsubstitute_recursive_mocks()
{
var monkey = Substitute.For<IMonkey>();
monkey.Keeper().AssignedMonkey.Returns(monkey);
monkey.Name = "Spike";
Assert.NotNull(monkey.Keeper());
Assert.Equal("Spike", monkey.Keeper().AssignedMonkey.Name);
}

Nice and clean.  No lambdas. Expressive and direct.  What more could you want :-)

P.S. At time of writing the recursive behaviour was only available when building from source.

 

The verdict? NSubstitute is the clear winner here.

 

Other posts in this series:

Jul 28, 2010

Mocking Comparison – Part 7: Exceptions

One of the great benefits of using mocks in your tests is being able to check how your classes behave when exceptions are thrown.  You do test for this sort of thing, right? :-)

Let’s say we have a persistence mechanism that talks to SQL.  What happens when we get a SQL primary key violation?  What if we are talking to web services and the connection drops or we get some other WCF exception?  How do we handle that?

More importantly, how do we get Rhino Mocks, Moq or NSubstitute to raise those exceptions.  Let’s have a look.

Rhino Mocks

[Fact]
public void Rhino_throwing_exceptions()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
monkey.Stub(m => m.Name).Throw(new ApplicationException());
Assert.Throws<ApplicationException>(() => monkey.Name);
}

This is pretty simple code.  Instead of returning a value, just throw an exception when the method is called.  Too easy!

Moq

[Fact]
public void Moq_throwing_exceptions()
{
var monkey = new Mock<IMonkey>();
monkey.Setup(m => m.Name).Throws(new ApplicationException());
Assert.Throws<ApplicationException>(() => monkey.Object.Name);
}

The Moq code is much the same as the Rhino code, except we still have that .Object. tax that annoys me so much.

NSubstitute

[Fact]
public void Nsub_throwing_exceptions()
{
var monkey = Substitute.For<IMonkey>();
monkey.Name.Returns(args => { throw new ApplicationException(); });
Assert.Throws<ApplicationException>(() => monkey.Name);
}

Not quite as clean as the other two frameworks here, though I’m sure the guys will rectify this soon enough.

 

And the winner: Rhino Mocks. Even though Moq requires slightly less code than Rhino it’s .Object. distinctions will always annoy me.

 

Other posts in this series:

Jul 26, 2010

Mocking Comparison – Part 6: Multiple Calls

Our comparison of Rhino Mocks, Moq and NSubstitute continues with a look at how multiple calls to a mock are handled and what you do if you want to alter the return values on subsequent calls.

Consider the scenario where you have a method you’re calling that you want to be successful the first time you call it, but where subsequent calls should fail (such as trying to save the same data twice, etc).  How do you do this?

Rhino Mocks

In Rhino Mocks to set return values for multiple calls to a method you simply specify the return value that many times.  To avoid duplication of code you can use the repetition syntax we saw in Part 5.

[Fact]
public void Rhino_multiple_calls()
{
var monkey = MockRepository.GenerateMock<IMonkey>();

monkey.Stub(m => m.TryAddFleas(1)).Return(true).Repeat.Twice();
monkey.Stub(m => m.TryAddFleas(1)).Return(false);

Assert.True(monkey.TryAddFleas(1));
Assert.True(monkey.TryAddFleas(1));
Assert.False(monkey.TryAddFleas(1));
}

Given that we have set up the return value 3 times, what happens when we make the call a fourth time?  At this point Rhino Mocks, having exhausted it’s known set of return values, will start returning the default value.

What if you just want to return the value True for every call?  We would use the Repeat.Any() method as shown here:

[Fact]
public void Rhino_multiple_calls()
{
var monkey = MockRepository.GenerateMock<IMonkey>();

monkey.Stub(m => m.TryAddFleas(1)).Return(true).Repeat.Any();

Assert.True(monkey.TryAddFleas(1));
Assert.True(monkey.TryAddFleas(1));
}

Moq

Moq doesn’t have a nice way of defining mixed return values (not that I can find anyway) however you can implement the same thing using callbacks, which we’ll cover in a later post.

By default setting a return value on a method call will always return that value no matter how many times it is called, so we can at least do the following:

[Fact]
public void Moq_multiple_calls()
{
var monkey = new Mock<IMonkey>();
monkey.Setup(m => m.TryAddFleas(1)).Returns(true);

Assert.True(monkey.Object.TryAddFleas(1));
Assert.True(monkey.Object.TryAddFleas(1));
}

NSubstitute

The code for mixed return values in NSubstitute is so nice.  To handle multiple calls providing different return values you can just provide multiple values or use an array in the return statement as follows:

[Fact]
public void Nsubstitute_multiple_calls()
{
var monkey = Substitute.For<IMonkey>();

monkey.TryAddFleas(1).Returns(true,true,false);

Assert.True(monkey.TryAddFleas(1));
Assert.True(monkey.TryAddFleas(1));
Assert.False(monkey.TryAddFleas(1));
}

And if you just want to always return true, then it works just like Moq does (with the benefit of not requiring the .Object. tax in the assert statements:

[Fact]
public void Nsubstitute_multiple_calls()
{
var monkey = Substitute.For<IMonkey>();

monkey.TryAddFleas(1).Returns(true);

Assert.True(monkey.TryAddFleas(1));
Assert.True(monkey.TryAddFleas(1));
}

If you can’t tell, my decision on the best framework to choose goes to NSubstitute.  It has such a simple and elegant way to handle both requirements, and again, not a single lambda in site.  Excellent!

 

Other posts in this series:

Jul 23, 2010

Mocking Comparison - Part 5: Repetitions

In Part 4 we looked at how parameter constraints are handled and in Part 3 we looked at how to do interaction based tests with mocks.  In this part we bring those two pieces together and add a little extra to check if a call was made a specific number of times.

Let’s just jump straight into the code shall we?

Rhino Mocks

Our test here is simply going to call a method a number of times and verify that the call was made the correct number of times using constraints to verify the correct calls.  It’s a completely useless test, other than as a vehicle to show you how to do this sort of thing

[Fact]
public void Rhino_repetitions()
{
var monkey = MockRepository.GenerateMock<IMonkey>();

monkey.TryAddFleas(5);
monkey.TryAddFleas(-1);
monkey.TryAddFleas(9);

monkey.AssertWasCalled(m => m.TryAddFleas(0),
options => options.Constraints(
Is.GreaterThan(3) && Is.LessThanOrEqual(10)
)
.Repeat.Twice());
monkey.AssertWasCalled(m => m.TryAddFleas(-1),
options => options.Repeat.Once());
}

Note the important part, the .Repeat.Twice() and .Repeat.Once() calls.  It’s these calls that define our expectations as to how many times the call should have been made.

Rhino also features a .Repeat.Time(n) call you can use as well if once or twice don’t cut it for you.

Moq

[Fact]
public void Moq_repetitions()
{
var monkey = new Mock<IMonkey>();

monkey.Object.TryAddFleas(5);
monkey.Object.TryAddFleas(-1);
monkey.Object.TryAddFleas(9);

monkey.Verify(m => m.TryAddFleas(
It.IsInRange(3,10,Range.Exclusive)
),
Times.Exactly(2));
monkey.Verify(m => m.TryAddFleas(-1), Times.Once());
}

Instead of using the word Repeat, Moq uses Times.  Apart from that there is little difference.

NSubstitute

Unfortunately NSubstitute doesn’t support this feature yet as it’s still a maturing framework.  If you really need to do this type of testing then you’ve got a few options – use Rhino or Moq, contribute to the NSubstitute project, or to just not do this type of testing :-).

Interaction based testing is valid at times, but it’s generally brittle and is usually a sign of “implementation verifying” tests rather than behaviour/specification verifying tests (but that’s an argument for another time)

 

Verdict: Moq wins out in this case simply because it’s constraint system is more terse than the Rhino one, but this is purely a personal taste thing.

 

Other posts in this series:

Jul 22, 2010

Mocking Comparison – Part 4: Parameter Constraints

Continuing with our comparison of Rhino Mocks, Moq and NSubstitute we now turn our eye to how constraints are managed.

Consider a method with parameters that was want to mock.  By default, a mock object will only return a specified value if the parameters passed to it exactly match the call signature specified to the mock object.  Maybe that’s a bit wordy, so as a quick example if you say Method(“a”).Returns(“Fred”) you’ll only get “Fred” back from the mock when you pass “a” as the parameter.  Passing “b” gets you nothing. Make sense?

OK, so what if I then want my mock object to provide different return values when different parameter values are used.  How do we avoid writing too much code or having to work out exactly what we should expect as a parameter every time?  And what if I don’t care what’s passed in, I just want a return value.

This is where constraints come into play.  Let’s look at some code to show how it works:

Rhino Mocks

Here’s a test where we try to add fleas to our monkey.  Adding positive numbers of fleas should succeed and adding a negative number of fleas should fail.

[Fact]
public void Rhino_no_constraints()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
monkey.Stub(m => m.TryAddFleas(5)).Return(true);
monkey.Stub(m => m.TryAddFleas(-1)).Return(false);
monkey.Stub(m => m.TryAddFleas(9)).Return(true);

Assert.Equal(true, monkey.TryAddFleas(5));
Assert.Equal(false, monkey.TryAddFleas(-1));
Assert.Equal(true, monkey.TryAddFleas(9));
}

Notice how the use of explicit parameter values in the stubs means we have to repeat a lot of code.  Hmm, that’s annoying.

Thankfully we can simplify this by just ignoring the arguments as seen in the code below.  Note that we still have to supply a parameter value for the TryAddFleas lambda in the Stub call, even though it’s ignored.

[Fact]
public void Rhino_ignore_arguments()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
monkey.Stub(m => m.TryAddFleas(0)).IgnoreArguments().Return(true);

Assert.Equal(true, monkey.TryAddFleas(5));
Assert.Equal(false, monkey.TryAddFleas(-1));
Assert.Equal(true, monkey.TryAddFleas(9));
}

However this will now cause the test to fail because we no longer return a false when passed -1 as a value.

So what we really want is to only return true when the argument value is in a certain range.  For fun let’s make that range between 4 and 9 inclusive.  Here’s the test now, using constraints:

[Fact]
public void Rhino_constraints()
{
var monkey = MockRepository.GenerateMock<IMonkey>();

monkey.Stub(m => m.TryAddFleas(0))
.Constraints(Is.LessThan(10) && Is.GreaterThan(3))
.Return(true);
Assert.Equal(true, monkey.TryAddFleas(5));
Assert.Equal(false, monkey.TryAddFleas(-1));
Assert.Equal(true, monkey.TryAddFleas(9));
}

Whilst this means I have 2 less Stub calls to make, it does tend to be a little verbose.  Expressive, but verbose.  For those wondering why I don’t have a constraint for the -1 argument, I’m relying on the standard behaviour of mocks.  If a constraint isn’t matched then the standard mock behaviour is to return the default value of the return type, being false in this case.

Moq

Here’s the same thing in Moq.  For brevity and completeness I’ve included a commented out line that shows what you would do if you just wanted to ignore parameter values

[Fact]
public void Moq_constraints()
{
var monkey = new Mock<IMonkey>();

//monkey.Setup(m => m.TryAddFleas(It.IsAny<int>())).Returns(true);
monkey.Setup(m => m.TryAddFleas(It.IsInRange(3, 10, Range.Exclusive)))
.Returns(true);
Assert.Equal(true, monkey.Object.TryAddFleas(5));
Assert.Equal(false, monkey.Object.TryAddFleas(-1));
Assert.Equal(true, monkey.Object.TryAddFleas(9));
}

The IsInRange method is quite nice, and the range constraints can be either inclusive or exclusive, however the code feels as verbose if not more so than the Rhino approach.

Note that if you use a test harness like MSpec (Machine.Specifications) then the “It” static class that Moq uses clashes with the “It” class used by MSpec for defining specifications which makes writing code a little painful at times.

NSubstitute

Finally the NSubstitute version.

[Fact]
public void Nsubtitute_constraints()
{
var monkey = Substitute.For<IMonkey>();

//monkey.TryAddFleas(Arg.Any<int>()).Returns(true);
monkey.TryAddFleas(Arg.Is<int>(count => count > 3 && count < 10))
.Returns(true);

Assert.Equal(true, monkey.TryAddFleas(5));
Assert.Equal(false, monkey.TryAddFleas(-1));
Assert.Equal(true, monkey.TryAddFleas(9));
}

This works much the same as the others with the difference being that you have a predicate as the parameter and you need to supply the parameter type to the Arg.Is<T> call.

From a readability perspective, there’s less code which is good and it’s easier to read than the other frameworks because of it’s terseness.  As a bonus, using the Arg.Any<T> call means we can avoid lambdas completely..

 

Verdict: I’ll go with the NSubstitute version as my first choice, though it’s more a matter of style choice than anything else. After that I prefer the Moq syntax over Rhino in this case.

 

Other posts in this series:

Jul 21, 2010

Mocking Comparison – Part 3: Interactions

Our comparison of Rhino Mocks, Moq and NSubstitute continues with a look at how these frameworks support interaction based testing (whether that’s a good idea or not is not going to be dealt with here!).

The idea behind interaction based testing is checking if our class under test makes appropriate calls to the mocked objects.  Typically this is done when testing against API’s that want things done in a certain order or where you want to check that the Save method of a repository was called, as an example.

For our example code we’re going to be testing that the Keeper makes the appropriate call to check the current flea count and then, because the flea count is low, does not try and clean the monkey, i.e. Monkey.Clean() should not be called.

Note that code for the classes under test is in Part 1 of this series.

Rhino Mocks

[Fact]
public void Rhino_method_was_called()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
var keeper = new ZooKeeper {AssignedMonkey = monkey};
monkey.Stub(m => m.CurrentFleaCount()).Return(0);

keeper.CleanMonkey();

monkey.AssertWasCalled(m => m.CurrentFleaCount());
monkey.AssertWasNotCalled(m => m.Clean());
}

So in the initial arrange section of our test we stub out the CurrentFleaCount() call to return zero fleas when called.

We then perform the Act part of our test and ask the Keeper to clean the monkey.

Finally in the Assert part of our test we check if the monkey had methods called on it – the CurrentFleaCount method should be called, and the Clean method should not be called.

If you’re playing along at home, you may have noticed that changing the monkey’s flea count to 100 still makes the test pass, when we should have expected the test to fail (because the clean method should now be called).  This won’t happen because there is a second guard clause in the keeper’s CleanMonkey() method that checks if the monkey is awake and this will always return false.

It brings up an important point with all mock objects in that the default response for any method not explicitly stubbed out is to return the default value, and the default for a bool is false.

Moq

[Fact]
public void Moq_method_was_called()
{
var monkey = new Mock<IMonkey>();
var keeper = new ZooKeeper() {AssignedMonkey = monkey.Object};

monkey.Setup(m => m.CurrentFleaCount()).Returns(0);

keeper.CleanMonkey();

monkey.Verify(m => m.CurrentFleaCount());
monkey.Verify(m => m.Clean(), Times.Never());
}

The code here is similar to Rhino’s code however you’ll notice that to configure our mock object in Moq we use Setup instead of Stub/Expect calls to set the return behaviour for the flea count method.

Moq uses a single Verify method to check that a call was made, so to check that something wasn’t called you have to check that it was called zero times – expressed in Moq syntax as Times.Never().  Make your own mind up over whether this is clear enough from a readability perspective, but I find that it feels a little like Yoda has been writing code and the Verify call seems like it should have a boolean statement for its parameter, not be asserting that a method was called.

NSubstitute

[Fact]
public void Nsubstitute_method_was_called()
{
var monkey = Substitute.For<IMonkey>();
var keeper = new ZooKeeper { AssignedMonkey = monkey };
monkey.CurrentFleaCount().Returns(0);

keeper.CleanMonkey();

monkey.Received().CurrentFleaCount();
monkey.DidNotReceive().Clean();
}

Firstly, not the lack of lambda methods anywhere in this code.  The mock object behaviour is setup by simply attaching a .Returns(0) to the monkey.CurrentFleaCount() method making the code more expressive and simpler to read.

As far as the asserts are concerned we check that a call was Received() by our mock object or that the mock DidNotReceive() the call.  I find this syntax better than the other frameworks in that there’s no lambdas, but it doesn’t read quite as well as I might like.

 

The verdict: It’s a split decision between Rhino and NSubstitute, with me wanting the best of both.

 

Other posts in this series:

Jul 20, 2010

Mocking Comparison – Part 2: Properties

Carrying on with our Rhino Mocks, Moq and NSubstitute comparison, let’s now look at how the various frameworks handle properties on their mock objects.

Rhino Mocks

So in RhinoMocks you have two choices for creating mock objects – using GenerateMock<T>() or GenerateStub<T>().  GenerateMock will create a mock object that tracks calls made to it so that you can assert expectations against which calls were made at the end of your test.  GenerateStub on the other hand creates a stub object, which is to all intents and purposes the same as a mock object just without any of the overhead for tracking which calls were made against it.  In other words you can do asserts to check if a call was made on a mock, but not on a stub.

Why does this matter?  Because properties on mocks and stubs have different default behaviours in RhinoMocks.

For a mock object we have to explicitly indicate if we want a property to have a standard getter/setter applied to it (i.e. just like an autoproperty implementation).  Here’s some code

[Fact]
public void Rhino_properties()
{
var monkey = MockRepository.GenerateMock<IMonkey>();
monkey.Stub(m => m.Name).PropertyBehavior();

monkey.Name = "Spike";
Assert.Equal("Spike", monkey.Name);
}

and the same test with a stub

[Fact]
public void Rhino_properties()
{
var monkey = MockRepository.GenerateStub<IMonkey>();

monkey.Name = "Spike";
Assert.Equal("Spike", monkey.Name);
}

Note that the stubbed object requires none of the property behaviour setup and so it feels more natural.  While some people may prefer to see properties being set up explicitly rather than implicitly I think that if it is a well known behaviour then it’s not going to be a problem that the property behaviour is implicitly provided on the stubbed objects.

Moq

The Moq code is much the same, however as you can see we have to pay the Moq .Object. tax on our calls.

[Fact]
public void Moq_properties()
{
var monkey = new Mock<IMonkey>();
monkey.SetupProperty(m => m.Name);

monkey.Object.Name = "Spike";
Assert.Equal("Spike", monkey.Object.Name);
}

This makes the code feel just a little more verbose and cumbersome than the Rhino and NSubstitute versions.

In fairness, if you want all properties to be automatically set to the standard autoproperty behaviour for a mock object you can use the SetupAllProperties method in Moq as follows:

[Fact]
public void Moq_properties()
{
var monkey = new Mock<IMonkey>();
monkey.SetupAllProperties();

monkey.Object.Name = "Spike";
Assert.Equal("Spike", monkey.Object.Name);
}

NSubstitute

The NSubstitute code shown below is much the same as the Rhino Mocks code when using stubs.  The difference being that NSubstitute doesn’t differentiate between stubs and mocks.

[Fact]
public void NSubstitute_properties()
{
var monkey = Substitute.For<IMonkey>();

monkey.Name = "Spike";
Assert.Equal("Spike", monkey.Name);
}

Overall, Rhino and NSubstitute are on par, however the more expressive mock object creation and reduced confusion over whether to use mocks or stubs in NSubstitute tilts the balance in favour of NSubstitute.

 

Other posts in this series:

Mocking Comparison – Part 1: The Basics

I recently gave a talk on mocking at the DDD Sydney conference called “You Look Like A Monkey and You Smell Like One Too”.  In it I not only got to call the audience various names and get mocked in return, but I also showed a number of features of mock frameworks and how they can be used.  Essentially it was a comparison of a number of mock frameworks, with a particular focus on a few of the main open source ones out there and a great new comer – specifically RhinoMocks, Moq and NSubstitute.

This is part 1 of series running through the code I had on screen and providing some thoughts on the pros and cons of each framework in the context of the scenario I’m showing.  This first part will be longer than the others simply because we have to put some ground work in place.  So, let’s stop wasting time and get started…

What Are We Testing?

All right, so in order to do any mocking we really should have an application to test.  For the purposes of this series the code is a simple class library.  It does nothing useful in the real world, but it does give us something to test.  It’s a pseudo “zoo” featuring monkeys, zoo keepers and tourists.

The IMonkey Interface

Every zoo needs a monkey, and thus we need a monkey right? Of course we do! However, because the monkeys haven’t been tested yet,  the monkeys aren’t yet on display (or at least we haven’t written the code for them yet) so all we have to interact with for now is an IMonkey interface as follows:

public interface IMonkey
{
string Name { get; set; }
bool TryAddFleas(int numberOfFleas);
int CurrentFleaCount();
void Clean();
bool IsAwake(DateTime timeOfDay);
void BananaReady(object sender, BananaEventArgs e);
event EventHandler<EventArgs> Dance;
IZooKeeper Keeper();
}

The IZooKeeper Interface and the ZooKeeper

Monkeys have Keepers, and since all staff in any organisation are just “resources” and thus completely interchangeable (grrr – don’t get me started on this kind of thinking!) we should have an IZooKeeper interface.  We are also lucky enough to actually have a standard ZooKeeper definition as well.  Here’s the code:

public interface IZooKeeper
{
event EventHandler<BananaEventArgs> OnBananaReady;
IMonkey AssignedMonkey { get; set; }
void CleanMonkey();
void FeedMonkeys();
}
public class ZooKeeper : IZooKeeper
{
public event EventHandler<BananaEventArgs> OnBananaReady;

public IMonkey AssignedMonkey { get; set; }

public void CleanMonkey()
{
if (AssignedMonkey == null) return;
if (AssignedMonkey.CurrentFleaCount() < 10) return;
if (AssignedMonkey.IsAwake(DateTime.Now))
AssignedMonkey.Clean();
}

public void FeedMonkeys()
{
if (OnBananaReady != null)
{
OnBananaReady(this, new BananaEventArgs(true));
}
}
}

public class BananaEventArgs : EventArgs
{
public BananaEventArgs(bool isRipe)
{
IsRipe = isRipe;
}

public bool IsRipe { get; private set; }
}

Tourists!

Every zoo needs tourists, and in this case we want concrete instances of tourists, not just some definition of what a tourist should be.  Our tourists are a little weird in that every tourist that comes to our zoo expects our monkeys to dance, and should that happen then they’ll take a photo.  Here’s the code:


public class Tourist
{
IMonkey monkey;

~Tourist()
{
if (monkey!= null)
monkey.Dance -= Look_ADancingMonkey;
}

public int PhotosTaken { get; private set; }

public void SeeAMonkey(IMonkey monkey)
{
if (this.monkey != monkey && this.monkey != null)
this.monkey.Dance -= Look_ADancingMonkey;

this.monkey = monkey;
monkey.Dance += Look_ADancingMonkey;
}

public void Look_ADancingMonkey(object sender, EventArgs args)
{
if (((IMonkey)sender).CurrentFleaCount() <= 100)
{
TakeAPhoto((IMonkey)sender);
}
return;
}

private void TakeAPhoto(IMonkey monkey)
{
PhotosTaken++;
Console.WriteLine(monkey.Name);
}
}

Record/Replay Is Dead

OK.  That’s done, now let’s get into it.  If you’ve used a mocking framework in previous years you may have used Record/Replay syntax.  The idea being you record the steps you want your mock to perform, then switch to replay mode, do the steps again and assert everything worked.  Here’s what Record/Replay looks like using Rhino Mocks in an XUnit test harness:


private string testName = "Spike";

[Fact]
public void Rhino_record_replay()
{
var repository = new MockRepository();
var monkey = repository.StrictMock<IMonkey>();

//In record mode at the moment
Expect.Call(monkey.Name).Return(testName);

//Now in replay mode
repository.ReplayAll();
var actual = monkey.Name;

Assert.Equal(testName, actual);
}

As you can see, we create a MockRepository object, ask the repository to give us a new mock object for a, call the method we want to mock out once, set the expected return value, then we switch to Replay mode and make the call again before finally doing our assertions.

From the small sample code you may be forgiven for thinking that the code isn’t all that bad, however the fact that you have to think about what mode you’re takes you out of the mode of thinking about what it is you’re actually meant to test, and that’s not a good thing. To make matters worse, this is a simple case.  When you get into complex cases, record/replay is downright painful and the amount of repeated code that has to happen in order to set up expectations is often overwhelming and confusing.

Oh, you’ll also notice that the sample code is using a StrictMock.  If this is what you’re doing, then stop it! StrictMocks are a great way to make your tests brittle and hard to maintain and should be avoided wherever possible.  Just search for why strict mocks are bad on your favourite search engine and you’ll no doubt find more information about it.

Use Arrange Act Assert Instead

Let’s now look at the same code using Rhino Mocks in AAA syntax mode.  In case you’re not aware of this, AAA syntax is a way of structuring the tests where you Arrange any pre-requisites for the test, Act on the class under test and then Assert that things occurred as expected.

[Fact]
public void Rhino_arrange_act_assert()
{
//Arrange
var monkey = MockRepository.GenerateMock<IMonkey>();
monkey.Stub(m => m.Name).Return(testName);

//Act
var actual = monkey.Name;

//Assert
Assert.Equal(testName, actual);
}

As you can see, the code is simpler, structured more cleanly and is also easier to read.  In large test libraries this makes things much more maintainable, which is a good thing.

The Moq Version

So far the code has used the Rhino Mocks framework.  Another very popular framework is Moq.  Here’s what the same test in Moq looks like:

[Fact]
public void Moq_arrange_act_assert()
{
var monkey = new Mock<IMonkey>();
monkey.Setup(m => m.Name).Returns(testName);

var actual = monkey.Object.Name;

Assert.Equal(testName, actual);
}

Moq became popular when it was released because no other test framework could do what it did in terms of setting up expectations via lambdas and providing such an easy way of performing AAA style tests.

Over time the other frameworks have caught up, and for me personally, the fact that Moq has a distinct delineation between the mock and the object being mocked (i.e. the monkey.Object code) adds clutter to the test code that I don’t like.  Others like that clarity, but I personally find it just gets in the way.

On the other hand, mock object creation is far less verbose, which is a good thing.

NSubstitute

Finally, the new kid on the block.  NSubstitute’s code looks like the following:

[Fact]
public void Nsubstitute_arrange_act_assert()
{
var monkey = Substitute.For<IMonkey>();
monkey.Name.Returns(testName);

var actual = monkey.Name;

Assert.Equal(testName, actual);
}

What you see is that the code is much terser than Rhino and Moq.  A mock is created as a “substitute for” a IMonkey and we don’t need to use lambdas to set return values for a mocked property, greatly improving readability of the test code and making test code feel more expressive.

My preference of the three for the basic syntax and coding? NSubstitute.

 

Other posts in this series: