“Smart Unit Tests” Are “Pinning Tests”

You’ve probably seen all the various announcements from Microsoft today, with the most notable being the release of Visual Studio 2015 Preview and the jaw dropping move to open source the .NET Core Framework. Fantastic stuff! Though I won’t go into it in detail here.

What I do want to cover are the “Smart Unit Tests” released with the VS2015 Preview, based on the Microsoft Pex research project. The messaging around it has created more than a little confusion. In fact, here’s what one of my colleagues on Twitter sent my way:

“Smart Unit Tests” is what happens when you don’t go outside. //cc: @rbanks54

— Pawel Pabich (@PawelPabich) November 13, 2014

During the MVP Summit I spent some time with the people who built this feature to better understand what the intent behind these so called “smart” unit tests. Is it simply so we can tell our managers that we can hit that magic 100% unit test coverage mark now, meet our KPI’s, claim our annual bonus for hitting an arbitrary metric, and still be able to produce rubbish quality applications or is there some other value to it?

It turns out that the purpose of the Smart Unit Test feature is to help people in their refactoring efforts. It’s good practice before you go and refactor code to ensure you have a healthy level of test coverage in place so that you can be confident that any changes you make to code don’t break the behaviour of the application. On existing, legacy code without tests, this is often a very expensive and difficult process, and as a result many refactoring efforts occur either without any test coverage at all or simply never happen because they’re too scary.

This is where “Smart Unit Tests” come into play. They help you do the “smart” thing and create unit tests that help you pin down the current behaviour of the code, before you do any refactoring.  These type of tests are known, unsurprisingly, as pinning tests.

Here’s a good description of a pinning test (source), emphasis added:

A “pinning test” is not a good test. It doesn’t try to be. It’s just the simplest, fastest-running test you can write that will allow you to refactor your code. It’s often an end-to-end test, but it could also look at your log files, monkey-patch a core library function, or do something similarly ridiculous.

The key here is that the pinning test lets you do your dozens-of-tiny-refactorings loop really quickly. As the code improves, you add high-quality unit tests. Once the tests are good enough, you get rid of the pinning test. Lather, rinse, repeat.

Hopefully that helps clear up some confusion. Oh, as a final note, Brian Harry mentioned that the Smart Unit Tests name is likely to change. Let’s hope so! :-)

Visual Studio 2013 Cookbook now available

About 18 months ago my Visual Studio 2012 Cookbook was published by Packt Press. Fast forward to a few weeks ago (March 25th in fact, I’m a little tardy on getting this post out) and an updated version of my book has been published, the Visual Studio 2013 Cookbook.

I’ve been a little time poor over the last 12 months so I need to give a big shout out to Jeff Martin who made a large contribution to the book and without who it wouldn’t have happened. Thanks so much, Jeff!

For those wondering what’s new, here’s some of the major changes in this version:

• Updates throughout the entire book to keep it in line with the various VS2013 changes (no surprises there!),
• A major update to the web development chapter based on the ASP.NET changes, editor improvements and new tooling such as Browser Link,
• A major update to the Team Foundation Server chapter, with particular focus on the new Team Explorer and git source control integration, and
• A new chapter on using TypeScript and Python with Visual Studio.

As with the previous edition, this book isn’t for everyone. It’s for those people wanting to get up to speed with the changes and improvements in VS2013 and is going to be more suited to those coming to VS2013 from older editions of Visual Studio, or those who are used to Eclipse or XCode and are wanting to know how to get to grips with an unfamiliar, and very powerful, IDE.

Go buy yourself a copy from Packt or your favourite book selling web site today!

Visual Studio Online (Monaco) prompting for credentials in IE

There’s a funny/strange behavioural difference when clicking the ‘Edit in Visual Studio Online’ link in the Windows Azure management portal between Internet Explorer, Firefox and Chrome.

In Chrome you will be taken straight into the development environment and the Monaco editor as expected. That said, if you watch the network trace in Fiddler you’ll actually see that there’s an initial failed initial authentication request before a second attempt is made with a different set of request headers.

On Firefox you get a slightly different behaviour and will be asked if you want to log in using an account based on your site name with a $ prefix, as shown here:

But in Internet Explorer it’s different again and you get prompted for your credentials.

If you try and log in by entering your Microsoft Account (Live Id) details it won’t work. You could try using the $ prefixed account that Firefox asked you about, but what password would you use? I’ve got no idea.

So what’s going on here?

It turns out that this is an IE security behaviour that occurs when the first auth challenge fails. Because Monaco in VSO uses same infrastructure as Azure git deployments (the Kudu project) to connect to the web site’s host virtual machine, it’s not your Microsoft Account but your Azure deployment credentials that you are being prompted for. I presume it’s your Microsoft Account details used in the first, rejected, request and then after that you need to put in your credentials. Why Chrome allows you to connect with a different account on the second request, I’m unsure. I’d expect that a security nerd could fill me in on all the gory details.

In any case, if you can’t remember what those Azure deployment credentials are (why aren’t you using a password manager!) then you can always reset them from the Azure portal. Be aware that this changes the credentials used for all your git based deployments on all your Azure sites.

I’m hoping that this is only a temporary difference in behaviour and that future updates fix the inconsistencies. Until then, enjoy using VSO and Monaco! :-)

A Visual Studio Online screencast

Last night at the Sydney Alt.Net User Group I presented a 30 minute session on Visual Studio Online, and more specifically how you can build an Azure hosted website without ever leaving the browser.

That session was recorded and is now available and embedded here for your viewing pleasure.

I hope you find it useful!

Visual Studio 2012 Quick Tips - Quick Launch

Visual Studio Quick Tips are a bit of an experiment to see if this is a useful way of presenting information and also as a mechanism for me to play around with video production :-)

So, to kick this off, here's a short overview of the Quick Launch feature in Visual Studio 2012



Feedback is, as always, very much welcome.

What’s Visual Studio’s Code Map All About?

In Visual Studio 2012 Update 1 (VS 2012.1) Microsoft delivered a feature for the Ultimate edition called Code Map with the goal of visualising relationships in code.

In Visual Studio 2012 Update 2 (VS 2012.2) they’ve extended the Code Map experience to include debugging support and the ability to generate code maps on the fly as a debugging session is conducted.

If you saw the announcement then you may have had one of the following reactions. Reaction 1 would be something like “Oh. It’s in Ultimate. I don’t have it. I’ll ignore it”. The second might be similar to mine: “Is this really going to be that useful?”. After ignoring it for a bit I decided to answer that question for myself.

For the purposes of this post, I’m used the excellent RestSharp project. You can grab it from https://github.com/restsharp/RestSharp. Oh, if you’re not aware of what it does, it’s a library that help’s you build REST based client and server applications by taking away a lot of the plumbing work you would otherwise have to do.

Getting Started

To start begin by asking a simple question. How does RestSharp make a client request? I’d never looked at the internals of RestSharp before so I wasn’t sure where to look.

Fortunately RestSharp comes with a set of unit and integration tests that I could use to explore the code with. I started with the integration tests and tried to find a suitable test in there to use and came across the Handles_Non_Existent_Domain() test. That seems a reasonable place to start. Note that if you want to follow along at home you’ll need the XUnit.Net Test Runner for Visual Studio 2012 extension installed since the tests are written for xUnit.

First thing to do? Set a breakpoint on the first line of the Handles_Non_Existent_Domain() test and start debugging the test. When the breakpoint is hit open a Code Map for the debugging session by either clicking the button in the toolbar (highlighted in the screenshot), choosing Show Call Stack on Code Map from the Debug menu, or pressing Ctrl+Shift+` if you’re using the default keyboard mappings.

Visual Studio will now split the document window and display a second document pane with the code map in it. Initially the code map will looks like this:

Not very interesting, right? Don’t worry, it gets better. Let’s start stepping into various methods to see how this test works.

Stepping Deeper

Press F11 to step into the RestClient constructor. As you do, you will notice the code map updating. Continue to step through the code until the first call to AddHandler() has completed. Your Code Map should now show something like the following:

Note that as we step back up the call stack that the colours of nodes in the Code Map change to reflect that methods we have called but that are no longer in the call stack. It helps us keep track of where we’ve been, not just where we are now.  This is kinda cool. I no longer have to mentally keep this tree in my head, and can focus more on what the code is doing rather than where I’ve been and which methods might be relevant later on.

Depending on your resolution, you may also start noticing the zoom level on your Code Map keeps resetting. If you want it to use a specific zoom value then go to the layout options and turn off “Automatically Layout when Debugging”

What Happens When You Skip Over Methods?

The code map only updates when execution pauses. It means that if you step over methods the Code Map will not show you what happened in the code you stepped over. It also means that if you use multiple breakpoints and after hitting the first breakpoint continue execution until you hit the next one, then the Code Map will only reflect the call stack from when the breakpoints were hit. You won’t see anything related to what happened in between breakpoints.

It helps keep the noise in the map down, and presumes that if you’ve skipped over code you’re also not interested in seeing it in the map. That seems to be a reasonable assumption to me.

Anyway, let’s get back to the test we were stepping through and continue execution until we get to the line that calls the Client.Execute() method.

As you step into the method the Code Map will update. Look at the tooltip for the newly added node and you will see details of the method itself based on XML Doc comments, as shown:

This can be handy when you’re deep within the bowels of something and can’t remember exactly how you got to where you are or which specific overload of a method was called.

If we then step into the switch statement, we see that we’re passing in a value of Method.GET. It might be handy to remember that; especially as both parts of the switch statement call Execute(). We can make a note of that on the Code Map by right clicking the node and selecting Add Comment (or hitting Ctrl+Shift+K)

At various point you’ll notice that the some calls go through external code first. The code map will reflect this when it occurs by marking the fact on the call stack arrow, as shown.

Way Down Deep

If you continue to step through the code you will eventually get to the RestSharp.Http.GetRawResponse() method and your Code Map will start to look somewhat busy, as follows:

A lot of the nodes in the Code Map are not that interesting now that we’ve stepped through things and are improving our understand of how it fits together. It’s fast becoming noise. To organise the map better we could simply select nodes we’re not interested in anymore and delete them or we can choose to group them by selecting a few, hitting the mouse right-click context menu and choosing Add Parent Group, to get something like the following:

We can then simply collapse groups to hide items we’re not really interested in at the moment.

With a little bit of judicious grouping I produced a Code Map as follows and now I have something that helps me better navigate the internals of RestSharp and get familiar with it.

Wrapping it Up

I can add to this map over time as I explore the code further and step through other tests to see how it works, I can also get a feel for which parts of the code are called often and what happens in those methods.

If I wished I could also save the code map and share it with the rest of team, or grab an image of it and post it in the team wiki for documentation purposes or put it on the wall as a reference.

While Code Maps might seem a little gimmicky at first, I’m finding them fast becoming an useful tool in helping me better understand code I’m wading through and in improving my knowledge of what exactly is happening when I’m using an application.

Give Code Maps a try (assuming you have a copy of Ultimate) and see what you think.

Git is now an option for TFS Source Control

Yes, it’s true. Hell has frozen over!

After much waiting the TFS team have today announced that Git is a fully supported source control choice for TFS, and that Visual Studio 2012 now has tooling available for working with Git repositories, built right into the VS shell.  I know, right?! Oh, and it’s not some Microsoft half baked git like implementation, it’s the full thing.

Here’s the announcement on Brian Harry’s blog: http://blogs.msdn.com/b/bharry/archive/2013/01/30/git-init-vs.aspx

And here’s a quick getting started post on the VSALM team blog: http://blogs.msdn.com/b/visualstudioalm/archive/2013/01/30/getting-started-with-git-in-visual-studio-and-team-foundation-service.aspx

So what’s stopping you? If you haven’t already created an account on http://tfs.visualstudio.com then get to it. Download the bits for VS2012, create yourself a team project with Git as your source control provider and see what it’s like.

P.S. In case you’ve forgotten, for those using a TFS server with the standard version control system, you can use Git-Tf to put a local git repository in front of it.

Visual Studio 2012 Cookbook is now available

I’m pleased to announce that my Visual Studio 2012 Cookbook is now available from Packt Publishing.  Amazon and other distributors should have it available shortly.

As a developer you should always know how to make the most of the tools at your disposal and the Visual Studio 2012 Cookbook is a great way to reduce the learning and discovery time for your shiny new IDE. The book is a “ramp up” book that aims to quickly familiarise you with the major new features of VS2012 and assumes you have a working knowledge of a prior Visual Studio version.

Because it’s a ramp up book it won’t be for everyone, however it’s priced so it can be more time efficient to buy the e-book (or tree-book if you must) and use it to learn the new features rather than trying to find the same information by either exploring the app click by click or scouring the web looking for “what’s new” articles.

Go ahead, buy a copy today, and don’t forget to get one for your Mum as well!  I’ve got starving children to feed!

How To Prevent Visual Studio 2012 ALL CAPS Menus!

For all those people who can’t stand the ALL CAPS menus in Visual Studio 2012 there’s a way to switch them to normal casing.

Crack open your registry editor and create the following registry key and value

HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\11.0\General\SuppressUppercaseConversion
REG_DWORD value: 1

Here’s what it looks like BEFORE

And here it is after the change

You’re all very welcome!  Now go tell your friends

The Visual Studio 11 Interface, With More Metro

Without getting into the “It sucks! No, It rocks!” debate about the new user interface there are some areas that people are particularly fired up about, including the use if colons as a placeholder for menu bars.

On the Visual Studio User Voice site there’s a request to tweak the UI and go “all in” with the Metro styling borrowing from the Zune interface (an interface I happen to particularly like).

Let’s have a look at the current UI as posted on the Visual Studio teams blog entry Introducing the New Developer Experience:

Now let’s look at the proposed user interface with it’s makeover (via http://i.imgur.com/Dh0zV.png):

Not to be left as the sole alternative, here’s a slightly different take on it (via http://i.imgur.com/EWH1R.png)

I’m not campaigning for votes for the feature (and at this stage the UI is going to be mostly set in stone in any case), but what I do want to do is point out that a consistent focus on the Metro design principles makes a difference.

If you go Metro, go full metro, not some halfway house. The alternate user interfaces work in that they remove as much window chrome and scroll bar noise as possible, letting developers focus more on the content and they realise that if typography is king, then make typography king!  Fix menu fonts, title fonts, and so forth and be consistent about it.  By improving the typography they also managed to retain the shouty tab names in such a way that they don’t grate anywhere near as much as the current UI does.  The UI also feels cleaner because the ::::: lines aren’t there adding unnecessary noise.

It’s interesting to see how much of a difference the Metro design language can actually make to a user interface, and definitely something to consider if you get asked by your users to create a Windows 7 or desktop application that feels like a Windows 8 Metro style application.

Improved Unit Testing with Visual Studio 11 Beta

There’s just so much new stuff in the Visual Studio 11 Beta! In fact, someone should write a book about it… Oh wait, I am! (more on that when the time is right). For now, let’s have a look at one feature that makes me so very happy: Visual Studio’s new and improved unit testing capabilities.
It’s widely recognised by those with a desire to do unit testing that there are better unit test frameworks out there than MSTest but given that Visual Studio has always been so tightly coupled with MSTest it’s always been more difficult than it should be to get other test frameworks working well in Visual Studio.  The TestDriven.NET and ReSharper test runners have helped, but the integration back to visual studio was always lacking.
That all changes with the Visual Studio 11. Now it’s a case of “Use the framework that makes you happy. We don’t mind”.
You want to use XUnit? No problem!
NUnit? Easy!
MSTest (without the baggage it usually has)? Sure thing!
QUnit for JavaScript? Bring it on!
Something random that we’ve never heard of? Write an adapter and it’ll work just fine!

Dogs and Cats Living Together!

Visual Studio 11 introduced a test adapter model so that any test framework can run inside Visual Studio if there is an adapter for it.  The adapter model also means that you can not only run XUnit, NUnit, or any other type of test but you can run them in the same test assembly if you really wanted to! Why? I don’t know! But you can :-)
Maybe you have a suite of MSTest tests but you also want use XUnit’s data driven test features because MSTest sucks for unit testing. You can do just that.  It’s really nice.
Note: In MSTest’s favour, in this release when MSTest is used in a plain old class library for unit testing the MSTest test adapter uses a cut down, light weight version of MSTest with just the features needed for unit testing and none of the baggage that it normally comes with, making it quite usable for most unit testing needs. For most developers with an existing investment in MSTest tests they will see an improvement in performance as a result.

Pre-Requisites

Get yourself started by loading the appropriate adapter for your unit test framework from the Visual Studio Extension Manager.  MSTest is already in the box so you don’t have to worry about that one.  In this screen shot I’ve loaded up the XUnit, NUnit and Chutzpah test adapters.

Running Unit Tests

Create a new C# Assembly project (NOT a test project) and add the XUnit and NUnit test frameworks to your project using NuGet

You can also add a reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework so that you can do MSTest based unit tests.
Here, I’ve started by adding a simple XUnit test, then building the code and running the unit test as shown:

As you can see in the output window there is now a “Discover test started” phase where Visual Studio looks at the assemblies and determines what tests are in the system so that it can spin up the right framework and execute the tests.
The Unit Test explorer on the left shows the tests that were run, the time it took and any error information for failed tests.
In the same code I have then added MSTest and NUnit tests as shown, however at this stage I have not yet built the project – take note of the unsaved changes icon in the document tab, indicating that the project is neither saved nor built as yet:

Run Tests After Build (Almost Continuous Testing)

Continuous Testing is the idea that as you do your work all the unit tests are constantly running in the background and giving you live feedback when there are problems code by highlighting where tests have failed and where your code is broken.  The immediate feedback cycle makes test driven development an even faster development process since there’s no waiting around for all the tests to run.
Visual Studio has taken a step towards this ideal with the “Run Tests After Build” option as shown in the image below. Turn that setting on and as soon as you compile your code Visual Studio will run the tests automatically on a background thread so that you don’t end up with a blocked UI and can get on with coding the next thing on your list.

If a test failed the unit test explorer goes red and it’s obvious that there’s a problem.  The interesting thing is that in smaller projects the tests often run so fast that you don’t even notice them happening!

As a tip, once you’ve been using the test after build feature for a while you will probably want to stop the Output window from popping up every time you build so that you don’t have to keep closing it.  You can do this in the Visual Studio Options as shown

Don’t Forget JavaScript Unit Tests!

OK, I won’t.  Make sure you have the Chutzpah test adapter Visual Studio extension installed (see above).
In a standard web project include QUnit or Jasmine in your project and then create a JavaScript file for your tests.  Once you have your tests written run them as you normally would and Chutzpah will do the tricky work of finding the tests and running them.  Here’s a screen shot of a web project with a QUnit test in it

Conclusion

So there you have it! A brief overview of the new unit testing features in Visual Studio 11 Beta. Go and get it now and start playing with it.
Having Visual Studio automatically running your tests each time you do a build will change your development workflow for the better and keep you more focused on coding and help you stay in the mythical zone if you ever get there.
Remember that since all tests run in the background and this removes the time spent waiting for tests to finish that you can have tens of thousands of tests taking minutes to run each time and you won’t even feel a delay in your development activities.  Fantastic!