[Test]
public void InsertTestNameHere()
{
var input = "a string";
Assert.Throws<FormatException>(() => int.Parse(input));
}
If we consider this from an Arrange-Act-Assert (AAA) perspective it’s pretty easy to see that the Act and Assert logic are in a single line. It’s a very common pattern. It works, it works well, and the readability is fine, but if we start using a BDD approach to our unit testing (e.g. with SpecFlow) or we want to explicitly keep the Arrange, Act and Assert sections of our test code separated then Assert.Throws gets in the way.[Test]
public void InsertTestNameHere()
{
var input = "a string";
var exception = Assert.Catch(() => int.Parse(input));
Assert.IsInstanceOf<FormatException>(exception);
}
In this case we’re catching any exception that int.Parse might throw. If we were more explicit and used Assert.Catch<FormatException>(), NUnit’s behaviour becomes much the same as Assert.Throws, and the test fails immediately if the expected exception isn’t detected. This isn't a behaviour we want, which is why we're using the generalised catch method. Now, since we have our exception in a variable, we can check if it’s the one we expected.[Fact]
public void InsertTestNameHere()
{
var input = "a string";
var exception = Record.Exception(() => int.Parse(input));
Assert.NotNull(exception);
Assert.IsType<FormatException>(exception);
}
The Record.Exception() method won't fail the test, regardless of what happens in the method. Unlike the NUnit approach, it merely records any exception arising from the call or returns null if no exception was thrown. Personally, I feel this approach is a better way to write tests around exceptions while still remaining consistent with the AAA approach. It also works well when using a BDD testing framework that separates the given/when/then step implementations as you can pass the result of the “when” method to the “then” methods with ease.
At the Sydney Alt.Net group last month we ran an Open Source Hack Night. The idea was to move from being a consumer of open source to being a contributor, to demystify what’s involved in an open source project, talk through some of the dos and don’ts, and the practical steps of getting started. Then we jumped in and did just exactly that over pizza and drinks. It was a fun night.
This is a recap of what we talked about on the night. I’m not going to walk through the mechanics of how to make a contribution, but will list some tips on how to find and get involved in a project and how to be successful at it.
More and more projects these days have issues that make for good “on-ramps” into the project. The web site http://up-for-grabs.net is an aggregator site for issues that meet this criteria. Have a browse of the projects on the site and see what interests you. Or just look at a project you use in your normal day to day work and see where/how you might be able to help.
They’re developers with the same time pressures and insecurities that you have. The project they’re working on was likely started to scratch an itch and then made public in the hope it would help others. Remember to interact and treat these people with the same respect that you yourself want to be treated.
If you see a project you might want to contribute to, look through the issues and pull requests to see how open the maintainers communicate and how open they are to contributions and different ideas. Lurk in the chat room (if they have one) and see what the banter is like. If you don’t like what you see, move along.
Read the contribution guidelines if they exist. And follow them! They might be in a readme.md or a contributors.md or on a wiki. If you ignore them don’t be surprised if your contribution is rejected. Pretty obvious, right
It reduces the time you spend before getting feedback from the maintainer(s) and they’re easier for maintainers to absorb. A large contribution that you have put a lot of time into only to have it rejected is extremely discouraging and can you off from ever making future contributions. Large contributions also carry a lot more risk for the project making maintainers leery of spending the time to look through if this is your first contribution. Remember that maintainers are just like you. What would you want to see from a new developer making changes in your work projects? Small, incremental changes or massive, cross cutting changes. It’s the same for open source maintainers. They need time to get to know you.
P.S. Check with your employer about making contributions on work time. Unsurprisingly employers expect you to be building the thing they’re paying you to build. You should also check what the rules are for contributing back code to projects you’re using for work. Some employers still have backwater policies about open source and IP protection.
Documentation is the one thing open source projects struggle with more than anything else. Maintainers would love it if you made documentation contributions. It’s just as critical to adoption as the code is, if not more so. After all, consider how you feel about using an open source project with no documentation versus one that has great documentation. A lot of projects now use https://readthedocs.org/ to host their documentation with the sources in git repositories. It’s easy to help write documentation.
You can also contribute by responding to questions/issues on the project site, on StackOverflow, or on other forum(s). You could also create screencasts, talk about the project at user groups, blog about it or do anything else you can to raise awareness. These are all useful and valuable contributions. Don’t be constrained by thinking it’s all about code.
Let’s learn from a an example of what not to do, shall we? Have a look at https://github.com/aaronpowell/db.js/pull/112 and you’ll see a number of things that have gone wrong here. To make it easier, let me give you a list:
Yeah. Don’t do this, OK? Great! We’re good to go, then.
Need to go a little deeper? Want specific “How To” information or other tips? Check out this selection of posts and articles others have written:
Most of all, dive in! Have a go! We’d love to see you involved. Good luck!
I’m in the middle of an unpleasant customer support experience with Lenovo at the moment and wanted to share my experience as a “caveat emptor” for other potential Lenovo customers (at least here in Australia)
We start our tale with an online purchase of a nice, shiny Lenovo Yoga 3 Pro for my daughter for school at the beginning of the year. Being a school laptop it gets transported back and forth to school in a backpack each day. Much like my work laptops get carried around in my backpack each day on my commute.
A few weeks back a number of keys on her keyboard started falling off. The 3. The 2. Right cursor. Left shift. M. And before you ask; no, there’s no obvious damage to the machine. It hasn’t been dropped (that I can tell). There’s no dents. No cracked screens. No warping. Nothing that I could detect that would explain why the key caps are dropping off. I should’ve taken a photo.
Now I will admit that I tried to push the keys back on, thinking they’d just click into place, but without success. They kind of half go on, but then fall off again with the lightest of touches. I guess it’s time for a support call, so I jump on the Lenovo site and start a warranty request.
I get an confirmation message (without a reference number) saying I’ll get an email from them shortly with the details for returning the machine. But there’s no email. Not even in my junk mail. I wait a few days wondering if their “eventual consistency” system will catch up (if they have one). Still no email. Great. I guess I’ll need to follow this up myself and call support. A job for tomorrow, since I’ll need to grab the serial number from the laptop again.
But then I get a call from Lenovo asking me why I haven’t arranged to send in the laptop yet. Huh? I explain I haven’t received any emails from them as yet and then they tell me “Oh, yeah. We had some issues with our email systems. Sorry”.
Well at least we’re getting somewhere, so we arrange the details for the laptop to be returned and I’m told I’ll get an email with the forms I need to include for shipping when the courier arrives.
You can guess what happens next. That’s right. Nothing. No email. I wait another day and decide this is ridiculous so I trawl through Lenovo’s web site until I can find a number to call (it’s not obvious). The service rep I talk to apologises, cancels the original service request and creates a new one, organising a new collection date with me over the phone. He also calls me back the next day to follow up and check I received the email. This time I did. That’s also showing signs of better service.
So now, we finally get the laptop collected and it’s returned to base for servicing.
3 working days later I get a status update email from Lenovo saying a quotation is ready for me to approve. A quotation for a warranty job? Hmm. That’s not a good sign, but I need the laptop back regardless. They’ve got my device hostage, so I figure I’ll accept the quotation and then call support once 9am rolls around to ask why they want me to pay since the explanation of what is wrong is unclear.
Here’s what happens. First I bring up the service request:
Yes, the keyboard is damaged. The keys were falling off. Duh! So I click the “Accept” button.
And confirm acceptance. Here’s the response:
Oh, yay! Great system there Lenovo.
So now I need to call support. I follow the prompts to check on the status of a job, but the phone system asks me to enter a 10 digit job number. Lenovo only gave me a 6 digit number. Um… genius. Way to go Lenovo.
So I follow more prompts through the phone system until I manage to talk to a human. I ask him why there’s a $200 charge for a warranty fix and I’m told it’s because of “induced damage”. Was I not meant to try pushing keys back on when they fell off? Would I have even tried to do this if they hadn’t fallen off in the first place? If I’ve somehow damaged the keyboard when trying to push the keys back on I could be at fault, though it’s some pretty poor customer service if that’s the case.
Unsurprisingly at this point I’m not a happy camper. Frustrated, I explain that even if I wanted to, I can’t accept their quotation because their systems are broken so I ask him to do it over the phone for me. The reply? I’ll call you back. What? Why? No thanks. I’ll wait on the line while you do it. It can’t take that long, surely.
Turns out that it can. It’s a “different department” that handles this and it will be 1-2 working days before someone calls me back to confirm. Are you kidding? 1-2 days to get someone to make a phone call? Can we drag out this repair any longer? I just want the laptop fixed and returned.
So here I am. Still waiting for Lenovo to learn how to make an inter-departmental phone call or to write an email to someone and I’m getting stonewalled. I’m reminded of our favourite insurance manager from the Incredibles.
I’m certainly not getting anywhere penetrating Lenovo’s bureaucracy at the moment so I’m doing the only thing I can and blogging about the experience instead.
As a note, my current work laptop is now 2 years old and due to be replaced. I was seriously considering a Carbon X1 since it looks like a great piece of kit but this experience has soured me. Support is really important, especially for someone like me who completely depends on my laptop for my job.
Lenovo, you’re definitely scratched from my list of approved vendors. Just fix my device, send it back and let’s hope we never have to deal with each other again.
There’s a aspect to owning a product backlog that a lot of Product Owners struggle with; how do they handle all the little requests? All the relatively tiny, inconsequential backlog items, customer requests and minor improvements that in and of themselves have very little value, but combined have a lot of value in that they help improve the overall quality and polish of an application. These are the “white noise” items, the constant background hum of a Product Owner’s life and can be the main thing that Product Owners have to do that feels like administrivia and can in turn make them want to stop managing a backlog at all.
For most Product Owners setting the delivery order of items with obvious business value and significance such as key initiatives or small but critical fixes and improvements is fine. It’s something that they want to do, that they can grasp and put their heads around, something that makes sense to them, but when that same Product Owner is faced with a bucket load of tiny requests each with small individual business value, then it’s not uncommon for Product Owners to simply throw their hands in the air, and out them at the bottom of the backlog effectively ignoring these items because ordering them is simply too difficult. “I’m meant to order my backlog by value and return on investment, right?”, say the Product Owner, “but these tiny items feel more like shuffling confetti by size and weight and none of them are more important than all these other larger, more obvious items I have”. What they’re forgetting is that the confetti sized items have potentially massive value when looked at as a whole. They’re the paper cuts that people feel with a product, the little niggling things that combine to give a product an overall sense of incompleteness, and ‘not quite what I expected’ quality.
So let’s get pragmatic about backlog ordering then, shall we?
In Scrum, a Product Owner is tasked with optimising the value of a product and of working with the development team to ensure maximum Return On Investment and a low Total Cost of Ownership. At the same time, an agile team should be learning constantly and aiming to be as productive and as efficient as they can be and spending a lot of time ordering these tiny backlog items is hardly efficient. How can we achieve both goals, and maximise value whilst staying efficient?
What I’ve done with a number of Product Owners now is to ensure that the PO is ordering all the non-trivial Product Backlog Items in the backlog as per usual, and then developing a working agreement/convention with the development team for dealing with the tiny items. During sprint planning the development team uses about 80-90% of their forecast for the planned, deliberately ordered Product Backlog Items, and then grab tiny items from the backlog until they think they’re good to go.
Of course, if the Product Owner specifically wants a tiny item worked on, they simply move it into the planned and ordered Release backlog section, just like every other item that they have placed there.
At the end of the day this doesn’t remove the responsibility from the PO to order the items or to focus on optimising value, but it does mean they don’t have to worry about ordering the metaphorical grains of sand in the jar, just the bigger rocks.
*P.S. For an ordering convention, I usually suggest alternating a small bug fix with a small improvement, then order by a value grouping such as Small, Very Small, Teensy, and then by date created, in descending order. As stated above, if the Product Owner wants a tiny item delivered in specific order, they just pull it up into the main explicitly ordered section.
I’m working with a customer who, for historical reasons, has a test lab with 20-something test machines of various speed and capacity and automated through code they’ve written themselves over the years. As part of the regular build process they distribute all their automated functional tests across these machines and a test run typically takes somewhere between 2.5 to 3 hours. They want this to be as fast as possible, so to figure out the fastest way they can complete a test run they have been pre-allocating tests to machines based on historical run times. They also have some tests that can only be run by some machines because of how the machines are configured, and they tie these tests to machines via a test category attribute.
In theory, this approach seems OK; you look at the previous execution times to work out the best distribution for the next test run, but in practice it doesn’t work out like that. Machines aren’t all the same capacity so test times can vary significantly between one machine and another, and there are new tests with no history being added all the time. The end result is some machines in the lab end up being idle for around 30 minutes, meaning test runs take longer than they should.
Now the obvious solution here is to move away from all the pre-allocated, predictive approach to distributing the tests and instead simply put all the tests in a queue. Test machines then grab the next available test from the queue and execute it. This way all machines will be busy up until the point the queue is empty and the final tests are awaiting completion by other machines in the lab.
Microsoft does exactly this with TFS. Machines in a test lab have a test agent installed on them and those agents communicate with a test controller that holds a queue of tests to be executed and farms them out to the agents based on certain criteria.
Borrowing from that concept, I decided to produce a proof of concept that my customer could borrow from and incorporate into their hand coded test lab environment.
That code is now available at https://github.com/rbanks54/DistributedTestRunner and I thought I’d share it in case you were interested.
The architecture is pretty simple.
1. Test Controller
The test controller is a set of REST endpoints (built in ASP.NET Web API) and a rudimentary UI that shows the status of a test run. The controller is a console app, running as a self-hosted OWIN server. No need for IIS here.
To start a test run the end user provides a path to an MSTest based assembly either via the API or the UI. The assembly is then parsed and all the tests placed in queues based on the category attributes, waiting for agents to start requesting tests to run.
2. Test Agents
An agent simply polls the controller for a test to run. Mulitple agents can run at once.
The controller will determine what test is next from the queue and return it to the agent. The agent then kicks off MSTest, passing the test name in as an argument, gathers the test results and sends a success/fail status back to the controller to indicate the test has completed, before then asking for another test to run.
Additionally, since we’re spinning up an instance of MSTest for each test we execute (I know, it’s not efficient) we do a little extra work to merge the individual test result files from each MSTest run into a single TRX file so that when all tests for a test run are completed, we can see the results in a single file.
It all works pretty well. Not bad for a small amount of effort!
So, feel free to have a look at the code if you like, and borrow from it as you will. If you like to experiment, feel free to take the code and extend it to make it more interesting and useful. It’s open source! I’d love to see what you do with it!
Just remember it’s a proof of concept at the moment. I made an assumption that the test assembly is in the same folder as the test agent/controller, I didn’t secure the API calls. I didn’t write unit tests (I know; practice what I preach, right?). I don’t send the TRX files back to the controller after the test run completes. I could’ve used SignalR for polling instead of the simple timer loop I used. These are all things you could improve on if you wanted to try your hand at something.
Personally, I found it fun and interesting to go through the process of putting it all together and then walking through the customer through how it works and the approaches I took. Maybe you’ll find it useful too. If not, don’t worry. It was fun to write and, after all, isn’t that why we do the job we do?
Happy coding!
As you may know, I help run the Sydney Alt.Net user group in Sydney with James Crisp. Jimmy Pelletier co-runs the Alt.Net group in Melbourne, and across Australia we’ve previously had groups in Brisbane and Perth as well.
Jimmy and I have been talking about the ‘branding’ of the user groups. Is Alt.Net still an appropriate name for the group? If we changed, what would make sense? Is Alt.Net still relevant in the current day and age we live.
First, a little history.
March 2007 - Microsoft releases Linq to Entities (aka Entity Framework). If we ignore Linq to SQL, this is Microsoft’s first serious attempt at an O/RM. Various developer MVPs look at it and don’t like what they see. They are used to NHibernate and are particularly upset about Entity Framework’s lack of persistence ignorance and the apparent disregard by the EF team for developers who might want to follow good design patterns in their application code.
There’s a good summary of the dispute for those who are interested.
April 2007 – David Laribee proposes the term “Alt.Net” as a name for the growing group of people who want to write good code, who are upset at Microsoft’s continuing long disregard for good development practices and those who are willing to look outside what Microsoft provides to see if there are better solutions. In other words, those people who don’t just blindly go along with whatever bad practices Microsoft was encouraging developers to follow and want to do things right.
October 2007 – An open spaces meeting for people who subscribe to the Alt.Net mindset is held in Austin, Texas.
November 2007 – Philadelphia holds the first ever Alt.Net user group meeting.
April 2008 – An Alt.Net open spaces event is held in Seattle. Microsoft’s home ground, It is well attended and further helps to define the Alt.Net movement and connect the people who are part of it.
June 2008 – Frustrated with Microsoft’s lack of change, and apparent disregard for those clamouring for improvements, the now infamous Entity Framework Vote of No Confidence appears. While this makes a number of people wonder if the Alt.Net movement is simply a home for ranty, angry people, the controversy further raises the profile of the Alt.Net movement.
September 2008 – The Sydney Alt.Net group starts (I can’t believe it’s been going for over 6 years now!). It’s safe to say that at this point there was a significant level of interest in the movement. Melbourne, Brisbane and Perth follow soon after. Similar groups are springing up worldwide.
March 2009 – The third Alt.Net open space is held in Seattle. Scott Hanselman points at the elephant in the room by asking the question “Alt.Net – Why so mean?” (video no longer available). The reactions from some of those in attendance show there is substance to the question and invokes some soul searching from many in the movement.
2009 – 2011 – A significant number of the people who started the Alt.Net movement continue to express frustration with Microsoft, and in particular EF and gradually drop out of the Microsoft ecosystem, generally moving to the new and shiny fad, Ruby on Rails. There’s a cargo-cult like spate of “I’m leaving .NET” blog posts that start appearing.
Fast forward to today
A majority of the people who started the Alt.Net movement have since moved on, either leaving the Microsoft ecosystem for other development pastures, or quietly dropping off the grid.
Even so, the spirit of Alt.Net continued on, gradually growing and spreading amongst developers so that the practices and approaches that were heavily talked about at the beginning of Alt.Net (i.e. SOLID principles, persistence ignorance, testability, etc.) have become all but mainstream.
We also see Microsoft making incredible adjustments to the way they treat developers. No longer is it about providing “monkey see, monkey do” tooling for the infamous “Morts”, They’re open sourcing their development, they’re taking community contributions for the core .NET framework, they’re out there in the open, developing completely naked and taking on all the feedback they can to ensure they give us, the customer, the products and tooling we want. This doesn’t yet apply to all teams or products, but the tide has turned. Unlike the dark days of 2007, Microsoft is not only listening to the community, but they are actively adjusting their plans based on what the community says. No longer does the stock standard response of “that’s good feedback” mean “I hear what you’re saying but I’m going to ignore you”, and that’s a great thing.
Credit for this cultural shift can be laid at the feet of Microsoft’s ASP.NET team and other vocal individuals within Microsoft who showed that an open approach and provision of tooling that supports good developer practices can result in much better products, higher engagement with the developer community and greater retention of developers in the Microsoft space. ASP.NET MVC, the Web API, SignalR and NuGet became shining examples of just how this works. Individuals such as Scott Guthrie (who announced ASP.NET MVC at an Alt.Net open space), Scott Hanselman, Phil Haack, Glenn Block, Jon Galloway and many others within Microsoft (too many to mention, sorry!) should be thanked for their tireless efforts swimming against the tide and bringing about the cultural changes we’ve seen.
With these changes, those of us who have been involved with Alt.Net for a long time are now wondering if the original mission is “done”. Is the movement finished? The momentum gone? Are those still involved just dinosaurs who have forgotten to move on? Is there a need to keep the Alt.Net name or should we change it? Does the rise of JavaScript make .NET far less important? So many questions! Time for some answers.
The Future of Alt.Net
I don’t see that the mission has changed, nor do I think the mission will ever be “done”.
Why? Let’s go back and look at the definition of an Alt.Net developer, shall we?
You're the type of developer who uses whatever works while keeping an eye out for a better way.
Check. I don’t think we’ll ever be done with that. I’m always looking for better ways, and as device types and user interface continue to grow and expand we’ll need to keep on top of this. Mobile development anyone?
You reach outside the mainstream to adopt the best of any community: Open Source, Agile, Java, Ruby. In no way does Microsoft or the .NET community have a monopoly on good software development.
Agreed. These days we’re looking at JavaScript and functional languages as the latest area for inspiration. What other patterns will appear over time? Would you rely on Microsoft to be the source of all the good ideas? No, neither would I – let’s keep using the tools and platform we love whilst looking elsewhere for great ideas. As a tip, I wouldn’t rely on Google or Facebook to have all the good ideas either.
You're not content with the status quo. Things can always be more elegant, more mutable, and of higher quality. We're all experimenting with techniques to more closely connect the coding and testing to the business domain.
OK, so some of the examples are no longer relevant, but the fundamental idea remains the same. We can always do better. We want to embody continuous improvement both in ourselves and those around us.
You realize that tools are great, but they only take you so far. It's the principles and knowledge that really matter. The best tools are those that embed the knowledge and encourage the principles (for example, ReSharper). Furthermore, you feel that the most important qualities of a solution are maintainability and sustainability. Maintainable code means good design. Good design arises from the skilful application of design knowledge. The .NET community has been placing too much focus on learning API and framework details and not enough emphasis on design and coding fundamentals.
Again, agreed. Tooling is good, frameworks are important. But it’s the principles that make the biggest difference in our success. Interestingly, the statement about the focus being too much on API and framework details is one that could now be levelled at the JavaScript/Web community in general. As an Alt.Net developer, the tooling ad language you use is far less important than understanding the principles and practices needed to produce fantastic, maintainable, high quality software.
Does this definition of developer still apply? Yes!
Is the mission done? While we still have developers who fit this definition, I’d say not.
Has the momentum gone? No, though it has definitely slowed now that Microsoft is being more responsive.
I think what we’re seeing is that Alt.Net has moved into a different stage of the community life cycle than when it started. This is explained at http://www.psawa.com/Community_life_cycle.html and describes the following stages:
Establishment –> Action –> Maintenance –> Self-Evaluation –> Consolidation/Growth –> (Action…) –> Death
Alt.Net was established in 2007, action saw Microsoft change, and now we’re in a period of maintenance and have been for some time, operating under a leadership vacuum. If you were to ask “who are the key people in Alt.Net” who would you point at? I’m not sure. Not me, for sure – I just organise a local user group. I actually don’t know who to point to either and there in lies a problem. One that will either result in a gradual decline into the “Death” stage, or one that will trigger a “self-evaluation” leading to a new future for Alt.Net. Thus this blog post.
I think we need to refine our definition of what Alt.Net is all about. The mission to change Microsoft isn’t what it was originally about, though it quickly appeared to become that fairly early on. The mission is simple - pushing and encouraging each other to grow and improve. To be better. To achieve more with less. It’s not us against Microsoft (nor should it have ever been), It’s not us against Apple, or Google, or Facebook, or anyone else. It’s us against our worst enemies; ourselves.
Without each other to prod us along, to challenge our thinking, to check we’re not being lazy, etc. it’s all too easy to fall back into bad practices, to willingly accept the status quo, to stop being curious about other ways to do things and to be plain ol’ lazy.
So the angry young men and the “why so mean?” people of Alt.Net have since moved on. Fair enough. That just means we’ve learned as a community. We’ve grown. We’ve matured. That’s awesome! I love it! It means we’re doing what we set out to do. But is that enough now? Should we now disband?
I don’t think so. The Alt.Net mindset still isn’t mainstream. I still need others around me who will hold me accountable when I invariably do something stupid. I still need others around me who have different experiences from me that I can learn from. I still need others around me that think about problems differently than I who I can be inspired by. I still need what the Alt.Net community provides.
Do you?
P.S. This isn’t the first time people in the community have self-evaluated. Ian Cooper wrote about this back in 2010 in “Wither Alt.Net?”
I thought this might be useful for some of you.
A question on StackOverflow asked if there’s a way to see any stats on git commits stored across a large number of repositories in TFS. The quick answer is “No, there isn’t. At least not directly”. The slightly longer answer is “Sure. If you’re willing to write a little SQL and maybe create a report to visualise the data”.
You may recall a little while back I posted on how TFS stores git repositories in its database. There are a number of tables in the database related to commits, and we can use this to get some useful information that we can then turn into stats. Here’s the query:
use [Tfs_DefaultCollection] select r.Name, u.FullName, m.CommitTime, m.Comment from dbo.tbl_GitCommitMetadata m left join dbo.tbl_GitCommit c on c.InternalCommitId = m.InternalCommitId left join dbo.tbl_GitRepository r on r.InternalRepositoryId = c.InternalRepositoryId left join dbo.tbl_GitCommitUser u on u.InternalId = m.CommitterId and u.PartitionId = m.PartitionId order by r.Name, u.FullName, m.CommitTime
Here’s an example of the output from within SQL Management Studio, looking at a RestSharp repository I pushed to my TFS instance, so you can get an idea of the result:
You’ve probably seen all the various announcements from Microsoft today, with the most notable being the release of Visual Studio 2015 Preview and the jaw dropping move to open source the .NET Core Framework. Fantastic stuff! Though I won’t go into it in detail here.
What I do want to cover are the “Smart Unit Tests” released with the VS2015 Preview, based on the Microsoft Pex research project. The messaging around it has created more than a little confusion. In fact, here’s what one of my colleagues on Twitter sent my way:
“Smart Unit Tests” is what happens when you don’t go outside. //cc: @rbanks54
— Pawel Pabich (@PawelPabich) November 13, 2014 During the MVP Summit I spent some time with the people who built this feature to better understand what the intent behind these so called “smart” unit tests. Is it simply so we can tell our managers that we can hit that magic 100% unit test coverage mark now, meet our KPI’s, claim our annual bonus for hitting an arbitrary metric, and still be able to produce rubbish quality applications or is there some other value to it?
It turns out that the purpose of the Smart Unit Test feature is to help people in their refactoring efforts. It’s good practice before you go and refactor code to ensure you have a healthy level of test coverage in place so that you can be confident that any changes you make to code don’t break the behaviour of the application. On existing, legacy code without tests, this is often a very expensive and difficult process, and as a result many refactoring efforts occur either without any test coverage at all or simply never happen because they’re too scary.
This is where “Smart Unit Tests” come into play. They help you do the “smart” thing and create unit tests that help you pin down the current behaviour of the code, before you do any refactoring. These type of tests are known, unsurprisingly, as pinning tests.
Here’s a good description of a pinning test (source), emphasis added:
A “pinning test” is not a good test. It doesn’t try to be. It’s just the simplest, fastest-running test you can write that will allow you to refactor your code. It’s often an end-to-end test, but it could also look at your log files, monkey-patch a core library function, or do something similarly ridiculous.
The key here is that the pinning test lets you do your dozens-of-tiny-refactorings loop really quickly. As the code improves, you add high-quality unit tests. Once the tests are good enough, you get rid of the pinning test. Lather, rinse, repeat.
Hopefully that helps clear up some confusion. Oh, as a final note, Brian Harry mentioned that the Smart Unit Tests name is likely to change. Let’s hope so! :-)
About 18 months ago my Visual Studio 2012 Cookbook was published by Packt Press. Fast forward to a few weeks ago (March 25th in fact, I’m a little tardy on getting this post out) and an updated version of my book has been published, the Visual Studio 2013 Cookbook.
I’ve been a little time poor over the last 12 months so I need to give a big shout out to Jeff Martin who made a large contribution to the book and without who it wouldn’t have happened. Thanks so much, Jeff!
For those wondering what’s new, here’s some of the major changes in this version:
As with the previous edition, this book isn’t for everyone. It’s for those people wanting to get up to speed with the changes and improvements in VS2013 and is going to be more suited to those coming to VS2013 from older editions of Visual Studio, or those who are used to Eclipse or XCode and are wanting to know how to get to grips with an unfamiliar, and very powerful, IDE.
Go buy yourself a copy from Packt or your favourite book selling web site today!
I just ran across the sad story of yet another failed software project (I missed it first time since it happened across the Christmas period). In this instance we see Avon writing off a $125 million project to revamp their sales systems and doing so after the initial pilot deployment was met with so much hate and derision from their people that sales staff were resigning from the business in massive numbers rather than working with the software.
This is a project that was started in 2009 and it wasn’t until December 2013 when the project was (mostly) killed off. That’s a 4 year investment of hard work and sacrifice from so many people to produce, well, nothing. Actually, not quite true. They did produce the software world’s equivalent of anti-matter, a substance that destroys everything it comes in contact with. It damaged corporate reputations for both Avon and SAP (well, maybe not SAP since their reputation is already heavily tarnished), it’s decimated Avon’s largely voluntary sales force, strangled sales revenue and cash flow, and burnt through $125 million and 4 years that could have been spent on building something that was actually valuable and useful for the business.
The culprit? It’s hard to point fingers anywhere other than Avon’s poor management and the poster child for expensive enterprise software that’s almost impossible to install and use without bus loads of specialist contractors; SAP. Of course, SAP aren’t taking any blame. No sir! They’re blame shifting to other vendors and trying to throw them under the bus instead. No doubt the same bus that is now shipping out all those expensive SAP contractors, of course. SAP are saying that they only built the back end and it was those other incompetents who people built a front end with poor usability that is at fault. Sure, SAP, you sit there and pretend that a back end doesn’t define the business processes and doesn’t impact usability.
"Many representatives couldn't even get logged into the new website, and once you got in, the system was not accepting orders, it wasn't saving orders properly, and it wasn't reserving inventory."
- Karen Edwards, Avon
Yep, sure SAP, that sounds like a ‘usability problem’ right there.
With so many people leaving, there’s a sadly high human cost to this failed project but combine it with statements like these, I start to get really upset with just how delusional Avon management and SAP must be:
"While the pilot technology platform [in Canada] worked well, the degree of impact or change in the daily processes to the Representative was significant, this resulted in a steep drop in the active representative count."
- McCoy, Avon’s CEO“Head office kept insisting that the system was working, but it was not”
- Edwards, Avon“[the] software was working as designed, despite any issues with the implementation of the project”
- SAP media statement
Hang on?! The CEO is trying to tell us that the project and the product are successful, even though it’s an obvious, abject failure? Could the management team’s collective head be stuck any further into the sand? Could SAP be any more uncaring as to the result of their product? I’m staggered!
We can add to this evidence of the ‘Failure is not an option” mindset within SAP. There was obviously no Plan-B. No rollback if things went pear shaped as they did. Instead, they’ll just keep pushing on with their rubbish.
“The company reported it would continue to use the software in Canada to avoid further problems in that market”
- Information Week.
Further problems? Like what? Putting the old system back in place and trying to bring back at least some of their disgruntled sales people? I’m crying right now.
Now, if this was a one-off then maybe it’s forgivable as a $125m rookie mistake. But they’d already failed this way before. In 2011, they screwed up with Oracle, and a logistics ERP rollout in Brazil that contributed to Avon’s then-CEO resigning.
“The roll-out of an Oracle ERP suite underpinning core supply chain and finance operations has posed huge challenges for the company’s operations in Brazil”
-IT DecisionsThe troubles around the implementation have resulted in large numbers of IT staff leaving the company in the last 18 months, according to the source. […] a separate large-scale ERP project was launched recently with a view of transforming other customer service areas. SAP was chosen to supply products for that body of work.
”The [SAP] project will be significantly more complex. And if we work on the assumption that the same difficulties around change will remain, it would be fair to say that this project will take six to seven years to complete”
- IT Decisions
2 years to learn from that failure and adjust how the SAP project was progressing and they did nothing. When will people learn that when it comes to building software, and especially complex software, that the failed methods of the past simply don’t work anymore? In fact, they’ve never really worked in the first place.
When will people learn to stop building large systems over many years and attempting to do a big-bang deployment, without a fall-back plan, and without ever genuinely engaging their user base first instead of just selling them on the promise of something awesome.
Sure, I get it. I’ve lived it myself. Large and complex systems are hard to build, but let’s not make it harder than it needs to be. Start by building something for a small subset of the end solution and prove it works. Incrementally grow from there to meet your goal. Get user engagement and feedback early, not just from the management team. If you do go down the wrong path and build something that turns out terrible you’ll know about it early and have time to adjust rather than forcing your steaming pile of effluence down people’s throats and watching them quit.
Identify the value in your system and deliver on the key high value items first, your minimum viable product. After that pile in all the other features you want over time and subsequent smaller releases. Who knows? Avon may have been able to save up to $50m if they’d failed with a much smaller product first. If they’d done that though, I suspect that we wouldn’t be talking about them at the moment.
Finally, be brutally honest and transparent with where a project is at. I can’t image the number of times people on the SAP project said there were problems and it was going to fail. I can’t imagine the countless times that project managers and senior execs who wanted to report that things were ‘green’ shut down anyone who voiced problems. I can, on the other hand, imagine the ego, politics and bluster that drove this project off the road, through the guard rails and over the edge, resulting in the disaster that somehow execs still believe is a success.
Oh, and for the love of all things good in this world, please stop using SAP and the horrible waterfall based project practices that seem to go hand in hand with that shambling behemoth.
References:
