Tag Archives: VS2010

Exploratory Testing With Microsoft Test Manager

Intro

Microsoft Test Manager is a great tool for executing manual tests over a set of known paths through an application. It allows you to create and action test scripts and record to minimise duplication of effort.  At first glance, this seems to be the extent of functional support for testing within the tool – however if we dig a little deeper there’s some real nuggets of gold hidden in there. In this article we’ll talk about how the practice of exploratory testing is supported by the tool and how you can turn an unplanned edge case test into a formalised, repeatable test in a matter of clicks.

Setting the scene for exploration

The test manager interface is primarily geared around the creation and organisation of test cases. Even though we’re not going to be executing a formalised test case in this instance, we’ll still need to have a test case to serve as an aggregation point for our results and collector data. So let’s start by creating a new generic test case. To do this, follow the normal steps for creating a test case up until you’d normally start filling out your steps.

Exploratory Test Case Setup

Setting up a test case for exploratory testing

At this point enter a single step describing what you’d like to explore. While this isn’t entirely necessary, it serves as a result indicator for the exploration which we can then use in the manager interface and our reports.

Once you’re happy with your test case let’s put on our safari hats and get to exploring…

Dr Livingstone, I presume?

The scenario from here is elegantly simple. We move into the test runner by running the test case we created and start exploring the application. When you kick off the test, ensure you choose to create an action recording so that you’re capturing the paths through the application you’re exploring.

Check the action recording box

Remember to check the action recording box

If you happen to come across a bug during the exploration you can mark the test step as failed and raise a bug against it.

This will capture the action log, and any other data from the collectors you’re running and associate it with the bug for the developer to help diagnose the issue. In this particular scenario, the action log serves a second purpose which we’ll get to shortly.

What was once random, is now repeatable

Once we’ve completed the test execution and are back in the test manager, we can navigate to the Verify Bugs screen of the Test tab.

Verify Bugs Link

Click the link to the verify bugs page

On this screen we’ll see a list of bugs based on a filter. The filter is adjustable but starts with an ‘assigned to me’ criteria.

Bug Filter Location

Change the filter to capture the bugs you require

If we click on one of these bugs the ‘Create Test Case from Bug’ button will activate.

Active create test case button

The 'Create test case' button should activate

If we click this button a new test case will pop up with a set of steps derived from our action recording pre-filled and ready to go. The steps are editable as a normal set, so we can remove any we feel are irrelevant or even apply parameters and generate a set of shared steps if we like.

Pre-populated test case

The test case should be largely pre-populated

Once you’re done editing the steps, save the test case and voila! There is a formal test case that can now be used to verify the path you’ve discovered a bug in. From here you can bring the new test case into the appropriate suite and ensure that as the application evolves this now known path is tested effectively.

Summary

Exploratory testing support in Microsoft Test Manager makes dynamically discovering new paths and bugs in your application more effective by enabling you to record your steps and generate formal test cases from them. This helps to eliminate those hard to reproduce edge cases where you can never quite remember what you’ve done to get to the bug, and will help you on your path to a higher level of software quality.

Advertisements

Quick Tip: Display unit test result on double click

In VS 2010 the default behavior for double clicking on a unit test result has changed. Previously when you double clicked on a failing test you’d be shown the test result window with the full result details for the failing test. Now when you perform this action you’re taken to the line of code that has failed in the test.

I find I prefer the previous behavior, so went and found the following setting under the Tools -> Options menu item to revert it…

VS2010 RC Intellisense Crash – Second Patch Available

A few weeks ago there was a patch released for a bug in Intellisense in the Visual Studio 2010 release candidate that was causing crashes. In most cases this patch resolved the issues experienced, however in a few cases a bug remained that caused Visual Studio to crash with a System.InvalidOperationException when performing QuickInfo. If you’ve installed the original patch (available here) and are still experiencing crashes when attempting to view an Intellisense prompt there is now a second patch available on connect to address the additional issue.

If you still experience issues after installing these patches make sure you log an issue on connect so that the right team is made aware of the problem.

Publishing Information to the Build Log in Team Build 2010

I’m currently working on a build activity for the Team Build 2010 contrib project. While debugging an issue I was having I thought it would be great to push some information into the build log. There’s already some great posts on customising build information on Patrick Carnahan’s blog (here and here), but I was looking for a simple way to write what was essentially a chunk of debug information into my log. I knew I had come across some extension methods for CodeActivityContext previously that made build information tracking much simpler, but couldn’t remember where they lived.

To access the CodeActivityContext extension methods, you need to first add a reference to the Microsoft.TeamFoundation.Build.Workflow assembly, then a using or import statement for the Microsoft.TeamFoundation.Build.Workflow.Activities namespace. By adding the reference and using/import statement you get access to TrackBuildMessage, TrackBuildError and TrackBuildWarning. These extensions give a quick entry point for basic build information logging and error reporting which I’ve found quite handy.

Visual Studio 2010 Quick Reference Guidance

Have you ever asked or been asked “Can you tell me about TFS or VS in 5 minutes?”. Did you struggle to fit all that content into an elevator pitch? Struggle no more!

Thanks to some great work by the Visual Studio ALM Rangers, there’s now a set of quick reference guides that provides a fast way to identify the answers to core questions about the VS and TFS ecosystem. This pack contains guidance on each of the roles supported by the VS ALM products, info on how the products transition from 2008 to 2010 and some great posters to hang around your workplace as reference points.

So if you’re looking to answer a “What is…”, “What is new in…” question or just need a quick reference sheet check out the codeplex project and download the guidance!

Canberra VSTS – February 2010 Meeting

0810 – 0830: VSTS Breakfast

Share a bite to eat with the Canberra VSTS group. Bring your VSTS queries and triumphs and share them! Quiz your peers on how they are getting the most out of VSTS, and maybe pick up some ideas you can take to work with you.

0830 – 0930 : Session Microsoft Lab Management
Mitch Denny – Chief Technology Officer, Readify

Mitch Denny has been working with Team System since early alpha versions in 2004. Since then Mitch has helped establish the ALM practice within Readify and has personally implemented Team System at many clients around Australia. Mitch is a VSTS MVP and now serves as the Chief Technology Officer at Readify establishing high performance project teams using Microsoft’s latest tools and technologies

In this session Mitch will introduce Visual Studio 2010 and Team Foundation Server 2010 then move on to demonstrating the advanced Lab Management features in the new product. In this session you will learn how an MTLM environment is structured and the various techniques you can use to get value out of the tooling.

Remember to submit your speaker evaluations at the end of the session for your chance to win a JetBrains ReSharper licence!

You can now register for this session at http://cvstsfeb10.eventbrite.com/

Team Build 2010 – Community Content

Something I regularly monitor in the “VSTS” space is the VSTS Feed Aggregator hosted by Accentient. It’s a great way to keep in touch with a group of MVP and team blogs without having to track down the feeds individually. One of the blogs I do subscribe to though is the Ed Squared blog. The product of Ed Blankenship and Ed Kisinger, the blog has recently seen an influx of content on Team Build 2010 from Ed B, including this mornings post which aggregates a bunch of great Team Build posts.

Also hidden in there is a link to the Team Build 2010 Contrib project on codeplex. The intent of this project is to be:

“a place for build engineers to share workflow activities, build processes, and tools for Team Build 2010.”

The project is currently empty, but the call is out for contributors, and content. I’ve signed up as a contributor, and have a couple of tasks in mind. If you’ve played with Team Build 2010 and have written some custom workflow activities, have a request for an activity or would like to add your expertise to the project why not head over to the site and either put in a request or get in touch with one of the coordinators and offer your assistance!