Category: Visual Studio

When and why to run Web Performance and Load Tests

A colleague from recently asked me about when should we use Load and Performance Tests? Well my answer would be: “All the time during the product development”, but there are specific stages where this is critical.

Some agile teams have a verification week before moving to the next iteration. During verification week, they spend first 2-3 days of the week verifying the product in their test environments. If everything is okay, by Thursday they deploy to preproduction environment. Once they have green light in pre-production, they deploy to production, which is targeted over the weekend or Monday morning.

So the tests that they run during this verification week are the same they run continuously in their working branch.

Load testing is a critical part of our software development process. We find many serious issues in load testing, everything from performance regressions, deadlocks and other timing-related bugs, to memory leaks that can’t be effectively found with functional tests. Load testing is critical to being able to confidently deploy.

Load Test Script Architecture

TFS has web service end points, rest end points and web front ends.

In order to generate the massive loads we need, we use Web tests to drive load to our web site and web pages, and unit tests to drive load to our web services.

Here a representation of the Load Tests Script Architecture.

All tests scripts use common configuration to determine which TFS instance to target in the test, and scripts can be configured to target an on-premises server or Visual Studio Online.

Types of Load Testing we do

Performance Regression Testing: executed every sprint to ensure that no performance or scale regressions are introduced.
Scale Testing:

Increasing the number of accounts and users to find new bottlenecks in scale. Some key counters to consider are:

  • RPS (requests per second)
  • % CPU AppTiers
  • Average Response Time
  • Current Server Requests
  • Active Team Project Collection Service Hosts
  • Private Bytes
Deployment Testing

We run a load tests while the upgrade is happening, so, as activity is constantly going against the service, we can detect outages through all the stages.

  • Stage 1: Deploy new binaries into web front ends and job agents.
  • Stage 2: Run jobs to upgrade the databases.
  • Stage 3: Run jobs to do necessary modifications of the data stored in the databases.
Directed Load Testing

Isolate a particular component or service for performance, stress and scale testing.

Testing in Production

These test are fundamentally focused on analysing the following data to look for regressions:

  • Activity log analysis.

    • Increases in failed command counts
    • Increases in response times, which indicates a perf regression
    • Increase in call counts, which indicates a client chattiness regression
  • CPU and memory usage.

    • Increase in CPU usage after deployment
    • Memory leaks
  • PerfView analysis: mainly driven by a tool called PerfView. Useful for finding memory and CPU regressions in production.

As you see, running performance and load tests is something that could be done anytime, but depends on the time where you do it, it will involve different techniques and data to capture.

I encourage you to create your first Web Performance and Load Tests following some of the next links and find yourself the answer of… When and Why to run Web Performance and Load Tests.

Generate and run coded web performance test

Step by Steps tutorial- Run performance tests on your app

Distributing Load Test Runs Across Multiple Test Machines Using Test Controllers and Test Agents

Analysing Load Tests Results Using the Load Test Analyser

Load Tests in the Cloud

Happy testing!

Eduardo Ortega

C# Programming resources

While running the Certified Scrum Developer Training at SQS London, we went through an amazing journey across TDD and BDD techniques using C# as a main programming languages.

For some of us, it was a good practice that helped us to refresh our rusty programming skills, for others was a way to move a Tester role to a Software Developer Engineer in Testing role.

Let me share with all of you some useful resources about C# programming that will make your journey on programming even more interesting.

C# Programming Guide:

[Video Training]C# Fundamentals for Absolute Beginners:

[Video Training]Programming in C# Jump Start:

[Video Training]Twenty C# Questions Answered:

[Video Training]What’s New in C# 6:

[Videos]How Do I? Videos for Visual C#:

[Samples]Visual Studio Samples:

[Free Book]C# Language Specification 5.0

[Books]Visual Studio Books:

More Visual C# Resources:
Get started with Visual C#
Asynchronous programming with Visual C#
Getting started with .NET and Visual Studio
Programming concepts

Last but not least, you can download the different Visual Studio versions from here
I recommend to download the Community Edition as it’s totally free and fulfil most of the needs you have about Coding.

I hope this info is useful and enjoy coding!

  • May the code be with you –

Eduardo Ortega

Exporting Fiddler2 sessions to Visual Studio Web Tests

While finishing the revisited and new “Web Performance Testing with Microsoft Visual Studio 2013” training at SQS Academy, I found a very interesting topic to cover, idea of my colleague Carl Bricknell, that is to migrate Fiddler2 sessions to Visual Studio generating new Web Tests.

As you know Visual Studio 2013 Ultimate and Visual Studio 2015 Enterprise have included the templates for Load and Performance Tests.
It’s as easy as go into VS and create a new “Web Performance and Load Test project”.

Usually, once the project is done, we have to record our session with the Web Test Recorder as follows:

And right after our session, we will be able to edit the recorded test steps on the WebTest view. This will allow us to create our validation rules that would be used to let the test pass or fail; also create, modify and delete web requests, their headers, parameters (dynamic and statics) and, the most fancy feature, to add data parameters in order to replay the tests several times with different data straight from an Excel Sheet, CSV file or SQL Database.

Many testers and developers use Fiddler2 in a daily basis, and this is because is one of the most powerful web traffic capture tools.

Fiddler2 includes the ability to capture web traffic (including AJAX requests) for later playback with the Visual Studio Web Performance Test feature. It can also be used to help debug your Web performance tests. Comparing your Fiddler2 recording and your Web performance test recording can help identify key missing headers that your Web performance test may not record, amongst other things.

We can summarize as:

Fiddler is at HTTP debugging proxy server application that captures HTTP and HTTPS traffic and logs it for the user to review.

Fiddler can also be used to modify HTTP traffic for troubleshooting purposes.

In order to export Fiddler2 recording sessions to Visual Studio Web Test Project we need to follow the next steps:

  1. Create a Visual Studio empty Web Test Project
  2. Record your session using Fiddler2
  3. Export the Fiddler 2 session to the Visual Studio Web Test Project

The exporting is very straight forward:

  1. Go to File à Export Sessions à All sessions

  1. Select Visual Studio Web Test as Export Format

  1. Select all the plugins installed by default:

  1. Find the generated web test in your hard drive and attach it to your Visual Studio Web Test Project:


From here, everything that happens is result of your creativity, it’s ready for testing!

  • Happy testing! –

Eduardo Ortega Bermejo

FREE Upgrade to Visual Studio Enterprise with MSDN!

There is no better time to get ready for the release of Visual Studio 2015. The new Visual Studio is coming in May 2015 and the prices have been reduced dramatically!
When you upgrade from Visual Studio Professional or Visual Studio Test Professional to Visual Studio Premium, the price is cut to 50%! And also get a free upgrade to Visual Studio Enterprise with MSDN automatically when Visual Studio 2015 gets released.

The offer is only valid for 2 months, expiring on June 30th so hurry up.
But there is much more. Visual Studio 2015 license system has changed. These are the versions that we will have now:

  • Visual Studio Community

    • Integrated Development Environment for building Web, Windows Desktop and cross-platform iOS, Android, and Windows apps
    • Ecosystem with thousands of extensions to choose form the Visual Studio
    • Free for open source projects, academic research, training, education and small professional teams
  • Visual Studio Professional with MSDN

    • Professional developer tool for building any application type
    • Powerful features to improve your team’s productivity such as CodeLens
    • Improve team collaboration with Agile project planning tools, Team Rooms, charts and more
    • MSDN subscription benefits including access to core software for dev/test, Team Foundation Server, Visual Studio Online Basic, $50/month in Azure credits, training and support.
  • Visual Studio Test Professional with MSDN
  • Visual Studio Enterprise with MSDN

    • End-to-end solution for your development teams, including the most feature-rich Visual Studio IDE for working on any type of project
    • Build quality applications at scale with advanced features such as Load Testing, automated and manual testing and new IntelliTest capabilities
    • Manage complexity and resolve issues quickly with features such as Code Map and IntelliTrace
    • Enhanced MSDN subscription benefits including comprehensive access to software for dev/test, Team Foundation Server, Visual Studio Online Advanced, $150/month in Azure credits, training and support

Customer price has changed as well:

If you have You will get New customer price
Visual Studio Community Visual Studio Community 2015 Free
Visual Studio Professional with MSDN Visual Studio Professional 2015
Your Visual Studio Professional with MSDN
subscription will continue without change
Visual Studio Premium with MSDN Visual Studio Enterprise 2015
Your subscription will be upgraded to
Visual Studio Enterprise with MSDN


Here you can find a comparison between the different versions of Visual Studio, including the Visual Studio Test Professional (that is the most relevant product for many of us).]


Supported configurations and platforms for VS Coded UI Tests

Coded UI automated tests! Sounds good, uh? Well, yes, it’s very exciting to see how a machine is simulating that is playing with your app while input some predefine values in your textboxes and performing some actions. Said that, it’s good to know where you can create and apply these Coded UI Tests.

First, supported Operating Systems:

  • Windows 7
  • Windows Server 2008 RD
  • Windows 8

Supported Architectures:

  • .NET 2.0, 3.0, 3.5, 4 and 4.5

Second, Platform Support:

  • Windows Phone Apps: Supported
  • Windows Store Apps: Fully supported including HTML5 in IE9 and IE10
  • Internet Explorer 8 and 9.
  • Internet Explorer 10 and 11: is only supported on the desktop, not Modern UI
  • Windows Forms and WPF: Fully supported
  • Internet Explorer 6 and 7: Not supported
  • Chrome: Recording of action steps is not supported. Coded UI Test can be played back on Chrome and Firefox.
  • Opera: Not supported.
  • Safari: Not supported
  • Silverlight Not supported
  • Flash/Java: Not supported
  • Windows Win32 and MFC: Partially supported
  • SharePoint: Fully supported
  • Office Client Applications: Not supported
  • Dynamics CRM web client: Fully supported
  • Dynamics AX 2012: Action recording and playback are partially supported
  • SAP: Not supported
  • Citrix/Terminal Services: Partially supported
  • PowerBuilder: Partially supported

Of course, when the talk about ASP.NET, HTML5 pages, these are included in the fully supported list of apps.

If you want to know how to Create, Edit and Maintain a Coded UI Test, check this out:

Happy support!

Eduardo Ortega​

Team Foundation Server 2013 Access Levels

*Info updated here:

The amazing world of licenses…when to use them, how to use them, how many computers, how many users, how many services available, on premises or online… These are few of the many questions that I receive every time a Dev or Test teams need to increase its number or its disciplines.

Let’s explain first how it works for those users that work with Team Foundation Server 2013 (the On-Premises server).

TFS 2013.3 Access Levels

First, a user, whatever kind of user is, needs to access to TFS2013 Update 3, so we need to specify which features need to get accessible. For that we have 3 levels (more info here):

  • Limited (Stakeholder access level)

    • View My Work Items
  • Standard (Basic access level)

    • View My Work Items
    • Standard Features
    • Agile Boards
    • Backlog and sprint planning tools
    • Chart Viewing
  • Full (Advance access level)

    • View My Work Items
    • Standard Features
    • Agile boards
    • Backlog and sprint planning tools
    • Request and Manage Feedback
    • Test case management (including running tests)
    • Team rooms
    • Agile Portfolio Management
    • Chart Viewing
    • Chart Authoring

Now that you know more about the licensing, let’s differentiate the licensing on TFS2013.3 from TFS2013.4 and Visual Studio Online.

In TFS2013.3 the access levels were Limited, Standard and Full.

In TFS2013.4 the access levels are Stakeholder, Basic and Advance.

In Visual Studio Online the levels are Stakeholder, Basic, Professional, Advance and MSDN Subscribers.

TFS2013.4 Access Levels

Let’s see now how the licensing changed from TFS2013.3 to TFS2013.4:

Access level

License required


TFS client-access license (CAL) or Visual Studio Professional with MSDN subscription


One of these MSDN subscriptions: Visual Studio Ultimate with MSDN, Visual Studio Premium with MSDN, MSDN Platforms, or Visual Studio Test Professional with MSDN.


No license required. Assign Stakeholder access to customers or stakeholders that you want to collaborate with but who aren’t on your team.

It means that the Stakeholder level (previously called Limited), will allow you to create Workitems (such as Bugs, Test Cases, Requirements, etc) but now, on the Update 4 for TFS2013, we have some extra features:

Stakeholder access level:

  • View and edit all work items (not only yours)
  • Standard features
  • Agile boards
  • Basic backlog and sprint planning tools
  • Agile Portfolio Management

If you go for the Basic access level (you would require a CAL or a license of VS Professional), you will get the next features access:

Basic access level (all stakeholder’s plus…):

  • Chart viewing
  • Build
  • Code
  • Administer account
  • Advanced home page
  • Advance backlog and sprint planning tools

Advanced access level (all basic’s plus…)

  • Chart Authoring
  • Request and manage feedback
  • Test case management
  • Team rooms
  • Advanced portfolio management

Visual Studio Online Access Levels

And last but not least VSO. If you don’t want to host the TFS server on your premises or just you want to avoid all the set-up of servers, SQL Server Databases, etc. Maybe the best option is to subscribe to an on demand service called Visual Studio Online (previously called Team Foundation Service). This service allows you to get licenses on demand and monthly, so you will save a considerable amount of money. Right now you can’t have divided your infrastructure half on premises, half on the cloud, so that means that your online users will work on team projects hosted on the cloud.

VSO features are slightly reduced in comparison to TFS (like reporting or SharePoint integration) but most of the features are available!

Let’s go through the features accessible through the different access levels

Stakeholder access level (free):

  • Work item tracking, queries, tagging
  • Alerts
  • Agile planning boards and backlogs
  • View Iteration and capacity planning
  • Portfolio management boards and backlogs
  • View query based charts 
  • Provide feedback
  • Track test progress and charts

Basic access level (5 free and $20/month new ones). All stakeholder access plus:

  • Team chat
  • Git repositories and TFVC
  • Work with Xcode, Eclipse, IntelliJ and others
  • Code Reviews
  • Enterprise Scale
  • Full Build and Deployment features
  • Web-based test execution. Test runner
  • Administer accounts, users, teams and projects structure

Professional access level (includes stakeholder and basic access levels). $45 per user per month.

  • Includes Visual Studio Professional

Advanced access level (includes stakeholder and basic access levels). $60 per user per month.

  • Request and manage feedback
  • Test planning: create test plans
  • Test authoring
  • Test suite management
  • Test tracking

As you can see it’s slightly different the On-Premise Server than the On-Line Services but in practice, this would be transparent for your developers, testers or team leads.

For more information follow the links below.

Happy licensing!

Eduardo Ortega


VSO Matrix:

Visual Studio Versions:

Work as a stakeholder:

Visual Studio Online Pricing:

Pay for users accessing your account:

Visual Studio Online Basic:

Hands On Labs and free book for TFS 2012. How to build a release pipeline?

For those that are into SDLC and ALM, here you have a fantastic guide. No matter what size business you have, customers now expect features such as real-time customer service and frequent releases.

The goal of this guide and HOLs is to provide you an excellent guide to the principles and practices of continuous delivery.

By continuous delivery, we mean that through techniques such as versioning, continuous integration, automation, and environment management, you will be able to decrease the time between when you first have an idea and when that idea is realized as software that’s in production. 

You will find this book an indispensable resource:

PDF and epub: Building a Release Pipeline with Team Foundation Server 2012
Kindle (Mobi): Building a Release Pipeline with Team Foundation Server 2012

Paperback versión (Amazon): Building a Release Pipeline with Team Foundation Server 2012

– Happy planning!



[#ALM]Why we use agile methodologies?

 In one of my favorite books , I read something concerning:

“In the Standish Group’s 2011 CHAOS Report found that more than half of software projects conducted between 2002 and 2010 were either de- scribed as challenged or complete failures; just 37 percent were classified as successful”

21% of these projects failed and 42% were challenged, and this is the real challenge.

We are wasting money, resources and time on bringing solutions and applications to the market just because we are not using and agile way to work.
Why are you not using AGILE? There are not excuses for don’t use agile methodologies anymore, and it is not anything new, as AGILE methodologies have been living with us during the last 10 years.

Still, when I’m visiting many of my costumers I see unstructured teams, following old methodologies like Waterfall or dinosaurian ones like CMMI in short projects. But when you have to talk about have a model adaptable, fast, responsive, incremental, collaborative and when you are realizing that 1 of every 2 projects that you are delivering have delays, causing low morale to your team and also you are struggling adapting to the constant changes from your customer,…. then is when you have to rethink on your strategy and maybe move to an AGILE approach.


Use an AGILE methodology like Scrum, doesn’t mean that we have to forget writing the documentation or do the planning on demand but it means that you have to be focus on the most important part of the project, the software.

We have to go back some years to 2001 when the Agile Manifesto was introduced. Since then, the Agile Movement, with all its values, principles, methods, practices, tools, champions and practitioners, philosophies and cultures, has significantly changed the landscape of the modern software engineering commercial software development in the Internet era.

The meanings of the manifesto items on the left within the agile software development context are:

  • Individuals and interactions – in agile development, self-organization and motivation are important, as are interactions like co-location and pair programming.
  • Working software – working software will be more useful and welcome than just presenting documents to clients in meetings.
  • Customer collaboration – requirements cannot be fully collected at the beginning of the software development cycle, therefore continuous customer or stakeholder involvement is very important.
  • Responding to change – agile development is focused on quick responses to change and continuous development.

The Agile Manifesto is based on twelve principles:

  1. Customer satisfaction by rapid delivery of useful software
  2. Welcome changing requirements, even late in development
  3. Working software is delivered frequently (weeks rather than months)
  4. Close, daily cooperation between business people and developers
  5. Projects are built around motivated individuals, who should be trusted
  6. Face-to-face conversation is the best form of communication (co-location)
  7. Working software is the principal measure of progress
  8. Sustainable development, able to maintain a constant pace
  9. Continuous attention to technical excellence and good design
  10. Simplicity—the art of maximizing the amount of work not done—is essential
  11. Self-organizing teams
  12. Regular adaptation to changing circumstances

There are hundreds of thousands of developers world wide  using Agile methodologies since then. There is no the right one, each one will be better for you and your team depending on your way to work, projects, timing, but all of them share the same principles.

Well-known agile software development methods and/or process frameworks include:

Just to finish these remarks, let’s introduce briefly ALM, as in Wikipedia is explained:

Application lifecycle management (ALM) is the product lifecycle management (governance, development, and maintenance) of application software. It encompasses requirements management, software architecture, computer programming, software testing, software maintenance, change management, project management, and release management.

This means that Modern Apps need a Modern Lifecyle, and for me here there is only one Winner, Visual Studio.

First) Because Microsoft is providing you a full set of integrated tools that cover the whole development cycle, and yes, INTEGRATED. It means that you don’t have to be installing tools from different vendors but also, if you want to install them, they will be integrated with the Visual studio Suite.

For example, I remember some of my projects in my previous company, Symphony Teleca, where we were implementing Scrum using “Open Source/Non Microsoft/Almost free” tools. For the source repository we used GIT (very good btw, truly recommendable), JIRA, Sharepoint and txt files 🙂

Our main IDE was Visual Studio, which you can integrate manually with GIT, but not with JIRA (there is a plugin from atlassian but very immatture and when you want to go for something more effective they are hidden costs on this “freeware” version), and the rest of the systems are disconnected. The TDD environment was isolated of the management and reporting tools and the team was also working in a “ghost” collaborative mode.

Nevertheless, opting by the Visual Studio approach, you get the full ALM cycle covered:
–  Set up: 
Set up TFS, create a team project, and add team member accounts.

–  Code: Share and build your code using Team Foundation version control (TFVC) or Git.

–  Work: Plan projects, track work, collaborate as a team, and report progress.

–  Build: Set up your on-premises build server and define your build processes. Or, set up continuous integration builds using Visual Studio Online.

–  Test: Test your application..

Also, Visual Studio includes some of the most common AGILE frameworks templates included like Scrum, CMMI, and Agile. But we will talk about TFS and Visual Studio in other posts.

Here some good readings about Agile methodologies to start the week with something new, I hope you like it:


 The Scrum Guide


Happy planning!
Eduardo Ortega



–  Scrum (Software Development)

–  Kanban for TFS

–  Kanban (Development)

–  TDD

–  Agile Manifesto





Visual Studio 2013 Update 3 RC

It has been a couple of weeks since Visual Studio 2013 Update 3 RC was available for download, so I guess that many of you already have the chance to play with it a bit, but just let me go through the new capabilities and features for those that didn’t explore it in deep.

First say that as part of the continue integration that Microsoft is doing with Visual Studio, we have had a bunch of improvements during the last 9 months. Most of these like the Update 2 were put in place to bring us access to the new Universal Apps and the support to Multi-Device Hybrid Apps for Visual Studio.

You have to realize that Microsoft is doing a lot of efforts bringing the multi-platform and universal development to Visual Studio, for example, with the Multi-Device Hybrid Apps pluggin you will be able to develop hybrid apps that run on iOS, Android, Windows and Windows Phone, using a single project based on HTML and JavaScript.



With Visual Studio 2013 Update 3 (RC), Microsoft is bringing you more new features and improvements:

  • CodeLens
  • Code Map
  • Visual Studio IDE improvements
  • Windows Store Debugging
  • Managed Memory Dump Analysis
  • Graphics Debugging
  • Performance & Diagnostics Hub
  • Test Case Management
  • Release Management
  • Sharepoint Apps and ClickOnce

Between the Visual Studio IDE improvements you can find also the multi-monitor configuration for Windows 8 apps debugging and a configuration option to put CAPS or no caps menus in the IDE.

We will be visiting one by one all these new features but let’s talk for a second about one of my favorites and also the most simple one, it is the Multi-Monitor configuration with Windows Store Apps.

When debugging a Windows Store app with a multi-monitor configuration, we used to have problems with the screens dragging all the time the app to the secondary screen, now, Visual Studio will remember which monitor your app was last run on. Example:



Let’s review all these features in further posts but in de meantime download the Update 3 RC and happy coding!

Eduardo Ortega




Creating documentation in C# using Visual Studio and Sandcastle

Imagine that you are creating an awesome application, let’s say for example for Windows Phone, but you didn’t pay so much attention to the initial documentation of your project, I mean, not very complete UML diagrams, no proper documentation on the entities definition or just, it’s a very short project where documentation is trivial but at least you want to make sure that if anyone wants to continue with this project, you will provide a proper documentation for the hand over.

Generated static printed documents no long work in an era when technology changes many times a year.

  • API documents has been replaced by IntelliSense and Go To Definition
  • Designs can be documented by  XML comments better than Word documents.
  • Books were replaced by Blogs and search engines

An easy way to create your own documentation with an elegant style (see CHM help documents from Microsoft documentation) is through the documentation options in Visual Studio + a great tool to export your code comments and documentation into a readable format.

Let’s split this into two parts.

1) Document your code

In Visual C# you can create documentation for your code by including XML elements in special comment fields (indicated by triple slashes) in the source code directly before the code block to which the comments refer, for example.

/// <summary>
///  This class performs an important function.
/// </summary>
public class MyClass{}

When you compile with the /doc option, the compiler will search for all XML tags in the source code and create an XML documentation file. To create the final documentation based on the compiler-generated file, you can create a custom tool or use a tool such as Sandcastle.

Another example:

/// <summary> 
/// Class level summary documentation goes here.</summary> 
/// <remarks> 
/// Longer comments can be associated with a type or member through 
/// the remarks tag.</remarks> 
public class TestClass : TestInterface
    /// <summary> 
    /// Store for the name property.</summary> 
    private string _name = null;

    /// <summary> 
    /// The class constructor. </summary> 
    public TestClass()
        // TODO: Add Constructor Logic here.

Now, that you now how to comment your code properly, you have to enable one option on the compiler.

Go to the Project Properties –> Build –> Output section –> Enable XML documentation file. See here:


After that build the project and get the generated XML file for the second step.

2) Generate a proper documentation

Now is time to generate a readable documentation through the XML file generated, for that SandCastle is our hero.

If you want to pull all the IntelliSense information out into HTML or compiled Html Help, your going to need some help.  Microsoft has Sand Castle to build those documents.

Let’s see what you need to create these CHM files:

  1. Download and Install Sand Castle via the MSI.
  2. Download and Install the Sand Castle Help File Builder (SHFB) MSI by Eric Woodruff
  3. Download and Install / Patch the Sand Castle Styles
  4. Run SHFB
  5. Add documentation sources (csproj or dll)
  6. Add references (csproj or dll)
  7. Set the help file format and other SHFB options (I recommend using the MemberName naming method to get html links with names instead of Guids)
  8. Run SHFB build, wait and done!

So, once you have Sand Castle Help File Builder GUI installed, you just have to add the references to the DLL and the references to the XML previously generated file and build the solution

image image

It will take time to get the final build but it worths it 🙂

– May the code be with you –


SandCastle Help File Builder:


XML Documentation Comments: