Category: Testing

When and why to run Web Performance and Load Tests

A colleague from recently asked me about when should we use Load and Performance Tests? Well my answer would be: “All the time during the product development”, but there are specific stages where this is critical.

Some agile teams have a verification week before moving to the next iteration. During verification week, they spend first 2-3 days of the week verifying the product in their test environments. If everything is okay, by Thursday they deploy to preproduction environment. Once they have green light in pre-production, they deploy to production, which is targeted over the weekend or Monday morning.

So the tests that they run during this verification week are the same they run continuously in their working branch.

Load testing is a critical part of our software development process. We find many serious issues in load testing, everything from performance regressions, deadlocks and other timing-related bugs, to memory leaks that can’t be effectively found with functional tests. Load testing is critical to being able to confidently deploy.

Load Test Script Architecture

TFS has web service end points, rest end points and web front ends.

In order to generate the massive loads we need, we use Web tests to drive load to our web site and web pages, and unit tests to drive load to our web services.

Here a representation of the Load Tests Script Architecture.

All tests scripts use common configuration to determine which TFS instance to target in the test, and scripts can be configured to target an on-premises server or Visual Studio Online.

Types of Load Testing we do

Performance Regression Testing: executed every sprint to ensure that no performance or scale regressions are introduced.
Scale Testing:

Increasing the number of accounts and users to find new bottlenecks in scale. Some key counters to consider are:

  • RPS (requests per second)
  • % CPU AppTiers
  • Average Response Time
  • Current Server Requests
  • Active Team Project Collection Service Hosts
  • Private Bytes
Deployment Testing

We run a load tests while the upgrade is happening, so, as activity is constantly going against the service, we can detect outages through all the stages.

  • Stage 1: Deploy new binaries into web front ends and job agents.
  • Stage 2: Run jobs to upgrade the databases.
  • Stage 3: Run jobs to do necessary modifications of the data stored in the databases.
Directed Load Testing

Isolate a particular component or service for performance, stress and scale testing.

Testing in Production

These test are fundamentally focused on analysing the following data to look for regressions:

  • Activity log analysis.

    • Increases in failed command counts
    • Increases in response times, which indicates a perf regression
    • Increase in call counts, which indicates a client chattiness regression
  • CPU and memory usage.

    • Increase in CPU usage after deployment
    • Memory leaks
  • PerfView analysis: mainly driven by a tool called PerfView. Useful for finding memory and CPU regressions in production.

As you see, running performance and load tests is something that could be done anytime, but depends on the time where you do it, it will involve different techniques and data to capture.

I encourage you to create your first Web Performance and Load Tests following some of the next links and find yourself the answer of… When and Why to run Web Performance and Load Tests.

Generate and run coded web performance test

Step by Steps tutorial- Run performance tests on your app

Distributing Load Test Runs Across Multiple Test Machines Using Test Controllers and Test Agents

Analysing Load Tests Results Using the Load Test Analyser

Load Tests in the Cloud

Happy testing!

Eduardo Ortega

Exporting Fiddler2 sessions to Visual Studio Web Tests

While finishing the revisited and new “Web Performance Testing with Microsoft Visual Studio 2013” training at SQS Academy, I found a very interesting topic to cover, idea of my colleague Carl Bricknell, that is to migrate Fiddler2 sessions to Visual Studio generating new Web Tests.

As you know Visual Studio 2013 Ultimate and Visual Studio 2015 Enterprise have included the templates for Load and Performance Tests.
It’s as easy as go into VS and create a new “Web Performance and Load Test project”.

Usually, once the project is done, we have to record our session with the Web Test Recorder as follows:

And right after our session, we will be able to edit the recorded test steps on the WebTest view. This will allow us to create our validation rules that would be used to let the test pass or fail; also create, modify and delete web requests, their headers, parameters (dynamic and statics) and, the most fancy feature, to add data parameters in order to replay the tests several times with different data straight from an Excel Sheet, CSV file or SQL Database.

Many testers and developers use Fiddler2 in a daily basis, and this is because is one of the most powerful web traffic capture tools.

Fiddler2 includes the ability to capture web traffic (including AJAX requests) for later playback with the Visual Studio Web Performance Test feature. It can also be used to help debug your Web performance tests. Comparing your Fiddler2 recording and your Web performance test recording can help identify key missing headers that your Web performance test may not record, amongst other things.

We can summarize as:

Fiddler is at HTTP debugging proxy server application that captures HTTP and HTTPS traffic and logs it for the user to review.

Fiddler can also be used to modify HTTP traffic for troubleshooting purposes.

In order to export Fiddler2 recording sessions to Visual Studio Web Test Project we need to follow the next steps:

  1. Create a Visual Studio empty Web Test Project
  2. Record your session using Fiddler2
  3. Export the Fiddler 2 session to the Visual Studio Web Test Project

The exporting is very straight forward:

  1. Go to File à Export Sessions à All sessions

  1. Select Visual Studio Web Test as Export Format

  1. Select all the plugins installed by default:

  1. Find the generated web test in your hard drive and attach it to your Visual Studio Web Test Project:

Done!

From here, everything that happens is result of your creativity, it’s ready for testing!

  • Happy testing! –

Eduardo Ortega Bermejo

Supported configurations and platforms for VS Coded UI Tests

Coded UI automated tests! Sounds good, uh? Well, yes, it’s very exciting to see how a machine is simulating that is playing with your app while input some predefine values in your textboxes and performing some actions. Said that, it’s good to know where you can create and apply these Coded UI Tests.

First, supported Operating Systems:

  • Windows 7
  • Windows Server 2008 RD
  • Windows 8

Supported Architectures:

  • .NET 2.0, 3.0, 3.5, 4 and 4.5

Second, Platform Support:

  • Windows Phone Apps: Supported
  • Windows Store Apps: Fully supported including HTML5 in IE9 and IE10
  • Internet Explorer 8 and 9.
  • Internet Explorer 10 and 11: is only supported on the desktop, not Modern UI
  • Windows Forms and WPF: Fully supported
  • Internet Explorer 6 and 7: Not supported
  • Chrome: Recording of action steps is not supported. Coded UI Test can be played back on Chrome and Firefox.
  • Opera: Not supported.
  • Safari: Not supported
  • Silverlight Not supported
  • Flash/Java: Not supported
  • Windows Win32 and MFC: Partially supported
  • SharePoint: Fully supported
  • Office Client Applications: Not supported
  • Dynamics CRM web client: Fully supported
  • Dynamics AX 2012: Action recording and playback are partially supported
  • SAP: Not supported
  • Citrix/Terminal Services: Partially supported
  • PowerBuilder: Partially supported

Of course, when the talk about ASP.NET, HTML5 pages, these are included in the fully supported list of apps.

If you want to know how to Create, Edit and Maintain a Coded UI Test, check this out: https://msdn.microsoft.com/en-us/vstudio/ff977233

Happy support!

Eduardo Ortega​