One of every two companies nowadays ask for a mobile application to integrate in their ecosystem of services and products. Most of the times is just for increasing the marketing or presence on the media, or to offer a better customer experience, or just to help their own employees to be more mobile. Whatever it is, one thing is clear, Mobile Apps are here to stay.

Microsoft Visual Studio offers its own Mobile Testing Platform, using frameworks such as Coded UI, MS Unit Test and few more included with the Visual Studio Suite. But most of the mobile test targets are Windows Phone apps. What about Android and iOS?

Xamarin is a multiplatform framework that allows you to write Mobile Apps for iOS, Android and Windows Phone using C# and Visual Studio or Xamarin Studio.

Recently they started offering a testing service called Xamarin Test Cloud. I’m not going to enter on licensing matters but in what you can do with it.

Xamarin Test Cloud automatically test your app on 1,000 devices in the cloud. Write your own tests, or have our engineers create and maintain a test suite for you.

Automated UI tests can be written using one of two frameworks:

• Xamarin.UITest – This is a framework that allows test to be written in C# using NUnit testing library. This is mainly for developers that write their own automated tests. Xamarin.UITest is available as a NuGet package and relies on NUnit.
• Calabash: This is a framework that allows you to write test in Ruby using Cucumber. This gives you an indication that this will be driven by Behaviour Driven Development.

With Xamarin.UITest, an interface shell called REPL is exposed to explore the views on a screen and automate interactions with the application. The best thing of this framework is it worksin any mobile app regardless of the platform it’s written in.

Each test is a method that follows the Arrange-Act-Assert pattern, as the one used in CodedUI with Visual Studio.

The process will follow the next steps:

1. Automate your app using one of the testing frameworks
2. Upload the test suite and run it on hundreds of real devices in the cloud
3. Receive a detailed report with the results, screenshots and performance metrics.

In terms of the capabilities of the Frameworks, here some details:

• Interaction:

  • Most of the gestures are allowed on the testing: tap, double-tap.
  • Build your own gestures.
  • Query UI elements and use CSS selectors to query these elements in hybrid apps.

• Device features:

  • Change GPS location, press physical buttons, activate camera and rotate the device.
  • Simulate 2G, 3G, 4G networks.

• Tools:

  • Write test in C# or Ruby with Cucumber.
  • Use Xamarin Studio and Visual Studio for C# tests.
  • Integrate with standard NUnit test runners

• Analyse app performance

  • Screenshots and video playback for every step of every test.
  • Receive p.erformance data and compare reports against previous runs to find regressions.

• Test continuously:

  • Collect test results in your CI system to include UI failures in your nightly reports
  • Integrates with TFS, Jenkins, TeamCity and post-build commands CI systems.

I didn’t have the luck to start playing around with it yet as it requires a license that I’m trying to get, so once is with me I will give you more details about how to perform and will do a comparison between Xamarin Test Cloud, SeeTest and Perfecto Mobile.

If you want to know more about how to write test with C# and Xamarin or Ruby, follow the links on the resources, there is a whole page plenty of guides, samples and API references here.

Happy testing!

Eduardo Ortega

Resources:

Xamarin Test Cloud: http://xamarin.com/test-cloud

Introduction to Xamarin Test Cloud: http://developer.xamarin.com/guides/testcloud/introduction-to-test-cloud/

Submitting Test to Xamarin Test Cloud: http://developer.xamarin.com/guides/testcloud/submitting/

Automate Xamarin.Forms testing with Xamarin.UITest and Test Cloud: http://developer.xamarin.com/guides/cross-platform/xamarin-forms/uitest-and-test-cloud/

[Video]Using C# and UITest to Create your first mobile test: http://blog.xamarin.com/test-cloud-video-series-using-c-and-uitest-to-create-your-first-mobile-test/

[Video]Uploading Test Scripts and View Mobile Test Results: http://blog.xamarin.com/xamarin-test-cloud-video-series-uploading-test-scripts-and-viewing-mobile-test-results/

[Video]A Quick Overview of Xamarin Test Cloud: http://blog.xamarin.com/video-a-quick-overview-of-xamarin-test-cloud/