blog

Let's talk about testing

Mobile website test automation with cucumber but without the hassle

This entry is part 9 of 10 in the series Test Automation

Introduction

In the test automation world the client often requests to run the tests on mobile devices, next to doing the testing on regular desktop. When running your automation in Cucumber, you have multiple possibilities to solve this. In this article I will talk about these possibilities and a new (simple) way to do it with Lapis Lazuli, a Ruby Gem that cooperates with Selenium Webdriver.

I will talk about the following solutions:

  • Lapis Lazuli device simulation
  • Using Appium
  • Using Browserstack
  • Using a phone via USB.

Lapis Lazuli device simulation

With the latest release of Lapis Lazuli (version 1.1.0), it is now possible to simulate a mobile device by providing:

  • Device name
  • User-agent
  • Screen width
  • Screen height

When starting your cucumber script, all you need to do is add DEVICE=iphone5. This will start the browser (Chrome or Firefox) with the dimensions and screen size of the defined device.

For more info, see testautomation.info/Lapis_Lazuli:Device_Simulation

Pros

  • Easy to set up
  • Easy to debug your scripts
  • Works with all your existing scripts
  • Any website will recognise the visitor as mobile.
  • Easy to integrate with your existing Continuous Integration
  • A free solution

Cons

  • Not as accurate as a real mobile device.

Appium

Another possibility is using Appium. This is a tool that allows you to connect to an emulated device, install or start an application and use it.

Pros

  • Uses the actual mobile device
  • Can also be used in combination with Cucumber
  • A free solution

Cons

  • Does not work with existing TA scripts
  • Is more focussed on testing apps, not websites
  • Difficult to set up in a Continuous Integration environment

Browserstack

This is a service that allows you to emulate any device. Browserstack also allows tools to connect to it and run automated scripts.

Pros

  • Reasonably easy to set up
  • Close to testing with the real device (emulation)
  • Usable in Continuous Integration

Cons

  • Not a free solution
  • Because your scripts are running externally, debugging is difficult
  • Remote testing increases the chance of invalid failure due to latency

Using mobile devices via USB

You could also connect a mobile device to your computer and run your scripts on the mobile device using Google Chrome. To be able to do this you will need to install the device drivers from Android Studio

I did not research a solution to do this with iOS, but most likely, it is only possible if you own a OS X computer.

Pros

  • Most reliable test results
  • Easy to do debugging
  • All Android tools are free

Cons

  • A lot of software to install.
  • Not useable for Continuous Integration. (Unless you physically connect mobile devices to your CI server)
  • Only Chrome is supported.

Summary

Chrome uses the same version for mobile and for desktop. Because Chrome has become the most common browser for mobile devices, a simulation like Lapis Lazuli offers, is now quite reliable.

For obvious reasons, Lapis Lazuli calls the mobile web testing “Device Simulation”. It is a solid and simple way to test websites for their mobile capabilities. Unfortunately, it is not the real thing. If using real devices is a priority for you and you want it to fit in your Continuous Integration, I would recommend using Browserstack. If you are only running your tests locally, I would recommend running your tests via USB.

Our solution for Continuous Integration, called Calliope, also supports remote browser testing. This function is still in beta testing, but soon you will be able to link your Browserstack account to your Test Automation solution and run all your scrips on any Browserstack device.

Gijs Paulides – spriteCloud – functional test engineer


  • Mark Duivesteijn

    These new functions make the gem so much more usable for Test Automation, I love it. Can’t wait for browser stack support.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.