blog

Let's talk about testing

Which Android versions should I test on?

This entry is part 5 of 5 in the series Android

Android statistics chart

It has been an interesting couple of months for Android enthusiasts. The highlight probably has been the release of the Samsung Galaxy S8 and S8 Plus. Next to releases of new devices, Android Nougat is being released by manufacturers to more devices. We can see this in the Android statistics for the last three months. Android Nougat has gained a market share of 7.1% compared to the 1.2% it had in February. Together with the growth of Nougat, we can see the stabilization of Android Marshmallow and Lollipop. Kitkat still holds a marketshare of 18.8%, losing only 3.1% over the last three months.


Fig.1 Overview by Android codename (click to enlarge)

Looking at specific version numbers we can see the same trend. Android 7.0.x and 7.1.x are gaining marketshare while Android 5.0.x, 5.1.x and 6.0.x are stabilizing. All older versions are slowly but steadily losing marketshare.


Fig.2 Overview by version (click to enlarge)

Based on these statistics, we can conclude that Android Nougat is really taking of now. For testing, it should be one of your priorities. Based on the trends we have seen in the past, Android Nougat will quickly gain more marketshare in the coming months. We expect it to be the largest Android version in about a year.

Source: https://developer.android.com/about/dashboards/index.html

Successful test automation with Cucumber in the Cloud

How can you make Test Automation succeed?

The biggest challenge to making a success out of test automation (TA) is maintainability. Maintainability of both your TA test suite as well as the TA infrastructure you are using is critical. I will explain both and describe how to get a successful TA setup on the API and UI level, using open source frameworks and the test automation platform Calliope.pro that can run your Cucumber TA suite in the cloud.

Maintainable TA suite

Things happen… A tester or developer goes on vacation or leaves your development team; changes in the tested application are made and break the test scenarios. When the TA suite is complex and hard to understand, in time and with a changing environment, it will soon require too much effort by the team to keep the suite up and running; it is doomed to fail. Using a scripting language such as Ruby, as opposed to a programming language, will keep the test suite easy to understand for non-developers. Using Cucumber (Gherkin language) as the top layer of your TA suite will make the test scenarios understandable for the business. On www.testautomation.info you can read about creating a stable TA suite from scratch using open source software. You can also find an Ecommere TA Suite template on github.

Maintainable TA infrastructure

Things still happen… Regular updates on web browsers and (test automation) frameworks get in the way and break your TA setup. The complete test automation infrastructure requires the ongoing attention of system engineers; but your system engineers are already busy maintaining the production environment… These factors affect whether you succeed or fail with your TA setup. Online TA platforms maintain the infrastructure for you, allowing you to put your effort elsewhere. spriteCloud Calliope.pro is such an online TA platform. It enables you to have a very cost effective test automation setup running all your Cucumber/Ruby TA suites.

Want to see for yourself how successful test automation in the Cloud works? Calliope.pro offers a demo functionality where you can see online test automation in action. Enjoy and don’t hesitate to contact us if you have any questions about test automation!

The testing recruitment Challenge

Monday, starting the week with another day in recruitment, where you hope that your inbox will be filled with good CV’s of software testers. Not only non-EU candidates but also Dutch speaking testers with test automation skills. Unfortunately for me, the market has flipped totally from a client driven market to a candidate driven market. And my inbox is almost empty.

Instead, the good software testers have an inbox full of incredible job opportunities. They get to choose between different companies to work for. Most testers nowadays get a headache of recruiters and are annoyed by constantly being phoned at work during the day.

As the talent acquisition specialist of spriteCloud in this market, recruitment is a time-consuming and grafting exercise. To find the testers I want to hire (YOU!) I need to fish and source extensively.

But after the fishing, sourcing and interviewing you every so often find someone who actually is a good fit, and has the skills we are looking for. Then you have to get them in. The negotiation starts and the offering of the right competitive contract is key with nice employee benefits. In this market it is not only about the money. Remuneration is one thing and every company has a cap. And lets be honest; beyond a certain amount of money, people are not motivated to work harder or perform better.

People are mostly motivated by secondary benefits like:

  • Flexible/part-time working hours
  • Training and personal development
  • The latest, most powerful hardware
  • Newest technical environments
  • Time off work to attend conferences
  • Etc. etc.

Fortunately spriteCloud’s secondary benefits have a competitive edge. And that makes my job rewarding instead of frustrating. I can tell our candidates that we have great clients: Heineken, adidas and G-Star RAW. That we do game testing and VR testing. That the opportunity to cross train in load testing or pentesting is possible. That we have our own open source test automation platform and our own test lab with a broad range of different devices. All assets to show why it is so much fun to work at spriteCloud.

So, if you are interested to know more and want to be part of the test community spriteCloud contact me, and I will tell you much more about our great company.

Patricia van Boxtel
Talent acquisition
0627107660
patricia.van.boxtel@spritecloud.com

Oedipus and the testing Sphinx

A long time ago, in ancient Greek, Oedipus was travelling to Thebe. On his way, he crossed the path of a Sphinx. The Sphinx stopped all travellers on the road to Thebe to ask them a riddle. If they were not able to answer the riddle correctly, they would be killed and eaten. If the travellers were able to answer the riddle correctly, they could continue their journey. The Sphinx asked Oedipus this riddle: ‘What walks on four feet in the morning, two in the afternoon and three at night?’ Oedipus took a short moment to think and answered: ‘Man: as an infant, he crawls on all fours; as an adult, he walks on two legs and in old age, he uses a ‘walking’ stick’. The Sphinx, puzzled by the fact that somebody actually was able to answer the question, allowed Oedipus to continue his journey to Thebe.

Now the testing Sphinx asks you: ‘What walks on four feet in the morning, two in the afternoon and three at night?’ You give him the same answer as Oedipus did. But she is not puzzled by the fact you are able to answer the question. Instead, she challenges you to give the smallest amount of test cases to test the theorem. If you are able to give the Sphinx the correct test cases, she will let you continue your journey. If not … you have to do overtime forever.

As a seasoned tester, you know what to do: use boundary analysis to identify the test cases. So first you check what the age groups are. The answer is 0-17, 18-64 and 65 and up. You define the minimum required test cases as 17, 18, 64 and 65. You choose these test cases, because you know that to test the boundaries, you will need to test the smallest possible step from the boundary, in this case 1 (year).

The Sphinx is puzzled by your testing knowledge and accepts your answer. You are allowed to continue your work without having to do overtime. But beware! The Sphinx may come back with another riddle, even more difficult. If you cannot answer the riddle … it is overtime forever for you.

Mobile website test automation with cucumber but without the hassle

This entry is part 9 of 9 in the series Test Automation

Introduction

In the test automation world the client often requests to run the tests on mobile devices, next to doing the testing on regular desktop. When running your automation in Cucumber, you have multiple possibilities to solve this. In this article I will talk about these possibilities and a new (simple) way to do it with Lapis Lazuli, a Ruby Gem that cooperates with Selenium Webdriver.

I will talk about the following solutions:

  • Lapis Lazuli device simulation
  • Using Appium
  • Using Browserstack
  • Using a phone via USB.

Continue Reading »

Which Android versions are most important for testing?

This entry is part 6 of 5 in the series Android


In the last few months Android Nougat (7.0/7.1) has started to gain market share according to the statistics that Google published. From previous new Android versions, we’ve seen that it takes about 6 months to gain more than 5% market share. Another sign of change is that Android 6.0 growth is slowing down. Here you can apply the same rule of thumb that within 6 months, this version will not increase its market share any more, instead the numbers will start to decline instead. And last but not least, Android 5.1 is finally showing a drop in market share.

Continue Reading »

Standard platform, operating system and browser recommendations (Feb 17)

This entry is part 11 of 11 in the series Standard Platform, Operating System and Browser Recommendations

Introduction

The following post updates our recommendations for platforms, operating systems and browsers to use when testing commercial web sites targeting consumers in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and our analysis of client needs. Since browser versions change frequently, we review and update these recommendations regularly.

Continue Reading »

spriteCloud placed 41 in the NRC Carrière monitor research for best employers

spriteCloud participated in the “NRC Carièrre monitor 2016/2017” survey and placed 41.

On 23 January 2017 the NRC Carrière Monitor published the results of their research at an event in Amsterdam. The evening included an inspiring presentation about Talent Management from industry-expert Prof. Lidewey van der Sluis.

Consultancy.nl plublished the post top 50 employers of NL according to employees that list the 50 top employers of the NRC Carierre monitor 2016/2017.

My year in review

Wait … what do you mean we need to get a Christmas tree? It seems that 2016 flew by and 2017 is already knocking on the door. As 2016 draws to a close, I would like to take a look back to see what happened and what we can expect for the year ahead.

As a company, it was an exciting year where we had a big growth, and became even more structured and performant. Due to our agile structure we were able to focus on upcoming technologies and for example started our VR Department with a dedicated test area.

Looking at the global testing trends of the past year, a few things caught my eye.

I worked on many projects this year for all kind of companies. Every time I started one, I heard the words ‘continuous’ and ‘deployment’ as I walked into the room. This is a trend that has been going on for quite some time already, but this was the first year that I heard this through almost everywhere. I notice that even the bigger & more hierarchal companies have jumped on the band-wagon too.

Another interesting development is the continuing and growing use of open source tooling. Selenium and SoapUI are showing up everywhere I go, while paid tools like QTP are seen less.
There is of course a big divide between multi-nationals and the smaller companies, but even the bigger companies seem to be using open source tooling now, especially since it can fit in with their needs more easily.

Within spriteCloud we saw a big rise in mobile & tablet testing requests in 2016. Companies and digital agencies are clearly shifting from the use of traditional media to the use of digital channels to reach their core target groups. With the rapid development of mobile hardware, a much more immersive experience can be delivered. This is a trend I expect we will see continuing to grow.

Lastly, I should not forget upcoming technologies. 2016 saw VR go mainstream. Oculus and Vive are selling top-notch hardware for the enthusiast who can afford it. Meanwhile, established companies have started to push out their easier to use, entry-level experiences.

Samsung brought out their Samsung Gear VR that can be used with their Galaxy series phones.
Google announced Daydream and are releasing their own headset.
Sony pushed out Playstation VR in the last quarter of this year, a system easily used by anyone owning a PlayStation 4 and having €400.- spare. Analysts expect around 1.6 Million units to be sold in 2016 alone(1).

As discussed in previous posts, VR presents exciting new challenges for testers everywhere.
Signing off, I would like to wish you all a lovely Christmas and New Year and an awesome start of 2017!

spritecloud_vector_clean_christmas

(1): https://technology.ihs.com/Biographies/400761/piers-harding-rolls

Which Android versions are most important for testing VR?

This entry is part 4 of 5 in the series Android


Continue Reading »

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.