blog

Let's talk about cross-platform testing

Standard platform, operating system and browser recommendations (Feb 17)

This entry is part 11 of 11 in the series Standard Platform, Operating System and Browser Recommendations

Introduction

The following post updates our recommendations for platforms, operating systems and browsers to use when testing commercial web sites targeting consumers in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and our analysis of client needs. Since browser versions change frequently, we review and update these recommendations regularly.

Assumptions

  • The majority of users receive new versions of Chrome, Firefox, Internet Explorer, Edge and Safari automatically.
  • New versions cannibalises the market share of old versions within three to six months.
  • Significant differences in web page display between versions are few and far between.

List of latest stable releases

Desktop browsers

  • Windows 7: Microsoft Internet Explorer (MS-IE) 11
  • Windows 8.1: Microsoft Internet Explorer (MS-IE) 11
  • Windows 10: Microsoft Edge 38 / EdgeHTML 14
  • Chrome 56
  • Firefox (FF) 51
  • Safari 10

Mobile and tablet browsers

  • Android 7.1 Nougat
  • iOS 10

Recommendations

General

  • Invest in the future, not the past. Focus project time and effort on optimising web sites for the next 6-9 months while making sure that popular older combinations are still viewable.
  • Use a range of mainstream devices to ensure coverage of most regular users everyday experience.
  • Avoid testing with specific minor releases unless there are compelling reasons to do so. Here’s an example of a compelling reason: a third-party augmented reality (AR) component is known to work with Safari 10.0.1 but not with Safari 10.0.2.
  • Do not invest in an operating system / browser combination if support is discontinued by the manufacturer. E.g. Google Chrome on Windows XP.
  • Do not support beta-release operating systems and browsers. Allow new versions time to stabilise, be released and become mainstream first. We define ‘mainstream’ as above 5% market share.

PC operating system recommendations

  • Test with Windows 7.
  • Test with Windows 8.1.
  • Test with Windows 10.

PC browser recommendations

  • Low priority: Test with MS-IE 11 on Windows 7.
  • Test with MS-IE 11 on Windows 8.1.
  • Test with MS-Edge on Windows 10.
  • Test with Chrome latest release only.
  • Test with Firefox (FF) latest release only.
  • Do not test with Opera. Market penetration is too low. Exception: Web sites targeted at Eastern Europe, especially Belarus.

Mac desktop operating system recommendations

  • Low priority: Test with OS/X 10.10 Yosemite.
  • Test with OS/X 10.11 El Capitan.
  • Test with macOS 10.12 Sierra.

Mac desktop browser recommendations

  • Low priority: Test with Safari 9.
  • Test with Safari 10.
  • Test with Chrome latest release only.
  • Do not test with Firefox. Market penetration is too low.

General mobile and tablet platform recommendations

  • Always test with the native browser.
  • Test in both landscape & portrait modes.

Apple mobile and tablet platform recommendations

  • Low priority: Test with iOS 9 using the Safari browser and Chrome browser (latest release).
  • Test with iOS 10 using the Safari browser and Chrome browser (latest release).

Android mobile and tablet platform recommendations

  • Do not test with Android 4.1 and 4.3 Jelly Bean. Market penetration is too low.
  • Low priority: Test with Android 4.2 Jelly Bean using the native browser only.
  • Test with Android 4.4 KitKat using the native browser only.
  • Test with Android 5.0 Lollipop using the native browser only.
  • Test with Android 5.1 Lollipop using the native browser only.
  • Test with Android 6.0 Marshmallow using the native browser only.
  • Test with one version of Android 7 Nougat using the native browser only. Market penetration is still too low to justify testing with both versions.

Other mobile and tablet platform recommendations

  • Consider testing with Chrome browser (latest release) if not native.
  • Consider testing on all platforms with Yandex Browser if targeting Russia.
  • Do not test with Firefox, Opera or any other non-native third-party mobile browser variants on any mobile or tablet platform. Market penetration is not significant. Exception: Consider testing with UC Browser and Opera Mini/Mobile browsers if targeting Asia.
  • Consider testing with Windows 10 Mobile only if penetration is significant in your target market.
  • Do not test with Blackberry. Market penetration is not significant.

Thanks for reading. Got comments? Questions? Let us know!

Sources


Comments are closed.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.