blog

Let's talk about cross-browser testing

Standard platform, operating system and browser recommendations (June 15)

Introduction

The following post updates our recommendations for platforms, operating systems and browsers to use when testing commercial web sites targeting consumers in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and analysis of client needs. Since modern browser versions change frequently, we review and update these recommendations regularly.

Assumptions

  • The majority of users receive new versions of Chrome, Firefox and Microsoft Internet Explorer automatically.
  • Each new version cannibalises the market share of previous versions within three months.
  • Significant differences in web page display between versions are few and far between.

List of latest stable releases

Desktop browsers

  • Windows XP: Microsoft Internet Explorer (MS-IE) 8
  • Windows Vista: Microsoft Internet Explorer (MS-IE) 9
  • Windows 7: Microsoft Internet Explorer (MS-IE) 11
  • Windows 8.0: Microsoft Internet Explorer (MS-IE) 10
  • Windows 8.1: Microsoft Internet Explorer (MS-IE) 11
  • Chrome stable release 43
  • Firefox (FF) stable release 38
  • OS/X 10.8 Mountain Lion: Safari stable release 6.2
  • OS/X 10.9 Mavericks: Safari stable release 7.1
  • OS/X 10.10 Yosemite: Safari stable release 8.0

Mobile and tablet browsers

  • Android 5.1 Lollipop
  • iOS 8.3

Recommendations

General

  • Invest in the future, not the past. Focus project time and effort on optimising web sites for the next 6-9 months while making sure that popular older combinations are still viewable.
  • Use a range of mainstream devices to ensure coverage of most regular users everyday experience.
  • Avoid testing with specific minor releases unless there are compelling reasons to do so. Here’s an example of a compelling reason: a third-party augmented reality (AR) component is known to work with Safari 8.0.1 but not with Safari 8.0.2.
  • Do not invest in a browser if support is discontinued by the manufacturer.¬†E.g. Firefox 3.6 and iOS 5.x. The Windows XP / MS-IE 8 combination will remain an anomaly for the rest of the year.
  • Do not support beta-release operating systems and browsers. Allow them time to stabilise, be released and become mainstream first. E.g. above 5% market share.

PC operating system recommendations

  • Low priority: Test with Windows XP with MS-IE 8 only – see below.
  • Test with Windows 7.
  • Low priority: Test with Windows 8 with MS-IE 10 only- see below.
  • Test with Windows 8.1.
  • Do not test with Windows Vista. Market penetration is low and declining.
  • Do not test with Linux. Market penetration is too low.

PC browser recommendations

  • Low priority: Test with MS-IE 8 on Windows XP. It is an old release with declining penetration and no support from the manufacturer. However, it remains significant for remaining users.
  • Low priority: Test with MS-IE 9 on Windows 7. It is an old release with declining penetration but remains significant for remaining Windows Vista users (sic).
  • Low priority: Test with MS-IE 10 on Windows 8.
  • Test with MS-IE 11 on Windows 8.1.
  • Test with Chrome latest release only.
  • Test with Firefox (FF) latest release only.
  • Do not test with Opera. Market penetration is too low.¬†Exception: Web sites targeted at Eastern Europe, especially Belarus.

Mac desktop operating system recommendations

  • Test with OS/X 10.8 Mountain Lion.
  • Test with OS/X 10.9 Mavericks.
  • Test with OS/X 10.10 Yosemite.

Mac desktop browser recommendations

  • Low priority: Test with Safari 7.
  • Test with Safari 8.
  • Test with Chrome latest release only.
  • Do not test with Firefox. Market penetration is too low.

General mobile and tablet platform recommendations

  • Always test with the native browser.
  • Test in both landscape & portrait modes.

Apple mobile and tablet platform recommendations

  • Test with iOS 7 using the Safari browser only.
  • Test with iOS 8 using the Safari browser and Chrome browser (latest release).

Android mobile and tablet platform recommendations

  • Low priority: Test with Android 2.3 Gingerbread using the native browser only.
  • Low priority: Test with Android 4.0 Ice Cream Sandwich using the native browser only.
  • Test with Android 4.1 Jelly Bean using the native browser only.
  • Test with Android 4.2 Jelly Bean using the native browser only.
  • Low priority: Test with Android 4.3 Jelly Bean using the native browser only.
  • Test with Android 4.4 KitKat using the native browser only.
  • Test with Android 5.0 Lollipop using the native browser only.
  • Test with Android 5.1 Lollipop using the native browser only.

Other mobile and tablet platform recommendations

  • Consider testing with Chrome browser (latest release) if not already native. The numbers are increasing.
  • Do not test with Firefox, Opera or any other non-native third-party mobile browser variants on any mobile or tablet platform. Market penetration is not significant. Exception: Consider testing with UC Browser and Opera Mini/Mobile browsers if targeting Asia.
  • Do not test with Windows Phone. Market penetration is not significant.
  • Do not test with Blackberry. Market penetration is not significant.

Watchlist

On our watch list for possible inclusion later this year:

  1. Desktop, tablet, mobile: Windows 10 / Microsoft Edge browser
  2. Desktop, tablet, mobile: OS/X 10.11 ‘El Capitan’ / iOS 9 / Safari 9 browser
  3. Tablet, mobile: Android M

… and finally

Consider testing on all platforms with Yandex Browser if targeting Russia.

Thanks for reading.

Comments? Questions? Let us know!

Sources

MRC


Comments are closed.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.