blog

Let's talk about testing

Standard platform, operating system and browser recommendations (updated)

Introduction

The following post makes recommendations for platforms, operating systems and browsers to use when testing commercial marketing web sites aimed at the general public in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and analysis of client needs. Since modern browser versions change frequently, we review and update these recommendations regularly.

Assumptions

  • The majority of users receive new versions of Chrome and Firefox automatically.
    • Each new version cannibalises the market share of previous versions quickly, i.e. within a quarter.
    • Significant differences between versions are few and far between.

List of latest stable releases

Desktop browsers

  • Windows XP: Microsoft Internet Explorer (MS-IE) 8
  • Windows Vista: Microsoft Internet Explorer (MS-IE) 9
  • Windows 7: Microsoft Internet Explorer (MS-IE) 10
  • Windows 8: Microsoft Internet Explorer (MS-IE) 10
  • Chrome stable release 27.0 (June 2013)
  • Firefox (FF) stable release 21 (May 2013)
  • Opera stable release 12.15 (April 2013)
  • OS X 10.6 Snow Leopard: Safari stable release 5.1.9 (April 2013)
  • OS X 10.7 Lion: Safari stable release 6.0.5 (June 2013)
  • OS X 10.8 Mountain Lion: Safari stable release 6.0.5 (June 2013)

Mobile and tablet browsers

  • Android 4.2.2 Jelly Bean (February 2013)
  • iOS 6.1.4 (May 2013)
  • Windows Phone 8.0.10211.204 (Portico)

Recommendations

General

  • Invest in the future, not the past. Focus project time and effort on optimising web sites for the next 6-9 months while making sure that popular older combinations are still viewable.
  • Use a range of mainstream devices to ensure coverage of most regular users everyday experience.
  • Avoid testing with specific minor releases unless there are compelling reasons to do so. E.g. a third-party augmented reality (AR) component is known to work with Safari 6.0.1 but not with Safari 6.0.2.
  • Do not invest in a browser if it is discontinued by the manufacturer. E.g. Firefox 3.6 and iOS 5.x.
  • Do not support beta-release operating systems and browsers. Allow them time to stabilise, be released and become mainstream first. E.g. above 5% market share.

PC operating system recommendations

  • Test with Windows XP, 7 and 8 – see below.
  • Do not test with Windows Vista. Market penetration is low and declining.
  • Do not test with Linux. Market penetration is too low.

PC browser recommendations

  • Do not test with MS-IE 6. It is an old release, market penetration is low and declining. Exception: Web sites targeted at China.
  • Do not test with MS-IE 7. It is an old release, market penetration is low and declining.
  • Test with MS-IE 8 on Windows XP. It is an old release with declining penetration but remains significant for XP users.
  • Test with MS-IE 9 on Windows 7. It is an old release with declining penetration but remains significant for Win 7 users.
  • Test with MS-IE 10 on Windows 8.
  • Test with Chrome latest release only.
  • Test with Firefox (FF) latest release only.
  • Do not test with Safari on the PC platform. The version is obsolete and any remaining market penetration is not significant.
  • Do not test with Opera. Market penetration is too low. Exception: Web sites targeted at Eastern Europe, especially Belarus.

Mac desktop operating system recommendations

  • Test with OS/X 10.8 Mountain Lion.

Mac desktop browser recommendations

  • Do not with Safari 5. It is an old release, market penetration is low and declining.
  • Test with Safari 6.
  • Test with Firefox (FF) latest release only.
  • Test with Chrome latest release only.
  • Do not test with Opera. Market penetration is too low.

General mobile and tablet platform recommendations

  • Test with native browsers only.
  • Test in both landscape & portrait modes.

Apple mobile and tablet platform recommendations

  • Do not test with iOS 5. It is an old release, market penetration is low and declining.
  • Test with iOS 6 using the native Safari browser only.

Android mobile and tablet platform recommendations

  • Test with Android 2.3 Gingerbread using the native browser only.
  • Do not test with Android 3.x Honeycomb tablets. Market penetration is low and declining. Exception: tablet-specific projects.
  • Test with Android 4.0 Ice Cream Sandwich using the native browser only.
  • Test with Android 4.1 Jelly Bean using the native browser only.
  • Test with Android 4.2 Jelly Bean using the native browser only.

Other mobile and tablet platform recommendations

  • Do not test with Windows Phone 7.5. The version is obsolete and any remaining market penetration is not significant.
  • Do not test with Windows Phone 7.8. It is an old release, market penetration is low and declining.
  • Do not test with Windows Phone 8. Market penetration is too small.
  • Do not test with Blackberry. Market penetration is too small.
  • Do not test with Blackberry 10 – too early to adopt and not yet mainstream.
  • Do not test with Firefox OS smartphones and tablet computers – too early to adopt and not yet mainstream.
  • Do not test with Firefox, Chrome, Opera or any other non-native third-party mobile browser variants on any mobile or tablet platform. Market penetration is too small.

Flash recommendations

  • Test with no Flash. At the very least, ‘no Flash’ should prompt the site visitor to download Flash.
  • Test with Flash 10.3.
  • Test with Flash 11.x (latest version).

Sources

MRC


Comments are closed.

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.