Date:
5 May 2020
Author:
Sami Ullah

About cross-browser testing

Cross-browser testing is an intrinsic part of web and mobile application testing. In the real world it's not possible to test a web app or native app on every device available in the market. We have covered the basics of this kind of testing in our blog Effective cross-browser testing.

BrowserStack and Sauce Labs

Both BrowserStack and Sauce Labs are cloud-based cross-browser testing platforms. From Google Trends, we can see that users have shown more interest in BrowserStack than Sauce Labs.

Browserstack and saucelab

Both platforms provide the following generic features:

  • Web application testing on multiple browsers across different devices and operating systems

  • The latest and older versions of browsers

  • The latest and older versions of operating systems

  • Options for changing the resolution

  • Local testing by a tunneling feature

  • Debugging and logging options

  • Reporting bugs via screenshots with proper annotation

  • App logs and network logs for native app testing

  • Integration for raising bugs on Jira and GitHub

  • API for automation testing along with screenshots and videos of executions

  • Bug reports

  • Analytics for tests executed (pass%, error% and number of tests executed)

  • An option for uploading installation files for the app for native app testing

  • Console logs, network logs and debugging options

  • A browser or app proxy option

  • A location or GPS option for testing apps/websites from a different location

  • Network throttling for testing apps or websites under different network conditions like 4G, 3G slow or 2G

  • Testing for simulated interruptions such as no network or wifi interruptions.

BrowserStack versus Sauce Labs

At Salsa we always analyste the tools before using them or bringing them into our standard practices. Currently we’re using BrowserStack mainly for cross-browser testing across different combinations of browsers and operating systems. We test various web components and user journeys on BrowserStack and provide results to our development team using Jira.

For one of our recent projects, we needed to test native iOS apps on multiple versions of iOS devices. Before performing actual testing we analysed both BrowserStack and Sauce Labs on various parameters, including:

  1. Pricing: Various pricing and plans should be available based on the number of parallel users (number of users that concurrently use the platform). The platform should also offer a trial for testing a few important features before subscribing.

  2. App installation option: If you’re manually testing a new native application you’ll either download the beta version from the app store or you can ask the developer to give you the installation file so that you can download the app on your device. Cross-browser platforms should provide the ability to upload such files or provide access to the app store or tools that hold these apps when under development, like Hockey or TestFlight.

  3. Usability: The platform should have a good user interface so end users can easily switch between different screens with no screen lag when testing the app.

  4. Logging: Whenever issues occur, web application developers quickly check browser or server logs. Cross-browser platforms should also provide features so that developers can easily debug the application they’re building.

  5. Network throttling and location options: The platform should have an option for changing network speed, such as switching between 4G to 3G. They should also have location changing options so that users can easily test an app from different locations.

  6. Device rotation: For testing a native app, the platform should provide the ability to switch between portrait and landscape modes.

  7. Interrupts: The platform should have an option for disabling the network or killing the running app. This helps the user to test it under such conditions easily.

  8. Reporting of bugs: Tool should have integration to directly log bugs on Jira and GitHub.

  9. Gestures: Native apps support device touch gestures. Platform should let users perform touch-screen gestures such as pinch, zoom, swipe, scroll, etc.

  10. Local tunneling: Local testing creates a secure, persistent tunnel between your local development/staging environments and the cloud platform. With local testing, all your web and mobile apps can be tested out-of-the-box, including those hosted on localhost, behind proxies and firewalls, and much more. Platforms should support this feature so that developers can test things easily.

Cross-browser tools should have most of the generic features listed above.

Comparing BrowserStack and Sauce Labs

Below is the comparison matrix based on the parameters above:

BrowserStack

Sauce Labs

30-minute free trial for users

14 days’ free access

iPhone XS and iPad Pro are provided during the trial period

Few iOS devices are provided during the trial period

Installation options provided from App Store, Hockey app, Test Flight and App Center along with the default file-upload option for uploading the app to the cloud device

Only file-upload option for uploading app installation file to test is provided during trial period

User interface looks sleek and shows installation options and devices available at single place

Browserstack

App installation options and devices available are controlled from different screens.

Saucelab

Data centre selection for this platform was not available

Sauce Labs provides two data

Centre options: US West and Europe

App live testing has following options:

  1. Console log access

  2. UI inspector

  3. Network logger

  4. Throttling

  5. Device rotation

  6. App install/uninstall feature

  7. Location and language changes

  8. App kill option

Browserstack

Live mode provides following options:

  1. Screenshot option

  2. ADB shell

  3. Wifi disable option

  4. Enable/disable system animations

  5. Paste content into device

Saucelab

Integration for logging bugs to Jira, GitHub, Trello, etc. are provided

Bug logging is provided using Jira and GitHub only

Local tunneling feature is provided

Local tunneling feature is provided

Provide videos for automated tests

Provide video recording for automated tests, as well as reports with number of bugs found, error %, etc.

For one parallel test user price is US$29 per month

See pricing

For one user plans start at US$19 per month

See pricing

Provides basic gestures like swipe, zoom, scroll, pinch and tap

Provides basic gestures like swipe, zoom, scroll, pinch and tap

Conclusion

Overall both seem feasible, but at Salsa Digital we use BrowserStack as our preferred option because it’s pricing is slightly more competitive and the stability of the interface on devices is good, with minimal lag.

Subscribe to Salsa Source

Subscribe to Salsa Source to keep up to date with technical blogs. 

Subscribe