02 Jul Cloud-Based Mobile Testing Platforms
The world is moving on to the metaphorical “cloud” of the internet which calls out a thorough Computing and Testing efforts.
What is Cloud?
Cloud as a metaphor to the Internet refers to the space where one can access or store data or information than on a hard drive. It acts as a centralized place for delivering various computing services, therefore the end user essentially avails the resources available on cloud like Servers, Virtual Desktops, Software Platforms, Applications, Storage/Data etc in a variety of ways.
Let’s discuss some of the cloud-based mobile testing tools in detail.
Kobiton is a cloud based platform for executing both manual and automated tests for web and mobile based applications. It supports automated tests with selenium for web based applications and appium for native and mobile applications. Kobiton provides testing the application with many languages so there is no boundation to learn a new language separately. Testing on new mobile devices is quite tough as it is practically not feasible to have a variety of mobile devices, but with Kobiton one can continuously test applications on a variety of mobile devices ranging from older devices to the latest available models. Testing apps with Emulator which sometimes fails to reproduce real user environments, while kobiton allows to execute automation scripts on real mobile devices. It also provides a good test report for devices tested with logs, Http commands and meta-Data, not only that screenshots and performance data can also be maintained using this tool. Appium being an open source framework is widely used for Automating mobile Apps and this tool is nattily built on this framework. Taking about the pricing it is available in three different plans: Individual $10/month or $100 annually, Team $199/month or $2000 annually, for growing Biz 499/month Or $5000 annually and a 15 days free-trial is also available. Over more than 160 Devices are available including iphones, ipads and google pixel devices.
- It is Built on Appium Framework.
- Tests on a large number of devices.
- Devices allow camera Access.
- Supports application files upto 500 MB
- Devices can’t be used to make phone calls or send SMS.
- Shows slow responses.
AWS Device Farm
AWS Device Farm is a cloud based platform for Testing Mobile Apps. Both manually and through Automation. It was developed by Amazon Web Services. It is basically an App testing tool that helps you test your Apps with real time devices hosted by AWS on the cloud. Device Farm supports testing of native, android and iOS applications Developed using various frameworks like unity, xamarin and phonegap. It allows the user to upload to their own test or perform a script-free compatibility test. Device Farm can be used in two ways:
Automated app testing: In Automation mode of testing it allows the user to upload to their own test or perform a script-free compatibility test as testing is performed parallely and test on multiple devices, as the test gets completed a test report gets generated comprising of High level Results, low level logs, screenshots and Performance analysis. Some automation test Framework supported by device farm are:
- UI Automator.
Remote Access Interaction: Remote Access Interaction which allows you to interact with various devices and maintains information of the device you interact with and generates a good report. Various Swipes and gestures are provided to interact with the devices. In remote access mode device farms capture the details of the actions the user performs on the devices. Logs with details of the actions and screenshots are available at the end of the session.
- It has both iOS and Android devices.
- It is easy to learn and adapt.
- Orientation of the device can be switched between landscape and portrait mode.
- Reports generated are available for 400 days.
- Device access is slow.
- Sometimes connection fails during testing.
- Gestures Like ZoomIn and ZoomOut are not available.
Firebase Test Lab
Firebase Test Lab is a cloud based applications testing platform for iOS&Android applications. It is developed by Google ,with a single operation you can test your android application with a wide range of devices available on the Google data centre and produces the result with Logs, videos and Screenshots too. It also provides email notification when your test gets completed. It also shows performance results. It is free for the first 10 virtual tests and 5 real device tests per day. Firebase allows you to integrate your existing workflow through Continuous Integration. It also has an Intelligence feature using Robo test which allows you to test the application with almost no code. The devices in the google data centre are provided with updated Api’s and customizable settings. It can also be integrated with Android Studio and devices available in the firebase are available with latest updated API’s and also with editable configuration settings. Testing in FireBase is done in two ways:
- Instrumentation Test that can be accomplished using the Robotium Or Espresso Framework.
- RoboTest which analyzes the apps structure then automatically simulates the user activities.
FireBase comes with three Plans: Spark which is free but with certain limitations, Flame wich is $25 per month and a Blaze plan that provides users all the features at its fullest.
- It is highly capable of managing Real-Time data.
- As data is stored in the cloud so can be accessed anytime from anywhere.
- It supports the Cross Platform API.
- Storage format is JSON unlike SQL so migration is not easy.
- It provides support for only Robotium Framework.
Perfecto is a web-based Software as a service that allows QA engineers to work services such as advanced automation and testing. For manual testing it provides support for all versions of the android and iOS and also provides real time devices for testing. The devices operate perfectly for almost all major mobile networks. It Also provides features to make calls and also send and receive messages while talking about automation the test scripts are easy to generate and use as with no code involved. Scripts are written using commands which are easily available in the form of widgets, test engineers just need to click on the commands to add it to the Script. It uses Keyword based Script technology. Testers can easily install their APK’s on the remote device using the Install widget. This tool automates on the basis of Optical Character Recognition (OCR) which analyzes the user Interface and tests it without any Limitations. It also generates a detailed report of the script executed with results, video recording of the script and graph statics. The scripts can perform functional, UI and Performance testings, same test scripts can be executed for different mobile devices irrespective of the OS or Platform. Perfecto comes with three plans: Basic $99/month or $1188 annually, Advanced $299/month or $3,588 annually and two hours free-trial.
- It is easy to learn and adapt as it uses Keyword based Script technology.
- Test reports generated can be shared with other users with different formats like HTML, XML, PDF, CSV etc.
- It also provides support for data driven approaches on devices.
- Testers can easily Install their APK’s on the remote device using the Install widget.
- Slow responses.
- If a device is allocated to a user then the other user can not use the same device at the same instance the user has to wait for the device to be free.
Sauce Lab is a cross platform testing tool which can test mobile applications over more than 600 browser and OS platforms. It can be used for both manual automation. Talking about Manual sauce lab accesses the servers that are not available publicly by creating a secure tunnel between the local network and sauce test platform. Talking about Automation Sauce labs can be used with Selenium to perform testing over various browsers/OS platforms. Test Automation can automate the mobile apps using Appium, Espresso or Robotium frameworks of which all are open source. It also provides features such as Continuous Integration(CI) and Continuous Delivery(CD) with Jenkins, Bamboo and many other platforms. 211 real devices are available on saucelabs out of 65 are iOS and 146 are android with Android versions from 2.2.2 to the latest version and iOS from 7.1.1 to the latest version. Talking about the price plans it provides a variety of plans : Manual $29/month, Individual $199/month, Expert $399/month, Professional $499/month, Small Team $699/month and an Enterprise Plan is also available. A 14 days free-trial is also available.
- Provide support for more than 600 browsers platforms.
- Provides support for both Appium and Robotium.
- Device Fragmentation is good.
- Continuous Integration is possible.
- Slower in performance when compared to Aws Device farm.
- App to be tested needs to be first uploaded on SauceLabs storage.
- Lacks support for multiple tests simultaneously.