Theme images by Storman. Powered by Blogger.

Software Testing

[best practices][feat1]






Most Recent

Random Posts




Wednesday, February 10, 2016

Mobile Testing Challenges for the Software Testers in 2016


1. Native vs Web-view

This has been a continuous problem that will most likely be around for a long time; the willingness to get an API ready and complete and not resorting to having to use web views and other clunky alternatives.

Testing a hybrid app in the sense that some parts are native and some parts are webviews is a particular pain in the butt. When using these two side-by-side you can pretty much always notice that native is faster so a webview will always look slower and buggy. There’s almost always a problem with the API and a webview talking to each other.

Solution: If you’re going to build an app, make sure you have all the APIs you need ready and raring to go before dev starts or early enough that something can be done about it.
If not go for all webviews and keep the app looking consistent in one form or another. At least this way it won’t be noticeably different as you traverse the app.

2. Mobile Test Automation

Mobile test automation isn’t in a great place – the market is dominated by either big player pushing sub-standard, expensive tools to ‘automate all the things’ or smaller open-source tools that are at the mercy of the mobile OS vendors.

Too many people still have an ‘automate all the things’ test strategy for mobile, not realising that doing this is even more fool-hardy than on desktop web. Mobiles are complicated and automating a lot via the app UI is not the best strategy at all.

Solution: Focus on understanding your application and the web services it most likely uses. Automate those first. Apply a small, light-weight automated UI test set using whatever automation tool suits you best. Be aware that the automation tool you use may stop working because an OS vendor cut’s access to APIs so have a plan B.

3. Fragmentation

There are soooo many Android devices out there that you can only test a small fraction of them. Add to that the different flavours of OS’s (Samsung adds something, other manufacturers do so too) and you have many potential risks.

Solution: I’d like it if only stock Android was used but hey…I can keep on dreaming I’m afraid. As far as I know, there’s no simple solution. You just have to engage with your customer and talk about which devices / OS’s must be tested (based on facts like most popular devices in your country etc).

4. Unexpected pop-ups

It’s hard to write a repeatable automation test script when at any given time, an ad or other alert-style window appears and obscures the screen below. This has been a problem since the creation of the GUI and is still a problem in 2016. (Ads are mostly the problem now.)

Solution: Create a routine that is always running in a background thread and scanning the screen for a list of “usual suspect” pop-up windows. And if found, perform the actions necessary to remove it — usually it’s sending an “Escape” key event. You also have to make the main script flexible enough to know to wait if it gets “stuck” by a pop-up.

5. In The Wild Testing

One of the biggest challenges software testers need to handle in 2016 is manual “In the wild testing”. The problem that I see, is that many software testers, test mobile applications in the same way as they would test web or desktop applications, always on the same place with the same Internet connection. But mobile apps are not the same like web or desktop applications. They are used in a total different environment with different Internet connections. If a mobile app is tested only with a fast and stable Internet connection lots of problems will not be covered during the testing and development phase.

Testing the app in the wild may find problems with:

  1. The light situation because of sun or rain.
  2. With the sensors because they are interfered by other elements such as buildings, cars or other obstacles.
  3. With the Internet connection. Due to the movement, uses have not always the same Internet connection and this has an impact to the mobile app under test. For example, the network switch from LTE to Edge may influence the UI or the information that will be processed between the backend and the mobile app.
The listed points are just some of the problems that will not be found in a lab situation.

Solution: The solution for this problem is pretty easy. Software testers must test outside the office. Depending on the use case of the app, the environment may change. For example if you are testing a business app, the app is likely used when people are commuting to or from work by train or car. Or if you are testing a sports app, that is tracking your workouts and activities, then you must test during this situations, too.

It is key for every software tester to test in the environment where the customers will use the app. I recommend to every software tester, to pack some devices in a backpack and to leave the office to test the app in the wild.

When testing in the wild it is important that the app under test is using a crash reporting tool like HockeyApp or Crashlytics to record any kind of problems that may happen during the test scenarios. Besides that I recommend to have just a pen and a piece of paper with you to make notes when discovering a problem.

0 on: "Mobile Testing Challenges for the Software Testers in 2016"