Fighting Android Fragmentation

There are thousands of Android devices on the market running 18 different versions of the Android OS. It’s impossible to test every combination of OS version, hardware, and screen size in house — but you have to face the reality that your app will run on all those con­fig­u­ra­tions once it goes live. And you will run into unforeseen problems.

How do you manage the frag­men­ta­tion problem and minimize the risk of problems when it goes live? I’ll give you a hint: it doesn’t need to be expensive or complicated if you use the right process.

Start With a Few Devices

Let’s assume you have a small budget to purchase devices for in-house testing. This is a common scenario for a freelancer or a small app dev shop. How do you decide which devices to buy? Here’s a couple tips that have worked well at Hudl and allowed us to maintain a high app rating in Google Play:

1. Buy one device from each of the Big 4 Manufacturers.

Hardware and software imple­men­ta­tions vary greatly between man­u­fac­tur­ers, but they will often be similar across a single manufacturer’s different phone models. For example, a bug that reproduces on theSamsung Galaxy S4 might also reproduce on the Galaxy Note 2. It’s a good idea to test your app on at least one device made by each of the Big 4 Manufacturers: Samsung, Motorola, HTC, and LG.

2. Buy at least one low-end device.

Manufacturers usually sell cheaper devices with lower hardware specs in addition to their flagship models. Test your app on at least one device with lower hardware specs to help discover issues related to processor speed and RAM usage (two resources that are often limited on mobile devices). Your low-end device should also run the lowest version of Android that you support. At Hudl, we use a Motorola Droid Milestone X running Android 2.3 with 512 MB of RAM and a 1 GHZ processor for this purpose.

3. Go used to save money.

Once you decide which devices to purchase, you don’t need to buy new. Most of our test devices were purchased used on eBay. You can save extra money by buying devices with bad ESN numbers. A bad ESN number simply means that you cannot activate the device on most major carriers. You can still use WIFI for your internet connection, which is all you need for development. These devices usually cost about 30% of their list price.

Let Your Users Be Your Guide

Once your app is live, a great way to expand your device base is to purchase the devices that your users are actually using. Google Play provides some basic stats on device usage for your apps, but other products provide much better information.

Event and Failure Based Analytics

At Hudl, we log device information with every web service call. We then use Splunk to analyze this log data and drill down by device type. For example, this allows us to see which devices and man­u­fac­tur­ers the video player is failing on most frequently. This type of information should be the the driving force behind what devices you purchase to test on.

There are also several free alter­na­tives to Splunk, which are great for independent app developers. My personal favorite is Flurry. It is an analytics service specif­i­cal­ly geared towards mobile apps, and it provides the same event based analysis that Splunk does.

Smash Bugs with Crash Reporters

Another essential tool for finding bugs is a crash reporter. Crash reporters are hosted services which store records each each time your app crashes due to an uncaught exception (commonly known as a ​“Force Close” in Android). They store the stack trace and device information, then aggregate issues to find the most common crashes.

If you find a crash that is reproducing frequently on a specific device or man­u­fac­tur­er, purchase that device and try to reproduce the issue in house. If you’re not using a crash reporter yet, start — they are indis­pens­able tools. Once again, Google Play provides basic crash reporting services, but I have found third party services like HockeyApp and BugSense to be much better.

Conclusion

Supporting a wide variety of Android devices can be a daunting task, but it’s manageable with the right process. Purchasing a reasonable selection of devices to test in-house and using the right tools to find problems in the wild will help you build a rock-solid app. Let me know how you manage Android frag­men­ta­tion in the comments below.