Testing your mobile apps
So you've tested your paper prototypes, and you've had feedback which has driven your work in a particular direction. Now that you have a working product, maybe you' ve used device emulators to do some further testing. Now, you'd like to test it with people and find out what the perceived quality of your offering is to your users. How do you do that with a mobile device?
Cut to the chase: the usual rules of user testing apply as always. The hard part is to get rigged up properly to be able to capture the users' actions. You will however be alert to a whole bunch of new potential issues...
Differences when testing on mobile devices:
- Context is much more important (indoors vs outdoors)
- Use of Location
- Device events (incoming messages, calls, network interruptions..)
- Use of Camera
- Environmental issues (glare, rain...)
- Device size (depending on the brand of device)
- Type of device (impacts the interface paradigm)
- User attention may be limited (multitasking)
- Orientation changes
- Fat fingers
Location:
Make sure you test your app in te environments that it will be used in as well. You can find most usability issues by testing in your office, and once those are fixed up, it's good to trial it on public transport. Usually, the user will lose the signal at some point, might not have earphone on them, are conscious of the other passengers, it might be noisy, they may be stood up in the aisle. Basically a whole bunch of new variables come into play. This is "stress testing your app': see if it can hold up even in the worst scenarios! You won;t be able to get as much data as in "lab tests" obviously, but it'll still be useful.
Use the actual devices you intend to release on. There's a lot of controversy around lab vs field testing for mobile apps. Some experts say that you don't really find any more issues in the field than you do in the lab, and others say the reverse. In my mind it makes sense to do some field testing, because it leads to further insights around how you can make your app really effective for people. Field testing can be much more time consuming than lab testing, so be prepared.
Things you want to capture in your tests:
- Body language
- Facial expressions
- Screen capture
- Voice capture
1. Hardware:
MOD1000 rig ($350) or you can make your own rig at a fraction of the cost.
2. Software:
Screen Recorder : Camtasia or CamStudio, Silverback or Morae.
App analytics (for when you go live):
TestFlight (iOS beta testing on the fly - Free SDK, and nice real-time analytics)
Mixpanel (also real-time analytics)
Final tips:
Don't test using Wifi, and if you are getting participants to use their own phones (which is ideal), cover their data costs.