When conducting mobile web usability testing (with a standard setup) you need your web host, internet, local network and test device to work as they should. But technology fails, and people fail. So how do you build a set-up that won't fail? (For under £100!)

1. How to build a failsafe
mobile usability testing set-up
Fabien Marry - @AlphabUX
1

2. I started my career working in TV…

3. Such control rooms have a large number of devices
working in a chain, any of which can fail at any moment.
Yet the live show must go on, so they have endless
amounts of contingency plans to immediately adapt to
most equipment failure.
The lesson I took away: technology fails.
This is the norm, not the exception. This is not an
excuse for your service to be interrupted.

4. Later on in my career, I was in charge of supervising
teams of qualified audiovisual professionals with much
more experience than me at some high-level events
involving heads of state.

5. Many times, these professionals let me (and my employer) down in a big way.
One VIP speech wasn’t recorded because they forgot to turn the volume back up after
turning in down to avoid hearing the noise of delegates settling in their chair before the
session. Another key 20 min speech recording was missing a part because the tape had
run out and they didn’t anticipate it.
Just like technology, people will fail. You need to anticipate that too.

6. These experiences taught me it’s unlikely that absolutely everything will
work as intended. So now, I always have a plan B (and often a plan C, and D).

7. + + +
Web host Internet Local Network Test device
When conducting mobile web usability testing (with a standard setup) you need all
four of these elements to work as they should.
Here two elements are completely out of my control: the web host and the location’s
internet connection. If they go down, I can’t do anything about it, and it’s game over.
This could leave me with confused participants, annoyed clients that wasted their
time, an expensive testing facility bill, and a project plan to redo.
We can do better!

8. + +
Your Mac Local Network Test device
You can use your own laptop as the web server (via the OS built in
servers or an application like Mixture).
By doing this, internet and web host issues can be fully prevented.

9. Now, let’s think about another part of the set-up. How do we
record the action?
Some use a document camera, requiring the devices to be kept
flat on their back on a table.

10. But this is not how people use tablet or mobile devices. They are meant to be
hand held.
Putting them on a table changes how they are used:
- The fingers used will be different, potentially hiding ergonomic issues
- The device will be positioned further away from the eyes than it would normally
be held, preventing you from testing legibility properly.

11. Test
Device Reflector Your Mac
Local
Network
Silverback
Observation
TV
+ + +
So here’s my initial setup: testing on an iOS device, using the Airplay
mirroring feature to send the screen content to a Mac running
@ReflectorApp.
The content of that Mac screen and the face of the participant
(captured via a webcam), are then recorded using @SilverbackApp,
and mirrored to the TV in the observation room via a cable.

12. Test
Device Reflector
Local
Network
Your Mac
Silverback
Observation
TV
+
+ +
Hue HD
camera
When recording hand gestures is important, as it often is for apps, I add another webcam.* The
picture from this is displayed on the Mac using Quicktime player (choose “File’ / ‘New Screen
Recording’, but don’t actually press record). Silverback grabs everything, including the
additional webcam, and displays it on the TV in the observation room.
* I use a @HueHD camera as it’s flexible neck is easy to attach to the participant’s chair.

13. Test
Device Reflector Your Mac
Local
Network
Silverback
Observation
TV
+ + +
Skype
Remote
client
When I can’t run a cable to the observation room TV, I use Skype
to share the screen and audio with a device in the observation
room. You can then connect your device (I use my iPad) to the
large observation TV / sound system.
With a group chat (sadly unsupported if one of the devices is an
iPad), this setup even allow one or several clients to observe the
sessions remotely.
However, this requires the internet, which we don’t control (see
slide 7), so I always flag that this as a nice bonus rather that
something to rely on.

14. So this is what the recordings look like (clockwise from top left):
- A view of the participant’s hand gestures (via the over-the-shoulder webcam)
- A perfect view of the screen content, unrestricted by the hand or the participant moving the
device away from the camera’s view (via Airplay mirroring to Reflection.app)
- The participant’s face (via the laptop’s built-in webcam)
Extra tips:
- Use a flat colour desktop with no visible icons. This will reduce the output video file size, and
help the viewer focus on the action.
- When recording a mobile phone screen, you don’t need the laptop full resolution, so I set my
laptop’s resolution to something low like 800x600, greatly reducing the space taken by the
recording files.

15. 15
But remember Sod’s law…

16. Local
Network
Test
Device Your Mac Silverback
Observation
TV
You are still relying on the local network. If it fails, a fall back option is
to connect the test device straight to the observation room TV via a
VGA or HDMI display adapter.
Of course, this means that your recordings won’t have the screen
mirroring, but you can somewhat compensate for this with an over-the-shoulder
webcam.

17. Another option: use your test iPhone’s
4G connection and the personal
hotspot feature to connect it to the
laptop.
You can even use a direct usb
connection for stronger interference
resilience.

18. Test
Device Reflector Your Mac
Your own
network
Silverback
Observation
TV
+ + +
So here you have it in full!
Reflector is £13, Silverback is £70. Assuming you already own a laptop
and your test mobile device, this set-up is basically a sub £100 full
mobile usability lab. And it fits in a backpack.

19. Fabien Marry - Senior UX consultant
!
fabien.marry@cxpartners.co.uk

20. Credits
!
Photos
Slide 2: http://tvmedia.hpage.co.in/course_37574641.html
Slide 3: Jérémy Dupeu, http://www.camera-forum.fr/index.php?/topic/8376-captation-multicamera/
Slide 4: The World Bank / Fabien Marry, http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/ORGANIZATION/
EXTPRESIDENT/EXTPASTPRESIDENTS/EXTOFFICEPRESIDENT/0,,contentMDK:20872180~menuPK:51408951~pagePK:
51174171~piPK:64258873~theSitePK:1014541,00.html
Slide 5: http://lolsnaps.com/news/27654/0/
Slide 6 plan A
Slide 9: Harry Brignull, http://www.90percentofeverything.com/2011/12/14/anatomy-of-a-hardware-usability-testing-rig/
Slide 10: huffingtonpost.com
Slide 15: Readee, http://s287.photobucket.com/user/Kailexanra/media/SodsLaw.jpg.html
!
Icons from the Noun Project
Wireless & iPad by Edward Boatman
Laptop by Olivier Guin
Cloud by Dmitry Baranovskiy
Television by Andy Fuchs
Bluetooth by Muharrem Senyıl