Skyscanner tests new hotels mobile app at local coffee shop. Senior UX Designer Chris Roy reports.
Taking a break from the more structured usability testing that we normally run, Skyscanner recently took on a whole new approach to usability testing for a prototype of our upcoming hotels mobile app.
Rather than formally recruit candidates for testing, we took to our social media accounts to rally together some Skyscanner fans – people whom we know have already established an interest in our products and who at some point may have used one or more of them. There was of course an incentive involved, and with the guerrilla testing taking place in a coffee shop, we were offering to pick up the tab on any drinks that our participants wanted.
Preparation
Admittedly, the whole thing was rather last minute. We had been given the all clear to use local ‘artisan coffee bar’ Brew Lab on the Friday, working to set things up for the following Thursday. Reaching out to our Social and Graphics team, we managed to pull together a creative for our mini campaign which was then sent out on the Wednesday.
Post-launch of our mini campaign I sat nervously monitoring the mailbox to see if we would either a) get enough response to fill every coffee shop in Edinburgh or b) get nothing. Fortunately it turned out to be something in between, and over the course of 24 hours we had around 20 local fans get in touch to take us up on the offer, of which eight were chosen to take part in the test.
On the day
The testing itself was fairly inexpensive in terms of equipment. To conduct each session, the participants were asked to hold the phone in front of a laptop which would record their actions and comments through the built in webcam and microphone. Often referred to as the hugging technique, this approach allows the users to hold and use the device more naturally. Just make sure that the phone remains visible in front of the camera!
Luckily, I had managed to position myself in a fairly quiet corner of the coffee shop, which was well equipped with plug sockets and seating space. In reviewing some of the videos, I might have more carefully considered the overhead lighting, as this sometimes created a glare on the mobile screen.
The test
Having worked on developing an interactive prototype using Axure, the mobile app (which was actually a web page behaving like an app) offered enough detail to demonstrate the key interactions and features that had been considered. Though limited in content and often struggling to imitate native app performance, the users were able to conduct various tasks which involved searching for: reviewing, sorting and storing hotels using the app.
Having the app built to this degree of fidelity meant that participants could get a truer sense of what it would feel like to use. Prior to this stage, there had been lots of sketches and high level interaction models which were tested and refined internally. Exposing end users to lower fidelity concepts is likely to yield less valuable feedback as it can be hard to truly visualise how things are supposed to work. For participants, it’s easier to show them how it would work rather than trying to explain "Well, this part would do this, and then that would move there…"
Feedback
Still a work in progress, I was keen to hold this testing ahead of any formal development of the new product as this would highlight any flaws in the app or any missed opportunities without the costs/time associated with finding these things out post-launch. As hoped, a lot was learned from the testing and some key issues were raised as a result. This has now given us some food for thought to review and revise some of the interactions and user interface (UI).
As an aside to the testing, it has also allowed us to connect with our local users like Doug Mitchell (pictured below), some of which were unaware that we were actually headquartered in their home town, The enthusiasm for the Skyscanner brand was very refreshing and I would very much like to continue building these relationships with the people who are using the products which we build.
Summary
All in all, the day could be described as a great success and I would like to thank the Social Media and Design team for their last minute help in publicizing the event. Also, huge thanks to Brew Lab for allowing us the use of their coffee shop for the event and to all of the participants on the day for their feedback. If anyone has any other approaches to testing, or would like to comment on this article, please drop us a note below.
Edinburgh, Europe, Scotland, United Kingdom, Product, user testing
