Successful remote user testing
One of the frequently highlighted tips for a successful user testing is to do it in person. It has a lot of advantages to do user testing face-to-face. Such as that it is easier to connect to your participants.
A personal connection reduces the stress level and makes your participant more willing to share their thoughts and opinions on the test subject. Also, you clearly can see facial expressions and reactions, which can be helpful to evaluate if what a person is verbalizing really matches their feelings. During our testing sessions we regularly run into users that say that the system was easy to use, but during the test they struggled for 3 minutes to finish a certain task. It is because humans don’t want to seem incapable and feel the user test is also about testing them.
Anyway, there are many reasons why sometimes it is not possible to set up a face-to-face testing and we need to go for a remote testing session e.g. the spread of the corona virus or simply that your target group isn’t available in your surroundings. Especially in our globalized world it is beneficial to have an alternative to F2F. Also, there are some advantages to remote testing e.g. that your participants test your product in their real-life environment including the device they would normally use and distractions they could probably face. We found that some users really struggle first time they are using our “Testing Mac”, and remote testing erases this issue completely.
But remote user testing requires some additional thoughts upfront to make it successful and generate outcomes as insightful as those of a face-to-face testing session. That’s why we collected our top 5 tips on how to set up successful remote user testing sessions.
Deciding on the right tools
To build a strong personal connection we recommend using Google Hangouts or Zoom for remote sessions. Seeing your counterpart instead of only hearing the voice helps to get on a more personal level and also allows you to see their facial reactions. Further, you can easily invite them to a meeting and explain them how they can share their screen. Anyway, be aware that not all participants are able to use it. Especially, when you have the testing during worktime some of your participant might face software restrictions. For this case, it is important to clarify upfront if this solution is suitable for everyone. If not Google Hangouts or Zoom there are plenty of other tools and your participant probably, they already has an alternative in mind.
Remote testing offers perfect solutions for documentation. We highly recommend recording both screen and audio off each session. You can use both to analyze your session in detail later on. Of course, with the written permission of the participant upfront. To record the screen, you can use QuickTime. It also allows you to record a limited part of the screen. Unfortunately, QuickTime alone is enough to record the audio and screen at the same time. But there is program to help with that: SoundFlower. It allows you to record audio via multiple channels.
Especially in remote testing it is important to have a sufficient technical set-up that allows you to
Build a strong connection with your user
Document your findings
Setting up the testing sessions
As remote participants do not visit our office in person it is easy to neglect the planning part. Still, it is super important to professionally set up the meeting. You need to follow certain steps to ensure everything is going as planned.
Set a date & time with your participant & note taker (mind different time zones)
Check with your participant if they can use the tools you are planning to use.
Get their written agreement for screen & voice recording as well as data usage. Depending on client and project also an NDA will be needed.
Send your participant a calendar invite including the link to the tool you are going to use.
If you plan the meeting more than a week ago do a friendly follow up the day before. This avoids that you participant forgets about the meeting.
Book a quite meeting room and book it a least 5 Minutes before the meeting starts and 15 minutes after the meeting is supposed to end. So you have enough time to set everything up and do not need to quite the meeting before everything is said. You will realize quickly that some participants have to say more than others.
Have a way to reach the participant via email or phone in case there are issues, or you run late.
Be careful which information you give the users upfront. Less is probably more. If the system is already live and you want to test with new users, it would be a pity if they check out the system upfront out of curiosity and already know their way around.
Get ready for taking-off!
Prepare for technical fails
Nobody is perfect. Not you, not technology. Therefore, you should always have a plan B for technical fails. Nothing is worse than coming out of a successful testing and realizing that the screen recording failed. Similar suffering is to realize that your PC does not start 1 Minute before the meeting.
One way to avoid that those issues result in 0 testing outcomes and frustrated clients is to have an alternative tool for every tool you use. For example, you can also record the audio with your phone or switch from Google Hangouts to Skype.
Having a note taker in your session can also be a lifesaver in case your PC is deciding for a 30 Minutes update shortly before a call. Plus, in case all recordings fail he has the key findings already written down.
Set the Right Atmosphere
To have the same quality as a f2f-testing one of the most important things is to create a warm atmosphere. It can be difficult to empathize with people over the web. Besides creating a digital f2f connection it is also important to take the time to thank them for their time, do some small talk upfront and get to know your counterpart a little bit better. Besides that it is especially important to let them know that everything that doesn’t work out is the systems fault – not theirs. Including this in your test on-boarding will help the participant to talk freely about their experience and thoughts during the testing.
Know when a remote testing does not make any sense
As mentioned above remote user testings can be a really good and time efficient alternative to physical testing. In some cases, however it is important to insist on face-2-face testings. This is the case if:
You need to observe the user’s environment/the testing is highly depended on a specific situation that cannot be recreated sitting in front of a laptop
You need to see how they interact with others or physical objects during the testing session
Your users are not comfortable with technology and will run into problems setting up the testing session on their side
For most products, apps and other software solutions these points are not applicable. But if you realize that those will influence the results of your sessions it makes sense to go back to good old f2f-testing and wait until the crisis is over.