Why does Vidéotron use Optimal Workshop's Reframer for its user tests?
We can never repeat it enough, user tests are essential to avoid errors and to improve your product. In addition to allowing to take a step back from your project, testing will save you time and money by creating a better experience for your users.
Vidéotron has understood this, as user research has been part of their daily routine for almost 3 years now.
Through interviews, card sorting and user tests on products, services, customer paths, sites, applications and even paper instructions, the value of research has proven itself in the company.
Requests quickly flooded in, to the point where it became difficult to satisfy them all.
The need for a user search assistance tool
The main problem was the lack of time because until then, all the note-taking and analysis was done by hand in Excel.
"It was working but it wasn't going fast enough, we had to find a way to increase efficiency without lowering the quality of analysis..." - UX Researcher at Vidéotron
Vidéotron quickly needed to turn to tools that could help them optimise their research process. After having looked at what was available on the market, they chose Optimal Workshop for the diversity of its products, its reasonable price and the quality of the data analysis.
Optimal Workshop is a platform for UX Researchers and Designers who want to improve their products through user research.
It currently offers 5 tools to support UX researchers in the research and testing phases:
OptimalSort for card sorting
Treejack to test tree structures
Chalkmark to identify the first clicks on interfaces
Questions for online surveys
Reframer to take notes and analyze user tests or interviews
I had the chance to accompany Vidéotron in their first uses of Reframer for online user testing. So let me share my feedback and opinion on this tool which in my opinion will change the way we analyse tests.
(The projects I worked on being confidential, the pictures presented here are just examples)
Simple and quick note-taking during the tests
The first step is the preparation before the tests.
As for a test protocol, you will be able to prepare the key elements: the objective of the study, the participants and their criteria, the link of the prototype, the tasks to be performed...
It is important to note that unlike platforms such as User Testing or Lookback, Reframer only covers note-taking and analysis. It is therefore necessary to complete its use with other tools (in my case MS Teams and Figma to record the test sessions and the prototype).
For note taking, there is no need to go from Excel cell to Excel cell. You type your observation, you press the Enter key, you start again... Nothing could be easier!
By adding the upstream tasks (they are found on the right during the note-taking), you are able to easily and quickly note the behaviours and reactions of the participant at key moments of the research and thus facilitate the analysis.
Recommendation: Use verbatims to ensure that you do not bias the participant's thoughts and actions by involving your own interpretations. These verbatims can also be very useful when presenting to business executives. Keep your leading quotes to illustrate what users are saying!
Reframer's strength, analysis
The importance of properly tagging your observations
The real strength of Reframer is the analysis. But to do it best, you need to tag the participants' observations first. The better tagged they are, the easier it will be to identify and organize the results later on.
By tagging each participant's observation, you will end up with a complete database ready to be analysed.
Problem identification and priorisation
This is where Reframer makes the difference and where the tagging work makes sense.
Thanks to the filters that include participants, tasks and tags, you can cross-reference the data to identify recurring problems.
You can then quickly prioritise them by looking at the number of participants who encountered the problem and the impact it had on their experience, which significantly facilitates the writing of the test report.
Sample test report result:
"3 out of 4 participants did not understand the included channels. 2 would have left the site at that point. "
In a nutshell
Reframer's strengths
Easy to learn – people without prior experience of test observation were able to help me in note-taking
Centralisation of information.
Facilitated collaboration, especially thanks to simultaneous note-taking.
Rapid analysis.
Re-use of tags for subsequent studies, and therefore time saving for the preparation and analysis of future tests.
An available and responsive support team. A big thanks to Paddy, Customer Success Manager at Optimal Workshop, who took the time to explain the tool and was open to all our remarks to improve it even more.
The weak points
We can't blame Reframer, but there is no magic tool today that can manage the recordings, the prototype and the analyses at the same time. So I had to complement its use with other tools like Figma and Teams.
The auto-save feature which - as you would expect - should save after each action is error prone. Quite frequently we end up with unsaved or duplicated tag entries. This is a problem that we have communicated to the support team.
Conclusion
After only 3 studies (and about 28 participants), I was able to learn quicky to use the tool and to save time in my research process. The analysis phase was simpler and more efficient, allowing to quickly build reports and to share them with all the project teams.
For Vidéotron, the difference was tangible: we went from an analysis that could sometimes take more than a week to an analysis completed within 1 to 2 days.
Optimal Workshop is not often mentioned as a platform to use for user research!
To make up for this, we contacted them to let you try the tool and see for yourself how Reframer can improve your research phase...
50% discount on a 3 month subscription : "ATECNAVIP"
Valid until June 30, 2021