When to Use Which User Experience Research Methods
In his Alertbox article entitled When to Use Which User Experience Research Methods, Christian Rohrer gives a good overview of various research methods and when to apply them.
The field of user experience, is blessed (or cursed) with a very wide range of research methods, ranging from tried-and-true methods such as lab-based usability studies to those that have been more recently developed, such as desirability studies (to measure aesthetic appeal).
You can’t use the full set of methods on every project, but most design teams benefit from combining insights from multiple research methods. The key question is what to do when. To better understand when to use which method, it is helpful to realize that they differ along 3 dimensions:
- Attitudinal vs. Behavioral
- Qualitative vs. Quantitative
- Context of Website or Product Use
The plotting of research methods according to these three dimensions is quite handy, as is his advice regarding combining multiple methods for any particular project.
Of particular interest is the “Context of Product Use” dimension. This dimension refers to whether or not you’re asking the participant to use the product in a certain way, as opposed to just observing how they might naturally use it. This can have a big impact on the method you would employ, and highly scripted approaches risk Asking Participants to “Pretend” in User Studies (which Jared Spool recently wrote about).
While this article should be quite useful in helping people to narrow down the most appropriate design research methods to use, it might not be enough. Inexperienced readers may still not be able to decide which method to use, even assessing all possible methods using these dimensions. What’s missing is the X factor regarding which factors are best for certain situations, but this probably comes down purely to experience and is a professional preference that develops over time (and would thus be difficult to capture in an article). All in all, a very useful article.
[Chris, if it's not on the blog, it's just not on :)]
Oh *the* blog, yea got it!
I thought this article was interesting, but fell short at the Phases of Product Development section and the final conclusions. The three phases of product development - strategize, optimize, assess - is overly simplistic, and lacks a) the notion of iteration/cycles; b) a clearly-defined ‘design’ phase; and c) clear indications of integration/interaction with other forms of research.
It felt like the author had some clear ideas around the three dimensions for evaluating when to use various research methods (Attitude/Behaviour; Qual/Quant; Context) but was scratching around for a way to actually *select* the methods and went with the relatively flat Phases model. They would have been much better off exploring in more depth the research questions, and assigning research methods to each; and then using the three dimensions to explore *why* it was appropriate, and the type of analysis involved.
Finally, and without wanting to tip this back into a methodology discussion, a don’t believe you can address the question of *which* research method to use without also discussing the question of deliverable (i.e. what’s appropriate), or flow from question to question (and, therefore, method to method).
Steve
Good points Steve, I felt the same way but was not able to articulate it quite as well as you (ie “X factor”). It was obvious after doing my own thinking around how to choose research methods that the decision is highly contextual and more complex than looking at a 2×2 chart (or 2×2x2!).
Is it a matter of not locking down the methods to be used for a project until a good initial understanding has been gained? I have done this in the past, undertaking some initial requirements gathering and needs analysis before even starting on the project proper. After that you can firstly decide whether to go ahead with the project, but also what further research should be undertaken. I usually refer to this as “successive levels of detail” to explain the fact that you might not know exactly what the best research methods will be up-front, but you do need to start somewhere, so you start at a high level and delve down.