All the tools used to collect data during a user testing session are useless if something isn’t done to turn the data into meaningful information. There are two different approaches we use, depending largely on available time and budget, but also on the nature of the product being tested and the stage of development it’s at.
This is the fastest and therefore most cost effective way to analyse the data collected. It largely requires an experienced analyst to review the video and audio a number of times using their knowledge to spot trends indicative of a problem, then use the number of people who experience that problem to work out how important it will be to fix it. It would be rare to make use of metrics, certainly not extensively, and it is not possible to calculate a Return on Investment as a result. Our entry level Rapid User Testing service uses predominantly qualitative analysis for speed and low cost.
For a more precise analysis that relies heavily on the scientific method of performing numerous metrics on the data, allowing stronger claims to be made about the findings (e.g. 3 in 5 people will experience problem X resulting in Y amount of time lost each year and a loss of revenue to the business of Z), there is quantitative analysis. It will generally take longer to collect and analyse the data in a quantitative fashion and is therefore largely not suitable for shorter testing projects, though it is ideal for the heavily customised Managed User Testing service where the metrics required can be selected to match the desired level of confidence in the results. It is common where possible to carry out a Return on Investment calculation as part of a quantitative analysis.