Meaningful Measurement and Measuring Meaning
The philosophical theory of empiricism states that all knowledge about the world is derived from sensory experiences. The development of empiricism comes from the larger question of how do we come to have knowledge in general, or what is known as epistemology. Empiricism emphasizes that all evidence needs to be derived through scientific exploration, often in the form of experiments. This led to the development of the scientific method. For anyone who has taken a research methods class, the scientific method still is emphasized as the primary way to conduct empirical research. Only in this way can we compare our observations of the external world against what is demonstrated through empirical (and presumably unbiased) investigation.
There are other schools of thought that don’t buy into empiricism as the only approach to understanding the world and our experiences in it. Nevertheless, the belief that quantification is the most valid and reliable evidence remains in research circles. Such an orientation leads to what we might call the cult of the quantitative, in which all experience needs to be turned into a numerical representation in order for it to be reliable and useful. But is this the best approach? Do we need to quantify everything to make it more reliable? Or do we just believe it is more reliable simply because it is numeric? Perhaps we need to use different approaches and rethink how we envision research and representation.
When I am teaching and doing workshops on Experience Ethnography, students and attendees often talk about how there is resistance to using qualitative methods and data because it is too ‘subjective’ and requires too much ‘interpretation.’ This kind of view loses sight of the fact that quantitative methods also require interpretation of results in order to render what they mean. The transformation of experiences and attitudes into numerical representations creates many levels of abstraction. The idea that qualitative is more interpretive and subjective just doesn’t hold up when you consider how experiences are simplified and turned into a single number.
Additionally, should subjectivity matter and be viewed negatively? Does leaving out subjectivity improve our understanding of meaning? Sociologist Max Weber spoke of verstehen, a German word for ‘understanding.’ In a social science sense, the goal is to understand “the interpretation of action in terms of its subjective meaning.” Before you can understand what an action can mean, you have to have some understanding of the meaning behind as ascribed by those who committed the action. Only in this way can we come to understand people’s lived worlds and their experiences.
Greg Kihlström’s new book “Meaningful Measurement of the Customer Experience” explores the methods currently being used by companies to measure experience, along with potential shortcomings that can be encountered relying on them. In it, he explores such questions as, “Does the data we passively collect and actively seek really give us a better understanding of the worlds in which our customers live, or do we construct a false sense of security around what it tells us?” With CX undergoing greater growth and visibility, it is a good time to revisit the methods we use to capture those experiences.
Toward this end, the book "provides a framework to create a system of meaningful measurement that can not only improve the customer experience by holistically looking at the customer journey, but provide a sustainable method for analysis and optimization with a more agile approach to continuous improvement.”
In talking with Greg on Experience by Design podcast, we explore the ways in which we conceptualize and try to measure experience. One of the points explored is how do we help create a space in which it is important to use qualitative and quantitative data. Beyond the data, it is important to have a qualitative mindset. A qualitative mindset involves:
Exploring outliers
Allowing the opportunity for exploration
Being open for ‘noticings’ to drive curiosity
Identify bias and perception inherent in all data analysis
Embrace multi-methodological approaches driven by discovering what is there, versus trying to prove what we want
By taking this more open-ended and open-minded approach, we also can see more of the ways in which experiences can be integrated and connected. It also allows us to explore new meanings of customer (and other) experiences that we didn’t see or know before. Ultimately, it can help us to de-silo experience design from beyond particular perspectives and allow for more inclusive opportunities to delight.
Experience by Design Episode with Greg Kihlstrom - https://www.experiencexdesign.com/episodes/meaningful-experience-measurment-greg-kihlstrom