In Search of Museum Audience Outcome Measures

This multi-institution, international research study builds on previous work by both the ongoing Museum Futures project and work of the ILI. The goal of this project is to, over the next 2 years, and in collaboration with 44 diverse museums from around the world, conduct research that will inform the development of a set of short-term audience outcome metrics that can be used as Key Performance Indicators (KPIs); KPIs that validly and reliably predict a museum’s long-term audience impact.

The vast majority of museums, i.e., somewhere in excess of 90%, do not currently utilize any kind of KPIs, and in fact, most have no process in place for assessing the impact they have on their users. Of the small percentage of museums that do collect KPI-type of data, most use indicators of audience impact that are short-term and almost exclusively output-focused, e.g., numbers of on-site and online visitors, visitor demographics, membership renewal rates, and % of return visitors. Even when museums do attempt to assess outcomes, most utilize indices that do not measure outcomes in an in-depth, valid manner, instead employing superficial measures of consumer satisfaction, visit dwell times, or critics’ reviews. There are a handful of institutions that have attempted to introduce serious outcome measures; measures purported to connect to a museum’s long-term impact goals for the public. However, even amongst these few, forward-thinking institutions, the measures in use typically lack rigor, and arguably validity. None of the measures currently in use, at least none that we are aware of, have been directly tested to determine whether the short-term measures used directly and significantly correlate with the long-term outcomes they purport to measure. This project is predicated on the belief that we can and need to do better.

To move toward this vision, towards a set of valid and reliable outcome KPIs, we need to begin by appreciating some first principles:

• Any measure of impact needs to be based on user’s sense of value, not provider’s sense of value. Only users can decide whether, or not, they perceive that an experience was of true value to them.
• Despite the singular focus of most museums on intellectual/learning outcomes, broadly defined, the public is actually attracted to museums for multiple reasons. The vast majority of museum users report experiencing a wide range of benefits that include but typically transcend intellectual/learning-related outcomes. This means that to be valid and reflect the needs of real users, museum outcome metrics need to reflect the full suite of outcomes/impacts museums support.
• Despite long-standing arguments about what the benefits are of museum use, considerable research has now been done in this area and suggests there is significant convergence on just a handful of known and readily definable and measurable outcomes. In addition, not only can all of these outcomes be defined and measured, as demonstrated by ILI’s recent research (see Falk, 2021; Finland Final Report and FINAL Measuring the Public Value of Art Museums TECHNICAL REPORT2.1), all such outcomes directly relate to one or some combination of four basic dimensions of human well-being; the desire for enhanced personal, intellectual, social, and physical well-being.
• The real value of museum experiences is not just the enhancement of short-term well-being but rather the enhancement of long-term well-being. Plus, importantly, longer-term benefits are perceived as more valuable than short-term benefits – both qualitatively and quantitatively. This reality makes the development and use of short-term measures complicated. Although it is obviously much easier to collect data immediately, while users are still at the museum, research has shown that the perceived benefits of museum experiences can and often do change over time. Thus, the task of assessing whether people have achieved desired outcomes is anything but straightforward.

In summary, then, the key challenge that this research will attempt to achieve is to develop a set of easily collectible, short-term outcome measures, e.g., KPIs, that both accurately capture the full array of user-perceived well-being-related benefits museums support and validly and reliably predicts the longer term well-being-related benefits museums aspire to achieve.

Project Team: John H. Falk Ph.D., Lynn Dierking, Ph.D.