I’ve long been a fan of developing clear measures of success or metrics in planning. Too often, though, boards and staff at historic sites only use total attendance or the financial bottom line to judge their success. Certainly, having no visitors is not a good sign, but is a large number of visitors a mark of success? Not necessarily, because high attendance may be due many factors, including some that may have nothing to do with advancing your mission in a significant manner, such as weddings rentals or dog walkers or corporate retreats. I’m not knocking those activities and they may be an essential part of your programming, however, what I’ve most often heard at board meetings are conversations like this:
Board chair: I heard we did well last month. What was our attendance?
Director: We had 2,500 visitors in March, double what we had in February.
Boardmembers (in unison): Wow, that’s great!
Director: And looking at the guestbook, we had people from 14 different states and 3 foreign countries, including Latvia.
Board chair: This must be a record for us. Okay, let’s have the financial report–looks like the bottom line is positive. Is there a motion to accept?
Don’t assume this only happens at the local historic house museum–it happens at the big ones as well. As I’ve often said, attendance and assets shouldn’t be the only measures of success and they aren’t the most reliable, yet we’re tempted to rely on them over and over again. If you’re just looking to increase attendance, think about offering a puppy circus.
I’ve discussed measures of success at historic sites at length elsewhere, but I’m currently reading The Lean Start-Up by Eric Ries and he put into words what has been rolling around in my mind for years, namely that metrics only have value if they demonstrate clear cause and effect. For example, if your attendance goes up, can you clearly point to the cause? And if you can, what can you learn from it to improve future performance? Typically, all we can say is that we had a special event that brought many visitors, but we can’t do much more. Did they come because of the weather, a newspaper article, the admission price, the event theme, a Facebook campaign, or the Latvian Tourist Bureau? That’s much less clear, so all we have is an attendance number that doesn’t provide any more direction than “have more events” (a sure way to burn out an organization). He calls these utterly useless and distracting measures of success, “vanity metrics“:
Vanity metrics wreak havoc because they prey on a weakness of the human mind. In my experience, when the numbers go up, people think the improvement was caused by their actions, by whatever they were working on at the time. That is why it’s so common to have a meeting in which marketing thinks the numbers went up because of a new PR or marketing effort and engineering thinks the better numbers are the result of the new features it added. Finding out what is actually going on is extremely costly, and so most managers simply move on, doing the best they can to form their own judgment on the basis of their experience and the collective intelligence in the room. . . .When cause and effect is clearly understood, people are better able to learn from their actions.
So think about your measures of success–do they clearly show cause and effect? Do your metrics give you clear guidance for decisions and actions? If not, they’re probably just skin-deep. Although they may seen beautiful and attractive, you’ll want to rethink them.