MEASURING UP
Many companies still struggle to use metrics effectively. It may be that fresh thinking is what really counts.
By Scott Leibs

Call it the Martinez factor.

Pedro Martinez – one of the highest-paid players in American baseball – was an ace pitcher for the Boston Red Sox who had a habit of choking at the decisive moment in key games. In the summer of 2003, he famously squandered his team’s lead, and ultimately locked the team out of the US World Series.

Martinez’s late-game collapse against archrival New York Yankees underscored what even casual fans knew. Whenever he had thrown close to 100 pitches during a game, his effectiveness declined markedly. Plotted on a line graph, it was plain to see.

But baseball managers don’t manage by line graphs. The Red Sox lost the game and several weeks later the team’s manager lost his job.

These events unfolded even as Michael Lewis’s book Moneyball, which described how a new breed of baseball visionaries used statistical analysis to assemble competitive teams on modest budgets, was proving to be not only a best seller, but also a business touchstone. Today, gurus on the business conference lecture circuit point to Lewis’s book and ask: If statistical analysis can level the playing field in baseball – why not in business?

Some companies are doing just that, extending metrics into areas that would seem to defy measurement. How, for example, do you measure innovation, or the future capabilities of your workforce, or the optimum menu of health benefits? It’s not easy, but some experts say the time is ripe for companies to move away from a managerial “feel for the game” in favor of more-rigorous, data-driven decision making.

It’s hardly a new idea. Harvard Business School, which will celebrate its centennial next year, took as the model for its initial curriculum the works of Frederick Winslow Taylor, the efficiency guru whose quantification of manual labor gave birth to what became known as “scientific management”.

A century’s worth of MBAs would seem sufficient to propagate a by-the-numbers approach into every nook and cranny of the business world, but it hasn’t worked out that way. By most accounts, companies have done a respectable job of mastering financial metrics, but have largely taken a flier on measurements of operations or intangibles such as customer satisfaction or brand loyalty. Fifteen years ago the advent of the “balanced scorecard” sought to redress this imbalance by demonstrating how nonfinancial metrics could be captured and used to help managers “see their company more clearly – from many perspectives – and make wiser long-term decisions,” according to its creators, Robert Kaplan and David Norton. But despite the popularity of that approach at a strategic level, many consultants and academics say it left thorny questions unaddressed at more tactical levels.

“Metrics are a powerful communications tool,” says Michael Hammer, the reengineering pioneer who recently described the “seven deadly sins of performance measurement” in an article of that name in the Sloan Management Review. One value of metrics, he says, is to provide real-time assurance that long-term improvements are on track. “Even a small company that has embarked on any kind of major process improvement faces a long haul,” he says. “But with the right metrics in place, you can measure results right away, which becomes a powerful driver.”

That was the case at Wells Fargo, which relies on what it calls a “happy-to-grumpy” ratio to assess whether its efforts to develop a more “engaged” workforce are on track. “We don’t want to just measure results,” says CFO Howard Atkins, “we want to measure what drives our results, and that includes team-member engagement. That measure might not get cited in your general ledger, but it can be quantified in a statistically valid way, compared over time to certain goals, and correlated to business outcomes.”

Atkins says that groups within the bank that have higher employee-engagement scores also rank higher on productivity and customer satisfaction. James Harter, chief scientist for the Gallup Organization’s workplace-management and well-being practice, says that while it can be complicated to connect employee attitudes to financial performance, it can be done. Gallup administers a 12-question survey on behalf of its clients (including Wells Fargo) that assesses, on a one-to-five scale, how employees feel about everything from their role at work to their co-workers’ commitment to quality.

“We focus only on things that affect performance,” Harter says. “Otherwise, managers become overloaded with too much information.” Wells Fargo incorporates its happy-to-grumpy ratio into a broader universe of measures that help it develop its workforce in a way that enhances corporate performance. Some critics argue that strong company performance may be what makes employees feel engaged, rather than vice versa. Harter agrees that there is some “reciprocal feedback” but says that employee engagement is more often predictive of financial performance than the reverse, in part because it predicts customer-service quality, employee turnover, and other outcomes that drive the bottom line.

Making connections

As companies extend metrics into various aspects of operations, they are often eager to discover links between them. What use is an improved customer-satisfaction score, after all, if there is no indication that it drove more sales? This attempt to relate one or more metrics to others is often labeled “business analytics” or “data-driven decision-making”. Tom Davenport, co-author of Competing on Analytics (Harvard Business School Press, 2007), says that “like metrics, analytics is not new, but in both cases what is new is basing your strategy on them.”

Davenport points to Hilton Hotels, which found that a 5% boost in customer-retention rates led to a 1.1% improvement in revenue in the following year. That, he says, is the emerging frontier: finding causal links between financial and nonfinancial metrics.

For example, retail giant Best Buy has been measuring employee engagement for a decade and customer satisfaction for three years. Says Joe Kalkman, vice president of HR capabilities: “We are just in the first year of understanding how these two metrics relate to one another.” Does having a more engaged workforce improve customer satisfaction, and does that in turn boost sales? If so, a company might reassess its investments in employees, confident that a dollar spent at one end will produce higher returns at the other.

Best Buy has already connected improved employee-engagement scores to store performance: it found that for every tenth of a point it boosted the former, its stores saw a US$100,000 increase in operating income. By looking for links between employee engagement and customer satisfaction, Best Buy hopes to “fill out a systemic view of what makes our entire business model work,” says Kalkman. Given the complexity of business today, he says, “single metrics are less and less effective.”

Run it like a business

At Pitney Bowes, the mailstream technology and services company, metrics played an important role in redesigning health benefits. The company has collected “de-identified” data on employee health claims, absenteeism, visits to on-site health clinics, and related measures for more than a decade, and several years ago decided that studying the patterns those various metrics provide might lead to insights about how to reshape its coverages to combat rising premiums.

Realizing that chronic diseases such as diabetes accounted for a huge percentage of health claims, the company redesigned its benefits so that preventive care is either free or very low cost. It even invented a sort of metrics system for employees, dubbed “Count Your Way to Health”, that provides targets ranging from 0 (that is, no smoking) to 100 (use a seat belt 100% of the time), with stops along the way for 1 (flossing every day) to 25 (maximum percentage of body mass attributable to fat) to 30 (minutes of exercise per day). “Health benefits tend to be consensus-based, or driven by your competitive position,” says Jack Mahoney, director of strategic health initiatives at Pitney Bowes. “We thought, why not run it like an internal business, by using metrics?”

While success stories sound simple enough – gather data, analyze it, act on the revelations – by most accounts companies botch metrics more often than they get them right. “I’m continually astounded at how poor the use of metrics continues to be,” says Brent Wortman, a senior partner in the financial-management practice at Deloitte. He cites as common problems the inclination to roll numbers up from the bottom, so the board and senior management are overwhelmed by irrelevant numbers; the lack of standards for linking operational and financial metrics; and the failure to put enough rigor behind hard-to-measure attributes, such as customer services, opting instead for either poorly conceived metrics or none at all.

Hammer agrees wholeheartedly. Among his “seven deadly sins” are vanity (emphasizing metrics that cast you in a good light), provincialism (allowing departments to measure themselves on their own narrowly defined goals, often at the expense of the organization as a whole), and “inanity”, which he defines as a failure to appreciate the consequences of a given metric. As an example, he cites a fast-food chain that focused on reducing the amount of wasted food; to improve that metric, restaurant managers stopped cooking in advance of anticipated demand. Waste was reduced, but service was delayed and sales suffered.

“Companies have not brought to bear a rigorous, analytical mindset about what they measure, how, and why,” Hammer says. “Nor do they regularly review what metrics they track and discard those that are outmoded.” Now would be the time to redress that situation, he says. “Why are metrics such a big deal in 2007? Because foreign competition, shareholders, customers, and other forces are collectively putting companies under more performance pressure than they’ve ever faced, and to be smart about performance you have to be smart about metrics.”

Scott Leibs is a deputy editor of CFO in the us.