Evidence-based management: the knowing/doing gap

KEY POINTS HR management practices are rarely based on evidence of what works and what does not - this is referred to as the knowing/doing gap. A good example of this is the use of performance-related pay systems. The use of HR metrics can help to close the gap by providing evidence of what works, and demonstrating how decisions have an impact on measures such as employee absence, engagement and productivity.

When implementing new HR practices, organisations often disregard the facts and act instead on belief, ideology and casual benchmarking. Will the use of HR metrics change this?

HR management practices are rarely based on academic evidence about what produces good organisational outcomes and what does not, and it is unusual for practices to be evaluated for their effectiveness. Jeffrey Pfeffer, professor of organisational behaviour at Stanford Graduate School in the US, describes this as the "knowing/doing gap" and asked some years ago: "Why does so much business research and education, so many books and articles, produce so little change in what managers actually do?"

From the 1980s onwards, academics have come forward with evidence that performance pay, for example, often does not work. Alfie Kohn, the American pay guru, found that incentive pay only buys temporary compliance, while US management theorist Ed Lawler found that merit pay increases are generally too small to act as any kind of incentive. But this did not stop the introduction of performance pay throughout the 1980s and 1990s, nor does it undermine its continuing popularity today. Ian Kessler, fellow in human resource management at Templeton College, Oxford, believes that performance pay became popular for reasons of ideology, fashion, acts of faith, or because it was seen as a symbol by employers of a new relationship with their workforce. The evidence that it may not work has been largely ignored.

The Chartered Institute of Personnel and Development (CIPD) has recently looked at the knowing/doing gap. In 2005 it held a symposium with the e-reward company in an attempt to understand and bridge the gap between academic evidence about what works and what HR managers actually do in their organisation¹. CIPD assistant director general Duncan Brown identified an implementation gap where employees simply do not understand or trust decisions on pay, for example, and managers lack the skills to deliver new reward systems effectively, particularly when it comes to improving performance. An explanation for the knowing/doing gap, he said, is lack of time to look at the evidence. A raft of CIPD surveys, including its 2005 Reward management survey, have revealed a mismatch between HR managers' beliefs that they need to spend time on such activities and actually being able to do so.

When it comes to evaluation, few organisations use quantitative measures, such as reduced recruitment costs, absence, employee turnover or the number of employment tribunal cases, and the use of softer measures, such as the impact on staff motivation, to measure the effectiveness of policies is also rare. This finding emerged from unpublished research carried out by University of Greenwich for the Department of Health, which looked at seven large unionised organisations in the UK that had recently introduced new pay and grading systems, all of which had at least some stated, measurable objectives for their change. When asked by the Greenwich researchers, few of the HR managers believed there was a correlation between changes in pay systems and business performance, and most were sceptical about the value of hard measures, given problems with linkage and background noise. Some HR managers took the view that an absence of negative reaction indicated success. The researchers explained these findings by suggesting that managers have such a psychological investment in the changes that they would have had difficulty admitting their objectives had not been met.

The knowing/doing gap

Pfeffer set out his latest thinking on the knowing/doing gap at the CIPD conference at Harrogate in October 2006. He defined evidence-based management as a way of thinking and making decisions based on facts, and said that decisions in other spheres - such as medicine, education and criminology - are increasingly evidence-based. He agreed that managers have little time to read and reflect, but said there was a reluctance to face facts. He suggested that an hour or so thinking about the assumptions underlying proposed new practices was time well spent. Basing decisions about reward on appraisal, for example, assumes the following:

the manager doing the appraisal is capable of being objective;

performance can be, and is, measured;

people are motivated by money; and

the organisation performs better if the workforce performs better.

All these assumptions are questionable. Research has shown that managers give higher appraisal ratings to individuals they have hired themselves, to people they like and also to people who someone else has described as a good performer. And if a competitor brings out a better or cheaper product or service or the stock market falls, company performance is going to suffer regardless of how hard the staff work.

In many cases, it is necessary to pursue facts, for example by conducting experiments, Pfeffer suggested. He advised HR managers to treat the organisation as an unfinished prototype and said there was a need to "try stuff". So, instead of implementing a policy throughout the organisation, managers should try it out in one department and see if it works.

Can HR metrics help?

Traditional HR metrics are simply HR data such as absence, labour turnover, revenue or profit per employee or the cost of filling a vacancy. Newer HR metrics combine data such as absence rates or labour turnover with employee responses in attitude surveys to questions which measure engagement, such as "Do you feel able to give of your best at work?" or "Do you know how well (or badly) your business is doing?" This data in turn can then be linked to customer satisfaction data. The advantage of using any metric is that it enables HR to converse with senior management in the language of business. Operational decisions taken by HR can then be based on hard facts and the value added by HR becomes more visible. This could become increasingly important as more and more functions attempt to justify their status as strategic business partners rather than merely cost centres. The use of metrics also underpins the old adage that "what you can't measure, you can't manage".

The rise of the new HR metrics has made it easier to link particular HR practices with good outcomes, so the evidence should be clearer and thus more compelling as a guide to action. John Purcell, professor of HR management at the University of Bath , speaking at the CIPD's e-reward symposium, said that employee surveys allow employers to evaluate the effects of their policies themselves. He demonstrates the use of HR metrics in his own work, which studies the links between a range of HR practices and organisational loyalty, which in turn impacts on business results. His study of 12 "excellent" organisations for the CIPD found that rewards and recognition, communication and work-life balance have a positive impact on commitment to the organisation across all occupational groups, while performance-related pay does not trigger organisational commitment.

Also at the symposium, Nationwide Building Society described how it uses HR metrics to improve business results. Its Project Genome seeks to identify, characterise and map the key drivers of employee engagement (see Building human capital investment at Nationwide). The tools used by Project Genome to obtain this information are an annual employee survey, which gets a response rate of 90%, and work in measuring human capital. The project has found that committed employees stay longer, have greater experience and skills and a positive service attitude. These attributes result in higher customer satisfaction scores, higher levels of customer commitment and retention, and ever-increasing sales.

Project Genome, for example, predicts that if Nationwide increases the average length of service of employees from 10.2 to 11.2 years, then this will improve customer commitment by 1%, improving mortgage completions against target by almost 2% and bringing in an additional £5.6m in revenue. Similarly, a 1% increase in employees' perceptions of coaching leads to almost a 1% increase in personal loan sales against target, generating an additional £136,000 in revenue.

¹Reward management: symposium report, available at www.cipd.co.uk/subjects/pay/general/_rwrdmngt.htm.

Why is it hard to use evidence-based metrics?

There's too much evidence. Some 30,000 business books are in print and more are published each year. There are hundreds of English-language magazines and journals devoted to business and management issues, and there are increasing numbers of web-based outlets. There is too much information for any manager to absorb.

There's not enough good evidence. The Bain Management Tools survey is the only long-running survey that exists on the use and persistence of various management tools and techniques, but even this just measures the degree to which practices are adopted and is restricted to subjective assessments of their value¹.

The evidence doesn't quite apply. Research tends to identify specific practices that work some of the time under certain conditions. For example, stock options only work well in small, privately owned start-up companies. This can leave managers wondering what will work for them.

People are trying to mislead you. Managers are constantly enticed to believe in and implement business practices that are inappropriate for their organisation. Jeffrey Pfeffer, professor of organisational behaviour at Stanford Graduate School , believes a big part of the problem is consultants, who are "always rewarded for getting work, sometimes rewarded for doing good work and hardly ever rewarded for evaluating whether they have improved things".

This article was written by Sue Milsome, a freelance writer on employment issues, s.milsome@btopenworld.com.

¹ Management tools and trends 2005 (PDF format, 784K).

Source: Jeffrey Pfeffer, Harvard Business Review, January 2006.