Are KPIs appropriate to the public service?

An article in the 25 October 2011 edition of The Age (see Ombudsman blasts child protection) cites a “scathing” recent Victorian Ombudsman’s report as saying that managers in Victoria's child protection system failed to investigate reports of children at risk so they could meet numerical targets.

Acting Victorian Ombudsman John Taylor said his investigations into the Department of Human Services child protection program in the Loddon Mallee region revealed serious and significant failures to provide safety and wellbeing to the state's most vulnerable children.

In his report tabled in the Victorian Parliament on 25 October Taylor said the failures were caused by deliberate policy decisions of certain managers to reduce the number of child protection reports investigated:

I consider the failure to investigate these reports to be a consequence of an intentional policy decision by the Bendigo office of the Loddon Mallee Region of the department to reduce the number of child protection reports that it investigates.

Despite receiving more reports in 2010-11 than the previous year, the region conducted less than three quarters of the number of investigations.

I believe a practice has developed where the drive to meet numerical targets has overshadowed the interest of children despite evidence that they may be at risk.

The managers deny this, however, I consider that the evidence speaks for itself.

It is not the first time – indeed, the article says that it is the fourth Ombudsman's report in two years raising serious concerns about the performance of Victoria's child protection system. Almost two years ago we saw Child protection workers fudging figures: report in the 26 November 2009 edition of The Age, which begins:

A scathing Ombudsman's report has identified gross deficiencies in Victoria's child protection service, with workers manipulating figures to cover up children neglected by the system.

This report was never released to the public but the then Brumby Government’s Community Services Minister Lisa Neville said that each of the Ombudsman’s 42 recommendations would be implemented. If they were, not much seems to have changed.

At a superficial level reports like this raise the question of what is wrong with the relevant agency, but at a deeper, more fundamental level they raise the question of whether the importation into public service of business practices like giving agencies and individual officers numerical targets (“Key Performance Indicators” – KPIs) is appropriate to the public policy and service delivery purposes for which public sector agencies are created.

Key Performance Indicators are a product of the business maxim, “What gets measured gets done”, which in itself is a warning to those responsible for leading our public sector agencies, for the corollary is that what doesn’t get measured doesn’t get done. KPIs tell staff in no uncertain terms that when it comes to the crunch, certain things matter and nothing else does.  Messages like that are barely appropriate in business. They have no place in the public sector.

KPIs work best when the required results are simple and easy to specify. It is easy to set KPIs for the distributor of a standardised manufactured retail item like, say, soft drinks or breakfast cereal, which might specify for example that the goods must be delivered to a retailer within a specified time of the order being placed, that the consignments must be accurately made up, that there must be no breakages or spillages, and perhaps that the quantity or value of goods sold in a particular distribution zone should increase by a certain percentage over the previous year.

The Soviet Union discovered a long time ago that even in as standardised a process as the manufacture of everyday items, the specification of required outputs requires great care. If you tell a manager to produce two tonnes of nails you will get two tonnes of 6 inch nails because they are easier to produce.  If you tell them to produce 10,000 nails you will get 10,000 panel pins, because they will require less metal.

When it comes to the delivery of complex services, these Victorian Ombudsman’s reports scream out the message that KPIs break down completely. They are simply inappropriate to measuring the performance of complex tasks.  What should be a high resolution colour photograph of a complex job, with all its nuances and subtleties, gets reduced to six or eight data points.

KPIs are also an open invitation to game the system.  Tell a manager that what is required for performance pay or for promotion is to meet certain numerical targets and they will be met, by a combination of concentrating on the easier cases and fudging the figures. There is money and position at stake, and people will duck, weave, dissemble and tell outright lies in order to establish that they have met their KPIs.  “Follow the money trail” is as reliable a guide to human behaviour here as elsewhere.

The result is that in an environment infected by KPIs managers at every level will have less knowledge of what is really going on than they otherwise would, and that in itself compromises organisational effectiveness.

To understand what is required to manage and measure performance in a large hierarchical organisation it is necessary to recognise three key principles.

The first is that a “manager” is by definition a person who is accountable for the performance of others. In the course of delivering on that accountability managers at every level are responsible for advising people of their duties and giving them the authority to act, ensuring that they receive adequate training to carry out their responsibilities, coaching them in the finer points, deciding what needs to be done and what doesn’t, allocating tasks to people who are equipped to carry them out, and allocating the necessary resources.

Recognise that central hierarchical accountability point and you are well on the way to understanding how a public service department in a Westminster system (and indeed any well run large organisation) is meant to work. The Department Secretary is accountable to the Minister for the performance of the Department as a whole. In turn, the Division Heads are accountable to the Secretary for the performance of their Divisions, the Branch Heads are accountable to the Division Heads for the performance of their Branches and so on. In a very real sense only the individuals at each level and their immediate supervisors have a full understanding of what the work in their local workplace is: what the current and emerging issues are, the workload pressures, how easy or difficult the people with whom they must interact are, and the priorities determined in the light of guidance from above.

There is not space to develop this line of thinking fully here, but what emerges from it is that the only KPI that has any meaning is that individuals at each level, whether they be managers or front-line staff, perform to the satisfaction of their immediate supervisor.  Supervisors being by definition responsible and accountable for the work of everyone below them, it matters not to people further up the line how the work gets done or who does it; what matters is whether it gets done, and to what quality.

The second is that performance at any level – the performance of managers and the performance of individuals – can only reasonably be measured in the light of the resources made available, a fact which is notoriously lost from sight when meeting KPIs is the order of the day.  It is not under-performance for an individual to be unable to carry out a task, or a series of tasks, or a volume of tasks, for which they have not been given the necessary resources, the necessary guidance or the necessary training. The November 2009 Ombudsman’s Report blamed lack of resources for poor service quality and said the failures it reported were not a reflection on the staff.  Child Safety Commissioner Bernie Geary said that the report reflected an overwhelmed department where beleaguered workers were often vilified for shortfalls in the system.  No doubt they are both correct. Governments are notorious for under-resourcing functions that they see as necessary but politically unglamorous, and then blaming the hapless staff and/or senior management when things go wrong.

The third essential principle is that senior people in any large organisation are paid to make judgements, and the performance of staff is quintessentially a matter for judgement, not something that can be reduced to a series of numbers that would enable a chimpanzee to assess performance.  Priorities change, resources that were meant to be available are delayed or unavailable, things beyond the control of the individual go wrong, other matters intervene.  Only someone in a position to observe closely – the immediate supervisor – can make a fully informed judgement about the performance of a subordinate unit or an individual, and making those judgements is a key part of their job.

Consequent upon this, another key responsibility of organisational leadership is to speak truth to power – to advise people up the command chain what resources are required in order to carry out the unit’s functions to an acceptable standard, and not allow themselves to be fobbed off with bromides about having to learn “to do more with less”.  Everyone should be striving to improve their personal effectiveness and that of their organisations, but there are limits to what any one human being can do.

So I say it is time to throw the whole silly business of public sector KPIs overboard. Statistics can inform management decisions but should not be allowed to substitute for them.  It is time to get back to the holding our politicians accountable for providing their agencies with the resources they need to carry out what is expected of them, for appointing mangers who have the skills and the authority to lead the organisation and ensure that it delivers, and holding those authorised managers accountable for their performance.

In conclusion it is worth noting that, aside from the business of giving everyone Stakhanovite KPIs, I fear we have learned another bad habit from the deservedly defunct Soviet Union – the selective enforcement of requirements, to the disadvantage of those who are out of favour.  When the Soviet Union began to become sensitive to international criticism of its nasty habit of locking up dissidents, it for the most part ceased locking them up for their dissident behaviour but pinged them for the sorts of economic sins that were endemic, part of making life tolerable in the centrally planned economy – crimes like acquiring a bit of foreign currency to buy small consumer items like jeans and Beatles records, or paying someone to give you their place on the waiting list for a fridge, a television set or a car. There were no political prisoners anymore, only people who had violated the foreign currency or other economic regulations.

KPIs are a marvellous vehicle for selective enforcement, a marvellous cover for favouritism or covert discrimination.  It is so easy to be understanding about all the extenuating circumstances that explain why the chosen ones didn’t meet their KPIs, but the advancement of those who are not favoured is easily explained – “they didn’t meet their KPIs”.

If you disagree with the above analysis, ask yourself whether you think that a quarter of a century of managerialism in our public sector has improved the quality of policy making or service delivery.  I do not think so.

Note:  A slightly shorter version of this piece was posted on the ABC’s The Drum on 3 November 2011 (see here).