Measuring moments that matter

14 Mar
March 14, 2013

There’s an old management mantra that states “What gets measured gets managed.”  This is also akin to the saying “If you can’t measure it, you can’t manage it.”  Either way, it’s clear that having some form of metrics, measures or KPIs can be a good thing.

When he was at Princeton, Einstein had a sign hanging over his office which read, “Not everything that counts can be counted, and not everything that can be counted counts.”

As Einstein suggested, it’s possible to measure the wrong things and I believe that within the People Profession we’re in danger of measuring the unnecessary.

Story

I speak from painful experience in this area.  For many years I worked within an organisation where (in hindsight, which is always easy to do), we were focusing on and measuring the wrong things!

When I joined the organisation we were trying to raise the profile of the training department (as that’s what they were called in those days) because we wanted people to take us more seriously in order to gain a greater slice of the limited corporate resources.

One of the measures that were bandied about the industry in those days was the ‘average number of training days per employee’.  A ‘leader’ in this field at the time was IBM who provided (from memory) an average of eight training days per year per employee.  The company I was working with felt that this was the ‘Holy Grail’ of training measures, and unless we delivered eight training days per employee per year then we were doomed to failure.  And clearly eight had to be the magic number because, after all, IBM was hugely successful and we thought this was because of all the training it provided to its employees.   We thought that if only we could provide as much training as IBM did then we too would be a hugely successful organisation.  This all sounds rather strange now but trust me, this was the key issue many years ago!

And yet, as we now know, this logic just does not hold true.  There are many reasons why organisations may prosper (or not) and providing a set number of training days per employee may not be such a great measure after all.  Perhaps the level of training provided could be a measure of success, but it could also be a measure of how complex the products were or how dim the employees were (unlikely) or indeed for a whole host of other reasons.  And for those reasons it’s of course a totally meaningless measure but we focused on this as well as measuring the amount of money spent on training per employee.  We were naïve; but we were happy!

Stand back for a moment and think – what the heck is a training day anyway?  Is it seven hours or eight hours?  Do two half day sessions really add up to a day?  What if someone’s done some e-learning; should that be measured in elapsed time or should we measure it some other way because we ‘feel’ e-learning is a more efficient way of training?

I used to think that these measures were the most important things – but no longer.  Now I realise they are not proving a real measure of learning and development effectiveness – certainly not when taken on their own.

Business context

Within the People Profession we have to get away from measuring ‘things’ and ‘perceived costs’ and start measuring value delivered to the organisation.  We have to begin measuring the ‘right moments’.

Unfortunately this is not likely to happen any time soon.  The Chartered Institute of Personnel and Development (CIPD) publish an annual Learning and Talent Development survey which in 2011 stated that “The median annual training budget per employee is £350.  The median number of days (training) employees receive per year is 5.0.  This represents an increase of one day compared with last year.”  This is totally meaningless and it’s a real shame that the CIPD, as a major professional organisation, focuses on these moments.  And to be brutally honest, as long as the CIPD continues to report these moments then others will follow, assuming that they are adopting ‘best practice.’

And shock, horror – the CIPD are not alone!  The BBC claims that via its Academy it delivers over 57,000 days of training each year to both BBC staff and the wider broadcasting industry.  Given that the BBC employs about 23,000 staff and assuming (for the purposes of this exercise) that it only trains BBC staff then this represents less than 2.5 days per employee per year.  That’s 50% less than the CIPD median – and we’re talking about the most respected broadcaster on the planet.  See what I mean? A totally meaningless moment!

I urge you to measure the moments that actually have impact on your business.  Never mind how big your budget is; the question always has to be: how much value are you adding?  Industry thought leader Jay Cross agrees.  He says that “the best proof is the link between your learning initiatives and your business results.”

Metrics and KPIs are great for monitoring your business, but what would you do if your metrics were 50% off the mark today?  Would you be able to explain in a rational manner why this is a problem for your business?  Would you know what this poor performance is costing you?  And would you know where to start looking for possible causes?  Would you know what to do in order to address a performance shortfall?

And what would you do if people began to manipulate your metrics and KPIs for their own reward?  Would you have a set of ‘wobble board’ metrics so that as one metric changed for the better others would show a negative impact to indicate they were being manipulated?  No?  Well, perhaps you should think about this.

Unfortunately, as soon as KPIs are introduced as a measure of reward they will be abused.  This is known as a perverse incentive, which is an incentive that has an unintended and undesirable result, contrary to the interests of the incentive makers.  Perverse incentives have dogged KPIs for centuries, for example:

  • In Hanoi, under French colonial rule, a programme paying people a bounty for each rat pelt handed in was intended to exterminate rats.  Instead, it led to the farming of rats for profit.
  • 19th century palaeontologists travelling to China used to pay peasants for each fragment of dinosaur bone that they found.  They later discovered that the peasants dug up the bones and then smashed them into many pieces to maximise their payments, greatly reducing their scientific value.
  • In 1696, the English Parliament adopted a tax under which dwellings were to be assessed according to the number of windows they had.  Although the tax was intended to be progressive in that it exempted houses with fewer than ten windows from the bulk of the assessment, in operation it exacerbated the gap in living conditions between rich and poor as landlords were incentivised to brick up tenement windows to reduce their tax liability, leaving working class tenants with insufficient light and ventilation.
  • Within the NHS, hospitals are able to claim money for each test, treatment and referral.  Surprise, surprise; lots of treatments are provided but there’s no focus on the overall health benefits!  Sadly people with medically unexplained abdominal symptoms are three times more likely to have their gallbladder removed, twice as likely to have their appendix or uterus removed and 50% more likely to have back surgery compared to a matched control group.  They are also significantly more likely to commit suicide after surgery.

Although KPIs can be manipulated in such a way as to have a negative impact on your business, they can also add real and lasting value.  KPIs can add a focus to your business and allow you to see the results of the work you’re doing.  They can also be a good way of demonstrating benefit to others.  But there is another way . . .

What about adopting “The love making approach”?

You know the story. You’ve just made love with your partner and one of you mutters those immortal words, “Darling, how was it for you?”  That’s right – simple and to the point.  After all, (hopefully) when you’ve made love you don’t get out a four-point Likert scale to assess the food, ambiance or pre-course reading!

You may think this is a little mad, but bear with me.  All too often we try and manipulate the benefit of learning and development within business; we try and put numbers to things that are notoriously difficult to measure and we try and sell success to our managers when often there is little more than a tenuous link.  Of course, if you can clearly measure benefits that the people function has delivered then do so, but bear in mind that asking the “How was it for you?” question can sometimes be far more powerful.

Conclusions

Asking senior managers this question immediately involves them in the results.  Why should you have to tell them how learning and development has made their lives easier? If they can’t see it for themselves then it could be argued it hasn’t worked! When you get the reply, listen and act upon it.  Rarely is anything perfect the first time and with the people function it can be a lifetime’s work to achieve deep changes within a business, so learn from what you’ve done, improve it and move on.

Call to action

Rather than spending hours developing meaningless – and potentially useless – metrics, why not simply ask your business “How was it for you?”

And finally

Today (14th March) would have been Einstein’s birthday.  There’s certainly much that he could teach us all about measuring the right things.

Tags: , , , , ,
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *