There is an old business saying: “You can’t manage what you can’t measure”. We have researched methods and tools for measuring the Return On Investment (ROI) in coach training programmes from the beginning to the end of the project, and how to express that value in quantitative terms as well as qualitative ones (quantitative being tangible measures like profit or reduced staff turnover, while qualitative refers to less tangible items like staff satisfaction and trust).
It is now widely recognized that coaching is a means of increasing employee motivation, retention and engagement, and that these factors are thought to contribute to raising profits or, in the case of not-for-profit organisations, better budget management. But the question on the lips of the financial controllers remains: ‘So everyone is feeling better – but are we making more money?’
Methods of measuring ROI
The main models used to calculate ROI in organisations have been identified by The Institute of Employment Studies as:
- Kirkpatrick
- Hamblin
- OEM (Organizational Elements Model)
- Indiana University Model
- Carousel (IS Carousel of Development)
- Phillips ROI
- KPMT (Kearns and Miller)
- CIRO (Context, Input, Reaction and Outcome).
The first ROI model was published by Donald Kirkpatrick in 1959. Later models mostly draw on and add to Kirkpatrick’s, which comprises the following four levels:
LEVEL 1: REACTION
The degree to which participants find the training favourable, engaging and relevant to their jobs.
LEVEL 2: LEARNING
The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training.
LEVEL 3: BEHAVIOUR
The degree to which participants apply what they learned during training when they are back on the job.
LEVEL 4: RESULTS
The degree to which targeted outcomes occur as a result of the training and the support and accountability package.
Planning
Kirkpatrick’s process makes sense in as far as it goes, but in a coaching approach we start by looking at where we want to get to before we set off, so another layer of research would have to happen at the beginning: What are our objectives? And how do we benchmark them?
The first steps in evaluating the benefits of a coaching or coach training programme need to take place long before it starts. A training needs analysis (TNA) will reveal the state of the current situation and what changes people would like to see in place. This can be carried out through a variety of means, from simply asking people, to an extensive series of surveys and meetings. The most important aspect is to identify all the possible stakeholders, from the top of the organization to the bottom, and ensure that all of their views have been sought and taken into account.
Pre-TNA:
The key is to ask the right questions of the right people, so before any TNA is undertaken, some research into staff surveys and customer reviews will be useful, as well as interviewing a small group of key stakeholders.
Training Needs Analysis:
As many people as possible should be asked out of those affected by the area of the training – the ones who will attend, their managers, their reports and possibly their customers. This can be done through organization-wide surveys of all staff, plus focus groups and interviews with key members. The questions can be based on the GROW model (described in our article The GROW Model)
The areas, topics and specific wording will be dictated by the nature of the organization and the various results along the way:
Goal:
- What do you want in the long term? (say one to five years)
- What do you want in the short term? (say three months to one year)
Reality:
- What is working well now?
- What challenges are you facing?
Options:
- What do you think should be done?
- What changes would you like to see in place?
- What suggestions do you have?
The Will stage is not necessary here as we are looking for suggestions and options rather than making decisions about what to commit to.
Identify objectives
By studying the results of the TNA we can identify trends and will be in a good position to formulate objectives which we know will engage most people throughout the organization.
Next to each objective we could place ways of measuring the results. If that proves difficult, it probably means the objective is not clear enough, and needs to be re-worded. Is it really a final objective or just a stage which must be completed along the way?
Some of the measurements can be qualitative, but there should always be some quantitative ones included as well. All the objectives should be framed in positive statements, for example: ‘increase staff attendance’ rather than ‘reduce absenteeism’. An effective goal names the destination we want to reach, not the one we are wishing to get away from.
Monitor progress
This can be done in the short and long term and follows the Kirkpatrick model:
- Feedback forms completed by participants at the end of each module
- Final feedback on the whole course or programme
- Follow up questionnaires say three to six months later to find out how well participants feel they have assimilated the learning
- Follow up questionnaires one to five years later, to find out the long term benefits of the programme.
The last two items can include asking people who the participants work with or for, to see how the learning benefitted them.
How to get quantifiable Return on Investment results from surveys
Most organisations run regular surveys with their workforce, whether about ‘soft’ items such as satisfaction and motivation, or ‘hard’ facts like absentee rates, target-meeting and turnover, and some of these may be adapted to measure the benefits of coaching. To ensure that those benefits solely attributable to the coaching programme are identified, and not confused with benefits from other changes that may be happening, the following question should be used: ‘What difference is due solely to the coaching programme?’
The answers can elicit more specific benefits and, where appropriate, it is possible to ask questions that place a figure on them:
- How much?
- How many?
- What percentage?
Questions can also be asked to find out what proportion of the benefit is due to the coaching programme. So if it is ascertained that there is a 20% rise in profits, and people have said that is 80% because of the coaching programme, then the result would be that the coaching programme has caused profits to rise by 16%.
Various surveys can be used to measure the change in culture. One such on-line survey is Richard Barrett’s Cultural Transformation Tools because it enables workers from all areas of the business to identify the strengths and weaknesses of leadership in an organization without pointing fingers at named individuals, or providing anonymous criticism that might be swayed by personal bias.
When conducted across sufficient numbers of employees, these surveys can show up trends. For example, if large numbers complain of confusion or bullying, a survey can be designed to work out what these factors are costing the business. The participants will estimate what percentage of say, turnover, is being lost as a result of the limiting factors. Taking the previous year’s turnover, this percentage can be translated into a figure. A result like the one in the table below, although not statistically accurate, will be enough to attract the attention of those in charge of finance and budgets:
Leveraging feedback for ROI
A key contribution to measuring the ROI in coach training can be made during the programme through collecting feedback, both on paper and verbally. Feedback forms can be filled in at the end of every day’s training, providing an excellent basis for a report at the end of the programme. If the participants agree, they can be quoted by name, which carries a great deal of weight if they are senior in the organization in terms of justifying the spend and encouraging other staff to participate in future training.
The sooner feedback is given the better, and we allot ten minutes at the end of a coaching programme, or each day, to fill out the forms, while the experience is fresh in everyone’s minds. If asked to complete feedback online or by e-mail after the course, probably most of the managers will not get round to doing it once they are back in their busy lives.
Some sample questions that might appear on a feedback form are:
- What are the 3 key benefits you have gained?
- Where will you use them in your work?
- What tangible benefits have there been for you?
- What are the tangible benefits for the organization?
- What intangible benefits have there been for you?
Verbal feedback is often more descriptive than what people put on a form. It can be noted by the coach or trainer and questions can be asked to draw out the specifics, including figures where possible. The answer can then be turned into a report to be presented to the organization, with the permission of the coachees or trainees concerned. All of this will help towards evidence of the return on investment in coaching.
Leveraging research for ROI
Another effective way of convincing organisations about the value of coaching is to present research. For example, the diagram below was compiled by US company Innovest:
The research shows that, whereas an organization’s worth might be expected to be gauged through inspecting its financial records, such as profits and share price, 60–85 per cent of its true value resides in the cultural capital – intangible drivers like relationships, communication, retaining knowledge and skill, behaviour within the company and to the external community, sustainability etc.
Various organisations including the Association for Coaching and CIPD publish relevant surveys from time to time, and these can be found through internet search engines.
Case history: ROI of coach training at the NHS
Since 2015 we have provided an extensive coaching and mentoring programme for the Royal Free London and several other NHS Trusts. The NHS has a robust system to measure staff satisfaction in place and the results of this reflect the return on investment in terms of positive change which the coaching programme has brought about. The process measures qualitative and quantitative results through identifying the areas where change needs to happen, and then designing questions which will accurately represent whether, and to what extent, changes have occurred.
The questions asked of staff throughout the NHS cover a broad range of topics including equality and diversity, health and wellbeing, quality of appraisals and staff engagement. These questions and the results of the surveys are available on https://www.nhsstaffsurveys.com, where it can be seen that the Royal Free London’s scores have improved since the start of the coaching programme, as shown in the graph below:
Caroline Mabey, Deputy Director of Organisational Development & Learning, concluded:
‘The hard data along with the narratives paints a picture of outstanding progress towards a coaching culture.’
Conclusion
There is no quick fix for measuring the return on investment in coach training, but it is possible to produce significant evidence of improvement, both qualitative and quantitative, through a combination of coaching, surveys, feedback and research, summarised below:
In practice I have often found it the case that although organisations often express a desire to measure the ROI during the planning stages of their coaching programmes, the changes that occur as managers start to use their coaching skills are so much in evidence, and clearly adding value to the organization, that intentions to put a structure to measure the return on investment in place are often set aside as being unnecessary.
An example of an organisation which recognises the value of intangible drivers is Virgin, and this quote was taken from its website:
“If we look after our people well,
they will look after our customers better, and
shareholders will benefit in the long term”
About the author
International speaker, writer and broadcaster Carol Wilson is Managing Director of Culture at Work and a Fellow of the Institute of Leadership & Management, the Professional Speaking Association and the Association for Coaching, where she is a member of the Global Advisory Panel. A cross-cultural expert, she designs and delivers programmes to create coaching cultures for corporate and public sector organisations worldwide and has won awards for coaching and writing. She is the author of ‘Performance Coaching: A Complete Guide to Best Practice Coaching and Training’, now in its third edition and featuring Forewords by Sir Richard Branson and Sir John Whitmore, and ‘The Work and Life of David Grove: Clean Language and Emergent Knowledge’. She has contributed to several other books and published over 60 articles including a monthly column in Training Journal.