Learning Design and Evaluating Learning Return on Investment

Learning Design and Return On Investment by Alec McPhedran

Evaluating the Return on Investment for Learning Interventions
 
Proving a return of investment for learning is quite a common challenge for those who manage a learning and development budget. Quite reasonably, the question posed is what do we get back for our investment? Here, learning specialist Alec McPhedran outlines a useful approach to developing learning while thinking about longer term evaluation. 
 
The evaluation of learning starts right at the beginning of any learning initiative. What needs to change and how will you measure success when you get there? The ROI of Learning model is a tool to help people think about planning the learning and how they can evaluate its progress at various stages.
 
Outcome
It’s useful to determine what you are looking to improve, change or introduce as a starting point. Sometimes it is helpful to state what the ideal outcome will be. For example, on a project, to reduce absenteeism levels in an organisation. it might have an outcome to improve the line managers approach to managing and reducing absence levels.’
 
Aim
Once you know what it is you are trying to do you can then set a clear learning aim or objective for the intervention. SMART based goals are a useful approach. In continuing with the absence challenge, a programme of learning aimed initially at line managers might have an aim to provide managers with the knowledge and skills necessary for managing and reducing absence levels within their team to support the company improvement target of reducing absence level averages per employee from 5.5 days to 3 days for 2020.
 
Performance Objectives
In developing a learning programme, it is incredibly valuable to consider developing performance objectives. Performance objectives describe the behaviours you would like to see, hear or feel happening following the learning. For example, a performance objective could be to consistently carry out a detailed and supportive WARM (Welcome back, Absence, Responsibility and Move on) return to work interview thoroughly following every absence with their team members. Essentially the performance objective is what they will do following the learning.
 
Learning Objectives
In putting together the learning, what knowledge will they need to carry out the activity? This helps to think about the learning objectives. For any learning intervention we need to consider what they need to know (new knowledge) and what they need to be able to do (new skills). Continuing with our absence programme one learning objective might be to understand the stages and application of the WARM return to work interview model.
 
Development Interventions
Once we know out aim, performance objectives and learning objectives we can then consider the most appropriate development options. This could include online learning, YouTube clips, classroom based learning, reading, coaching, smart phone apps and so on. When we start selecting and organising our development options, we can now consider how we will start measuring learning during and after the programme. An example of a modular learning programme on absence management for line managers could be pre-work through self-reflection, two online video clips, an article to read and a highlighted section of the ACAS website to read through before attending a one day workshop.
 
Evaluation
There are many ways to evaluate learning but a useful model as part of this is the Kirkpatrick Phillips Model of Evaluation. As a tool to prompt learning design thinking relating to measuring impact and success, it is one method that creates useful reminders of areas to evaluate in learning.
 
If we continue our absence management programme, the following ideas could be considered using the Kirkpatrick Phillips model:
 
Level One – Reaction
We need to identify how we can check how well the learning is being received with activities that check the reaction to the learning content. Ideas include:

• A short pre-online learning questionnaire
• A smart phone app for comments, questions or polls
• Self-reflection and recording of thoughts and challenges with absence management
• Activities during the workshop to test understanding of theories and concepts such as role plays, questionnaires, games, group and individual feedback activities
• Evaluation sheet or online evaluation at the end of the workshop to gauge initial thoughts and reflections

 
Level Two – Learning
When identifying candidates for a learning programme, sometimes it is useful to ask delegates before attending to identify their learning needs and behaviour needs. This helps the individual, sometimes with their line manager, identify what they need to learn and how this can both be supported and reviewed following the programme. In some cases, this could be linked to their personal development objectives or CPD plans. Reviewing learning could include many of the activities similar to Level One Reaction along with discussions with the line manager or post programme tests or short reports.
 
Level Three – Behaviour
Evaluating behaviour can take quite a while after a learning event. Essentially it could mean behaviour change and this is not instant. There are a number of theories around how long it takes to change one behaviour ranging from 21 days to 55 days and in some cases, up to 255 days. So, evaluating behaviour needs planning that will take place soma time after the learning event. An example might be a one to one with the line manager some three or four months later. For example, how are your WARM interviews going? What have you changed and what has happened since? Sometimes, with behaviour questions, it is down to observations, feedback from others or discussions looking for competence-based responses.
 
Level Four – Results
When looking at the one to one discussions or recording of behaviour improvement for Level Three, this presents a great opportunity to identify areas that have improved within the individuals actions with their performance. This contributes to Level Four Results evaluation. Linking this to absence management, a manager might acknowledge that since the training, in the past six months, average days absence in their team of 15 has fallen from 6.2 days last year to 3.1 days so far this year following the return to work interviews approach was introduced. This is a great result likely linked to the learning intervention. Of course, one of the challenges in evaluating the ROI from learning is how can you definitely claim this as a direct success of the learning? However, the more examples from individuals that can be collated, the stronger the claim can be that the event has made a difference for the better. The results on behaviour improvement can then be checked back to the initial programme aim or objective.
 
Level Five – Return on Investment
If we collate as much of the qualitative and quantitative data from a selection of candidates from the learning event, we can build up quite a good case for demonstrating both business improvement and a return on investment from the initial budget for the learning event. With some solid thinking, we can often turn those improvements into financial figures. From the previous example we can identify that over six months, with a team of 15 people, we have an average of saving 3.1 days salary x 15 people. That is quite a good return on investment, especially if we revisit it over a twelve month period. If we collect a sample range of between 20% to 30% or more from course delegates some three to six months later, we can gain a total guide on potential return on investment. Looking for figures such as increased sales, reduced wastage, quicker response or down time, etc it can nearly always be monetised. The total can then be used to reflect on achieving the initial need for the event based on the outcome identified. The value of the ROI figure is that this helps justify value in investing in learning If a training programme for 40 managers, at a cost of £32,000 in total, can be seen to save the business up to £197,000 in twelve months through reduced absenteeism, then why would we not want to invest in more development.
 
Of course, the above is simplifying it but none the less, the role of the person responsible for learning events is to demonstrate the value and positive business impact of their services. Identifying the return on investment from learning should be a regular part learning provision.
 
Alec McPhedran Chtd Fellow CIPD, Chtd Mngr CMI, MCMI  is a recognised creative sector coach and mentor. He specialises in one to one talent coaching, facilitated learning and team development. For further information, visit www.mcphedran.co.uk.

Copyright © Alec McPhedran 2024