We need to have a conversation on why and how to measure the ROI of your programmes
It’s the number one question asked at budget time: What difference has your learning and development strategy, or your most recent choice of training programme, made to workplace performance?
The ROI pressure is on…
Did you measure the impact on your organisation beyond collecting the ‘happy sheets’ from newly enthused teams? Furthermore, did you measure the return on investment for that training and its impact?
If not, you’re not alone. According to the DDI Global Leadership 2018 report, only 18% of organizations measure the ROI of their L&D programmes.
You may feel that measuring the cost-effectiveness of training choices is out of reach, but here’s why organisations do it, and how to go about it.
Organisations may choose to measure different areas, but generally, ROI calculations of training programmes aim to work out:
- Are trainees gaining new knowledge and skills so that they can increase efficiency and/or reduce costs in the workplace?
- Can we measure the training cost against the benefits to the individuals and the organisation?
It seems cold when we know for sure that good training can transform how people approach their work, but the bottom line is that the Board or C-suite need numbers. They’re less interested in how exciting or uplifting the training programme is. They want to know how the training they’ve invested in is affecting workplace performance and, ultimately, they want to see tangible returns from that impact.
But measuring ROI in training - beyond the questionnaires and the typical post-programme mood uplift - is complex.
Organisations need to create systems and metrics that work. That can be a tough call when the clearest benefits are intangible, or when even the ‘tangibles’ require acres of cross-functional data to show a meaningful shift in business performance.
According to LinkedIn’s 2017 Workplace Learning Report (showing insights from 500 learning pros from the US and Canada), 92% of CEOs don’t see the business impact of L&D programmes, and only 4% see the ROI.
We suspect this number is related to the 82% of L&D departments that struggle to measure the ROI of learning, its business impact and the long-term productivity gains of their programmes.
Reality check: ROI measurement is necessary to justify future investment
Measuring ROI may be difficult, but it’s necessary if you want to secure future budget (and what L&D pro doesn’t want to expand their programmes?)
Although not every type of training requires an ROI analysis, being able to measure the ROI of larger or longer-term training initiatives gives leaders confidence in L&D choices and helps organisations optimise future training budget spending. If the training programme you commit to has a clear business connection, it will stand the test of time.
Of course, most L&D managers - by the very nature of their career choice - view training as intrinsically valuable. They might not always think in terms of ROI at the outset.
Even though clear metrics can help trainers and talent development professionals justify their budgets, it’s not always the first thought when scouring for inspiring programmes that will motivate teams to perform better in their roles.
Let’s get started: what are we measuring?
It will help throughout this article to think about three sets of objectives: learning objectives, specific application objectives and impact objectives.
Ask yourself: what’s missing from your current practice, how would employees benefit from getting better at it, how will they apply it in their roles and what impact will that have on individual and business performance? Being able to answer this will help identify your ROI.
Early metrics might include:
• qualitative feedback from employees about the training
• general engagement surveys after specific time periods
• qualitative feedback about behavioural changes
• increase in the number and range of skills developed
More monetary measures might include team or business metrics like deals closed, customer satisfaction and repeat business. Other typical measures are time saved, productivity increases, retention, promotions and increased innovation.
Why does ROI feel like a minefield?
Here are some common stumbling blocks to measuring the ROI of learning and development programmes. The good news is, they can be overcome...
- Unclear training objectives and unclear training needs
- Unclear desired outcomes e.g. desired impact on participants AND the colleagues, clients or projects affected by their work
- The ROI metrics used are not aligned with business goals and therefore less credible to C-suite
- Lack of expertise: from knowing what to measure to designing surveys, error margins and drawing conclusions, this is specialised work. It takes time AND money: you need the right expertise for a trustworthy result
- Lack of meaningful data. When L&D teams are asked to provide analytics on linking performance issues with the training programme, they often cite lack of existing cross-functional data. This might be sales data, customer satisfaction levels, productivity and output or a detailed breakdown of training costs
- Funders may perceive even the most obvious intangible benefits (vital performance factors like well-being, engagement and motivation) as less important than tangible (monetary) outcomes
- Confusing time metrics: even tangible monetary equations need strictly defined timeframes for any meaningful calculation. It will look different for different teams e.g. sales vs. product managers
- Time: cross-functional data analysis takes a lot of time out of an executive’s day
- The ROI methodology is not well enough understood by those who need to understand it.
Who can help? Three common ROI models…
Let’s look at three of the most popular ROI methodologies: the Kirkpatrick Model, the Phillips Model and the ROI Institute’s ROI Methodology®.
The Kirkpatrick Model of Training Evaluation
The de-facto model of training evaluation in the 1970s and 1980s, this simple, practical approach was developed in 1959 (since updated twice) by Donald Kirkpatrick and is based on four measurements:
Level 1: Reaction. The degree to which participants find the training favourable, engaging, and relevant to their jobs. The feedback here enables improvements to future training programmes.
Level 2: Learning. The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training.
Level 3: Behaviour. The degree to which they apply what they learned when they’re back on the job. In this stage, changes in morale, motivation, or employee engagement can be measured.
Level 4: Results. The degree to which targeted outcomes occur. This might include ‘impact’ KPIs and metrics for company goals, such as increased profits, reduced turnaround time and costs, customer and employee retention, accreditation, and innovation.
(Level 4 measurements are critical to the ability to calculate actual Return on Investment.)
The Phillips Model of Training Evaluation
Jack Phillips (author of the 1997 book ‘Return on Investment in Training and Performance Improvement Programs’) expanded the Kirkpatrick model to include considerations for actual return on investment (ROI). The Phillips model includes a fifth level focused specifically on ROI.
Using various calculations, such as a cost-benefit analysis, level 5 looks at the link between training and business results e.g. sales generated compared to the cost of the training. It maps impact data to tangible monetary benefits and a set of intangible benefits.
The ROI Institute
The ROI Institute’s ROI Methodology® is now the most used evaluation system, favoured by over half of America’s Fortune 500 companies and many governments, nonprofits, and public institutions throughout the US and 70 countries. Over 6,000 organisations now use its 12-step, six-area-process (covering Why?, How?, What?, How Much?, What’s it Worth? and So What?).
Like the Phillips model (Jack Phillips is chairman of the ROI Institute), the ROI Institute categorises evaluation data into five levels, and has four process stages to ensure the appropriate data is collected from the proper sources at the right time.
As shown below, the process consists of sequential, logical steps that lead to data categorised by five levels of outcomes.
This diagram, the ROI Methodology® is taken from the ROI Institute website, and you can find a more detailed version of it on page 5 of the Insights Indeed ROI Impact Report
Insights and the ROI Institute
In 2023 Insights concluded a study into the ROI of our flagship psychometric tool, Insights Discovery.
We chose to retain the ROI Institute for its dedicated team of top minds in HR, Leadership Coaching, Talent Management and Human Capital Analytics.
The Insights/Indeed ROI Report published in late 2023 evaluates the application, impact and ROI of the Insights Discovery learning journey at the American employment technology company, Indeed.
Initiated by Insights, the study sought to demonstrate the programme’s impact on team effectiveness and ROI more credibly and thoroughly. Indeed sought to understand the value the Insights Discovery programme has delivered to their organisation. At the time of print, our longstanding partnership since 2014 had generated over 15,278 personal Insights Discovery profiles at Indeed.
The study’s remarkable conclusions showed not only that the respondents at Indeed reacted positively to Insights Discovery and successfully applied what they learned, but that Indeed experienced significant impact as a result, citing $10m in benefits and an ROI of 2063% (for every dollar invested, that dollar was returned plus an additional $20.63).
Important intangible benefits were delivered as well as clear tangibles, including reduction in days to proficiency for new employees, time savings, reduction in time to promotion and improved innovation and agility.
The ‘intangible’ benefits of our long-term collaboration - as we would expect with a programme focused on improving self-awareness and awareness about others - included increased emotional intelligence, communication, collaboration, and teamwork. These results are of course every bit as important as the ‘tangibles’ and are the foundation for great workplace performance!
Final thoughts on ROI measurement
For L&D pros reading this article, we don’t need to tell you that the last few years in the industry haven’t been easy ones. From layoffs to burnout, anyone who works in the people function have had their hands full, and as economic uncertainty continues to cloud the future, it’s possible that L&D programmes will continue to be questioned.
While ROI measurement is complicated, it’s a necessary part of proving the impact that our industry has on organisations’ bottom lines.
This is why we choose to measure the ROI of Insights Discovery, and it’s why we encourage all L&D professionals to think about how they might measure the ROI of the good work they do within their own organisations.
