Measuring Mentoring Impact Measuring Mentoring Impact
How to show what a mentoring program changed, and for whom.
What this presentation covers
What this presentation covers
What this presentation covers
Agree on results first
Agree on the results before launch
Ask sponsors which change would show them the program worked.
- Retention who stays a year on
- Promotion who moves up
- Time to productivity new starters
- Engagement survey scores
Kirkpatrick’s four levels
Measure at four levels, not one
Kirkpatrick’s model runs from reaction to learning, behaviour and results.
- Reaction was it useful?
- Learning what did they gain?
- Behaviour what do they do now?
- Results what changed for the organisation?
Surveys before and after
Survey before the first meeting
A baseline shows change, not only a final score.
- Before the first meeting goals and confidence
- Mid-point is the pair working?
- At the end the same questions again
- Six months later did it last?
Use a comparison group
Compare with people not yet mentored
In one trial, mentored pupils did better at school than a control group.
Follow up after the end
Look again after the program ends
In that trial, the academic gains did not last into a second year.
- End of year one better than the control group
- Year two the gains were gone
Mentoring and retention
Link mentoring to who stays
Among Army officers, mentoring went with less turnover, tracked ten years later.
Pay and promotion effects
Expect small effects on pay
A meta-analysis found career benefits, with small effects on objective outcomes.
- Promotions rate, mentored vs not
- Pay salary growth, mentored vs not
- Career satisfaction survey score, before vs after
What to ask at check-ins
Ask the same questions at each check-in
MENTOR asks programs to use one standard procedure at every contact.
- What the pair did
- Progress for the mentee
- Quality of the relationship
- Impact on both people
- A record of each meeting
Reporting to sponsors
Report on one page, in their terms
Lead with the agreed results, then the evidence and its limits.
- Goal agreed with the sponsor
- Reached pairs and meetings
- Change before vs after
- Compared with people not mentored
- Limits what the data cannot show
Sources
The research behind this presentation
- Kirkpatrick, D. L. & Kirkpatrick, J. D. (2006) Evaluating Training Programs: The Four Levels, Third Edition Berrett-Koehler Publishers, San Francisco
- Herrera, C., Grossman, J. B., Kauh, T. J. & McMaken, J. (2011) Mentoring in schools: An impact study of Big Brothers Big Sisters school-based mentoring Child Development, 82(1), 346-361
- Payne, S. C. & Huffman, A. H. (2005) A longitudinal examination of the influence of mentoring on organizational commitment and turnover Academy of Management Journal, 48(1), 158-168
- Allen, T. D., Eby, L. T., Poteet, M. L., Lentz, E. & Lima, L. (2004) Career benefits associated with mentoring for protégés: A meta-analysis Journal of Applied Psychology, 89(1), 127-136
- MENTOR: The National Mentoring Partnership (2015) Elements of Effective Practice for Mentoring, Fourth Edition MENTOR, Boston
Decide what counts first
Then measure it before, during and after the program.