Beyond the survey
How evaluation insights and reporting create lasting impact
Why Evaluation Matters
Like anything in life, it’s not just the experience that matters… it’s what we learn from it. In facilitation and leadership development, the real value lies in how we assess and evolve our work. That’s why evaluation (done insightfully) is not just a checkbox. It’s a powerful tool for improving impact and demonstrating success.
As a passionate learning designer, facilitator, and leadership development professional, with decades of experience in the public service, I’ve developed deep skills in evaluation. I have expertise in measuring whether an intervention delivers on its intended inputs, outputs, outcomes, and longer-term impact. This isn’t just technical; it’s essential for knowing whether the work we do truly makes a difference. I invest in this because evaluation significantly enhances how effective any facilitated event can be.
Common Pitfalls: Quick Fixes and Shallow Insights
In today’s world, we have many tools at our disposal to evaluate the success of what we do… from word clouds to easy online surveys that offer instantaneous feedback. These digital solutions have real value when used to capture surface-level indicators. They are relatively inexpensive, fast, and easy to administer.
However, they rarely capture innovative insights.
Why? Because they often rely on character-limited free-text boxes or pre-set multiple-choice options. These formats limit the opportunity for participants to share unexpected or deeply personal feedback. The kind of insights that can only emerge when people are free to express what really impacted them. Without this data, you can't take meaningful action or uncover what you didn't even think to ask.
At my most cynical, I sometimes wonder if such evaluations are designed more to create the appearance of listening, rather than a genuine process for incorporating participant feedback. When surveys lack free-text options, I question whether feedback is truly valued… or whether the system to act on it even exists.
How to Capture Data That Enables Understanding
Whenever I design or redesign a learning experience, I always evaluate both quantitative (Likert scale) data points and qualitative free-text feedback.
Here’s what I’ve learned:
Likert scale metrics are useful in two main ways:
Identifying trends across multiple interventions.
Illustrating a group’s general growth against learning objectives (e.g., pre- and post-workshop confidence).
These stories are helpful for assessing quality and for reporting back to stakeholders or delegates who commissioned the work. But to truly understand what was working for or not working for a participant, you need their own words. Free-text feedback gives voice to ideas and experiences that I may never have anticipated.
The richness of lived experience cannot be fully captured by pre-populated surveys or character-limited boxes. This is why free-text answers are essential for genuine evaluation.
The Evaluator’s Mindset: Human Understanding
Effective evaluation is about more than just collecting data. It’s about appreciating the human understanding behind the numbers. This includes:
Data cleansing
Codifying data (so that it can be meaningfully analyzed)
Analyzing data to ask: What story does this tell?
These are the essential steps for drawing insights that matter. When these processes are handed off to shallow technological tools, we lose the nuance, connections, and human stories that drive improvement.
If we care about the performance of the products and interventions we deliver, we should care enough to invest energy into thoughtful evaluation.
Challenges and How to Address Them
I often hear, “James, it’s hard enough to get people to fill out evaluations—let alone write free-text comments!”
In my experience, when participants trust that I genuinely care about their input, and see that I will use it, they respond. Consistently.
My evaluation response rates are 95–100%, and free-text response rates range from 25–45%—even higher when I take time to explain why their feedback matters.
The takeaway? Without free-text questions, you can only test your assumptions. You can’t get new data or new perspectives. And without new perspectives, you can’t uncover blind spots.
Evaluation as Respect
Thoughtful evaluation is more than good practice. It’s a sign of respect for your participants and a commitment to continuous improvement. If you want your facilitated events to deliver lasting impact (not just outputs, but outcomes) invest in meaningful evaluation.
I’d be delighted to help you design evaluation approaches that are robust, human-centered, and useful. Let’s make sure your good work gets the feedback it deserves, and the results you want.
