Beyond the survey
INSIGHTS ARTICLE
Evaluation as learning, respect, and stewardship
Most facilitated events end in much the same way. Participants are thanked. A link appears on the screen. People are asked to complete a short survey before returning to full inboxes and competing priorities.A week later, the results arrive. The average score is encouraging. A few comments are copied into a report. The event is declared successful, and attention moves to whatever comes next.There is nothing inherently wrong with this. A well-designed survey can tell us a great deal. But increasingly, I think the most important question is not whether we collected feedback. It is whether we were genuinely prepared to learn from it.Because evaluation is not the administrative task that follows the work.Done well, it is part of the work itself.
What the numbers can tell us
Quantitative measures are useful. I regularly use Likert-scale questions to understand how participants assess their capability, confidence, or understanding before and after a learning experience. Across repeated programs, these measures can reveal patterns that are difficult to see through anecdote alone.They help us ask practical questions. Did participants report movement against the intended learning objectives? Were some parts of the program stronger than others? Are results changing across cohorts? Is an intervention delivering consistently, or does its value depend too heavily on a particular facilitator, group, or circumstance?For organisational leaders, this information also supports accountability. It helps distinguish activity from outcome. It provides a clearer basis for deciding what should continue, what should change, and where further investment may be justified.But numbers are signals. They are not the whole story.
What the numbers cannot tell us
An average score can tell us that participants valued an experience. It cannot necessarily tell us why.It may not reveal the moment a participant reconsidered an assumption, found language for something they had been struggling to explain, or felt safe enough to contribute for the first time. Nor will it necessarily show us the unintended consequence: the activity that confused people, the example that excluded them, or the part of the design that worked for one group while quietly failing another.These insights often emerge in participants’ own words.This is why I place considerable value on carefully framed free-text questions. Not because qualitative comments are automatically deeper, but because they create room for people to tell us something we did not already know to ask.A multiple-choice question tests the options we have imagined. An open question creates the possibility that our assumptions are incomplete.
Listening for what we did not think to ask
There is a temptation in evaluation to make participation as frictionless as possible. Short surveys improve completion rates. Pre-set answers simplify analysis. Automated dashboards produce immediate results.All of these things can be useful. Yet convenience has a cost when it narrows the field of what can be heard.At my most cynical, I sometimes wonder whether some evaluations are designed to create the appearance of listening rather than the conditions for it. If the questions allow only predictable answers, if comments are constrained to a few characters, or if nobody has the authority or appetite to act on what emerges, then the process may be collecting data without creating learning.The issue is not technology. It is intent.Are we seeking confirmation that the work was well received? Or are we willing to discover that something important needs to change?
From feedback to learning
Collecting richer feedback is only the beginning. Thoughtful evaluation requires us to work with the material carefully. In practice, this usually involves several connected disciplines:
Clarify the purpose. Know what decisions the evaluation needs to inform and what evidence would actually be useful.
Collect proportionate evidence. Use quantitative and qualitative methods that match the scale, risk, maturity, and intended outcomes of the work.
Clean and organise the data. Remove duplication, correct obvious errors, protect confidentiality, and prepare the material for responsible analysis.
Look for patterns without erasing difference. Code themes and compare results, while remaining alert to minority experiences, contradictions, and unusual responses.
Interpret with humility. Ask what the evidence suggests, what it cannot establish, and what other explanations may be plausible.
Act and close the loop. Use the findings to improve the work and, where appropriate, tell participants what was heard and what will happen next.
This final step matters. People are more likely to offer thoughtful feedback when they believe their contribution will be treated thoughtfully. In my own programs, response rates have often been between 95 and 100 per cent, with substantial free-text participation. I do not think this is because people are unusually enthusiastic about surveys. I think it is because time is taken to explain why their feedback matters, how it will be used, and what it may influence.Trust changes the quality of the evidence we receive.
Evaluation as respect
The deeper purpose of evaluation is not to prove that our work was successful.It is to become more capable of understanding what the work actually did.That includes recognising what was valuable, confronting what was ineffective, and noticing outcomes we did not anticipate. It means treating participants as more than recipients of an intervention. They are witnesses to its effects and, often, partners in improving what comes next.Seen this way, evaluation becomes an expression of respect. It says: your experience matters; your interpretation may reveal something I cannot see; and I am willing to let what I learn change the way I work.For leaders, facilitators, coaches, and learning designers, this is also a question of stewardship. We are often entrusted with people’s time, organisational resources, public money, professional hope, and sometimes a surprising degree of vulnerability. Measuring outputs is not enough. We need to ask whether the work created useful outcomes, for whom, under what conditions, and with what unintended effects.
Questions worth carrying into the work
Before designing the next evaluation, it may be useful to ask:
Purpose: What decision, learning need, or responsibility is this evaluation intended to serve?
Voice: Whose experience is easy to capture, and whose may be missing or obscured?
Assumptions: Are our questions genuinely open to surprise, or are they designed mainly to confirm what we already believe?
Use: Who will interpret the findings, who can act on them, and how will participants know their contribution mattered?
Proportion: Are we gathering enough evidence to support the claims we intend to make… without imposing an unnecessary burden on participants?
The quality of an evaluation is not determined by the sophistication of the platform or the visual polish of the dashboard. It is determined by the quality of attention we bring to the evidence, the honesty of the questions we are willing to ask, and the care with which we respond to what people tell us.A survey can measure satisfaction.Evaluation, at its best, helps us learn whether the work mattered; and how to make the next iteration more worthy of the people it is intended to serve.
Continue your exploration
Related Insightful Path writing
The real risk in capability work — Why depth, trust, reflection, and practice matter more than reaching the largest possible audience.
Facilitation as a practice — An exploration of facilitation as embodied, ethical stewardship rather than a collection of techniques.
Putting these ideas into practice
Facilitation and its design — Explore how Insightful Path designs facilitated work with learning, participation, and meaningful outcomes in mind.
Beyond Insightful Path
Australian Government Commonwealth Evaluation Policy : The Commonwealth framework for embedding evaluation in policy and program design.
BetterEvaluation Rainbow Framework — A practical map of the tasks involved in planning, managing, and using evaluation.
OECD evaluation criteria — A framework for considering relevance, coherence, effectiveness, efficiency, impact, and sustainability.
If you are designing a learning experience, facilitated process, or leadership intervention and want the evaluation to generate useful learning (not simply a completion score) I would be glad to help you think it through.
If you would like a pdf copy of this essay - please hit the button below:
