Multi all over...
Multi-component, multistakeholder, multisite, multi-actor, multi methods and many more such conceptual characteristics are lately cropping up in the subject description of the terms of reference for evaluators. In simplified terms, they express the interconnectedness and apparent complexity of a subject to be reviewed, suggesting that any type of assessment would necessarily have to adopt a comprehensive and systemic multi-methods approach to capture the development quality or progress.
This explanation is plausible at first sight. However, the inflationary use of the term "multi" may also be an indication that comissioners of evaluations are - for whatever reason - not in a position to precisely define the boundary of the task, to differentiate value chains and relate these to specific targets or to relativise the potential scope and likely impact of an intervention. Under such uncertain expectations which do relate to incomplete reflection at the visioning and planning stage, all imaginable aspects become potentially relevant. The sky appears to be the limit for the succepted scope of influence of a development intervention. In today's often breathless and squeezed real world of development management, the approach of an all-inclusive type of TOR is kind of a time saving short-cut. It transfers the entire responsibility for narrowing down the focus and tracing the core components for learning on to the consultants. This leads normally to some sort of relevant findings for the responsible programme managers in a pick-and-choose fashion. And for the consultant, this approach leaves room for assigning one's own selective focus within the allocated time and resources, without being over-guided by a set of air-tight evaluation questions. So is this mutual benefit a classical win-win?
Probably not in terms of generating deeper insights, in situations where the relevance and strategic learning about the appropriateness of a chosen approach are at stake. A meaningful external contribution to strategic guidance in development depends fundamentally on the way in which the external advice was solicited. The mandating agency, in close collaboration with other concerned stakeholders, should invest in preparing the best perspective for an external input, in setting the most pertinent objectives for purpose, in crafting the right guiding questions and designing an appropriate evaluative process. An all-embracing TOR will not suit the specific expectations for reflection and learning.
The answer to the win-win question is probably yes in those situations where an outside birds eye view over the entire range of components of a development intervention is needed, without preemptively problematizing any aspect of a programme.
It is well worth to address the character of a planned review well ahead in order to clarify generous expectations, to differentiate ambitions and to avoid disappointments.
| Martin Sommer
Managing Consultant devolutions Ltd.