Designing a Systemic Methodology for Program Evaluation

Abstract

Evaluation is commonly seen as a systematic process to determine merit, worth, value or significance. When engaging in program evaluation, evaluators use research methods to systematically investigate the effectiveness of social intervention programs adapted to the political and organizational environment surrounding them. However, aside from having a systematic character, evaluation has at its core a systemic and a critical character as well, as it is based on the establishment of judgments and the inclusion of stakeholders, both of which inevitably affect what will be seen as an improvement. Critical in terms of not taking for granted predefined assumptions about the evaluation and systemic in terms of a dynamic attitude towards the establishment of what and who should be considered in the evaluation and the acknowledgment the existing relationships of those involved in the evaluation. Thus, the systems theory of boundary critique (about how to explore value and boundary judgements) is relevant. For this reason, we seek to propose a methodological development for conducting social program evaluations. Our methodological proposal, seeks to contribute at a theoretical and a practical level as we not only seek to present a methodology that can be widely applied in the realm of social program evaluation through a practical case but we also seek to contribute to enriching the literature that links systems thinking practice and evaluation, focusing primarily in the contributions that critical systems thinking can make to the practice of evaluation. We examine different stages of the evaluation process and show how boundary critique can be used in each one. A practical example will be provided of an evaluation of a program for teaching alternative conflict resolution techniques to children in vulnerable areas of Bogota, Colombia.