The challenge of monitoring and evaluating psychosocial support
Louise Juul Hansen is a Communications Officer at the International Federation of Red Cross and Red Crescent Societies (IFRC) Reference Centre for Psychosocial Support. Here she explains how their recently published M&E framework was developed and what sets it apart. To download the framework documents, visit the Resource Page.
Monitoring and evaluation (M&E) is not always a subject that brings smiles on the faces of psychosocial support practitioners. When the IFRC Reference Centre for Psychosocial Support (PS Centre) talks to colleagues in the field and in academia about M&E of psychosocial support interventions there seems to be a wide consensus on two points: 1) M&E is very important and we need more of it. 2) Nobody really knows how to do it well.
Developing good, practical and usable M&E systems would be a huge step forward for the field of psychosocial support. It is necessary to document results and learn from them in order to improve interventions. It is also important for accountability purposes to be able to demonstrate that interventions actually improve the lives of the people we support, and that we are spending money appropriately. Having good reliable data from the field combined with the growing mass of research from academia on the field of mental health and psychosocial support (MHPSS) helps the community of practice to build a good advocacy case for incorporating MHPSS in humanitarian work and for demonstrating to donors that their money is well spent.
The PS Centre started developing M&E frameworks in 2013. The first framework was focused on the response phase of emergencies, but it was clear that a broader framework was needed to encompass the great diversity of the interventions and organizational context of the Red Cross Red Crescent (RCRC). So the work of the IFRC Monitoring and evaluation framework for psychosocial support interventions commenced.
At the same time, the Inter-Agency Standing Committee (IASC) Reference Group for Mental Health and Psychosocial Support started working on an M&E framework for emergency settings based on the IASC MHPSS guidelines: “A common monitoring and evaluation framework for mental health and psychosocial support in emergency settings”. The development of the IASC monitoring and evaluation framework has been a collaborative inter-agency and multi-year effort including several different member organizations at HQ and country-level, including the IFRC PS Centre.
It may look like a double effort to develop two seemingly identical frameworks, but there are crucial differences between the two, making both relevant and complementary.
IASC vs IFRC
-
The IASC Framework is an inter-agency level framework focusing exclusively on emergency settings. Thus, the intended target group is very wide, but the focus of emergency settings is quite narrow. The IFRC framework on the other hand has a much narrower primary target group (RCRC National Societies), but a wider focus.
- The IASC framework is deliberately kept fairly generic and only operates on the higher levels of goal and outcome for both mental health and psychosocial programmes. This means that it is easy for organizations to adapt it to their organizational and situational context while keeping within a large, internationally agreed upon framework that is underpinned by the IASC Guidelines. It aims to increase the standard of M&E for MHPSS programmes by forcing organizations to report at impact and outcome level rather than focusing their energy at output (or activity) level.
- The IFRC framework on the other hand is more specifically designed for the Red Cross Red Crescent where non-emergency interventions, capacity building and caring for staff and volunteers are key activities. This framework also has indicators on the output level, suggestions for mean of verification and a substantial toolbox to support with data collection
The IFRC M&E framework was tested in early 2016 in a mature draft format and underwent a rigorous review in late 2016. During this review much effort was put into making sure that whenever possible, the IFRC framework is aligned with the IASC framework (e.g. IASC framework indicators are included in the revised version), so that the two frameworks complement each other rather than compete with each other.
Components of the Framework
The IFRC M&E framework consists of various tools to help build the capacity of National Societies in developing a systematic approach to M&E of their PS programmes:
The guidance note
The guidance note provides an overview of monitoring and evaluation approaches and principles as key components of the programme management cycle. Psychosocial programme objectives and indicators are covered, including quantitative and qualitative indicators, and guidance is given on the development of M&E plans. The guidance note also details ethical principles and other fundamental requirements in preparing for M&E, including building the capacity of National Societies’ staff and volunteers in conducting M&E activities.
The indicator guide
The indicator guide presents a set of sample indicators which are broadly applicable to National Societies (NS) PS programmes of various kinds. Although each PS programme will be unique to the context where it is implemented and the people who are involved, certain key priority areas are shared among different kinds of PS programmes. The indicator guide outlines sample indicators that capture key aspects of change that PS programmes hope to achieve. They are drawn from experience in M&E of PS programmes within IFRC, National Societies and amongst the global community of PS programme implementers and evaluators. Indicators from IASC MHPSS RG ‘A common monitoring and evaluating framework for mental health and psychosocial support in emergencies’ are also included in the indicator guide.
The indicator guide provides a broad understanding of changes that can result from PS programmes at goal, outcome and output levels. The indicators developed for each level are phrased in such a way that they can be tailored to specific programmes – that, is, for a particular type of target group or problem. The indicator guide is a road map for developing an M&E system and indicators for PS programmes. The end result should be a set of indicators which are appropriate to the specific activities of a programme and its target group and the cultural context and local understandings of psychosocial well-being in which it is being implemented.
The toolbox
The toolbox contains guidance and tools for data collection in M&E of PS programmes. They are all suitable for PS programmes. Many have been drawn from existing PS programme M&E tools, but they are not an exhaustive list. NSs may find that new or additional tools may have to be developed for a specific programme. Local tools may already exist or other tools may have already been adapted to a specific context and situation that can be used instead. Whatever the source, it is vital that tools are adapted to the specific PS programme, and relate to the target group, activities and scope, etc.
The tools provided are those listed in the means of verification (MoV) column of the indicator tables in the indicator guide. The tools are focused mainly upon MoV for outcome levels. However some of the tools can also be used as MoV on the output level (particularly programme management cycle tools and supervision reporting tools).
