Evaluate Peer Support

Evaluating peer support is important for designing peer support programs (e.g., training outcomes of peer supporters), understanding what peer support looks like (e.g., delivery of peer support events), and demonstrating program impact (e.g., quality of life, health outcomes). Knowing “when, how, and what to evaluate” is very important. In many cases, answers to these questions can be critical to program sustainability.

In general, program evaluation starts with planning to make sure what you are going to measure answers not only the question of “did peer support work”, but also “how it worked”.  Overall, it is crucial to identify reliable measures, indicators, tools, and instruments that are relevant to overall program goals, objectives and activities.

Learn more about how to evaluate peer support

In this section, you will find information and resources that help

• Identify a relevant and reliable set of indicators that measure the benefits of peer support:  Peers for Progress Consensus Evaluation Measures
• Plan your evaluation: RE-AIM evaluation framework
• Evaluate changes in behaviors, and health status or clinical outcomes: Impact and Outcome Evaluation
• Document peer support delivery: Process evaluation
• Find example protocols and tools for program evaluation

To learn more about how to evaluate training outcomes of peer supporters, please visit Train Peer Supporters.

Measuring the Benefits of Peer Support for Self Management in Diabetes:  Peers for Progress Consensus Evaluation

Starting in 2009, Peers for Progress funded eight evaluation grants to conduct their own research projects testing the effectiveness of peer support in management of type 2 diabetes.  These individual sites also pooled data into a cross-site evaluation.  Investigators and key staff collaborated with the Peers for Progress Program Development Center at the University of North Carolina at Chapel Hill to identify key evaluation indicators of their peer support programs that could be applied across all projects.  The aim was for a core set of shared evaluation indicators that could strengthen evidence from, yet not add burden to, their individual and collective projects assessing the impacts of peer support.

The following consensus set of shared measures from the Peers for Progress Evaluation Grants include clinical, behavioral, quality of life, process evaluation, mediator/moderator, and costs items.  These measures are selected to be (or be readily modifiable to be) of general applicability in adult health care, chronic disease management, and health promotion, not just diabetes management.

Here you can view a complete version of Peers for Progress Consensus Evaluation for Research on Self Management and Peer Support in Diabetes.

Back to Top

Planning Your Evaluation: RE-AIM Model

This evaluation framework (outlined below) includes and goes beyond a sole focus on the “did it work?” question to help expand our understanding of and translate effective health promotion and behavior change interventions.

According to RE-AIM, evaluations should examine five dimensions:

• Reach into the target population;
• Efficacy or effectiveness of the intervention;
• Adoption by target settings or institutions;
• Implementation-consistency of delivery of intervention; and
• Maintenance of intervention effects in individuals and populations over time.

Researchers and community leaders can visit the RE-AIM website for additional background and resources for evaluation.

Learn more about the RE-AIM Model by viewing this presentation: Glasgow Presentation 2007

View the RE-AIM Planning Tool to help in planning your evaluation.

In addition to RE-AIM, the following resources may also be helpful for a comprehensive approach to evaluation:

• The World Health Organization site provides a Workbook on Planning Evaluation, Implementing Evaluation, and Needs Assessment Evaluation.
• The University of Kansas Community Tool Box offers a section on Developing an Evaluation Plan.
• The W. K. Kellogg Foundation outlines a blueprint for conducting project evaluations

Impact and Outcome Evaluation

According to a 2007 WHO report, there are several key indicators of the effectiveness of a peer support program. These indicators of success help assess:

• The immediate effect a program has on target behaviors and their influencing factors (internal or external): Impact Evaluation
• The effect of a program on health status and quality-of-life indicators: Outcome Evaluation

Impact Evaluation Indicators identified by the WHO report are:

• Self-reported quality of life and emotional distress
• Adherence to behavioral and medication prescriptions
• Knowledge, attitudes, self-efficacy, autonomy and ability to function in their life roles and at work or school

For diabetes peer support programs, tools that can be used to assess behavioral measures include:

• Revised Summary of Diabetes Self-Care Activities Scale (SDSCA)
• Perceived Diabetes Self-Management Scale (PDSMS)
• Self-Efficacy for Diabetes Management
• Diabetes Support Scale (DSS) (in appendix of article)
• Diabetes Empowerment Scale (outlined in Table 3)

Tools that are often used to assess Quality of Life include:

• WHO Quality of Life Scale
• Diabetes Distress Scale (in English and Spanish)
• Problem Areas in Diabetes Questionnaire (PAID)
• Patient Health Questionnaire, also known as the PHQ-9 (Depression)

Outcome Evaluation Indicators identified by the WHO report can be categorized into:

Clinical parameters such as:

• Presence or absence of symptoms (hypo or hyperglycemia)
• HbA1C
• Cardiovascular risk factor control
• Hospitalizations and emergency room visits
• Health care resource consumption
• Presence or development of long term diabetes or cardiovascular complications
• Expenses (costs) and savings (benefits) associated with the program

Additional indicators such as:

• BMI (Body Mass Index)
• Cholesterol
• Blood Pressure

Learn more about Impact and Outcome Evaluation

• The World Health Organization provides a Workbook on Outcome Evaluation, Cost Evaluation, Client Satisfaction Evaluation, and Economic Evaluation.
• The Free Management Library provides a Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources. The document provides guidance toward basic planning and implementation of an outcomes evaluation in nonprofit organizations.
• This evaluation article reports on peer support in type 2 diabetes among adults in Ireland (2007).
• This evaluation article reports on an evaluation approach for examining Aboriginal Diabetes Prevention and Care (including peer support).

Back to Top

Documenting Peer Support Delivery: Process Evaluation

Process evaluation assesses how a program is implemented. It takes into account all program inputs (e.g. theoretical foundations, goals and objectives, resources, etc.), activities (e.g. training, peer support interactions, etc.), and reactions of participants and stakeholders. Some areas you might consider assessing during a process evaluation are:

• Context: what are aspects of the community or environment that might influence the program?
• Reach: who is participating?
• Dose Delivered: how are peer supporters trained?
• Dose Received: what happens in interactions between peers supporters and participants?
• Fidelity: was the program delivered as planned (training and peer support interaction)?

Some ways you might measure process evaluation questions are:

• Interviews or questionnaires with peer supporters, participants, or stakeholders
• Observation of trainings or peer support interactions
• Administrative bookkeeping

Learn more about Process Evaluation

• The Stanford Self-Management Programs’ Program Fidelity Manual provides resources and guidelines for monitoring fidelity.
• The World Health Organization provides a Workbook on Process Evaluation.
• Process Evaluation for Public Health Interventions and Research is a book chapter by Linnan and Steckler providing an overview of process evaluation. ( * See below for the complete reference)
• This article of peer support and smoking among youth outlines a comprehensive process evaluation approach for detecting change.
• Appendix 17 of Peer Support Resource Manual provides sample process evaluation tools and other resources.
• The Georgia Department of Human Resources, Division of Public Health, provides a workbook for designing process evaluations.
• Chapter 3 of the Community Toolbox, Assessing Community Needs and Resources, describes focus groups and other needs assessment strategies.
• This University of Illinois report describes a process evaluation of a peer certification training program.

* Complete reference for the book chapter by Linnan and Steckler

1. Linnan, L., and Steckler A. (2002). Process evaluation and public health interventions: An overview.  In: Steckler, A., and Linnan, L (Eds.),      Process Evaluation in Public Health Interventions and Research. San Francisco: Jossey-Bass Publishers. (pgs. 1-23).
2. Steckler, A., and Linnan L. (Editors). (2002). Process Evaluation in Public Health Interventions and Research. Jossey-Bass Publishers. (400 pages)

Back to Top

Example Protocols & Tools for Program Evaluation

• The Peer Education Evaluation and Resources Center (PEER Center）is a  national resource and  evaluation center for people living with HIV and organization interested in PEER education training programs. Its section on Resources for Peer Programs has evaluation instruments.
• Peer Outcomes Protocol Project’s administration manual was developed as a way to evaluate community-based, mental health peer support programs. Each module in the manual describes how to conduct interviews, use questionnaires, and analyze the data collected in order to better focus on improving quality of life and peer supports for people with psychiatric disabilities
• The National Diabetes Program Evaluation Framework describes how to design an evaluation of a multifaceted public health education program. This framework has helped program planners and evaluators develop measurable short-term and long-term outcomes.
• This 2006 article from Prevention Chronic Disease describes methods and approaches to program evaluation.
• Section K: Program Evaluation (pg. 57) of the Mentoring Partnership Program Manual describes how to develop a plan for program evaluation.
• Annex 2 and Annex 3 of this peer mentor training manual include example pre- and post-training tests to rate the quality of the training and also peer educator and trainer evaluation forms.
• Appendix 1 (pg. 176) of this peer supporter training manual includes a checklist for observers evaluating peer supporters in training.
• The Robert Woods Johnson Foundation Diabetes Initiative provides resources on project participant assessment, pre-test and post-test questionnaires and other program evaluation tools
• The University of Kansas Community Tool Box provides a number of evaluation resources including Evaluating Community Programs and Initiatives, Developing Training Programs for Volunteers and Evaluating the Trainees, and a Trainee Evaluation Form and Checklist.
• The U.S Centers for Disease Control and Prevention Evaluation Working Group provides a host of descriptive information and practical tools for a program evaluation framework.
• A Tool to Assess Cultural Competency in Mental Health Peer-run Programs and Self-Help Groups
• Community Health Worker Assessment and Improvement Matrix (CHW AIM) – A toolkit for improving CHW programs and services, published by USAID Health Care Improvement

Back to Top