Advocacy Planning and Evaluation Program

Advocacy Evaluation Breakfast Series

Upcoming Events

The Effectiveness of Global Health Policy Networks with Jeremy Shiffman, Ph.D.
Tuesday, June 5, 2012
8:15 - 9:45AM
The Aspen Institute
One Dupont Circle NW, Suite 700
Washington, DC 20036
Please RSVP here.

Globally, why have deaths declined more rapidly from some conditions than others? Typically, efforts to answer this question focus on whether effective medical interventions exist.  But this is not the whole story.  Missing is consideration of the actors: the individuals and organizations linked by a shared concern for a condition - what might be termed its ‘global health policy network’.  Presumably some networks are more effective than others in attracting attention to the issue, in generating funding, in discovering interventions, and in convincing national governments to adopt policies and carry out programs.

The Global Health Advocacy and Policy Project (GHAPP) is a three-year research initiative funded by the Bill and Melinda Gates Foundation designed to understand why some global health policy networks are more effective than others in achieving these outcomes.  It compares the effectiveness of networks concerned with six global health problems: maternal mortality, newborn deaths, tuberculosis, pneumonia, tobacco control and alcohol abuse.  The talk will highlight a set of factors that help to explain why some networks are more effective than others.

Jeremy Shiffman is Associate Professor of Public Administration and Policy at American University.  A political scientist by training, he researches the politics of health policy and administration in poor countries.  He has a particular interest in health agenda-setting: why some issues receive priority while others are neglected.  Among other topics, he has investigated maternal survival, newborn survival, family planning, donor funding for health and health systems reform.  His research has been funded by the Gates, MacArthur and Rockefeller Foundations, among other organizations.  His work has appeared in multiple journals, including The Lancet, The American Journal of Public Health, Social Science and Medicine, Population and Development Review, The British Journal of Obstetrics and Gynaecology, The Bulletin of the World Health Organization and Health Policy and Planning.  Previously he was on the faculty of the Maxwell School of Syracuse University, where he received four teaching awards.  Prior to working in academia he served as an executive with the international public relations firm Burson-Marsteller, and as a social worker, working with Vietnamese boat people.  He received a BA summa cum laude from Yale University in philosophy, an MA from Johns Hopkins University in international relations, and a Ph.D. from the University of Michigan in political science.

Recent Events:

05.08.12 Expanding the Range of Advocacy Capacity Assessment
The George Gund Foundation's Marcia Egbert and Alliance for Justice's (AFJ) Sue Hoechstetter will present AFJ's new Advocacy Capacity Tool— for assessing organizational capacity.  Almost seven years ago AFJ, in collaboration with Mosaica and the George Gund Foundation, developed the Advocacy Capacity Assessment Tool to help funders, nonprofits, and evaluators determine the skills, knowledge, and other resources organizations need to engage in effective advocacy over time.  After collecting input from users of the tool and a third party evaluation, AFJ has revised its survey instrument, broadening the range of capacities covered in four sections: Advocacy Goals, Plans, and Strategies; Conducting Advocacy; Advocacy Avenues; and Organizational Operations to Sustain Advocacy. Marcia provided a brief background on how her foundation used the original tool as well as feedback from Gund Foundation grantees who have tested the new version. Sue presented an overview of the 18 indicators in the new version of the tool and how this free, online resource can be used to analyze an organization’s capacity and to compare the findings with that of other organizations.

12.15.11 Measuring Success: Advocacy Evaluation and Performance Management Implementation at the United Nations Foundation
The United Nations Foundation (UNF) is currently focused on finding the best ways to quantify and measure impact, which they are doing through a new Performance Management Plan. UNF is already benefiting from the implementation, with more informed decision-making and strategic planning specifically related to their global health advocacy work. Central to the plan is the concept of a “Champion Index”, which helps quantify the action, or inaction, of key policymakers on the Hill with regards to global health policy as well as a "Constituent Commitment Curve", which helps track grassroots engagement. Guest Mike Beard, Director of the Better World Campaign, discussed the conceptualization and use of these tools, and Kate Dodson, Director of Global Health, and Andrew Axelrod, Executive Director of Planning & Learning, talked about overall best practices and lessons learned in implementing a performance management plan.

10.04.11 From Community Clinics to Statewide Policy Change: Evaluation Lessons from The California Endowment’s “Clinic Consortia Policy and Advocacy Program”
In 2001, The California Endowment (TCE) pledged $10 million to help build the advocacy capacity of 19 clinic associations in its “Clinic Consortia Policy and Advocacy Program.”  Spread across California and targeting diverse populations, the grantees aimed to increase access to high quality and affordable health care for underserved Californians.  TCE believed that strengthening the capacity of these associations to support the policy and operational needs of community clinics could help lead to substantial improvements in public health, for example, by promoting policies to   expand clinic services for low-income populations.  Three years later, in 2004, TCE invested an additional $8.8 million to continue funding 18 of these clinic associations.  Annette Gardner, PhD, MPH, was the Principal Investigator leading the evaluation of the “Clinic Consortia Policy and Advocacy Program.”  At this breakfast, Dr. Gardner provided an overview of the evaluation findings and described some methods and tools that evaluators and advocates can readily use to assess the impact of advocacy strategies.  She also discussed the relevance of the evaluation for researchers and evaluators of non-profit advocacy organizations, including the application of a logic model framework to developing data collection instruments and suggestions for addressing stakeholder information needs.

07.15.11 The Elusive Craft of Evaluating Advocacy
In the Summer 2011 issue of the Stanford Social Innovation Review, Steven Teles and Mark Schmitt argue that evaluating advocacy is a fundamentally different task than evaluating services. Evaluating advocacy is, to put it bluntly, evaluating political activity, and it is the nature of politics that events evolve rapidly, often unpredictably and in a nonlinear fashion. Consequently, it is unclear if there are any 'best practices' in politics or useful metrics to judge progress. Advocacy evaluation should, therefore, be seen as a matter of trained judgment rather than method, one requiring deep knowledge and feel for the politics of issues, strong networks of trust, the ability to assess organizational quality, and a sense for the right time horizon against which to judge activity. In fact, the more ambitious the effort at political change is, the more judgment matters and the less any abstract method is useful. Guests Steve Teles and Mark Schmitt discussed their findings and Jackie Williams Kaye, Director of Research and Evaluation at Wellspring Advisors, offered a response.

01.14.11 Changing the World through Local to Global Advocacy: Scaling and Assessing Impact from the Campaign to End Pediatric Aids
The Campaign to End Pediatric HIV/AIDS (CEPA) is a CIFF-funded initiative managed by the Global AIDS Alliance (GAA). Launched in May 2009, CEPA seeks to achieve 80% coverage for comprehensive prevention of mother-to-child transmission and pediatric treatment services, with an initial focus on six sub-Saharan African countries: Kenya, Mozambique, Nigeria, Tanzania, Uganda, and Zambia.  Specifically, the campaign is working to overcome key policy and implementation bottlenecks, and to hold government and multilateral stakeholders accountable for their commitments. GAA coordinates local-to-global advocacy campaigns. CEPA also works on maternal and children health, sexual and reproductive health and rights, and universal basic education.  In collaboration with GAA, CIFF, and other partners, iScale has developed an innovative monitoring and evaluation system for CEPA.

12.10.10 From Evidence to Influence: The Sequel
If alleviating global poverty depends on successful pro-poor policies, then CARE, like other international humanitarian organizations, can promote these policies by presenting evidence based on decades of working in more than 60 countries. With Gates Foundation support, CARE is testing this hypothesis via two initiatives. CARE's LIFT UP grant aims to build organizational capacity to more systematically use country-level evidence to influence U.S. policymakers. CARE’s Learning Tours grant provides Members of Congress and influential media and “grasstops” leaders with firsthand experiences aimed at increasing their support for improving maternal health and child nutrition globally.  Working with external evaluators Innovation Network and Continuous Progress Strategic Services (CPSS), CARE is assessing the effectiveness of these approaches. Speakers discussed how to measure the effect of country-based evidence on policy change, and highlight how CARE’s overlapping evaluations inform each other’s work and increase CARE's ability to influence policy change.

6.8.10 Building African Advocacy Through Evaluation with Rhonda Schlangen
Advocacy can be particularly challenging when it is conducted in contested political space by organizations constrained by resources and experience. Effective monitoring and evaluation can support or hinder those efforts. It can provide a mechanism for real time learning in rapidly changing political environment.  But it can also be a distracting exercise conducted exclusively for the benefit of donors. Rhonda shared lessons learned from advocacy and evaluation work with non-governmental organizations and coalitions in the global south, with particular attention to sub-Saharan Africa. She discussed the challenges facing small-scale, community based health service providers - as well as multi-sector advocacy coalitions -- as they add evaluation to their policy change advocacy work.   Rhonda Schlangen is an independent evaluation consultant specializing in the evaluation of advocacy projects and campaigns.

12.11.09 Unique Methods in Advocacy Evaluation with Julia Coffman
Like all evaluations, advocacy and public policy evaluations can draw on a familiar list of traditional data collection methods. But advocacy and policy work is often complex, fast-paced, and dynamic. That makes data collection challenging. And advocacy efforts often aim for outcomes that are hard to operationalize and measure, So evaluators are developing new and innovative methods specifically for assessing advocacy and policy efforts. Julia Coffman of the Center for Evaluation Innovation will introduce us to new methods designed to be relevant, timely and efficient: call it the trifecta of practical evaluation for busy advocates.

9.25.09 Measuring Political Will and Defining and Measuring Policymaker Champions with Craig Charney and David Devlin-Foltz
APEP convened colleagues and friends from the world of policy advocacy and evaluation for the first of our Advocacy Evaluation Breakfasts from 8:30 to 10:00 on Friday, September 25 at The Aspen Institute, Suite 700, One Dupont Circle NW.  The session featured political scientist Craig Charney, president of Charney Research, on Measuring Political Will.  He explored how to identify and quantify the holy grail of advocacy efforts.  APEP Director David Devlin-Foltz presented on defining and measuring what it means to be a "champion" for an issue.