Upcoming Events:

Assessing the Capacity of Community Coalitions to Advocate for Change

Wednesday, May 22, 2013 | 8:15 - 9:45AM

The Aspen Institute, One Dupont Circle NW, Suite 700, Washington, DC 20036

To RSVP, please complete this form by May 21, 2013.

Research has shown that high-capacity coalitions are more successful in effecting community change.  While a number of coalition assessment tools have been developed, documentation is scarce regarding how they are implemented, how the results are used, and whether they are predictive of coalition success in collaborative community change efforts.  This session will focus on a coalition assessment tool that was designed by Innovation Network to assess changes in coalition capacity over time.  Developed for a health promotion initiative of a major health foundation, this tool is designed to assess coalition progress in eight key areas across twelve different community coalitions, over the course of a three year initiative.  Presenters will share lessons learned from the first year of the initiative about developing and deploying the assessment tool, as well as what these tools can—and can’t—tell you about a coalition’s capacity in conducting community change work.  In addition presenters will share how information collected from this assessment can be communicated back to the coalitions using data visualization approaches to effectively communicate the data.

Presenters:

Veena Pankaj is a Director of Innovation Network, Inc., and also leads many of the organization’s consulting and research projects.  Veena leads and manages evaluation and strategy consulting projects with clients such as the Kansas Health Foundation, Share Our Strength, CARE, and the Congressional Hunger Center.  She has over a decade of experience leading organizations through the evaluation process, and she works closely with foundations and nonprofits to answer questions about program design, implementation, and impact.  Ms. Pankaj completed a Certificate in Organization Development from Georgetown University, and has received a B.A. in Sociology from University of Virginia, and an M.A. in Social Sciences from University of Chicago.

Johanna Morariu is a Director of Innovation Network, Inc., and also leads the organization’s consulting and research projects, working with organizations such as the Annie E. Casey Foundation, the Ford Foundation, the Connecticut Health Foundation, the Center for Community Change, and others.  Johanna is a fierce believer in the power of the social sector to improve people’s lives, and the role of the social sector as innovator, experimenter, and advocate for effective approaches to societal challenges.  In her work, she seeks to magnify the impact of her partners by providing timely, high-quality information to bolster decision making.  She directs the organization’s research agenda, including the ongoing State of Evaluation project (www.stateofevaluation.org).  Ms. Morariu received her B.A. in International Relations and History from Penn State University and her M.A. in Political Science from George Mason University.

Kat Athanasaides is an Associate with the Innovation Network team.  She brings skills of evaluation design and implementation, advocacy evaluation, SROI analysis, data collection tool design, and technical assistance provision.  Kat consults with organizations such as the Ford Foundation, the Kansas Health Foundation, the Center for Community Change, and others.  She received her B.A. in Sociology and Anthropology from Swarthmore College, and a Master of International Affairs (M.I.A.) from the School of International and Public Affairs at Columbia University, where she focused on international economic development.

Ann Emery is an Associate with Innovation Network.  Ann works with a variety of social sector organizations, including the Annie E. Casey Foundation, Center for Community Change, Connecticut Health Foundation, and the Kansas Health Foundation.  She also contributes to research projects such as State of Evaluation 2012: Evaluation Practice and Capacity in the Nonprofit Sector.  Ann brings a well rounded skill set, including evaluation design, data collection, analysis, data and information visualization, technical assistance, and training.  Ms. Emery received a B.A. in Psychology and Spanish from the University of Virginia and is currently pursuing an M.S. in Educational Psychology from George Mason University

Recent Events:

04.25.13 Ten Years of PEPFAR: A Conversation about Evaluation, Advocacy and Grantmaking

Since the inception of the President’s Emergency Plan for AIDS Relief (PEPFAR) in 2003, the Center for Health and Gender Equity (CHANGE) has contributed to advocacy for policy changes that would advance sexual and reproductive health and rights of women and girls.  CHANGE and its allies sought to eliminate the Congressionally-mandated "abstinence-only" budget allocation (or earmark), replace abstinence-focused prevention guidance with more comprehensive guidance, and focus greater attention on gender inequalities and violence against women.  The advocacy effort helped change U.S. law and policies on these matters.  In February 2013, the Institute of Medicine (IOM) of the National Academies published a comprehensive and rigorous assessment of the program.  This evaluation included an assessment of how PEPFAR policy and guidance concerning gender issues, abstinence and HIV prevention evolved, and examined the effects of PEPFAR-supported activities in partner countries.  At this Advocacy Evaluation Breakfast, Kimberly Scott (Senior Program Manager at IOM), Serra Sippel (President of CHANGE), and Shira Saperstein (Deputy Director and Program Director for Women's Rights and Reproductive Health at the Moriah Fund) discussed the report’s findings, the impact of policy advocacy in the US on the lives of women in PEPFAR countries, and the role of funding organizations in the advocacy process.

02.28.13 Using Public Opinion Polling in Development Evaluations

Evaluators have multiple tools at their disposal, including polling of the public in target communities.  For some evaluations, public opinion polls and surveys can provide critical information about the outcomes—and possibly the impact—of an intervention.  For advocacy evaluators and others tracking complex social change efforts, these polls may clarify how awareness campaigns influence public attitudes and behaviors.  For example, carefully-crafted surveys can yield valuable evidence for development project evaluators tracking attitudes towards gender-based violence, or women's access to microcredit programs.  Dr. Craig Charney, a polling expert and President of New York-based Charney Research, spoke about the value that public opinion surveys can add to evaluation design.  Drawing from his extensive experience working in the developing world, Dr. Charney discussed the role of surveys in better assessing needs, developing strategies, evaluating progress, and increasing impact.

01.30.13 Lessons Learned from a Donor's Collaborative: Managing and Sustaining Effective Coalitions

Since its inception, The Connect U.S. Fund has used its program of grant-making and operations to foster strong collaboration and advocacy among NGOs seeking to advance responsible U.S. global engagement in an increasingly interdependent world.  Over the last eight years, the Fund has been a key player in the promotion of multilateral engagement and strong U.S. leadership on human rights, climate change, nuclear non-proliferation, and development.  As a result of an 18-month long strategic planning process, the organization decided to wind down its grant-making and operations by February 2013 and share its lessons learned with the NGO and philanthropic communities.  Executive Director Alexandra Toma shared best practices from their eight years of a “donor’s collaborative,” including best practices on managing effective and impactful collaboratives, conducting rapid response grant-making, and launching and sustaining an effective donor's collaborative.

06.05.12 The Effectiveness of Global Health Policy Networks

Globally, why have deaths declined more rapidly from some conditions than others?  Typically, efforts to answer this question focus on whether effective medical interventions exist.  But this is not the whole story.  Missing is consideration of the actors: the individuals and organizations linked by a shared concern for a condition - what might be termed its ‘global health policy network’.  Presumably some networks are more effective than others in attracting attention to the issue, in generating funding, in discovering interventions, and in convincing national governments to adopt policies and carry out programs.  The Global Health Advocacy and Policy Project (GHAPP) is a three-year research initiative funded by the Bill and Melinda Gates Foundation designed to understand why some global health policy networks are more effective than others in achieving these outcomes.  It compares the effectiveness of networks concerned with six global health problems: maternal mortality, newborn deaths, tuberculosis, pneumonia, tobacco control and alcohol abuse.  Presenter Jeremy Shiffman, Associate Professor of Public Administration and Policy at American University, highlighted a set of factors that help to explain why some networks are more effective than others (Please right click and "save link as").  

05.08.12 Expanding the Range of Advocacy Capacity Assessment

The George Gund Foundation's Marcia Egbert and Alliance for Justice's (AFJ) Sue Hoechstetter will present AFJ's new Advocacy Capacity Tool— for assessing organizational capacity.  Almost seven years ago AFJ, in collaboration with Mosaica and the George Gund Foundation, developed the Advocacy Capacity Assessment Tool to help funders, nonprofits, and evaluators determine the skills, knowledge, and other resources organizations need to engage in effective advocacy over time.  After collecting input from users of the tool and a third party evaluation, AFJ has revised its survey instrument, broadening the range of capacities covered in four sections: Advocacy Goals, Plans, and Strategies; Conducting Advocacy; Advocacy Avenues; and Organizational Operations to Sustain Advocacy. Marcia provided a brief background on how her foundation used the original tool as well as feedback from Gund Foundation grantees who have tested the new version. Sue presented an overview of the 18 indicators in the new version of the tool and how this free, online resource can be used to analyze an organization’s capacity and to compare the findings with that of other organizations.

12.15.11 Measuring Success: Advocacy Evaluation and Performance Management Implementation at the United Nations Foundation

The United Nations Foundation (UNF) is currently focused on finding the best ways to quantify and measure impact, which they are doing through a new Performance Management Plan. UNF is already benefiting from the implementation, with more informed decision-making and strategic planning specifically related to their global health advocacy work. Central to the plan is the concept of a “Champion Index”, which helps quantify the action, or inaction, of key policymakers on the Hill with regards to global health policy as well as a "Constituent Commitment Curve", which helps track grassroots engagement. Guest Mike Beard, Director of the Better World Campaign, discussed the conceptualization and use of these tools, and Kate Dodson, Director of Global Health, and Andrew Axelrod, Executive Director of Planning & Learning, talked about overall best practices and lessons learned in implementing a performance management plan.

10.04.11 From Community Clinics to Statewide Policy Change: Evaluation Lessons from The California Endowment’s “Clinic Consortia Policy and Advocacy Program”

In 2001, The California Endowment (TCE) pledged $10 million to help build the advocacy capacity of 19 clinic associations in its “Clinic Consortia Policy and Advocacy Program.”  Spread across California and targeting diverse populations, the grantees aimed to increase access to high quality and affordable health care for underserved Californians.  TCE believed that strengthening the capacity of these associations to support the policy and operational needs of community clinics could help lead to substantial improvements in public health, for example, by promoting policies to   expand clinic services for low-income populations.  Three years later, in 2004, TCE invested an additional $8.8 million to continue funding 18 of these clinic associations.  Annette Gardner, PhD, MPH, was the Principal Investigator leading the evaluation of the “Clinic Consortia Policy and Advocacy Program.”  At this breakfast, Dr. Gardner provided an overview of the evaluation findings and described some methods and tools that evaluators and advocates can readily use to assess the impact of advocacy strategies.  She also discussed the relevance of the evaluation for researchers and evaluators of non-profit advocacy organizations, including the application of a logic model framework to developing data collection instruments and suggestions for addressing stakeholder information needs.

07.15.11 The Elusive Craft of Evaluating Advocacy

In the Summer 2011 issue of the Stanford Social Innovation Review, Steven Teles and Mark Schmitt argue that evaluating advocacy is a fundamentally different task than evaluating services. Evaluating advocacy is, to put it bluntly, evaluating political activity, and it is the nature of politics that events evolve rapidly, often unpredictably and in a nonlinear fashion. Consequently, it is unclear if there are any 'best practices' in politics or useful metrics to judge progress. Advocacy evaluation should, therefore, be seen as a matter of trained judgment rather than method, one requiring deep knowledge and feel for the politics of issues, strong networks of trust, the ability to assess organizational quality, and a sense for the right time horizon against which to judge activity. In fact, the more ambitious the effort at political change is, the more judgment matters and the less any abstract method is useful. Guests Steve Teles and Mark Schmitt discussed their findings and Jackie Williams Kaye, Director of Research and Evaluation at Wellspring Advisors, offered a response.

01.14.11 Changing the World through Local to Global Advocacy: Scaling and Assessing Impact from the Campaign to End Pediatric Aids

The Campaign to End Pediatric HIV/AIDS (CEPA) is a CIFF-funded initiative managed by the Global AIDS Alliance (GAA). Launched in May 2009, CEPA seeks to achieve 80% coverage for comprehensive prevention of mother-to-child transmission and pediatric treatment services, with an initial focus on six sub-Saharan African countries: Kenya, Mozambique, Nigeria, Tanzania, Uganda, and Zambia.  Specifically, the campaign is working to overcome key policy and implementation bottlenecks, and to hold government and multilateral stakeholders accountable for their commitments. GAA coordinates local-to-global advocacy campaigns. CEPA also works on maternal and children health, sexual and reproductive health and rights, and universal basic education.  In collaboration with GAA, CIFF, and other partners, iScale has developed an innovative monitoring and evaluation system for CEPA.

12.10.10 From Evidence to Influence: The Sequel

If alleviating global poverty depends on successful pro-poor policies, then CARE, like other international humanitarian organizations, can promote these policies by presenting evidence based on decades of working in more than 60 countries. With Gates Foundation support, CARE is testing this hypothesis via two initiatives. CARE's LIFT UP grant aims to build organizational capacity to more systematically use country-level evidence to influence U.S. policymakers. CARE’s Learning Tours grant provides Members of Congress and influential media and “grasstops” leaders with firsthand experiences aimed at increasing their support for improving maternal health and child nutrition globally.  Working with external evaluators Innovation Network and Continuous Progress Strategic Services (CPSS), CARE is assessing the effectiveness of these approaches. Speakers discussed how to measure the effect of country-based evidence on policy change, and highlight how CARE’s overlapping evaluations inform each other’s work and increase CARE's ability to influence policy change.

6.8.10 Building African Advocacy Through Evaluation with Rhonda Schlangen

Advocacy can be particularly challenging when it is conducted in contested political space by organizations constrained by resources and experience. Effective monitoring and evaluation can support or hinder those efforts. It can provide a mechanism for real time learning in rapidly changing political environment.  But it can also be a distracting exercise conducted exclusively for the benefit of donors. Rhonda shared lessons learned from advocacy and evaluation work with non-governmental organizations and coalitions in the global south, with particular attention to sub-Saharan Africa. She discussed the challenges facing small-scale, community based health service providers - as well as multi-sector advocacy coalitions -- as they add evaluation to their policy change advocacy work.   Rhonda Schlangen is an independent evaluation consultant specializing in the evaluation of advocacy projects and campaigns.

12.11.09 Unique Methods in Advocacy Evaluation with Julia Coffman

Like all evaluations, advocacy and public policy evaluations can draw on a familiar list of traditional data collection methods. But advocacy and policy work is often complex, fast-paced, and dynamic. That makes data collection challenging. And advocacy efforts often aim for outcomes that are hard to operationalize and measure, So evaluators are developing new and innovative methods specifically for assessing advocacy and policy efforts. Julia Coffman of the Center for Evaluation Innovation will introduce us to new methods designed to be relevant, timely and efficient: call it the trifecta of practical evaluation for busy advocates.

9.25.09 Measuring Political Will and Defining and Measuring Policymaker Champions with Craig Charney and David Devlin-Foltz

APEP convened colleagues and friends from the world of policy advocacy and evaluation for the first of our Advocacy Evaluation Breakfasts from 8:30 to 10:00 on Friday, September 25 at The Aspen Institute, Suite 700, One Dupont Circle NW.  The session featured political scientist Craig Charney, president of Charney Research, on Measuring Political Will.  He explored how to identify and quantify the holy grail of advocacy efforts.  APEP Director David Devlin-Foltz presented on defining and measuring what it means to be a "champion" for an issue.