Measuring the Impact of Youth Voluntary Service Programs:

An International Experts Meeting

ICP partnered with the Children & Youth Unit at the World Bank to convene an international experts meeting on measuring the impact of youth voluntary service programs. The meeting will was held on May 8-9 2008 at the World Bank headquarters in Washington, DC. It brought together over 50 researchers, policymakers and practitioners from the field of youth civic engagement to examine existing evidence for the impact of voluntary service on participants, explore different evaluation methods, identify gaps in the research, and develop a research agenda to address these gaps.

A full report based on the proceedings of the meeting will be compiled and made available here and on the World Bank website shortly. Please see below for details on the meeting, including background, objectives, and the agenda. Links to presentations delivered at the meeting are also available below.

Click here to see pictures from the meeting.
Click here to download biographies of speakers at the meeting.

Background

Youth voluntary service programs exist in dozens of countries around the world, and new programs and policy initiatives are currently being developed in many others, often with the help of international organizations like UNICEF, UNV/UNDP, and International Youth Foundation. This will result in many more young people being offered the opportunity to engage in youth service. Already, fully one quarter of young respondents to the World Values Survey in 2000 report some voluntary action with religious or secular organizations that promote social welfare, conservation, human rights, and many other goals. These programs attract a great deal of attention, and potentially enormous resources. In December 2007,

presidential candidate Barack Obama said that expanding opportunities for national service would be �a cause of (his) presidency�. In 2008,

Time magazine made the case for National Service in its cover story and a call was issued in the American Prospect for a national youth service program, to enroll one million young Americans by the year 2020, at a cost of more than $14 billion the per year, in today�s dollars. We � the community of researchers, policymakers, and practitioners � have an obligation to make sure that we are doing the right thing in supporting these programs, and that we have the tools necessary for evaluating their impact.

Overview

The World Bank and ICP organized this meeting as an opportunity to bring together, researchers, policymakers and practitioners from the field of youth civic engagement and particularly youth service, to assess the existing research on the impact of youth service programs on young people, explore different evaluation methods, develop a draft evaluation framework, identify gaps in the research, and develop a research agenda to address these gaps. This meeting was intended to initiate a conversation among an international group of researchers, practitioners and policymakers about the evidence base for youth service as a strategy for positive youth development.

Objectives:

� Discuss methodological approaches for evaluating the impact of youth voluntary service

� Outline the main challenges in evaluating the impact of youth voluntary service programs

� Review the current research and available evidence on whether and how service contributes to youth development

� Identify the gaps in existing research on the impact of youth voluntary service programs

� Develop a draft framework for evaluating youth service programs, and

� Determine a plan to expand the evidence base in ways that are consistent with the highest standards in quantitative and qualitative research and also useful to policymakers and practitioners in promoting best practices in youth service; identify the roles of different actors in this process.

Outputs:

� Draft framework for evaluating youth voluntary service programs, including discussion of appropriate methods, indicators and outcomes

� Synthesized recommendations, action plan, and endorsed commitments for moving the youth voluntary service research agenda forward

� Final meeting report summarizing presentations, discussions, and research agenda to be published and distributed through WB and ICP website and in print

� Dedicated space on World Bank Children and Youth and ICP�s website will be used as a tool for building an international community of interest around the youth voluntary service program evaluation and for distributing knowledge

� Establishing a working group to carry on the work and keep the topic in the agenda of all the organizations that commit to working on it

Outcomes:

Short Term

� Shared understanding of the current evidence base

� Consensus about what evidence is lacking

� Development of a framework for evaluating youth service programs

� Creation of a roadmap for expanding the evidence base

Long Term

� Increased resources for research on the impact of youth service as a youth development strategy

� Rigorous evidence on whether and which programs are most successful

Agenda

Day One � May 8, 2008

I. Welcome & Setting the Stage: background and objectives of the meeting

1. Opening Remarks

- Wendy Cunningham, Advisor, Children and Youth unit, The World Bank

2. Definition and characteristics of youth service programs along the continuum of volunteering and service. Potential positive impacts of youth service on participants and the importance of measuring these impacts more effectively.

- Susan Stroud, Executive Director, Innovations in Civic Participation. Presentation

3. Framing the discussion on evaluation of youth civic participation and youth voluntary service programs. The importance and challenges associated with conducting evaluations to produce rigorous evidence of a program's net impact. Why are youth service programs not considered �proven� to be effective in the World Development Report 2007?

- Mattias Lundberg, Senior Economist, Children and Youth unit, The World Bank

II Evaluation Methods and their application to Youth Voluntary Service Programs

Chair: Shahrokh Fardoust, Sr Adviser, IEGDG, The World Bank

Presentations:

Ariel Fiszbein, Chief Economist, HDN, World Bank. Controlled Impact Evaluation. Presentation
Michael Patton, Director, Utilization-Focused Evaluation, Alternative Methods for measuring the impact of Youth Voluntary Service Programs. Presentation

Topics: What needs to be proven? Can it be proven and if so, how? What are acceptable standards of evidence?

Discussants:

Eva Mysliwiec, Founder, Youth Star Cambodia. Practitioner�s perspective. Presentation
Busani Ngcaweni, Senior Policy Analyst, Policy Coordination & Advisory Services, Office of the President, South Africa. Policymaker�s perspective
Victor Karunan, Senior Advisor, Adolescent Development, and Participation (ADAP), UNICEF. International Agency�s perspective

Open Discussion

Topics: What is the distinction between methods and approaches? What are the needs for evaluation from practitioners�, policymakers� and funders� perspectives? What needs to be proven and what are acceptable standards of evidence? What are the challenges?

III Existing Evaluations and Challenges

Session A

Chair: Varun Gauri, Senior Economist, DECRG, The World Bank

Presentations:

1. Kevin Cramer, Deputy Director, Research and Policy Development, Corporation for National and Community Service. Presentation

2. Ami Thakkar, Program Director, International Youth Foundation. Presentation

3. Licio Palazzini, Director, Arci Servizio Civile. Presentation

Presenter-Discussant:

4. Steve Powell, Country Researcher, PROMENTE, Bosnia. Presentation

Plenary Q&A

Session B.

Chair: Ron Kassimir, Associate Provost, The New School

Presentations:

Manish Subharwal, Director of Monitoring and Evaluation, and Nicholas Hartley, Director of Operations & Programmes, Student Partnership Worldwide. Presentation

Ryan Cooper, Evaluation Coordinator, "Un Techo para mi Pais" project in El Salvador and Peru. Presentation
Tim Cross, President, YouthBuild International/COO YouthBuild USA. Presentation

Presenter-Discussant:

Peter Levine, Director, The Center for Information and Research on Civic Learning and Engagement, CIRCLE

Plenary Q&A

Chaired Small Group Discussion:

Discussion on the existing evidence base and challenges of evaluating the impact of volunteer programs. Each group will focus on one of the main research topics identified during the preceding plenary discussion. Total of 6 groups.

Day Two � May 9, 2008

I Knowledge and Research Gaps

Review on the existing evidence base and key research questions remaining
Comments and inputs on the presented review, and identifying the main gaps in knowledge and research

Chair: Andrew Furco, Associate Vice President for Public Engagement, University of Minnesota

Discussants:

1. Constance Flanagan, Professor of Youth Civic Development, College of Agricultural Sciences at Penn State University

2. James Youniss, Professor of Psychology, Catholic University

3. Yuanzhu Ding, Director, Research Center for Volunteering and Welfare, Peking University. Presentation

4. Lonnie Sherrod, Professor of Psychology and Director of Applied Developmental Psychology Program at Fordham University. Presentation

Facilitated Discussion

Outcomes: Assessment of the existing evidence base and gaps in it

II Evaluation Clinic for upcoming youth service projects

Case studies to be reviewed and commented by a panel of experts

Clinic I: South-African National Youth Service

Chair: Arianna Legovini, Lead Monitoring and Evaluation Specialist, AFTRL, World Bank

Presentation on the basic features of the project,

� Busani Ngcaweni, Senior Policy Analyst, Policy Coordination & Advisory Services, Office of the President, South Africa. Presentation

Considerations for an Evaluation Design,

� JoAnn Jastrzab, Principal Associate, Abt Associates

Open Discussion: floor opened by discussants' comments on the evaluation:

� Michael Baizerman, Director of Youth Programs, University of Minnesota

� James Radner, Executive Director, The Boreal Institute for Civil Society, Munk Centre for International Studies, University of Toronto.

Clinic II: Youth Star Cambodia Program

Chair: Markus Goldstein, Senior Economist, AFTPM, World Bank

Presentation on the basic features of the project

� Eva Mysliwiec, Founder, Youth Star Cambodia. Presentation.

Considerations for an Evaluation Design

� Felipe Barrera, Senior Education Economist, World Bank. Presentation.

Open Discussion: floor opened by discussants' comments on the evaluation:

� Alan Melchior, Deputy Director and Senior Fellow, Center for Youth and Communities, Brandeis University

� Chris Blattman, Yale University

III Research & Evaluation Framework and Agenda

Evaluation Framework:

Presentation on the draft Evaluation Framework for building further evidence

� Charmagne Campbell-Patton, Program Associate

Research and Evaluation Agenda:

Discussion on the Research and Evaluation Agenda: Roles and Responsibilities and Next Steps
Small Group Discussions on the Agenda
Inputs and suggested changes on the Agenda and Commitments to moving the Agenda forward.

Outcomes: New draft of the Evaluation Framework and Agenda for moving forward

Closing Remarks

Mattias Lundberg, Senior Economist, Children and Youth unit, The World Bank
Susan Stroud, Executive Director, ICP