 |
 |
 |
||
 |
 |
 |
|
|
 |
|
|||
 |
 |
|
||
 |
|
|||
 |
|
|||
You
are here:
Taking Stock of Our Future
Program Evaluation and Assessment of Global Issues Education
by Kim Rakow Bernier and Gilda Wheeler
Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. - Albert Einstein
Albert Einstein reminds us that the business of evaluating programs is a complex and at times daunting endeavor. Despite the challenges, determining the impact of education programs and curricula on teachers and students is no longer optional. Organizations providing resources to teachers must be able to show that their materials measurably enhance the learning experience.
Facing the Future: People and the Planet is a nonprofit organization that develops and distributes global issues curriculum and action opportunities to teachers and students throughout the United States and the world. Facing the Future's curriculum teaches students to think critically about the relationships among environmental degradation, social equity, population, consumption, and peace and conflict. It promotes understanding of how our personal choices affect these issues locally and globally, and empowers students to take positive action to solve community problems.
Facing the Future has integrated a multifaceted evaluation process into its program to gather information from its stakeholders, and to ultimately drive program and organization direction. The evaluation process is designed to collect information from teachers and students at varying points during their contact with the program.
Designing Program Evaluation
Facing the Future's program evaluation process is iterative and includes defining desired program outcomes, program design, implementation, assessment, and revision. These steps are described in the remainder of this article with sample results from the assessment we conducted in 2003. The assessment includes surveys of all educators who receive Facing the Future curriculum and a more in-depth evaluation of teachers and students engaged in our Washington Teacher Partnership Pilot Program. Figure 1 describes the assessment tools we use to provide us with a range of quantitative and qualitative data on the efficacy of our global studies program.Facing the Future Assessment Tools and Uses
One page survey Workshop attendees At the end of each workshop (approx. 1,000 workshop attendees per year) On-line survey sent as a link via email Educators who have downloaded, purchased or received FTF curriculum at a workshop or from our website Two times per year (approx. 2,500 per year) One page pre/post workshop surveys Washington State Lead Teachers One 8 hour workshop (20 Lead Teachers) On-line survey after using FTF curriculum and service learning project Washington State Lead Teachers One time at program completion (20 Lead Teachers) One page pre/post survey Washington State Lead Teachers' students Ongoing - (at least 500 students) Email and in-person interviews Washington State Lead Teachers and students During '03-'04 school year Figure 1: Facing the Future's assessment tools
Linking Evaluation to Program Outcomes
Before developing a program or activities, an organization has to define its vision of the impacts and outcomes it hopes to achieve. Facing the Future's evaluation process is built around an assessment of our four major goals. They include:1) Teachers embrace global issues education and teach about the interconnections among them
2) Students and teachers internalize the interconnections among population, poverty, consumption, the environment, and conflict
3) Teachers use service learning and action as a tool to empower youth
4) Students engage in the world through actionWe incorporate these goals into our organizational inputs, activities, outputs, and outcomes to help guide the development of indicators and survey tools. For example, in order for us to meet goal #1 (see above), we have to know if Facing the Future curriculum is an effective vehicle to teach about the interconnections among global issues. The survey results indicate that 97% of educators believe that Facing the Future offers effective tools to teach about global issues and their interconnections.
In the context of the federally mandated 'No Child Left Behind' Act, an equally important indicator of Facing the Future's efficacy is its ability to meet national and state education standards. 73% of educators believe Facing the Future curriculum fits within their state's educational framework and helps meet state standards (25% said they didn't know) and 79% believe our curriculum increases students' critical thinking skills – a skill that is addressed in multiple standards.
Figure 2: Percentage of educators that believe Facing the Future curriculum is aligned with standards
Another assumption that we are able to test by surveying teachers after our workshops is whether or not teachers have the knowledge needed to teach about global issues in their classrooms. In the teacher workshops we conducted during the first half of the 2003-2004 school year, 96% of educators agreed that the workshop increased their knowledge about the interconnections between the global issues of environment, human populations, poverty, consumption, and conflict, 99% of those educators learned effective and engaging strategies for teaching about the interconnections of global issues, and 97% learned effective strategies and tools to empower their students to take action on global issues.
Designing an Informed and Relevant Program
Like many nonprofit organizations, Facing the Future's program and materials were in use prior to implementation of the current evaluation process. As such, the evaluation process is used to verify that the program is meeting the organization's desired outcomes, and if not, to guide program redesign.Facing the Future strives to have global issues education taught in a variety of subject areas, as we believe that addressing global issues and sustainable living demands an interdisciplinary approach. Before the 2003 survey, we lacked comprehensive evidence of the subject areas in which Facing the Future curriculum was being used. The study's results showed that our global issues and sustainability curriculum is being used by teachers in the following subject areas (see Figure 3).
Figure 3: Subject areas where Facing the Future curriculum is in use
These findings will inform our future plans for curriculum modifications and program marketing and expansion.
Assessing and Revising the Program
Program implementation and assessment are integrated in Facing the Future's program. This allows us to change our materials and programs mid-course as well as plan for future program redesign that incorporates our learning. We are using data collected from the surveys as we begin a more comprehensive review, update, and edit of Facing the Future student issues guides in 2004. Teachers have provided us with concrete ideas about activities and content they feel could enhance our curriculum activity guide which will also inform its update in 2005.
Student at John Hay Elementary, Seattle, WA, studying FTF book
One especially significant and useful finding is in regard to the service learning component of the program. Of the teachers we surveyed, 93% believe that service learning is an important and effective teaching method, but only 31% are using service learning in their classroom. Teachers report that the main reason they are not doing service learning is that they lack time to research service learning opportunities and organizations. We were able to respond to this important need by providing previewed and engaging service learning and student action opportunities on an online database that links action with the curriculum. Our current data indicates that 80% of teachers believe Facing the Future's curriculum increases students' belief that they can make a difference in solving global problems. Now that we have launched the service learning database we will be able to collect more concrete information about how service learning impacts students' beliefs and actions about their role in creating a better future. We will also be able to determine how the service learning database is being used and how to build upon the pilot database in Washington as we expand to other regions.
The Role of Qualitative Data
As we are reminded by Einstein, not everything that counts can be counted. Ultimately our work is about students and the effect our program has on their learning. The words of teachers and students who have used Facing the Future curriculum and resource capture that impact in a personal and provocative way. For example, two students comment positively after participating in our sustainability activity, Fishing for the Future:I learned a lot about the delicacy of our environment. If we treat it with respect and only take what we need we will be supplied for generations to come. (5th grade student, Stevenson Elementary, WA)
What I found most interesting about the Facing the Future unit on global issues was how much impact each American has on the environment compared to people in much poorer countries. (6th grade student, Washington Middle School, WA)
Students at Seattle Girls School in Seattle, WA doing "Fishing for the Future" activity
Looking to the Future
We are currently engaging in a comprehensive evaluation through our Teacher Partnership Program and are exploring the possibility of conducting a longitudinal study to more fully address the impact of Facing the Future curriculum on student achievement. As a learning organization, evaluation, assessment, and ongoing monitoring will continue to shape the growth and development of our global issues curriculum and programming.More detailed survey results can be found on Facing the Future's website, www.facingthefuture.org.
Kim Rakow Bernier is the Outreach Director at Facing the Future: People and the Planet. She holds a Master in Public Administration and Certificate in International Development Policy and Management from the University of Washington. Before joining Facing the Future she worked as a coordinator for the Peace Corps Master's International Program at the Evans School of Public Affairs. She has conducted evaluation work and program design and development for two Seattle nonprofits on an educational technology initiative called Global Classmates, a project for which Kim was nominated for The Betty Jane Narver Degree Project Award at the Evans School. Kim taught math and science in a community high school in the Solomon Islands as a Peace Corps Volunteer and also worked at an independent K-12 school in Southern California. As Outreach Director, Kim conducts teacher workshops and oversees the educator, community, and partner outreach program. Email: kim@facingthefuture.org
Gilda Wheeler is the Program Director at Facing the Future: People and the Planet. She is the primary author of Facing the Future Curriculum Guide: Classroom Activities for Teaching about Global Issues and Solutions. Gilda oversees Facing the Future's education and training programs while continuing to develop curriculum and conduct educator workshops. She has led over 50 teacher workshops around the country and developed hands-on experiential curriculum pieces on global issues. Before joining Facing the Future, Gilda spent four years teaching language arts at an independent high school in California. Prior to her teaching career, Gilda worked for 10 years as an Environmental and Land Use Planner with the County of Santa Barbara, California. She holds an undergraduate degree in Geography, and a Masters in Education from the University of California, Santa Barbara, as well as California and Washington State Teaching Certificates.
© March 2004 New Horizons for Learning
http://www.newhorizons.org
info@newhorizons.orgFor permission to redistribute, please go to:
New Horizons for Learning Copyright and Permission Information
