A Case Study from Plimoth Plantation

How Do We Know Our Students Are Engaged?

A Case Study from Plimoth Plantation

by Lisa Neal and Kim Van Wormer

Plimoth Plantation's You Are the Historian online learning center (OLC) teaches about "The First Thanksgiving," which the museum more accurately refers to as the 1621 harvest celebration. In designing the OLC, we grappled with how to develop engaging materials which encouraged exploration and learning. We started off trying to find an appropriate balance between learning and fun. We soon realized, however, that it wasn't a question of balancing learning and fun, but rather making learning fun.

Plimoth Plantation, located in Plymouth, Massachusetts, is the "living history" museum of the English colonists, the Wampanoag Native People, and the groups' interaction in 1620s New England (Figure 1). The museum employs immersive, recreated environments where visitors can encounter colonial "interpreters" who are dressed in period costumes, speak in dialect, and talk about their lives, and Native American interpreters who talk about and demonstrate Native life in the past and today.

Plimoth Plantation received Federal and private foundation funding to provide standards-based online education for children and teachers. The team that was formed to accomplish this, which included museum staff, teachers, and external consultants, faced two main challenges. One was how to bring the "powerful, personal experience" of physical visitors online, so that the OLC reflected the unique characteristics, as well as the expertise and perspectives, of Plimoth Plantation. Another was how to design a site that was engaging, fun, and educational.

Teaching for Understanding
To help overcome these challenges, it was decided early on to design the OLC using a pedagogical framework called Teaching for Understanding (TfU) [Perkins] that is based on the philosophy that understanding is built over time through a series of active "performances" of understanding. Using the TfU framework, the OLC team developed Understanding Goals that were tied to the 3rd and 5th social studies standards for elementary school children in the Commonwealth of Massachusetts and the US. The Understanding Goals drove the design of the OLC, ensuring that online activities were relevant and led to increased student understanding.

You Are the Historian
While it is important that children learn facts to meet social studies standards, it is arguably more important that they develop an understanding of how historians learn about and interpret history. The OLC team weighed the importance of this notion while noting the success of goal-based and scenario-based learning, where the learner is given a problem to solve [Schank]. With this in mind, we decided to challenge a visitor to the OLC to become a historian. In this role, they could investigate what really happened in 1621 by interpreting information about people and events, rather than just by reading text or performing isolated activities. The name of the site, "You Are the Historian: Investigating the First Thanksgiving," reflects this focus.

As historians, children learn how historians use primary sources and oral histories and how historical events are interpreted and reinterpreted over time. After completing the activities on the site, rather than taking the multiple choice quiz common in e-learning, children were asked to develop an online museum exhibit of captioned images. This goal-based approach is motivating to children since they work toward a defined goal and have tangible results which can be printed and shared.

Children as Guides and Interpreters
The OLC team wanted to replicate the immersive experience of Plimoth Plantation by incorporating people into the site. We thought that our target learner population of children would identify more readily with and be more curious about other children. Hence, two children, a Wampanoag boy and Colonial girl, act as guides in their modern attire (Figure 4) and as interpreters in their 1621 attire (Figure 2).

Figure 2: 1621 Children

"Visit the Expert"
In addition to the children, Plimoth Plantation's expertise is embedded in the site, with museum personnel offering their relevant perspectives on every page of the OLC. "Visit the Expert" provides information that would rarely be available to an actual visitor, and provides the user with a deeper understanding of the work of historians. This feature uses short audio clips supplemented by text and a photograph (Figure 3).

Figure 3 "Visit the Expert" on "Fact or Myth" page

Engagement and Fun
The OLC team critiqued online courses and web sites for children, looking for what was engaging, fun, and educational. While the successful ones varied considerably in their design, they drew the user in and had an emotional appeal. Norman points out that "fun and pleasure are elusive concepts" [Norman]; while engagement is accepted as important for effective e-learning it is similarly elusive. In a rubric used for evaluating the quality of web sites for children, ten criteria are specified but "engaging" is the only one without an explanation; it is circularly defined as "process engages the learner" [Blue]. The discussions about how to find a balance between learning and fun led to a realization that our goal was ultimately to make learning fun.

Some themes emerged in addition to challenging children to conduct an investigation as an historian, using children as guides and interpreters, and including expert opinions. These included the extensive use of graphics, audio, and video to provide visual and auditory richness; interactive and purposeful activities; variety; and rewards and surprises. These themes both borrowed from the best examples we found and avoided what seemed ineffective.

Specifically, in the latter case, we wanted to avoid the downfall of much e-learning: linear, heavily text-based courses offering little, if any, interactivity. We found many activities that were educational but not fun or fun but not educational. For example, in addition to educational materials, a web site about early US explorers includes a game in which children shoot at buffalo running across the Plains. It was fun but was neither educational nor seemingly relevant to the site's educational goals.

Extensive visual and auditory components were considered important to draw children in and to capture the essence of Plimoth Plantation. The site uses a rich color palette, a carefully chosen font, images, photographs, video, and extensive audio.

Figure 4 Modern Children on home page

Exploration
Most pages provide layers of information, in contrast to the linear approach more common to e-learning. Hidden information and directions are discovered by moving the mouse over text (known as mouseovers). The "English Colonists" page has the most layers and mouseovers, starting with a photograph of the Plimoth Plantation village. When one of the houses is selected (through a hidden link), the next screen shows the interior of a home, where mouseovers provide more information about objects and Q&As provide further information about aspects of daily life. There is information about a limited number of topics; these were selected as the ones most likely to appeal to children's curiosity about life then and include information about food, sleeping arrangements, children's work, and even chamberpots.

Rewards and surprises
Some user actions are rewarded by audio, video, text, or animations, such as after an activity is complete or when an object is selected or dragged to another location. "The need to add fun elements that relate to critical features of an existing activity" increases engagement [Dix] and contributes to a sense of accomplishment, as well as providing variety. Rewards are common, if not mandatory in games, but are not often used in e-learning.

The rewards differ on each page. For example, in Figure 3, the "Fact or Myth" page, a turkey is visible on the left. Once the captions are all correctly placed, the turkey runs across the screen saying "Gobble, gobble," providing a humorous reward. In the "Primary Sources" page, one activity uses sticky notes to provide explanations of phrases in a primary source document. When all of the explanatory notes have appeared, a final sticky note appears with a smiley face that winks once. In the "Wampanoag" page (Figure 5), when a child has selected all twelve stones, he or she is rewarded by a video.

Figure 5 the "Wampanoag page"

Activities
Each page is interactive in the sense that there are links to select and activities to do. Each page differs in the type of activity provided, and some provide multiple activities. The "Primary Source" page, for example, includes a "magic lens" that can be dragged over a primary source document, showing, within the lens, a translation of the text to modern English. On the "Wampanoag" page (Figure 4), which emphasizes the importance of giving thanks as a part of daily life throughout the seasons, children select stones to learn about an aspect of Wampanoag life through text, audio, and pictures.

The Impact of Formative and Heuristic Evaluations and Usability Testing
Evaluation was considered an essential part of the design and development process. A formative evaluation was done early in the process. A heuristic evaluation was undertaken once development of the site was nearly complete. Usability testing was done at the end.

Formative Evaluation
A formative evaluation of the first version of the OLC, done in collaboration with the Harvard Graduate School of Education, led to significant redesign of the site. The formative evaluation was done by students, using almost 100 children and teachers, under the supervision of Dr. Ilona Holland. The evaluation found that many aspects of the OLC were effective at achieving the Understanding Goals as well as our goals of developing an appealing, engaging, fun, and educational site.

Reactions were very positive to the visual richness and the use of audio. However, we also found out that children and teachers wanted more activities, more audio, and a stronger role for the children as guides and interpreters. Children who took part in the formative evaluation identified with the children in the OLC and were curious to find out more about them. They liked the juxtaposition of the 1621 and modern children. Children liked the audio, especially when they were not strong readers. They reported that the one "game" that was developed was fun but they wanted more games and more they could actually do.

The subsequent redesign incorporated this data, enhancing the existing features that were especially appealing and addressing problems. The two children were given more prominent roles on every page. More activities were designed, so that each page had something that a student could do. These activities were carefully designed to allow children to be active, rather than passively reading text or listening to audio, and to be engaged in an activity or exploration that helped meet one or more of our Understanding Goals.

Heuristic Evaluation
Further evaluation was necessary as the project neared conclusion to insure that the redesigned site was usable by the targeted audience and that the learning objectives and goals were met. Three heuristic evaluators who had significant expertise with the design and testing of children's web sites and software, looked at the individual components of the site as well as the overall site. The features they were most positive about were the colors, layout, children, graphics, and audio. They made many minor suggestions for improvements, as well as a few more significant ones. For example, the two children were given more prominent roles on every page and were photographed again in more active poses to further engage children. The heuristic evaluator's in-depth understanding of how children think and learn, as well as how children approach web sites, proved invaluable for improving, and especially for increasing the interactivity of, the OLC.

Usability Testing
Once the changes resulting from heuristic evaluation were made, final usability testing was done using children and teachers. Children were watched carefully to see what they did, how they reacted, and if they encountered problems. As an education site, we were interested in what testers had learned, not just how they reacted or where they encountered difficulties. Before and after their use of the site, we asked children and teachers to tell us what they knew about the events of 1621 and to define terms such as "myth", "historian", "Wampanoag", and "primary source". We were pleased when a child demonstrated greater or deeper understanding following their use of the site.

Conclusions
From the initial phases of design, we grappled with the best way to make learning engaging and fun for children. In redesigning the OLC following formative evaluation, we struggled with how to add the additional activities children requested and that we knew were needed. Some of their suggestions from the formative evaluation were skewed toward increasing interactivity rather than learning. For instance, one child wanted "a runaway turkey game…you have to press forward and backward to run away from the bullets." Another requested "a game that teaches you to sail a ship or a game where you guide the ship through obstacles." While these and other ideas were loosely related to Plimoth Plantation or Thanksgiving, they did not further the OLC's education goals.

The final version incorporated new activities that brought in the desired interactivity in a manner that was consistent with our Understanding Goals and with the goals for the site. Other enhancements, to further engage children, include increasing the sense that the child is investigating a mystery and drawing children in using an intriguing question, puzzle, or riddle on the loading pages. We avoided the more linear approach typical of e-learning sites: each page was different, providing activities, rewards and surprises, and layers of information. We feel that this approach draws the learner in, and does not sacrifice learning for fun or fun for learning. The act of learning, itself, becomes fun.

The OLC, available at www.plimoth.org/OLC, was released in Fall 2003. Feedback from the actual use of the OLC has been extremely positive to date. While we have received extensive feedback from educators, parents, and librarians, we especially enjoy the feedback from children. One child wrote in the feedback form, "This web site rocks. I got a lot of information from it and it helped me with my report. Thank you!!!!!!!!!!!!!!!!!" Another wrote, "I loved your website it was the best! I am 12 and I don't like history so it helped me enjoyed history! Thanks!"

The OLC has received awards, including the Massachusetts Interactive Media Council (MIMC) 2003 award for best education site and has been "Site of the Day" or "Site of the Week" in a number of places. While we appreciate these opportunities to have more people find out about the OLC, we are especially gratified since this recognition shows the perceived quality of the OLC.

Acknowledgments

We thank the many people involved in the design, development, and evaluation of the OLC. This project was supported by a congressionally directed grant administered by the Institute of Museum and Library Services, by a grant from the Arthur Vining Davis Foundations, and by other generous donors.

References

Blue Web'n Site Evaluation Rubric, http://www.kn.pacbell.com/wired/bluewebn/rubric.cfm.

Dix, A. (2003) Being Playful – Learning from Children. In Proceedings of the 2003 Conference on Interaction Design and Children. New York: ACM.

Norman, D. (2004) Emotional Design. New York: Basic Books.

Perkins, D. (1993) "Teaching for Understanding" In American Educator, v. 17 n3, Fall, pp. 8,28-35.

Schank, R. (1997)Virtual Learning: A Revolutionary Approach to Building a Highly Skilled Workforce. New York: McGraw-Hill.

About the authors

Lisa Neal is Editor-in-Chief of eLearn Magazine and an Adjunct Professor in The Department of Family Medicine and Community Health at Tufts Medical School. Lisa consults on e-learning projects with museums and corporate, academic, and government clients. She is a frequent presenter and tutorial instructor at conferences. Lisa was awarded the 2003 U.S. Distance Learning Association (USDLA) Award for Most Outstanding Achievement by an Individual in Corporate/Business. She holds a Ph.D. in Computer Science from Harvard University. Email: lisa@acm.org

Kim Van Wormer is director of education at Plimoth Plantation, P.O. Box 1620, Plymouth, MA 02362 USA. Email: education@plimoth.org

© March 2004 New Horizons for Learning
http://www.newhorizons.org
info@newhorizons.org
For permission to redistribute, please go to:
New Horizons for Learning Copyright and Permission Information