Earthkeepers: The Relationship between Instructional Strategies and Program Outcomes

The study focuses on the relationship between selected characteristics of the outdoor environmental (earth) education program Earthkeepers and its impacts on students' environmental values and knowledge. It is rooted in the authors' previous research that is applied and reflected here. For providing empirical evidence, it uses data collected by the environmental center Lipka that implemented the program in the Czech Republic in 2020-2021 (N=80). The program successfully influenced students' environmental values and understanding based on the analyses. Students described the programs' instructional strategies as highly encouraging, experiential, empowering, and straightforward. At the same time, student descriptions of the program correlated with the environmental values they held before the program; students who viewed the program more positively also tended to be more positively influenced by the program. Implications for the practice of outdoor environmental education are also discussed.


Introduction
Environmental education programs are not a black box. Their effects are shaped by an interplay of internal and external factors: place and settings, individual and collective features of its participants, teachers' competence and personality, and qualitative features of the programs themselves (Činčera et al., 2021).
This article provides a case study of an outdoor environmental (earth) education (OEE) program, Earthkeepers. It is based on previous research analyzing the impact of selected instructional strategies (program features) used by programs at several Czech centers. Based on this, we begin by summarizing the findings from already published research. Then we switch to the Earthkeepers' instructional design description, focusing on the instructional strategies analyzed in this study. Finally, we provide the findings from the present study. We hope this analysis will help enrich understandings of the importance of carefully designing educational programs and spark further research in this important area.
The study focuses on selected instructional strategies that are crucial for establishing an OEE program (Real World Learning Model, 2021;Menzies et al., 2017). Specifically, these strategies focus on distributing power among the program stakeholders, communicating the programs' themes and overlying values, and facilitating experiential and conceptual learning.
On the general level, the study aims to contribute to the discussion on how OEE programs can be designed to develop students' environmental values and understanding.

Distribution of Power over the Program
The question of "who shapes the program" provides one of the fundamental dilemmas of OEE programs. There is evidence of the benefits of the "emancipatory" (Wals et al., 2008) approach for some environmental and sustainability education programs, primarily those based on whole-school or community-based approaches (Cincera et al., 2017;. However, the context of a few daylong residential outdoor environmental education may call for a different approach. Many experts recommend sharing power over the outdoor program among program leaders, teachers, and students (Menzies, Bowen-Viner, & Shaw, 2017, Real World Learning Model, 2020Thomas, 2010). These recommendations include asking program designers to discuss the program's aims and activities with teachers and then adjust them to the needs of the teachers and allowing students to make decisions about the specific program activities and their implementation.
Based on our previous research, this approach seems to be uncommon in outdoor environmental education programs. Most teachers prefer the "destination" model instead of active collaboration on the program. Only one of five programs we investigated students had a real opportunity to shape the program by their decision-making (Cincera, Simonova, Kroufek & Johnson, 2020). Neither students' nor teachers' satisfaction seemed to be much affected by the level of their control over the program. At the same time, we found a complex dynamic of power among the program designers, leaders, teachers, and students, when all the parties reported some dissatisfaction with the existing constellation and preferred more control. However, this aspiration was not necessarily connected with the program activities, as students seemed to prefer having more control over their free time in the residential program (Cincera, Simonova, Kroufek & Johnson, 2020).

Framing the Program
Other authors have suggested that programs should be straightforward in communicating their central theme, highlighting the values of universalism (Real World Learning, 2020). The idea of values communication is not without controversy; some authors are worried about the potential for indoctrination implied by values education (Sanera, 1998). However, other authors argue that no education is value-free, and environmental and sustainability education is by nature value-laden (Cairns, 2002;Jickling, 2010;Kopnina, 2012).
In our previous research, the program leaders and designers often apply well-prepared "surface frames" to motivate students to their program. However, the surface frames are often not well connected with the "deep frames," the value-laden themes to be communicated to students, or the program designers do not explicitly formulate the deep frames. Program leaders are, in some cases, unaware of what message their program should communicate and unsure if the program should or should not promote particular values (Cincera, Johnson, Kroufek & Simonova, 2020).

Experiential and Conceptual Learning
The learning process is another salient feature of OEE programs (Real World Learning, 2020). Students appreciate when the learning process is "experiential," while that is not always understood by the learners in the same way.
Teachers' theories of what "experiential learning is" seem to be in play, encapsulating the idea of promoting robust and transformational experience, learning by authentic experience emerging in the program, and learning from the process of elaborating and applying new knowledge emerging from program activities .
Finally, OEE programs typically include a goal of promoting conceptual understanding. The nature of these relatively short programs does not support the gradual process of uncovering and challenging students' alternative scientific theories, required by contemporary theories of conceptual change (diSessa, 2014;Duit & Treagust, 2003;Vosniadou, 2013). As a result, the OEE programs often apply various strategies such as "transferability" (linking the learned concept with other phenomena and with students' experience) or communicating the main idea the concept before engaging the students in an activity to assimilate it and then apply it (I-A-A model) (Johnson & Cincera, 2019;Cincera et al., 2021).

The Earthkeepers Program
Earthkeepers is a two-and-a-half-day residential program for 10-12 years old students. It focuses on developing students' ecological understanding and values to promote their responsible behavior (Van Matre & Johnson, 1988).
Designed to be a "magical learning adventure", Earthkeepers consists of an initial 2.5 day "springboard" experience at a center away from school with follow-through back at home and school to complete the program (Johnson & Manoli, 2020). The ecological concepts of energy flow, materials cycling, interrelationships and change are taught in participatory, outdoor activities designed to bring these abstract concepts into the concrete. Feelings are a second major component with their own outdoor activities. The follow-through at home and school engages students in adopting behavior changes to use less energy and fewer materials and to engage more with the natural worlds as well as sharing their experience with others.
The Earthkeepers program has been evaluated in many countries with a repeatedly demonstrated positive impact on students (Cincera & Johnson, 2013;Johnson & Cincera, 2015;Bogner, Johnson, Buxner & Felix, 2015;Dettman-Easler & Pease, 1999;Manoli et al., 2014). In this study, we focus on how specific program characteristics influence students' learning. While the program was included in the original research, these characteristics could not be elaborated in detail due to the anonymization of the programs in the study. While the present study uses newly collected data, it also uses the authors' experience with the program.

Distribution of Power in Earthkeepers
In the project, we identified five main stakeholders in sharing the power of the programs: program designers, program leaders, students, teachers, and parents (Cincera, Simonova, Kroufek & Johnson, 2020). In Earthkeepers, the power is firmly positioned on the side of the program designers. The program has been designed and developed by the Institute for Earth Education in the United States and shared with other centers worldwide. The program leaders are supposed to use all of the program storyline, organizers, and activities. Teachers, students, or parents cannot influence the components of the initial part of the program, but they can make many decisions on the followthrough portion, necessary for completing the program.

Framing Earthkeepers
The Earthkeepers program has an elaborated set of frames, linking the surface and deep frames. On the general level, the program applies an apprentice-master model when students are offered to play the role of "apprentices" of a mysterious wise "master" Earthkeeper named E.M., revealing to them the fundamental principles for a responsible life. Moreover, the program applies a set of additional frames for each activity, linking the sometimes fairy-tale-like stories (surface frames) with their deeper meaning.
The program is very explicit in communicating the values it wants to inculcate; the most vital communication seems to be the value of universalism (Schwartz, 2012).
The effect of this strategy seems to be two-fold: to precisely communicate the program messages and underlying values and to motivate students to participate in the program and its activities. Based on the previous program evaluation, participants tend to be highly satisfied with the program, and even a year after their participation, recall it as a significant and unforgettable adventure (Johnson & Cincera, 2015).

Experiential and Conceptual Learning in the Earthkeepers
The program applies some experiential and conceptual learning principles, interpreted in an original way. The program provides limited opportunities for reflecting and elaborating on the experience, such as reflective diaries for students' written reflection. However, the program strongly resonates with the transformational interpretation of experiential learning theories. The extraordinary nature of what students experience in the program is supported by elaborate mechanisms, such as night rituals, mysterious features of E. M., and other elements described in the associated earth education publications as "magic" (Wohlers & Johnson, 2003;van Matre, 1990).
The program applies a specific instructional model for conceptual learning -I-A-A (Van Matre, 1990). Students are initially introduced to the concept, then participate in an activity to assimilate it, and finally apply it to the natural environment close at hand. In addition, all of the main concepts in Earthkeepers (energy flow, material cycles, change, and interconnectedness) are mutually connected, linked with the program frames, and supported by visual means. The effectiveness of the program on conceptual learning has been shown in several studies (see Cincera & Johnson, 2013).
At the same time, the I-A-A model does not correspond with the recent conceptual change theories (diSessa, 2014;Duit & Treagust, 2003;Vosniadou, 2013). Students are not asked to reflect on their initial understandings of the concepts or the way those change after the program. In a previous study, we found that students' understanding of the learned concepts may be inadequate, mixed with alternative theories, or regress to the original theories in time (Johnson & Cincera, 2019).

Methodology
In this study, we wanted to apply the previously investigated findings to better understand the relationship between the program characteristics and effects in one particular program. Specifically, the study analyzes a) the effect of the Earthkeepers program on students' environmental values and knowledge and b) the effect of the applied instructional strategies on the program's outcomes.
We cooperated with the environmental education center Lipka, which started to implement the Earthkeepers program in 2020 on the data collection. All the data come from the students who participated in the program implemented by this organization in 2020-2021. Because of the pandemic, the program was implemented with only five groups, three times as a summer camp and two times as a residential program for school classes. For practical reasons, the data were collected immediately before the program (pretests) and immediately after implementing its initial, residential part (posttest).
After cleaning the data from incomplete answers, we paired data from 80 respondents who participated in the program in the observation period. In both pretest and posttest, we used the 2-MEV scale for measuring students' environmental values of nature preservation, utilization of nature, and appreciation of nature (Bogner, 2018). The instrument's composition and reliability are carefully described in Johnson and Cincera (2021).
To measure students' environmental knowledge in both the pretest and posttest, we used the Ecological Concept Questionnaire (ECQ) instrument, which is often used in earth education research (Cincera & Johnson, 2013;Manoli et al., 2014). The instrument consists of 13 closed response (multipole choice) items about the concepts of energy flow, materials cycling, interrelationships, and change.
In addition, in the posttest we added scales focusing on students' perception of the program characteristics and their overall satisfaction with the program. The scales focused on program clarity (associated with analyzing the effects of framing the program), students' participation at the decision-making in the program (empowerment), leaders' encouragement of students to act pro-environmentally (encouragement), quality of experiential learning (experiential learning), and students' overall satisfaction (satisfaction) with the program. The scales have been thoroughly described and their reliability demonstrated in Johnson and Cincera (2021).
The findings are based on a relatively low number of students participating in the program in one center, limiting their generalizability. The strategy to collect data in a brief period between the pretests and posttests provides other limitations: the programs' effects on students' learning (namely on their understanding) would likely be lower when evaluated a few weeks later. At the same time, the minimal time allowed us to capture the students' immediate interpretation of the instructional strategies, unaffected by other external factors (e.g., parents' opinion on how students reflect on it).

Findings
Overall, students viewed the program very positively. The most salient features were • the way it encouraged them to enact responsible environmental behavior, • the applied experiential way of learning, and • the way it linked their experience with further aspects (transferability).
As can be seen in Figure 1, students were highly satisfied with the program, with a mean score of 4.35 on a 5-point scale for overall satisfaction with the program. They felt that the program contained all of the progam characteristics in our study, though some were stronger that others (see Fig 1). Encouragment (m=4.57) was highest, followed by experiential learning (4.35), transferability (4.26), empowerment (3.90), and meaningfulness (3.69).

Figure 1 Students' views of the programs' characteristics
Note: 1 indicates the lowest level of the perceived characteristic, 5 the highest After the program, students' nature preservation scores increased significantly and utilization of nature decreased (see Table 1), both moving toward a more environmentally friendly value. This supports the expected positive impact of the program on students' environmental values. The size of the effect could be interpreted as moderate in both cases. There was no statistically significant changes for the values of Intent of Support and Appreciation of Nature, which began as environmentally friendly. Similarly, the knowledge test revealed a positive impact (see Table 2). The effect size can be considered as modest while still statistically significant.  Note: *statistically significant, p<0.01 Before the program, students' ecological knowledge and the values of preservation positively correlated (r=.260), while their knowledge and values of utilization correlated negatively (r=-.336). After the program, the correlations were stronger, preservation and knowledge (r=.318) and utilization and knowledge (r=-.507). Students with more environmentally friendly values of preservation and utilization scored higher on the ECS knowledge test, especially after the program. There were no statistically correlations between knowledge and the values of Intent of Support or Appreciation of Nature.
There were no significant correlations between ecological knowledge and the program characteristics perceived by the students. However, the program characteristics correlated with the students' pre-program values of preservation and appreciation and with their post-program values of preservation, intent of support and appreciation but not utilization. The correlations were stronger after than before the program. Students who viewed the program characteristics more highly also had more environmentally friendly values (see Table 3).
While the program's instructional approach does not correspond with the existing recommendations for facilitating the conceptual change, it effectively develops students' ecological knowledge. There may be a few possible explanations. First, the instrument used for assessing students' knowledge may not fully reflect the complexity of students' scientific theories. It may mean that students learn some aspects of the concepts but not all aspects. As a result, students may hold two contradicting theories together -such synthetic models may explain why, in the previous research, some of the students did not keep the new concepts and came back to their original alternative theories (Johnson & Cincera, 2019). We assume the program plays a significant role in developing this knowledge, while the full potential is likely undercut by the limited length of the residential program. Another explanation is that the instructional strategy, based on transmitting, assimilating, and applying the concepts, works for teaching scientific concepts in outdoor settings. Such explanation may highlight the importance of plurality in environmental education.
The relatively strong negative correlation between students' values of utilization and ecological knowledge resonates with other research (Baierl, Johnson & Bogner, 2021;Schneiderhan-Opel & Bogner, 2021). Students who support a more anthropocentric perspective (values of utilization) may be less interested in learning ecological concepts. However, the opposite may be true, that the lack of ecological understanding supports the anthropocentric values held by students.
This study further supports the crucial relationship between the program characteristics, students' values, and program effects. As found in a previous study (Johnson & Cincera, 2021), students with more pro-environmental values tend to rate the program's instructional strategies more highly than do students holding more anthropocentric values. This relationship works for environmental values before the program and even more so after the program. It shows the complex nature of the role of the applied instructional strategies. It seems that the way students interpret the instructional strategies is related to how the program content corresponds with their values. If such a correspondence exists, the program seems to support these values; if not, its effect is likely insignificant.
At the same time, while some of the instructional strategies seem to be more influential than others, identifying the salient factors for program design is not straightforward. As we can see, students rated all the perceived strategies as relatively high, regardless of their actual implementation in the program. This is obvious for the "empowerment": while students had a relatively limited chance to decide "what to do" in the program, they felt the opposite. There are several possible explanations for this. Students might not feel the need to be more involved in decision-making in the program due to their overall satisfaction with the program, or they felt that they were provided with enough opportunities for choice within the constraints of a set program. Based on this, other program characteristics, such as the emphasis on experiential learning (highly evaluated by students), other instructional strategies not investigated in this study (e.g., the intensity of the program), or other internal or external factors (e.g., students' age, aesthetic values of the area) might reduce students' need for a more participative approach; they could be happy with the somewhat limited opportunity for decision-making the program allowed. In addition, from the perspective of students with initial high levels of nature preservation values, the program offered what they wanted, and they had no reason to change it. Furthermore, compared to the typical instructional strategies used in school classrooms, students might feel even the limited opportunity to participate as relatively high.
It seems that no single instructional strategy is crucial for enhancing students' learning in OEE programs. Instead, the effect seems to result from an interplay among students' characteristics and the complexity of mutually influencing instructional strategies, likely further influenced by other factors not investigated by this study (group cohesiveness, area, weather condition). It may be the consistency of the strategies, the way they support each other, and the strategies alone that make a difference. As we can imagine, if Earthkeepers consisted of "boring" lecture-type presentations; it would not be evaluated as participative, even if the actual level of students' involvement in decisionmaking remained the same. Similarly, we may hypothesize that if the program did not communicate its messages, it would not correspond with the environmental values of its participants, and they would not interpret it as they did. If this theory is correct, it shows the relatively low value of the lists of quality criteria such as thoseprovided by the Real World Learning Model (2021) or other documents (Menzies et al., 2017). Based on this, the complexity, not the single strategy, matters.
An additional comment is relevant regarding students who prefer the more anthropocentric value of the utilization of nature. Based on the theory mentioned above, the complex instructional strategies supporting the environmentally-focused students may not as deeply affect the students whose environmental values would be the most meaningful to develop. It opens a question on if and how this could be changed. While more profound changes in the instructional design (such as the value-neutrality in the program frame) may be positively received by this group of students, it may also compromise the program's effectiveness on the pro-environmentally thinking participants. If no easy solution can be found, it might lead to a pragmatic solution "not to do the programs for everyone" and do not aspire to have a more profound effect on those who do not share the values communicated by the program. However, the pragmatic solution may have broader implications, as it may support the divisions and polarization in contemporary society ever since a young age. Besides, as can be seen, the level of utilization of nature decreased after the program in the group. To assess this, further research focusing specifically on students with high values of utilization of nature would be needed.
The way outdoor environmental education programs might deal with this dilemma needs to be investigated in further research.

Conclusion
The study highlights the importance of a complex approach for designing and investigating outdoor environmental education programs. It supports the understanding of the programs as a complex web of relationships, rather than a black box, or even the black box interpreted as a set of specific quality criteria and recommendations for practice. It seems that no single instructional strategy or program characteristic is vital for the program per se; the way the applied strategies support each other and correspond with the participants is what matters.
At the same time, the study confirms the crucial role of students' initial environmental values in interpreting the program quality and their environmental learning. Students are "black boxes"what they bring to the program helps them resonate or oppose what they will experience. As a result, the study highlights the unexpected implication: those who could and should learn more are usually not those who learn most.