Assessing the Efficacy of an Experimental Alternative Middle School: A Case Study, 2(13)
David C. Whaley
whaley@cahs.colostate.edu
Director, Educator Licensing Program
Colorado State University
Peggy Lofquist
lofquist@CAHS.Colostate.edu
Colorado State University
William Lamperes
Principal
Centennial High School
Guest editors from Nipissing
University
for partnership theme articles:
Douglas
R.
Franks
Mary
Ross Hookey
Helen
G.
Langford
Abstract
The Alternative Middle School represented a collaboration between two school districts, city governments, a post secondary school, and a university. Findings revealed that although career awareness was one of the original three goals set for the school, student outcomes in this area were marginal. This is significant since the school was purposefully situated on the university's research farm in order to enrich students' contact with the world of work.
Perceptions of school climate and level of operational support greatly influenced the school stakeholders. The research protocol and the results obtained herein may suggest useful direction for the establishment and assessment of similar schools.
Levine and Levine concluded that "the problems which young people encounter in their transition to adulthood and the high incidence of destructive behavior, such as crime, drug abuse, and teenage pregnancy, are causes for serious concern" (1996, p. 183). Alternative schools are one means by which society has attempted to address the unique needs of at-risk learners. Yet, although alternative high schools have long existed for older at-risk adolescents, similar educational venues have not generally been available for younger children.
Within a local community which is served by two school districts and encompasses a mid-western land-grant university, it was recognized that a substantive body of younger at-risk students existed who could benefit from an alternative school setting (Task Force Committee, 1995). With funding from both school districts, the university, and a private corporation, an experimental alternative middle school was created to address the unique needs of these learners. The Alternative Middle School (AMS) was purposefully located on the university's agricultural research farm to offer the youth a safe and tranquil environment in which they could develop new behaviors and appropriate coping skills. Initial goals set for the students included (a) acquiring academic skills and study strategies, (b) enhancing interpersonal skills and strategies, and (c) enhancing career awareness. The daily contact with agricultural production practices occurring on the university's farm was intended to enrich these students' appreciation of the world of work.
Hughes and Barrick found that agricultural education can "help early adolescents make the transition from childhood to adolescence and be better prepared to make career-related decisions..." (1994, p. 21). The authors emphasized that, as with high school-age youth, knowledge and practice in agriculture can assist middle school-age youth with social skills and personal values, understanding the world of work, intellectual development, and understanding of agricultural concepts and career information.
The AMS opened its doors to the first group of 32 students in fall, 1995, but soon was faced with a series of probing questions posed publicly by community constituents regarding the efficacy of the school. Initially, broad goals had been adopted for the school, yet what were the specific expectations that various constituents held for the school? And, to what degree were these goals and expectations being met? What factors contributed to the AMS success? Did the agricultural setting enhance the school's success? Finally, how might the experiences and lessons garnered through the planning and initial operation of the school aid in the establishment of other similar alternative middle schools? The purpose of this study was to assess the efficacy of this experimental school by exploring these research questions.
Research Objectives
- Identify and describe the primary stakeholders of the school by their age, gender, and affiliation with the school.
- List specific expectations of the AMS held by the stakeholders and determine the degree to which these expectations were perceived to have been met during the early stages of the school's operation.
- Assess stakeholder perceptions on the school's educational climate, level of operational support, and extent of involvement of its community partners.
- Identify factors (predictors) which led to the perceived attainment of the school's broad goals and specific expectations.
Procedures
In order to accurately assess this study's research objectives, the investigation encompassed two distinct data collection strategies: a modified Delphi protocol, followed by a descriptive survey. The Delphi was used to establish consensus on the expectations held for the AMS, while the descriptive survey was used to assess the remaining research objectives.
Modified Delphi Protocol
The utilization of the Delphi strategy enabled the researchers to generate a list of expectations held for the school within the context of group interactions by the stakeholders. Features of this Delphi included anonymity of individual responses, control led feedback, and a descriptive statistical presentation of group responses (Sutphin, 1981).After a comprehensive review of the literature in each goal-area, 28 expectations were preliminarily suggested. The expert membership of the governing council reviewed this list for content validity and for consistentcy with the original goals. Through the recommendation of the governing council three of the expectations were eliminated and substantive changes were made to the wording of five others, resulting in 25 final expectations. The round one Delphi instrument afforded respondents the opportunity to rate (four-point scale) the importance of each expectation held.
A ten-day time interval occurred between the original mailing of the Delphi instrument and the follow-up mailing. The initial mailing in round one was made to all 84 of the stakeholders nominated by the governing council. One follow-up mailing and a corresponding telephone reminder were made to each non-respondent, resulting in a total of 52 usable questionnaires (61.9%) returned in round one. The 52 respondents who participated in round one were then mailed the modified instrument in round two. A similar follow-up strategy was followed for non-respondents which resulted in a round two total of 32 (61.5% for "round two") usable questionnaires.
The instrument in round two contained all items utilized in round one and, in addition, reported statistically the results from round one on the overall importance of each expectation, how the respective individual had responded to each item in round one, and a synthesis of the subjective comments received. In round two, respondents were first asked to consider the new information and then to re-evaluate their judgments. Data were treated as ordinal-level and were analyzed and reported using frequencies, percentages, means, and standard deviations.
Descriptive Survey Process
After the expectations were formulated through the Delphi protocol, the stakeholders were then asked to respond to a descriptive survey instrument intended to assess the degree to which the broad goals and specific expectations were met and to suggest strategies for improvement to the school. Using the Dillman (1978) procedure for mailed questionnaire administration, these judgments were solicited, along with personal characteristics of respondents and the perceptions of the respondents on the school's educational climate, level of operational support, and extent of involvement of its community partners.Initially, the respondents to this descriptive survey were intended to be only those members of the stakeholder group who had responded to the second round of the Delphi administration. Yet, the lower response rate in the Delphi protocol was deemed by the researchers to be of concern. Thus, the researchers returned to the total group of nominated stakeholders (N=84) for their participation in the descriptive survey phase of this study. Although appropriate procedures for handling non-responses were followed for both the Delphi protocol and the descriptive survey phases of this research, reviewers are cautioned that the findings re ported from the Delphi protocol reflect only those perceptions of stakeholders from rounds one and two of the Delphi (N=84 and n=52 respectively), while the descriptive survey results reflect perceptions of <I>all</I> nominated stakeholders (N=84).
All parts of the descriptive survey instrument were reviewed for content validity by the expert membership of the governing council. Members concurred with the dimensions, categories, definitions, and examples of items in the instrument. The internal reliability of the descriptive survey instrument was established on the three major groupings of scaled items employed in the instrument. Cronbach alpha coefficients were calculated on each of the three scales and these ranged from a low of .72 (academic study skills) to a high of .96 (career awareness). Nunnally (1978) suggested that the appropriate standard for reliability estimates should be above .70. Using this standard, all scales demonstrated an acceptable level of internal consistency.
Non-response error was assessed by comparing early and late respondents (Miller & Smith, 1983), with the late respondents designated as surrogates for non-respondents. No significant differences (p>.05) were found between the two groups of respondents in the three major groupings of scaled items. Therefore, results were generalized to the population of stakeholders (N=84). An initial mailing, and one follow-up mailing with an accompanying telephone contact were utilized to gather descriptive data. All respondents were assured of complete confidentiality. Ten-day time intervals occurred between the original mailing and the follow-up procedure. Data collected were treated as ordinal-level.
Objectives one, two, and three were analyzed using standard descriptive statistics, and one-way analysis of variance, Fisher's LSD, and t-tests. Objective four was analyzed using stepwise multiple regression. In keeping with customary practice, a significance level of .05 was set.
Results
Research Objective One: Characteristics of Stakeholders
Data revealed that the stakeholders were mostly older and experienced in the public and/or private sector. Only 7.3% of the respondents were younger than 30 years, while more than 70% were 45 years of age or older. Approximately 50% of the respondents wer e female. The respondent group had similar representation from the two participating school districts (39.0% and 41.5).Approximately 25% of the respondents were principals, 20% were school board members, 20% were counselors and teachers, and 25% of respondents were parents or guardians. The average length of association of respondents with the Alternative Middle School was slightly more than one year (13.8 months).
Research Objective Two: Expectations Held by Stakeholders
Through the use of a two-round modified Delphi protocol, twenty-five expectations which were formulated around the three original goals for the school were examined (Table 1). The expectations which were associated with Goal 1 (academic skill development) and Goal 2 (interpersonal skill development) were rated overall more important (M=3.34 and M=3.85 respectively) than Goal 3- development of career awareness- (M=2.73). Further, the expectations which addressed the positive impact of the agricultural setting of the school on student achievement were perceived less importantly than most other expectations. All items were rated on a four point scale (4=Extremely important, 3=Important, 2=Only slightly important, 1=Not important).For Goal 1 (academic skill development), most of the expectations which were rated important (e.g., improve communication skills, and practice and apply academic principles) were also rated as being met. However, the expectations for meeting academic standards in math were perceived as being more poorly met (M=1.22). This was notable since this expectation was rated as "important" by the stakeholders.
Within Goal 2 (interpersonal skill development), expectations were favorably perceived as being met. Mean scores for these expectations ranged from a high of 3.00 (learning social skills and practicing conflict resolution skills) to a low of 2.50 (taking responsibility for individual actions). It is interesting to note that the expectation which scored the lowest for being met (taking responsibility for individual actions) was rated as the most important of all interpersonal skill expectations.
Perceptions on the expectations held for Goal 3 (career awareness development) were generally less favorable than for either Goal 1 or Goal 2. The mean scores on the degree to which the expectations were being met ranged from a high of only 2.35 (using agricultural principles to learn) to a low of 1.32 (preparing a portfolio). Again, it is interesting to note that the expectation which scored the lowest for being met (preparing a portfolio, M=1.32) was rated as the most important of all expectations (M=3.53) in this goal area.
Table 1. Comparison of Expectations: Their Importance and the Degree to Which Each was Met
| Goals | Importancea | Degree metb | ||
| Goal One Expectations (academic skill development) | Mean | S.D. | Mean | S.D. |
| - improve communication skills | 3.81 | .39 | 3.20 | .67 |
| - develop study skills | 3.75 | .51 | 2.57 | .86 |
| - meet academic standards in reading/writing | 3.72 | .46 | 2.59 | .89 |
| - practice and apply academic principles | 3.56 | .56 | 3.50 | .73 |
| - participate in interdisciplinary writing projects | 3.55 | .57 | 2.90 | .82 |
| - meet grade-level academic standards in math | 3.34 | .65 | 1.22 | .94 |
| - improve speaking through interdisciplinary projects | 3.21 | .71 | 2.83 | .78 |
| - enhance use and knowledge of technology | 3.13 | .72 | 2.84 | .83 |
| - meet grade-level academic standards in science | 3.03 | .75 | 2.70 | .89 |
| - develop knowledge of individual learning styles | 2.84 | .51 | 2.22 | .85 |
| - meet grade-level academic standards in history | 2.84 | .72 | 1.65 | .71 |
| - meet grade-level academic standards in geography | 2.81 | .70 | 1.56 | .65 |
| Goal Two Expectations (interpersonal skill development) | ||||
| - take responsibility for individual actions | 4.00 | .00 | 2.50 | 1.13 |
| - learn social skills | 3.94 | .25 | 3.00 | .68 |
| - practice conflict resolution skills | 3.94 | .25 | 3.00 | .68 |
| - develop problem solving skills | 3.87 | .34 | 2.59 | .81 |
| - set goals to reach personal and professional ambitions | 3.52 | .77 | 2.56 | .89 |
| Goal Three Expectations (career awareness development) | ||||
| - prepare an academic/career portfolio | 3.53 | .62 | 1.32 | 1.08 |
| - complete projects for the community | 3.10 | .75 | 2.10 | .99 |
| - prepare a career plan | 3.03 | .75 | 1.62 | 1.00 |
| - explore careers and establish vocational goals | 3.00 | .63 | 1.70 | .89 |
| - use agric. principles to learn academic principles | 2.68 | .93 | 2.35 | 1.04 |
| - learn about agricultural career opportunities | 2.26 | .82 | 2.10 | .97 |
| - gain an overall appreciation of agriculture | 2.09 | .73 | 2.17 | .97 |
| - learn agricultural terminology | 1.94 | .72 | 2.29 | .97 |
bScale: 4=Fully met, 3=Met, 2=Slightly Met, 1=Not Met
Research Objective Three: Assessing School Climate, Operational Support, and Involvement of Partners
A series of questions were posed in order to seek judgment on the school's educational climate, level of operational support, and extent of functional involvement of its community partners. Table 2 reflects that the educational climate was perceived most favorably, with a mean score of 3.1, while the extent of functional involvement of the school's community partners was perceived less favorably (M=2.4). All items were rated on a four point scale (Scale: 4=Strongly favorable, 3=Favorable, 2=Unfavorable, 1=Strongly unfavorable). A one-way analysis of variance revealed a statistically significant effect for type of affiliation of the respondents with their perception of the extent of involvement of its partners, F(2,31)=4.44, p>05. A Fisher's LSD analysis revealed that respondents from one of the school districts were significantly (p>05) less favorable (M=2.1) than all other respondents towards the level of functional involvement of the school's community partners.Table 2. Perceptions on School Factors
| Perceptions on School Factors | Mean | S.D. |
| School's educational climate | 3.1 | .44 |
| Level of operational support | 2.8 | .31 |
| Extent of functional involvement of its community partners | 2.4 | .49 |
Research Objective Four: Predictors of Attainment of School Goals
Possible predictors of the three broad goals held for the AMS were investigated (Table 3). One variable -- perception of the school climate -- explained significant amounts of variance (p>05) and was found to account for 42.7% of the variance for G oal 1 (academic development) of the AMS. Two variables -- perceptions of the overall school climate and the functional involvement of the school's community partners -- explained significant amounts (p>05) of variance in Goal 2(development of life skills). Perception of the overall school climate explained 39.4% of the variance, and when combined with the involvement of the school's community partners (9.8%), a total of 49.2% of the variance for Goal 2 was explained. Two variables -- perceptions of the overall school climate and the level of operational support -- explained significant amounts of variance (p>05) for Goal 3 (enhancement of career awareness). Perception of the school climate explained 40.0% of the variance, and when combined w ith the perception of operational support (13.6=%), a total of 53.6% of the variance for Goal 3 was explained. Readers are cautioned to note that in every case, although significant amounts of variance were explained, these amounts accounted for less than 54% of the total variance. Therefore, considerable amounts of variance remained unexplained.Table 3. Predictors of Attainment of Goals 1-3 Determined Through Stepwise Multiple Regression
|
|
|
|
|
|
|
|
|
|
|
|
|
.427 | 1 | 19.41 | .0002 | Climate | 3.482 | 4.406 | .0002 | 0.427 |
| Intercept | -4.532 | -1.87 | .0730 | ||||||
|
|
.492 | 2 | 12.13 | .0002 | Climate | 4.586 | 4.82 | .0001 | 0.394 |
| Community Participation | -1.921 | -2.20 | .0373 | 0.098 | |||||
| Intercept | -2.489 | -1.07 | .2930 | ||||||
|
|
.536 | 2 | 13.88 | .0001 | Climate | 3.516 | 3.57 | .0016 | 0.400 |
| Operational Support | 3.670 | 2.65 | .0139 | 0.136 | |||||
| Intercept | -15.935 | -3.83 | .0008 |
Conclusions
Findings revealed that student achievement of the basic academic skills
of reading, writing, and math were viewed as most important to the stakeholders.
The achievement of essential communication skills (listening, speaking,
and writing) were also considered as very important. Stakeholders responded
that most of the important academic expectations which were important were
being met. It appears, however, that achievement in math, although important
to the stakeholders, was considered not well met at the school. It is notable
that knowledge of individual learning styles was regarded as less important
(M=2.84). Yet, the development of study skills was regarded highly
(M=3.75).
Educational literature reflects many examples of how knowledge
and application of learning styles is relevant in the development of appropriate
study skills and habits (Kellough & Kellough, 1996;
Slavin, 1997). Therefore, a more positive relationship
in the findings between the importance of these two variables (learning
styles and study skills) might have been expected. Further investigation
into the relationship of these two variables with other independent variables
might yield some clarity on this issue. Respondents also indicated that
although the preparation of a portfolio was important towards enhancing
career awareness, this particular expectation was not well met.
The development of interpersonal skills was quite important to the stakeholders. Students learning to take responsibility for their own actions, learning social skills, learning to resolve conflicts, and setting personal and professional goals were important interpersonal skills identified in this study.
Interestingly, the original expectations which provided for greater awareness and appreciation of the ongoing agricultural enterprises occurring on the university's research farm were rated less important (agricultural principles, M=2.68; careers in agriculture, M=2.26; appreciation of agriculture, M=2.09; and agricultural terminology, M=1.94). This is especially noteworthy since the school's farm location was purposefully selected in order to provide a learning environment that was conducive for students to have regular contact with agricultural production practices. A basic assumption of the governing council was that "this direct contact would enhance the students' abilities to develop responsibility and skills related to the care of animals and the land, which transfers to broader personal development" (Task Force Committee, p. 2, 1995).
None of the factors age, gender, or length of affiliation with the school impacted the perceptions of the stakeholder group on the school's educational climate, operational support, and involvement of its community partners. Yet, stakeholders from one school district were significantly more critical of the involvement of the school's community partners. This could infer an uninformed or unclear understanding between all stakeholders on the expectations and initial outcomes expected for the school.
The perceptions of the stakeholders on the overall school climate had a significantly positive relationship to the attainment of each of the three goals. As anticipated, favorable perceptions of teachers and students were significant factors in predicting the favorable achievement of academic, life, and career skills. The perception of the level of participation of the school's community partners was important in predicting the achievement of interpersonal skills for students. The perception of the level of operational support for the school was significant for predicting enhancement of career awareness for students. These data reflect that a positive sense of school climate is important to the achievement of program goals, that the school must be strongly community based for the successful achievement of interpersonal skills, and that there must be adequate levels of support for the school in order to successfully establish career awareness.
Recommendations for Enhancing the Success of the AMS and Other Similar Schools
All members of the partnership must have open and informed communication. In this study, stakeholders from one school district were significantly more critical of the involvement of the community partners than were other stakeholders. Was there different information possessed by the different stakeholder groups to cause this difference in perception?
More understanding and commitment on the role of learning styles is needed in order to support the appropriate development of study habits and problem solving strategies for the students. In this study, knowledge of learning styles was perceived as not important, yet developing study skills and developing strategies for effective problem solving were important.
The school must be community-based and its community partners actively involved if it is to experience positive interpersonal skill development in its students. In this study, interpersonal skill achievement was acknowledged as important by those stakeholders who perceived a stronger functional involvement of community partners.
There must be adequate levels of support if career awareness is to be achieved. In this study the belief that career awareness was being achieved was closely tied to a sense that there were good levels of operational support for the AMS.
References
Hughes, M., & Barrick, R. K. (1994). Building the case for middle grade agricultural education. The Agricultural Education Magazine, 67(4), 20-21, 26.
Kellough, R. D., & Kellough, N. G. (1996). Middle school teaching: A guide to methods and resources. (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall, Inc.
Levine, D.U., & Levine, R. F. (Eds.). (1996). Society and education. (9th ed.). Needham Heights, MA: Allyn and Bacon.
Miller, L. E., & Smith, K. (1983). Handling nonresponse issues. Journal of Extension, 21, 45-49.
Nunnally, J. (1978). Psychometric theory (2nd ed). New York: McGraw Hill.
Slavin, R. E. (1997). Educational psychology: Theory and practice. (5th ed.). Needham Heights, MA: Allyn and Bacon.
Sutphin, H. D. (1981).Positions held by teachers, teacher educators, and state supervisors about selected national issues in agricultural education. (Doctoral dissertation, The Ohio State University, 1981). Dissertation Abstracts International, 42 4257.
Task Force Committee. (1995, March). Teen Learning Center curriculum framework. Fort Collins, CO: Author.
Author Notes
David Whaley
Professor and Director of Educator Licensing
School of Education
Colorado State University
Fort Collins, CO 80523
Phone: (970) 491-5292
Fax: (970) 491-2632
whaley@cahs.colostate.edu
IEJLL - Educational Partnerships Theme Series
External Reviewers
(In addition to the usual members of the IEJLL Editorial Team)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|