The Next Generation of Basic Education Accountability in Alberta, Canada: A Policy Dialogue, 5(19)

Education Manager, Alberta Learning¹

---

 

Abstract

The action research reported here considers extant issues regarding the design and implementation of the accountability model for basic education in Alberta, Canada. The paper reports a dialogue between accountability policy implementers and policy makers and considers how accountability processes can evolve into a more collaborative process. While accountability has long been a focus in schools, statutory requirements contained in the Alberta Government Accountability Act have altered perceptions and practice. Accountability would be most effective if the field and government purposes for accountability were congruent and an inherent component of practice. Similarly, a more dialectic approach to accountability would be facilitated by redesigned approaches to program evaluation and data analysis premised on richer, shared data that held meaning for practitioners as well as policy makers. The model of accountability illuminated in this paper, because it is more professional, comprehensive and transparent, points to the benefits that are achievable when those educators who are accountable feel personal and professional ownership for the accountability model mandated by the state.

---

 

Introduction

The 1995 Alberta Government Accountability Act established a new model of accountability premised upon greater planning and reporting articulation between Alberta (Ministry of) Learning, school boards and schools than previous accountability initiatives. Implementation of the new accountability model began in 1996 with school boards and by the following year schools were required to be developing three-year education plans followed by annual results reports.

An action research project investigated the early implementation of the new Alberta Accountability model in the 1997-98 school year from the perspectives of a school principal, a central office administrator, and a ministry senior manager responsible for monitoring the jurisdiction-ministry accountability relationship (Burger, Bolender, Keats, & Townsend, 2000). A key observation of this action research was that accountability can be at most a necessary, but not a sufficient, condition for excellence in schools. The study concluded that the provincial accountability model should be premised upon the responsibility for and commitment to continuous improvement that professional educators believe in.

The Burger et al. (2000) action research became instrumental in helping to reshape the accountability relationship between the Ministry and school boards whereby the emphasis began to be shifted from compliance to collaboration and excellence, with accountability playing a support role. Ministry officials expended considerable effort beginning in early 1999 to demonstrate that the accountability relationship between school boards and the Ministry was indeed a two-way process, i.e., a process that empowered school jurisdictions to influence provincial planning and priorities for basic education, rather than solely a mechanism for the province to monitor jurisdiction results.

Despite efforts to increase school and jurisdiction-based ownership of the provincial accountability model, feedback to Alberta Learning through analysis of school board accountability documents (Alberta Learning, 2000a & 2001) demonstrated that key challenges must be addressed for true field-based ownership and the next generation of accountability to emerge.

Analysis of school board education plans and results reports raised several key questions about the assumptions underlying the accountability model. These questions were the subject of dialogue between Ministry representatives and College of Alberta School Superintendent (CASS) members at Fall, 2000, College of Alberta School Superintendents’ zone meetings. Among the questions discussed were:

Q1. If you were to prioritize the key purposes of accountability in order of importance, what would this ranking look like?

Q2. Is the balance between compliance and collaboration about right? For example, do provincial priorities sufficiently reflect jurisdictions’ priorities and vice versa?

Q3. Should the accountability framework maintain a core set of required measures beyond achievement tests, diploma exam results, and participation rates; to provide for a stronger empirical base and comparability of measures between jurisdictions?

Q4. Do provincial standards for achievement on Provincial Achievement Tests and Diploma Exams provide meaningful jurisdiction targets for all student achievement tests and diploma exams, or should targets vary in relationship to past performance?

The Purpose of this Paper

This paper was informed by the feedback received from the College of Alberta School Superintendents (CASS) zone meeting discussions of the above noted questions regarding accountability. The CASS dialogue was supplemented with a subsequent parallel dialogue with three practising Alberta central office administrators, two Ministry officials, and a private consultant. This authors group formed the action research team for exploring the above-noted questions and other accountability issues in a more iterative and intensive format in relationship to the ongoing accountability work of the group members. The purpose of this paper is to report the outcomes of this dialogic process and link the perspectives of practising administrators and policy implementers in the field with policy makers in the Ministry.

The Purposes of Accountability

Alberta Learning’s Fall 2000 analysis of school board education plans and results reports not surprisingly indicated that the purposes underlying the provincial accountability policy were not clear. Not only is accountability a complex and ambiguous concept, it is also a highly politicized term, which resonates with emotion. Accountability has multiple layers of meaning understood in the context of the recently "restructured" Alberta educational system.

"Accountability" as a concept was at the forefront of the unprecedented wave of top-down, seemingly ideologically driven package of educational reforms that swept over the Alberta educational landscape in the mid-1990s. "The explicit goal of the provincial government was to increase the accountability of public education, both fiscally and academically" (Webber & Townsend, 1997, p. 174). Given the province’s achievements in developing a comprehensive accountability system over the preceding decade, beginning in 1984, it is little wonder there is some confusion in the field about the meaning of the term. In the mid-1980s, Alberta’s "policy driven" system tied to the Management Finance Plan implemented annual curriculum-based, standardized tests for all grade three, six, nine, and twelve students.

Evaluation was the cornerstone of this accountability system, which required every school authority to establish student evaluation policies and to enact cyclical teacher and school evaluation programs. School jurisdictions themselves were scheduled for evaluation every four to five years. The utilization of systematic planning approaches and provincially mandated Annual Reports were well entrenched. A prominent and effective feature of this eighties approach was the notion of "interactive accountability" (Fullan, 1982). This was achieved through ongoing, reciprocal, and mutually beneficial communication links between members of the superintendency and government managers who staffed the network of Regional Offices of Education across the province. Significantly, students did well under this approach and parental satisfaction rates were consistently close to 90%.

It appears that the government intended something more with the new accountability policy exemplified by the phrase, "to increase the accountability of public education." As several scholars (Barlow & Robertson, 1994; Kuchapski, 1998; Manzer, 1994; Townsend, 1998) have observed, the heavy reform agenda was tied to at least three additional purposes. First (and most explicit) was the concentrated effort to eliminate the provincial deficit. Second was the re-establishment of provincial control through the consolidation of school jurisdictions and the institution of provincially approved superintendent appointments. Third was the swift transition to an emphasis on choice evidenced by a market ideology in many aspects of the system. Additional challenges included across-the-board salary and funding reductions, the mandated implementation of School-Based Decision-Making (SBDM), and the establishment of increased roles for School Councils. Barlow and Robertson (1994) characterized the government’s intentions thus:

The basis of these reforms is neither pedagogical nor fiscal, but ideological and political, consistent with the ultraconservative beliefs about the role of government (as small as possible), the role of the private sector (as large as possible) and a deregulated marketplace. (p. 219)

Evidence suggests that this massive restructuring agenda had little positive impact on student learning (Burger et al, 2000; Townsend, 1998). As Ginsberg and Berry (1997) indicate, "much of the externally driven demand for accountability has not been terribly effective" (p. 45).

Nevertheless, for a variety of reasons, tensions have eased over the past few years and the system finds itself in a period of relative calm. With improvements in the economy, salaries and general school board funding have risen. In many instances, consolidated school jurisdictions have become stable and successful. Earmarked funding initiatives of wide variety have channeled efforts toward provincial priority targets. In many respects, a renewed spirit of collaboration and openness is becoming established between Alberta Learning and stakeholders in the education community.

For many, the province’s implementation of the Alberta Initiative for School Improvement (AISI) in the Spring of 2000 was a significant turning point. Under the leadership of a new Deputy Minister, representatives of the teachers’ association, the Superintendents’ group (CASS), the universities and trustees’ associations succeeded in designing a forward-looking and collaborative program to provide additional learning supports to students. With AISI, the ministry appears to be embarking on a path more likely to achieve positive learning results, in keeping with the following conclusion reached by researchers Michael Fullan and Andy Hargreaves (1998):

If governments are really interested in substantial improvements for all students, they must engage with the evidence regarding reform strategies which really do deepen and make a difference to student learning and avoid self-defeating strategies which, despite repeated use, have had virtually no positive effects at all in the past. Government leaders who recognize the power of capacity-building strategies, along with frameworks of accountability, will achieve significantly more change "on the ground" than those that employ distant hierarchical methods of compliance. (p. 121)

Within this emerging provincial-local dynamic of increasing collaboration, re-emerging trust and heightened optimism, members of (CASS) were asked to discuss accountability in their Fall 2000 regional meetings. They were also invited to prioritize the three purposes of accountability outlined in provincial policy. While members of CASS voiced frustration with the initial bureaucratic and compliance-oriented nature of the accountability framework as well as the sheer volume of measures, two major themes surfaced from their conversations. As a first theme, it is evident that members of the superintendency understand and support the need to hold themselves, their districts, and their schools accountable. In this respect, their comments suggest adherence to two of Kuchapski’s (1998) undergirding principles and elements that are meant to clarify the meaning of the term accountability. First, ensuring their operations and structures and decision-making processes are clear and open for observation and participation. The second of Kuchapski’s principles is "disclosure" or the timely reporting of program and financial performance (p. 187). Sinclair (1995) describes these as political and public accountability. Alberta superintendents expressed strong belief in the need for these approaches to accountability. They see these as important aspects of their personal and professional responsibilities (to reference two more of Sinclair’s typologies), thus establishing a more legitimate basis for the current Alberta accountability initiative. They do not want unnecessary paper exercises and needless complexity of measures.

Superintendents seek to focus on one major purpose in their accountability efforts. CASS members overwhelmingly believe that the purpose of accountability should be to guide efforts to improve teaching and learning in their schools. They clearly articulate a conception of accountability "with a human face," as Schmoker (1996) conceives it:

We must make sure that accountability focuses on improving results. The information we collect and analyze should help us understand and improve instructional processes that help get better results. Our constituents should be informed of our work so that they know how we are doing and can help us figure out how we can all work together more effectively. (p. 70)

School system leaders in Alberta have indicated they are eager to move forward with the agenda of continuous improvement, capacity building, and open reporting. They embrace the opportunity for greater accountability and interactivity and to engage on a more equal footing with the Ministry as it formulates plans and communicates outcomes to the public. With a new era dawning, a challenge shared by CASS and Alberta Learning is to conceive of and institutionalize collaborative planning and reporting processes that will serve both local and provincial interests simultaneously. Perhaps what is sought is that 1980s notion of "interactive accountability" premised on the aforementioned ongoing, reciprocal, and mutually enlightening accountability dialogue.

Pushes and Pulls of Accountability in a Southern Alberta School System

There is no argument on the part of central office administrators and school-based staff with the concept of accountability. Accountability is a given because it is a fundamental element of our democratic system that links the various orders of government held accountable for decisions made and actions taken.

Since January of 1995, there have been some subtle and some not so subtle changes affecting the accountability framework for basic education in Alberta. Trustees have lost the right to local taxation, except through a special plebiscite, and the province has taken control over taxation as a means to fund education equitably. Although the monetary impact on the taxpayer in most cases has not changed significantly, there has been a symbolic shift of responsibility from the local trustee to the provincial government, and associated with the shift, there is a perceived loss of accountability by trustees. The taxpayer can no longer hold the local trustee accountable for the mill rate in relationship to their perceptions about the quality of educational services being provided.

The right to local taxation has been minimized, but school boards are still responsible for how they use the funds made available by the province. School boards are expected to establish measurable goals and implementation strategies to monitor progress, evaluate and report the results, and make modifications as new goals and plans are established. School boards must also incorporate into their planning process provincial goals and measures for which they are accountable. Ideally, the provincial and school system goals and accountability measures have been developed through collaborative processes based upon grassroots input and ownership for what has been determined to be important.

The Alberta Auditor General formally defines accountability as an obligation to answer for the execution of one’s assigned responsibilities. This definition raises the question for educators: "What do you believe you are accountable for–primarily for the provincial goals and measures (even though there may not be ownership for the goals and measures) or for providing the best possible education for students based on your own professional expertise and judgment of educational goals?" Ideally, there should not be any incongruity in these two conceptualizations of accountability; however, given the local perspectives of students, parents, and community members and their judgment of how well our schools are doing, these goals may not always be congruent or compatible.

Alberta Learning has also mandated implementation of SBDM with many different approaches being taken in translating the philosophy and principles of SBDM into reality. Superintendents are responsible for making meaning of SBDM and the accountability framework. This requires superintendents to find a balance between the provincial perspectives on monitoring, alignment, and compliance with provincial goals and accountability measures with the expectation that they will provide for more local decision making and control at the school and community level while also coming to terms with accountability expectations from the school board.

How do superintendents meet what are frequently diverse expectations in a more democratized and complex environment? If provincial goals are to be carried out, they must be clearly understood and be perceived as worthwhile and meaningful, and there must be ownership at all levels in the school system. To accomplish this, the emphasis in one southern Alberta school system has been on making education plans "living documents." A significant amount of time and effort is spent in working with administrators and teachers in considering the provincial, school system, and school-based goals in terms of how they best can be implemented. Through what is referred to as the "think tank" process, at the school and school-system levels, opportunities are provided to reflect on the education plan, describe what is working well, and identify areas for further attention. This process provides for the active involvement of students, parents, school staff members, administrators, trustees, and community members. The process of collaboration and dialogue promotes a greater appreciation for the goals and accountability measures and a stronger commitment to achieving them. At the same time, the process identifies grassroots issues and concerns that should be addressed provincially in keeping with the top-down/bottom-up concept of planning and communication.

In this southern Alberta school jurisdiction, the school principals’ annual reports and the school system annual report are seen as opportunities to share information and to celebrate accomplishments in regard to the full range of indicators and goals that have been identified. The annual report is made available through the school system website, newspaper supplements, division office, and each school community. The report provides a better understanding of the context of specific issues in the implementation process and success stories, as well as areas that require further attention. Student and staff support and satisfaction are assessed through satisfaction surveys and are reported to the school board. As well, principals are encouraged to seek feedback on an on-going basis from school staff members through what are described as "fireside chats" (informal one-on-one discussions with the principal and each staff member) that take place several times during the school year. The discussions centre on the teacher’s professional growth plans and the staff member’s perceptions of what is working well and areas for improvement in the school and the school system.

The concept of building strong linkages between schools and their communities and developing a shared understanding of what is important and what schools should be held accountable for is being emphasized in this southern Alberta school system. At each board meeting, two school communities, with representatives of the administration, school council, staff, and students, make a presentation. The school presentation provides an opportunity for sharing information and identifying areas of concern. The superintendent of schools and trustees also attend school council meetings and meet with representatives of each school council two or three times during the school year to discuss goals, measures of success, and emerging issues.

Despite provincial priorities, emerging local issues sometimes take priority over everything else. An example is the issue of school bus safety. This question arose following two serious school bus accidents. Parents expressed the view that the school system was accountable for the safe transportation of students and raised the question of whether safety could be enhanced through seat belts and school bus safety information presentations. In response, information meetings were held and, although it was determined that equipping school busses with seat belts may not necessarily be the answer, there were a number of steps that could be taken in each school community to promote school bus safety. This is an example of an emerging accountability issue that was not reflected in the school system or provincial plan, and it demonstrates how the perspective of parents and community members can quickly reorient the priorities of a school jurisdiction.

Local re-prioritization is also evident in the think-tank process, where the view has been expressed that, more than anything, parents are more concerned about having their children happy in school, feeling safe, secure, and supported with a positive self-concept as learners than they are with test scores. Parents, students, and teachers have expressed a concern with the perceived pre-occupation with test results to evaluate the success of schools and the school system. There is a feeling that other aspects of an education such as experiences in the fine and performing arts, school athletics, student leadership, and the non-academic subject areas have been overshadowed by a preoccupation with provincial achievement test results.

Another example of conflicting accountability issues for teachers and principals is the use of provincial achievement tests and diploma examination results as a key accountability component. Booi (2000), provincial president of the Alberta Teacher’s Association, describes the teachers’ perspective and how they feel they are being unfairly judged by provincial examination results. He makes reference to the misguided obsession with standardization and accountability as "narrowing the curriculum" to what is easy to measure and observes that what gets tested is what gets taught. He refers to the accountability movement as a desire for a "quick fix" in education based on a simplistic application of business methods. Reflecting Booi’s concerns and their own anxiety, some teachers have expressed a desire to teach outside of the provincial examination grades because they feel the pressure of being held unfairly accountable for student results, when in their view, other measures are more important.

Principals and superintendents need to address teachers’ anxiety over provincial testing and emphasize the research that demonstrates that, by teaching the curriculum in holistic ways, the tested curricular domains will be well covered. In this school system, the practice is not to compare schools or to focus attention exclusively on the provincial examination result. However, outside agencies such as the Fraser Institute have, through the publication of the ranking of schools in Alberta, created the perception that the examination results can be used to determine which schools provide the best educational experiences for students.

In this school system, the provincial examinations are recognized as serving a very useful purpose in terms of providing a benchmark and useful information for teachers in analyzing the success of the programs and educational experiences they are providing their students. The emphasis is on using the results collectively with the involvement of teachers from all grade levels to determine the areas of strength in programs as well as areas in which there is room for improvement in providing educational experiences. It is recognized that the context, participation rates, and students’ past performance must be considered in interpreting examination results.

Narrow interpretation of testing for accountability purposes contradicts the philosophical orientation of teachers who subscribe to the belief that the real measure of success of a teacher is the degree of growth that the student has experienced. Accountability processes must also appreciate the complexity of the learning and teaching process and recognize that one cannot quantify all that is important in providing students with a well-rounded education.

It is very important from the provincial, school system, and school-level perspectives that these fundamental elements of learning and teaching are recognized and the misuse of standardized test results eliminated. The professionalism of educators at all levels of the educational enterprise must be recognized and promoted. The provincial policy on teacher professional growth and evaluation is an example of a very significant step in the right direction. Provision is made for teachers to have a professional growth plan outlining what they describe as their areas of strength, as well as a plan to maintain a life-long learning focus and to continue to grow professionally. Teachers share this plan with their colleagues and are given an opportunity to become part of a community of learners with a network of support. There has been a very positive response to the provision that is being made for teachers to have ownership for their professionalism and their professional growth, and there is wide acceptance of the belief that this personal approach is the ultimate measure of accountability. Everyone wants to do a good job, and everyone should be striving to improve. All individuals in various roles, including classroom teachers, administrators, trustees, Alberta Learning staff, and the Minister of Learning, must internalize the belief of being accountable as life-long learners.

What evidence is there that this professional ownership of accountability is occurring? Support for this premise can be found in the improving quality of school and school board education plans and results reports (Alberta Learning, 1999, 2000a) and in the improved availability of these documents on school board websites. There is also evidence of improved public involvement and openness to stakeholder participation in planning processes (Burger et al., 2000). The reorganization of the Alberta Ministry of Learning and the concurrent emphasis on seamless, lifelong learning provide a highly supportive context for collaborative accountability. However, the real measure of success will be the impact of accountability on students who will ultimately benefit from teacher professional growth and responsibility firmly rooted in accountability.

Given these dynamics, principals, similar to superintendents, sometimes feel that they are caught in the middle of the push and pull of accountability. They are required to submit education plans and annual reports on behalf of their school community, describing goals, implementation strategies, and accomplishments, and they are held accountable for what takes place at the school level. As well, in keeping with the site-based decision-making philosophy, principals are expected to give staff members a more active and meaningful role in the decision-making process. The principal, just as the superintendent, must spend a great deal of time in establishing a clear vision for the school community, articulating the vision, and developing a sense of ownership and commitment to the goals and priorities that have been established as a school community. The principal and superintendent must be team builders in bringing staff members together to work collaboratively and as facilitators in assisting staff members in addressing the goals and accountability measures.

When these dynamics create conflict and dissonance, one can appreciate why accountability is not seen as a friend. It is only when the tools of accountability provide principals and superintendents with positive support for balancing out the competing interests in the education community that the accountability requirements can be embraced. And, for this reason, it is imperative that the goals and outcomes of the provincial accountability model fully reflect the priorities of the school jurisdictions as much as the priorities of the government.

Getting Collaboration and Compliance Right: Developing a Common Language

School jurisdictions have tended to view accountability within a framework that reflects the bureaucracy of government and the expectation of compliance. Therefore, when the ideology of the Ministry around accountability shifted to a more collaborative initiative, confusion arose. Entrenched points of view are difficult to reverse. An invitation to share opinions, to provide input, to be heard seriously, is often met with suspicion and disbelief. Distrust is evidenced by the comments collected from CASS in late 2000. Responses suggest that opportunities for local input to the provincial accountability model have been too limited and that compliance outweighs collaboration (Alberta Learning, 2001).

Why do some superintendents view the accountability model as too prescriptive and oriented to provincial priorities at the expense of local priorities or needs? Clearly, Alberta Learning has not yet achieved the optimal balance between collaboration and compliance. Surveys continue to show that jurisdictions do not yet feel sufficient ownership for reporting results within the present use of language, format, and process, but it must be noted that responses do not indicate an objection to planning nor to reporting. However, the accountability implementation process is beginning to demonstrate that planning and reporting are essential components of a journey aimed at growth, progress, and excellence. Ownership for success is being proudly and competently displayed in both planning and reporting documents.

Jurisdictions are using summary brochures and newspaper announcements as marketing and advertising connectors with communities. Provincial planners use findings from results reports to revise direction and allocate resources. Expectations have been clearly communicated. Standards and targets are known across the province. Jurisdictions are requiring more accountability from their schools. There is a commonality among school authorities in the reporting of issues, trends, provincial test results, budgeting, special-needs programming, and infrastructure accomplishments. An element of consistency is emerging, and a focus on continuous improvement is evident. An increased understanding of the purpose and results of planning, measuring, and strategizing is acknowledged. After only four years, these are credible accomplishments to be recognized and celebrated.

Despite these gains, Alberta Learning managers who work with superintendents are identifying a need to clarify the educational planning process. The recent process generates plans that are shared with multiple stakeholders and serve more than one form of accountability. Educational leaders are complying with political, public, and managerial forms of accountability, three of the five identified in Sinclair’s (1995) research, while struggling to locate this with their own professional and personal senses of accountability. In fact, superintendents who take responsibility for professional and personal accountability may be in the best position to define the appropriate balance between compliance and collaboration.

Is the collaboration/compliance balance concern really about imposition or is it about autonomy? Are these mutually exclusive or can a picture be enhanced by its frame? Peters (1991) describes simultaneous loose-tight properties as "the co-existence of firm central direction and maximum individual autonomy." He claims that successful organizations can be on the one hand rigidly controlled and at the same time allow autonomy, entrepreneurship, and innovation from the rank and file. Peters recognizes that the essential ingredient in making this concept work is the adherence to a culture rooted in mutual values. Within Alberta Learning’s vision for Albertans of Optimizing Human Potential, core values expressed are trust, respect, integrity, openness, and caring. These are elaborated in the principles of service: accessibility, accountability, responsiveness, innovation, equity, and collaboration. Successful collaborations require that the collaborators trust each other. In the present climate, trust is still being built. Perhaps before assuming that a culture of collaboration is a realistic goal, the level of trust must be ascertained.

As described earlier in this paper, jurisdiction administrators sometimes feel that the priorities of the larger provincial picture are different from their local needs. In addition to the historical, cynical image of government intent and the mistrust of the consultation and collaboration continuum, the language, format, and process of educational planning are fuelling dissonance. Although Alberta Learning managers have worked to explain the process as a reciprocal system of communication and information exchange, some gaps in understanding are still apparent in superintendents’ perceptions.

The provincial planning cycle and accountability framework assume a collective and reciprocal professional and personal responsibility for the continuous improvement of performance. If the planning loop is incomplete, provincial planning occurs in a vacuum. The perception appears to be that jurisdiction priorities must fit within provincial goals when in fact provincial priorities are most effectively abstracted from jurisdiction reports.

In addition, the current format of the paper transactions between the Ministry and jurisdictions misses this important concept. Communication from the Ministry through the paper trail is particularly vulnerable since the translation of intent is in the mind of the translator. Current summary templates of school board plans and results reports capture only minimal levels of meaning. The message of the minimum is perceived by some jurisdictions with relief, while others envision, create, measure, and record the whole enterprise with great expectations, only to be disappointed by the quality of the dialogue.

If the attitude to planning is that of compliance, the internal usefulness and the process of the planning journey will remain a resisted and disconnected endeavour unworthy of the energy and time committed to it. When partners are informed and consulted, when dynamic processes involve divergent thinkers, when vision and strategy are aligned, the results are owned and activated reciprocally.

What are the next steps? What impediments exist? If collaboration is the goal, the next steps must come from a meeting of minds. Issues to be addressed will include:

Language

• A study and/or continued dialogue around the language of accountability and the resulting, diverse mental models it evokes

• Preparatory documents that emphasize simplicity, giving local jurisdictions autonomy, recognizing integrity, while still retaining the framework of Provincial consistency
• Revisions to the Guide to School Board Planning and Reporting to reflect the above

Format

• Alternatives to the present summary and review templates of school board plans and results reports to include equal space for recognition of local priorities
• Clearer articulation to show transitions and alignment between provincial priorities and those of jurisdictions
• Clarification that "rolling three-year plans" means minor revisions annually and more significant revisions on a three-year cycle

Process

• An improved understanding of and agreement about the need for both autonomy and alignment
• An opportunity for discussion and sharing of jurisdiction planning and reporting formats, tools, and processes
• A study of optimal timelines within the accountability cycle, including the usefulness of the Annual Education Results Report based on the previous plans’ old goal structure when new provincial goals have been announced
• Clearer and more frequent communication of the use of the school board plans and reports by Alberta Learning
• A sharing of follow-through suggestions with the community for plans produced
• A strengthening of trusting relationships–between Alberta Learning and jurisdictions and between managers in the branches of Alberta Learning

A continued commitment to addressing these points by school boards and the Ministry of Learning will result in an inevitable meeting of minds and purposes around the functioning of the accountability model.

Making Accountability Measures Meaningful

The professionalization of teaching has been strongly advocated by Hargreaves (1994), who cites many recent developments to support advocacy for the profession. He views the profession built upon shared vision, commitment, and critical dialogue. The accountability in this discussion is embedded in the responsibility to learning and to students. The Government of Alberta essentially restructured education when it passed the Government Accountability Act in 1995 that clearly established a set of criteria to which it held Alberta school systems accountable. The accountability initiative required school jurisdictions to report on goal achievement and to demonstrate alignment with Alberta Learning's planning and reporting. School systems found themselves reporting to the Minister and the public on achievement or results-based criteria and measures largely supported by provincial achievement test and diploma examination data and locally administered satisfaction surveys.

Against this backdrop, the Government of Alberta passed a Ministerial Order (1998) which set out a clear Teaching Quality Standard and related competencies for teaching in the Province of Alberta and, in so doing, clarified the expectations for teaching performance.

To support the Ministerial Order, Alberta Learning regulated that teachers no longer be subject to a regular cyclical evaluation process as a means to assess competency. In its place, the Minister of Learning established a supervision practice supported by a requirement that teachers annually develop professional growth plans that outline a research and development strategy relative to a growth area identified by the teacher. School boards were required to implement the growth and supervision practices by developing local policy to drive the new initiatives. As noted earlier, this model of teacher evaluation provides a good basis for supporting accountability. This observation is critically important as it recognizes that change occurs at the individual teacher level, and it has a strong relationship to Sinclair’s (1995) notion of personal and professional accountability.

Another south-central Alberta school jurisdiction adopted a process that built a language of common understanding around teacher professional development. Its growth, supervision, and evaluation policy made it possible to incorporate everyday practice into a context to inform discussion and judgments. To address accountability requirements, rubrics were developed in collaboration with teachers and principals to identify standards in teaching practice, criteria for professional growth plans, and for principals’ supervision practice. The rubrics were a reflective tool with which professionals could enhance their practice, identify growth areas, and recognize excellence. As intended, a common language emerged in the implementation process as a means for creating a critical dialogue to enhance professional practice.

Feedback from Alberta superintendents suggest that superintendents have continued to respond to Alberta Learning's reporting requirements by focussing on narrowly crafted measures that typically address student achievement and locally developed satisfaction survey data. Superintendents have voiced their dissatisfaction with these measures, citing frustration with the emphasis on compliance in the reporting language. Superintendents' criticism ranges from disenchantment with narrowly focussed, test-based achievement measures to frustration with the challenges in using disassociated survey data to create critical dialogue. Some school systems have settled on using provincial targets for achievement and, in so doing, have crafted a reporting system measuring achievement insensitive to local context.

Eisner (2001) challenges educators to imagine a set of criteria that identifies quality schools based on criteria broader than test results in core subject areas. He advocates for a conversation in the classroom, risk-taking, exploration, uncertainty, and speculation as better quality indicators of school success. He speaks of a higher purpose in public education that is ultimately linked to how well one does in life rather than in school. Eisner's concept of a quality education is more connected to processes than it is to outcomes, and, thereby, it raises issues for a system immersed in a culture that emphasizes measurement as a component of accountability.

Hill and Crevola (1999) suggest that standards by which we measure school system success be built around a culture of improvement. They cite beliefs and understandings as the glue of a system and that "proximal" factors–those closest to classroom practices–are more important than "distal" factors such as governance and organization.

If Eisner (2001) and Hill and Crevola's (1999) ideas indeed relate to a higher form of accountability, a natural conclusion is that governments should adopt a more professional view of reporting or establishing measures. One way of improving the reporting process while recognizing that school systems are complex is to create measures that are based upon widely recognized criteria or values such as those espoused by Eisner, Hill and Crevola, and others. These criteria are indicators of a school system that is meeting a professional standard. Alberta Learning has identified some of these standards in its own three-year plan for education. For example, empowerment, fair allocation of resources, partnership with parents, and policy-driven practice are cited as components of the restructured system.

Perhaps there is merit in looking at models such as this south-central school jurisdiction’s accountability rubric development as a way to establish the common language and broad-based criteria for measuring and reporting system-wide success for school jurisdictions’ accountability processes. Rubric development begins with an understanding of a professional standard. An example of such a standard might be the fair allocation of resources in support of Alberta Learning's goal for High Quality Learning Opportunities cited in the Guide for Planning and Reporting March, 2000. A possible four-point holistic rubric (with 4 representing the highest level of development) for allocation of resources using these values follows:

4. The system enhances and supports learning goals by creatively directing appropriate resources to encourage learning opportunities. Equity is recognized and celebrated such that the resource allocation enhances a critical dialogue, shared vision, and common beliefs.

3. The system makes learning a priority by allocating sufficient resources to support a variety of learning opportunities. Funds are distributed equitably such that equity issues are mitigated. The resource allocation is consistent with the school system's goals.

2. The system allocates its resources to schools without regard for unique circumstance and learning needs of students. Linkages between goals, beliefs, and allocation of resources are not immediately apparent.

1. The system allocates its resources based on convenience and history. Inequities prevail throughout the schools, and resource allocation is managed and guided by convenience only.

This rubric is an example of the way that a school system could assess itself in terms of its values, goals, and practices against a common standard. Furthermore, it would serve as a self-reflective tool for the system and subsequently as a way to identify growth areas and strengths. A common set of rubrics would need to be established for each of the provincial outcomes and strategies emanating from provincial goals. In so doing, the measuring process could address broader standards, higher order values, and simultaneously develop a common language across the province, while at the same time embedding measurement in practice.

Making Target Setting a Collaborative Process

Alberta Learning (2000b) defines a target as "a desired level of measurable or observable performance to be attained by a specified time" (p. 7). The Ministry further states that, by identifying immediate, reachable outcomes, targets encourage teachers, students, administrators, and their community to believe that distant goals are attainable. Alberta Learning encourages jurisdictions to set realistic and achievable targets based on past accomplishments, including past performance on provincially mandated achievement tests and diploma exams. Some superintendents reflected this desired approach to target setting, for example:

• We see the provincial standard as a long-term goal when we are below it. But each school analyzes its own results, sets its own targets, and really pushes its program to address problems.
• We support the process of looking at one’s own achievement trends and then setting realistic targets based on that history.
• Target setting needs a process focused on improvement, building from the school level up.
  • Provincial standards are meaningful, but individual schools should set their own targets. I am happy if we are moving in the right direction.

The Ministry’s belief in the importance of targets is sufficiently strong to require school jurisdictions to set targets for the provincial achievement tests and diploma exams in the school and school jurisdiction education plans. However, the Ministry’s Guide to School Board Planning and Reporting notes that, "It is possible that local targets would be different from provincial standards." And here is the source of tension in the target-setting issue: Is it reasonable to expect school principals or jurisdiction superintendents to openly and publicly acknowledge that their targets vary from provincial test standards? One superintendent noted, "Schools are comfortable with setting targets higher than the provincial standard, but are uncomfortable with setting targets that would be below the standard." Another superintendent stated, "We use the provincial standards because we got burnt by public response when we varied from the provincial standard."

When asked if provincial standards for achievement on Provincial Achievement Tests (PAT) and Diploma Exams provide meaningful jurisdiction targets for all student achievement tests and diploma exams, responses suggested there is a need for clearer technical information on PAT and diploma exam trend data comparability. Examples of superintendent concerns regarding these aspects include:

• Our jurisdiction does not put much faith in provincial targets. Department staff have said that provincial achievement tests are comparable from year to year, but that diploma exam results are not. So what does this mean? If we can’t compare diploma exams [over time], then how do we set targets?
• There are lots of things done to statistics that have led to the public being disillusioned by them.
• Target-setting creates considerable consternation among principals. There is no buy-in to target-setting; not part of the culture of accountability.

It is clear from the dialogue around this issue that the Ministry must demonstrate leadership in provincial target-setting where provincial level results historically depart from the provincial standards. Also, the Ministry will need to explore through continued dialogue with superintendents how target-setting can add value to accountability processes in ways that contribute to local ownership and control over accountability. Central to this dialogue will be the concept of balance in target-setting whereby the local context is carefully and fully represented in the target-setting process. Target-setting is one of those accountability tasks where the process (which results in greater clarity and understanding of a school’s or jurisdiction’s strengths and weaknesses) is more important than the product.

The Next Generation of Accountability

Key objectives of this action research project and the related process of writing and discussing accountability processes were to search for ways to understand the impact of educational reform and to capitalize on the belief in accountability. None of the authors would deny or abrogate the need for accountability–and indeed our belief in its contribution to education. Significantly, this observation is congruent with the input from CASS.

However, at the same time, we had all experienced in ourselves and in other educators the powerful feelings of distrust, powerlessness, and fear of misplaced or unjustified negative judgment. Furthermore, these reactions were not restricted to the new accountability policy. As we looked back, we could see them as a part of the recent history of education in Alberta. How can educators make sense of the contradictions in our history?

Some answers seemed obvious. Diploma exams and provincial achievement tests don’t in themselves take into account the complexity of the classroom, school, and community context. Recent changes in reporting structures, jurisdiction mergers, and the funding cuts of the mid-1990s have contributed to the sense of disenfranchisement. The lack of collaboration and consultation in these changes increased resistance and resentment. A format of reporting that is not perceived to acknowledge local issues adequately furthers the feelings of powerlessness. However, these points do not fully explain the strong emotions associated with accountability or tell us how to define and design accountability measures that acknowledge and advance professionalism.

We wanted to define the appropriate balance between collaboration and compliance and between provincially mandated and local accountability processes. We are looking for what Schrage (1990) calls "collaborative architectures" that are "… designed to support the various processes of interaction — conversations, sketches, arguments, agreements — and not to predestine or predetermine any specific set of results." But this is only a part of the solution. There has to be a balance between these architectures and the need for a "… structure that can hold the context and contents that these collaborations generate" (p. 67).

The search to make sense out of the reactions and to provide alternatives led us to consider carefully the work of Sinclair (1995, pp. 219-237) on accountability of CEOs. She summarizes the five roles played by CEOs: political, public, managerial, professional, and personal. However, the roles alone did not explain what was interesting and problematic about accountability. She sought to understand the shifts in affect, language, and contradictions in each CEO’s "… construction of their self-identification as accountable."

Sinclair (1995, p. 224) chose a research methodology that could inform an understanding of these shifts in meaning. She used discourse analysis to identify the structural and personal discourse associated with each of the five roles. Each discourse is distinguished by "…different patterns of words and associations, different emphases and ways of relating experience and understanding." Accountability in structural discourse is considered to be the "technical property of a role or contract, structure or system." It is "… unproblematic, able to be delivered, demarcated or exacted independently of personalities, politics, or fate." In the researchers’ written accounts of experiences with accountability and in the feedback we considered from the College of Alberta School Superintendents, this form of discourse was apparent. Statements such as "Governments are held accountable for decisions that are made and actions that are taken" are examples of structural discourse.

On the other hand, personal discourse is confidential and anecdotal. Sinclair (1995) points out that in this discourse "… accountability is ambiguous with the potential to be something that is feared or uplifting … [it] is about exposure and vulnerability … risks and failures, exposure and invasiveness with which accountability is experienced." In our dialogue as action researchers, there are personal statements such as "what we see the government doing is mandating growth plans–this feels like mandating that you will be professional, as opposed to acknowledging what you do is good–now let’s build on it." And "… there is reason for optimism but a lot of hesitancy … there is fear that we may fall back on the disenfranchised model … we may again experience a lack of voice" or "there is a lot of fear–why? Someone from outside the profession will judge and this is not the same as accountability" (p. 224).

As we have written and discussed accountability within Sinclair’s (1995) framework, we have become more aware of the complexity of the roles which overlap, contradict, and support each other. As one researcher said, "At any given time, the [accountability] flow may vary. Sometimes one aspect of the role is more important than the other. What you are trying to focus on is the professional orientation of the processes and policies."

Through our dialogue and writing, it became clear that all five roles and both structural and personal discourses are present at all times, but not always or often consciously connected. However, there are multiple switches from structural to personal discourse both within as well as from one role to another. Sinclair (1995) states that it is the moment of shifting between discourses that "… enable CEOs to feel themselves accountable: neither overwhelmed by vulnerabilities, nor so detachedly an agent of structure that they were unable to feel accountable." Acknowledging these interrelationships and shifts is an essential device in the process of defining and taking ownership for accountability.

Applying Sinclair’s (1995) research framework and findings to our writings and conversations has produced a clearer understanding of our context and our understanding of accountability. A key insight suggests that, like many other current organizations, Alberta Learning has tried to implement policy without adequately attending to the emotions experienced at all levels of the Department and the field. Bridges (1991) states that strategic planning (policy development) focuses on the outcomes. People, however, begin the process of change once they decide to let go of what is currently comfortable. The process of letting go, which he calls transition planning, is contradictory, hopeful, frightening, and moves at its own pace. And, it must be attended to for implementation to be fully accepted. Sinclair’s article notes that the subject of emotion in organizations has not been studied. She points out that "fear, vulnerability, and fealty are some of the emotions that contribute to a ‘feeling’ of accountability" (p. 233). To acknowledge and respond to them is critically important for successful implementation of accountability policy.

Secondly, as several of our authors have indicated, accountability needs a language, ideology, values, and ethics of its own. They further assert–both in writing and in interviews–that it is about ownership, responsiveness, and going beyond the idea of simple managerialism. Managerial models of government have framed the search for ways to make administrators more accountable. They define accountability as "… a matter of imposing program budgeting, performance monitoring and tighter audits, ‘leaving managers free to manage’" (Jenkins et al., in Sinclair, 1995, p. 234). Sinclair (1995) says clearly "… the solution is not to institutionalize one form of accountability, legitimized according to a single ideology" (p. 234). Instead, as is evident in these writings, to acknowledge and strengthen what is already there: namely a strong sense of responsible accountability–by building a more grounded, professional and shared sense of ownership for the education of children and youth.

Finally, the relational basis of accountability in education must be explored. As one author stated, "there is no longer a we/them attitude in the classroom. How can we apply similar positive relationships to the work of accountability? Somehow, the experience of partnerships in classrooms and schools must also take place between the Department and the field." How can this happen? In the words of one researcher, "Dialogue is essential." Accountability initiatives must be re-thought in ways that ensure a balance between the structural interests of government and the personal discourse around the local issues of educators in the field.

In summary, the key strategic issues that will shape the structure and function of the next generation of accountability will include the following.

• Goal compatibility: While there is widespread acceptance of the concept of being accountable, educators frequently find themselves in conflict with mandated accountability goals and pursuing what they believe as professionals to be in the best interests of the students they serve. Ideally, there will be a great deal of collaboration in the development of school system and provincial goals, and they will be congruent with the professional values and beliefs of the educators who are responsible for implementing these goals and with the values and beliefs of the broader society.

• Meaningful measures: The concern among educators that accountability as reflected in provincial testing has resulted in a narrowing of the educational experiences of students needs to be balanced with holistic models of education and assessment. To be meaningful, there must be a culture of improvement and ownership of what has been established as an accountability measure. Measures have to link to what people value in the outcomes. For example, if local authentic assessment measures achievement, then that also warrants reporting. Essentially, a professional model of measurement is needed where worthy teaching and learning drives worthy assessment.

• Professional foundation: There has been a positive response to provincial policy for teachers to assume ownership for their professional growth. There is great potential for building on life-long learning and professionalism as the ultimate basis for personal accountability supportive of the other forms of accountability.

• Relational accountability: An effective accountability culture must be rooted in mutual values, which requires the requisite time, commitment, and dialogue between the Department and educators in the field in order to recognize and compensate for the emotional responses to accountability, and to build clarity in language, format, and process.

Notes

1. The views expressed in this paper are those of the authors and not necessarily of the organizations they represent. Back to top. 

References

Alberta Learning (1998) Teacher Growth, Supervision and Evaluation Policy News Release, February 26, Edmonton. AB.
http://www.learning.gov.ab.ca/news/1998nr/Feb98/nr-teacher%20policy.asp

Alberta Learning. (1999). Policy, regulations and forms manual. Edmonton, AB: Author

Alberta Learning. (2000a). Analysis of school jurisdiction 1998-99 annual education results reports. Edmonton, AB: Author.

Alberta Learning. (2000b). Guide for school boards planning and reporting results. Edmonton, AB: Author.

Alberta Learning. (2001). Analysis of school jurisdiction 2000-2003 education plans: CASS input on accountability issues. Edmonton, AB: Author.

Barlow, M., & Robertson, H. J. (1994). Class warfare: The assault on Canada’s schools. Toronto: Key Porter.

Booi, L. (2000, Winter). Helping good people get out of a bad system. ATA Magazine. Edmonton, AB: Alberta Teachers' Association.

Bridges, W. (1991). Managing transitions: Making the most of change. New York: Addison-Wesley.

Burger, J., Bolender, M., Keates, V., & Townsend, D. (2000). Implementation of the Alberta Accountability Framework. Canadian Journal on Educational Administration and Policy. University of Manitoba. Retrieved from http://www.umanitoba.ca/publications/cjeap/

Eisner, E. (2001). What does it mean to say that a school is doing well? Phi Delta Kappan, 82, 367-372.

Fullan, M. (1982). The meaning of educational change. Toronto: OISE Press.

Fullan, M., & Hargreaves, A. (1998). What's worth fighting for out there? Toronto: Ontario Public School Teachers' Federation.

Ginsberg, R., & Barry, B. (1997). The capability for enhancing accountability. In R. J. S. Macpherson (Ed.), The politics of accountability (pp. 93-104). Thousand Oaks, CA: Corwin Press.

Hargreaves, A. (1994). Changing teachers, changing times: Teachers' work and culture in the postmodern age. New York: Teachers College Press.

Hill, P.W., & Crevola, C.A. (1999). The role of standards in educational reform for the 21st century. [Electronic version]. In D. Marsh (Ed.), Preparing our schools for the 21st century, Association for Superivision and Curriculum Development 1999 Yearbook. Alexandria, VA: Association for Supervision and Curriculum Development.

Kuchapski, R., (1998). Accountability and the social good: Utilizing Manzer’s liberal framework in Canada. Education and Urban Society, 30, 531-45.

Manzer, R. A. (1994). Public schools and political ideas: Canadian educational policy in historical perspective. Toronto: University of Toronto Press.

Peters, T.J. (1991) Thriving on chaos: Handbook for a management revolution. New York: Harper Collins.

Schmoker, M. J. (1996). Results: The key to continuous improvement. Alexandria, VA: Association for Supervision and Curriculum Development.

Schrage, M. (1990). No more teams! Mastering the dynamics of creative collaboration. Toronto: Doubleday/Currency.

Sinclair, A. (1995). The chameleon of accountability: Forms and discourses. Accounting, Organizations and Society, 20(2/3), 219-237.

Townsend, D. (1998). The impact of educational reforms on the operations of Alberta’s public education system: A comparison of the perceptions of superintendents, principals and teachers. Lethbridge, AB: University of Lethbridge.

Webber, C., & Townsend, D. (1997). The comparative politics of accountability of New South Wales and Alberta. In R. J. S. Macpherson (Ed.), The politics of accountability (pp. 172-184). Thousand Oaks, CA: Corwin Press.

---

Author Notes

Dr. John Burger is a Senior Manager in the System Improvement and Reporting Division of the Alberta (Ministry of) Learning in Edmonton, Alberta. Email: John.Burger@gov.ab.ca

Mr. Art Aiken recently retired as Superintendent of Schools with Prairie Land Regional School Division located in Hanna, Alberta. Art is currently an advocate of leadership orientation based on service and support for effective teaching and learning. Email: nola.aitken@uleth.ca

Mr. Jim Brandon is Associate Superintendent of Learning Services in Foothills School Division. The central office is located in High River, Alberta. Email: brandonj@fsd38.ab.ca

Dr. Patricia Klinck is the Principal/Owner of KeyLinks International Consulting Ltd. that operates in Canada and Australia on issues of leadership, organizational design and change. Email: pklinck@keylinks.ab.ca

Dr. Garry McKinnon is Superintendent of Schools, Golden Hills School Division No. 75. The central office is located in Strathmore, Alberta. E-mail: garry.mckinnon@goldenhills-schools.com

Ms. Sylvia Mutch is an Education Manager with the Field Services section of the Alberta (Ministry of) Learning and is located in Calgary, Alberta. Email: Sylvia.Mutch@gov.ab.ca