Appendix 3

Summary of Recommendations for - COE-1998

Item l

Students and faculty had some difficulty with this item. It was unclear to many respondents what or who was being rated (the instructor, the lectures etc.). Secondly, if the item was referring to the instructor, then there seemed to be some confusion as to whether the students were supposed to rate the performance of the instructor (delivery, organization) or the qualities of the instructor (ability, skills, knowledge).

It is recommended that Item I be reworded from:

The instruction in this course was:
To:
The overall instruction in the course was:
Or:
Overall. how would you rate the instructor of the course?

Anchor Points (Item l only)

Although some respondents wanted fewer anchor points, the majority of the responses to this scale were positive. However, the anchor "terrible" was of concern to a significant number of students and faculty. Some suggestions include replacing this word with "completely unsatisfactory" or "unacceptable".

It is recommended that the anchor points on the scale be changed as follows:

From:

Terrible, Verg Poor, Poor, Satisfactory, Good, Very Good, Excellent
To:
Unacceptable, Very Poor, Poor, Satisfactory, Good, Very Good, Excellent

Item 2

The majoriry of respondents understood the question and had no suggestions for revisions. Some respondents felt that the word "enough" was vague and should be replaced with "clear", "sufficient" or "accurate" to improve the clarity of the item, although the concerns over vagueness would likely remain with any of the suggestions offered.

Therefore, it is recommended that Item 2 remain as it stands:

The course outline or other materials provided enough detail about the course (e.g., goals, readings, topics covered, assignments, exams, due dates, grade weightings).

Item 3

This item generally was viewed as the course having followed the syllabus or outline. When asked for suggestions for revisions. most indicated that it was fine as it stands, but a number wanted clarification of "consistent with" and
"other materials".

While it is not a strong recommendation to do so, the Task Force may want to consider rewording Item 3 from:

The course as taught was consistent with the outline and other materials.
To:
The course as taught followed the outline and other materials.

Item 4

Both the students and faculty felt this question needed some clarification. Some respondents felt it was referring tO the logical flow of lecture material, others thought it referred to the overall organization and preparedness of the instructor. Basically, the subject of the statement needed to refer to something specific - the instructor. Although it is recognized that the Task Force wanted to stay away from the term "instructor" on the instrument, this issue might be reconsidered.

It is recommended that Item 4 be reworded from:

The course material was presented in a well-organized manner.
To:
The instructor presented the course material in a well-organized manner.

Item 5

Many respondents felt that the sentence needed a "subject". Secondly, "student input" and "appropriate" were cited as being too vague. However, given that most of the respondents understood the meaning of the question, the change suggested focuses only on clarifying "input".

It is recommended that Item 5 be reworded from:

Students input was treated appropriately.
To:
Student questions/comments were responded to appropriately.

Item 6

Again. the respondents felt that the question would be more clear with an obvious "subject" conveying the enthusiasm. The Task Force may want to reconsider allowing " instructor" to be part of the statement. In that case Item 6 could be reworded from:

The subject matter was communicated with enthusiasm.
To:
The instructor taught with enthusiasm.

Item 7

The majority of respondents both understood this question and offered no consistent su~gestions for further revision. One minor suggestion was that the internet be added in the example list.

It is recommended that Item 7 be changed from:
 

Opportunities for course assistance were available (e.g., instructor office hours, out-of-class appointments, e-mail, telephone).
To:
Opportunities for course assistance were available (e.g., instructor office hours, out-of-class appointments, e-mail, telephone, internet).

Item 8

The most common responses of students and faculty were consistent with what the item intended to ask. The most common concern was, once again, to identify the instructor in the statement. This concern was less apparent than in some of the previous questions, so that it is recommended to leave Item 8 as it stands:

Students were treated respectfully.

Item 9

Most of the respondents perceived this question as having something to do with "fairness". While most thought the item was fine as it stood, a few made comments that it should be simplified to focus on fairness of evaluation methods. The Task Force might consider simplifying to item to reflect what people think it means; that would mean that Item 9 would be reworded from:

The evaluation methods used for determining the course grade are appropriate to assess student learning.
To:
The evaluation methods used for determining the course grade are fair.

Item 10

The majority of faculty respondents were concerned about this item. Comments centered around the vagueness of the word "promptly" and the issue of fairness regarding the use of "promptly", considering class size and assignment type. Some students and many faculty felt the question should include "in a reasonable amount of time". Although this was a concern for the faculty in Part 1 of Phase II, the quantitative data in Part 2 did not support this concern. However, given that this was an issue during Phase I. it was a concern for the Task Force after Phase I, it is recommended that Item 10 be reworded from:

Students' work was graded promptly.
To:
Students' work was graded in a reasonable amount of time considering class size and type of assignment.

Item 11

The majority understood the question and felt that the intent was straightforward. However, a substantial number had trouble with the phrase "a lot". tJnfortunately suggestions for changes to "substantial" or "expanded knowledge greatly" have the same problem. Therefore, it is recommended that Item 11 remain as it stands:

I learned a lot in this course.

Item 12

The vast majority of respondents both mlderstood the question and had no suggestions for revision. Some mentioned deleting the first "course" as it was redundant and made the question more cumbersome.

It is recommended that Item 12 be reworded from:
 

The course support materials (e.g., readings, audio-visual materials, speakers, field trips, software, etc.,) used in this course helped me to learn.
To:
The support materials (e.g., readings, audio-visual materials, speakers, field trips, software, etc.,) used in this course helped me to learn.

Item 13 (Additional Items)

In terms of additional items, the majority of both student and faculty respondents had no suggestions. The suggestions that were made consisted largely of unique ideas expressed by one individual. The only suggestions made by any "grouping" was from faculty who were critical of the preceding 12 items rather than adding any new items. Therefore, there is little basis for recommending the addition of new items to the Instrument

Item 14 (Student Instructions)

Most of the students thought no changes to the Instructions were necessary. However, 10 were concerned about reprisal issues with their giving out their student ID numbers, 5 suggested that the thank you statement was important to keep and 5 students wondered how the results of this form will be used to help students choose classes. An additional 6 students mentioned the issue of "ID number" in their response. The "ID number issue" is a very important one that could bias student responses on the Instrument. It needs to be given very careful consideration, and should be tested empirically.

Helpful comments included underlining or using bold for the disclaimer statement that also asks for the ID number, including the Dean or Department Head's phone number so the students had a contact and the irrelevancy of using the term "separately" with regard to responding to each item.