Appendix 9

What Can I Learn from the Published Information on the Universal Student Ratings of Course Instruction?  

The Universal Instructional Student Ratings Instrument is administered in University of Calgary courses to gather information for three main purposes: 1.) in published form, to assist students in making course selection, 2.) by administrators as part of their  evaluation of instruction, and 3.) as feedback to course instructors to help them enhance their instructional activities.   Students should recognize that course ratings can be affected by many variables beyond the instructor's control, including class size, the number of times that instructor has had the opportunity to teach the course, the inherent difficulty of the course material, and the composition of the class (e.g., mostly majors or non-majors).  The information that the published ratings provides, some of its limitations, and how it can be used are discussed in more detail below.

Course Background Information

1. Number of times that the instructor has taught the course.  In teaching, as in most human activities, practice and constructive feedback can contribute to improved performance.   Ratings in the first few times that an instructor teaches a course are likely to give you a less accurate estimate of what you might expect in the course in the future, than those for courses that the instructor has taught many times before.

2.   Number of respondents and response rate.  The reliability of ratings of instruction increases with the number of students responding.  Thus, some caution should be exercised in interpreting the rating results from classes with either a low number of respondents, or those based on a low response rate.

3.   Instructor's comments.  For a variety of reasons, including changes in instructional approach, class size and composition, instructor experience, assignments and learning materials, past student ratings of instruction may not be indicative of instruction in future offerings of the course.  Instructors can use the comments section to inform you about such issues, both helping you to understand the factors contributing to past rating information, and to give you a better idea about what to expect in future offerings of the course.

4.   Course workload rating chart.  The workload chart will give you some idea of how students in the course perceived the workload relative to other university courses of the same general type.  While such workload information provides a context for understanding student ratings, alone, it is not a good basis for course selection.

5.   Class composition chart.   Ratings of instruction can be affected by students' general interest in the course topic, and also by its relevance to their program of study.  The class composition chart will tell you if the course ratings are based on the responses of students in a situation similar to your own.   For example, if the rating results are based primarily on the responses of majors in the same area as the course, they are likely to be more meaningful for other majors than for non-majors.
 

Rating Information

Overall student rating information for the course is presented separately for each of the 12 items on the University of Calgary Universal Ratings Instrument.  Each rating item is identified only in brief form.  So that you know the specific issues to which the students were actually responding, it is important that you review the copy of the original instrument that is provided.  For each item, the information specific to the course in question is followed by comparative information for all courses at the Department (if any) and Faculty level.

1.   Graphical Frequency Distribution.  The frequency distribution for each rating item provides a visual representation of the percentage of student respondents rating the course instruction at each of the seven rating levels available, from low of 1 to a high of 7.  Because all students responding to a given item are represented in the frequency distribution, the total of the respondents across all seven levels is always 100%.  The frequency distribution provides valuable information regarding the degree of consistency or variability between student raters regarding instruction in the course.

2.   Mean Rating.  The mean is the most commonly used measure of the "central tendency" of a distribution of scores.  It is what most people mean by the term "average", and is calculated by adding everyone's responses from 1 to 7 on each item, and then dividing the resulting sum by the number of respondents.  Because it is based on the actual value of all the scores in a distribution, the mean is often used to determine if one set of scores is higher or lower than another set.  Such a comparison of ratings across items in the same course or different courses is more meaningful if the scores in the distribution are grouped symmetrically around the mean, than if the distribution is asymmetrical and/or shows great variability between raters (see the preceding Frequency Distribution and the following Standard Deviation sections).

3.   Standard Deviation (S.D.).  The standard deviation is the most common measure of variability in a set of scores, and is based on the degree to which the scores differ from the mean of the distribution.  If the scores are very different from one another (i.e., many extreme scores), the S.D. will be high; if the scores are highly consistent, the S.D. will be low.  By providing information about the variability in a set of scores, the S.D. is useful for evaluating the differences between the means of two or more distributions; as the S.D. increases, the difference between means must be greater to be meaningful.