University of Calgary
UofC Navigation

Survey Design

The Design Team produced a survey that:

Consists principally of items that are known to be related to important college outcomes. NSSE intends to provide information about the extent to which different colleges exhibit characteristics and commitments known to be related to high-quality undergraduate student outcomes. To that end, The College Student Report is relatively short and contains items directly related to institutional contributions to student engagement, important college outcomes, and institutional quality. The Design Team had three general criteria in mind when selecting items that might be used, including: (1) Is the item arguably related to student outcomes as shown by research?; (2) Is the item useful to prospective students in choosing a college?; and (3) Is the item straightforward enough for its results to be readily interpretable by a lay audience with a minimum of analysis?

Items on actual student behaviour and perceptions of the extent to which the institution actively encourages high levels of engagement are included in The College StudentReport. In general, the questions fall into three broad categories:

  1. Institutional actions and requirements: include specific items about the curriculum (e.g., how much reading and writing have you done?) and about faculty behaviour (e.g., have you worked with a faculty member on a significant scholarly task such as a research project?).
  2. Student behaviour: includes items about how students spend their time inside and outside of the classroom (e.g. have you worked with other students outside of class to prepare class assignments?).
  3. Student reactions to college: include items that seek students' perceptions about the quality of their own experiences (how would you rate the overall quality of your experience here?). This last category also includes questions about self-reported gains in skills that students feel they have developed as a result of attending college (e.g., has college helped you to develop your ability to think critically and analytically?).

Concentrating on only a few carefully-chosen items in a special-purpose survey should both focus attention and promote high rates of response. The emphasis on student engagement and good practice is intended to shift the focus of current conversations about quality away from resources and inputs and toward outcomes,while being specific enough about processes to indicate concretely the kinds of improvements in which colleges should invest. The ability to compare results among peer institutions to identify best practices is also an important feature.

Be administered to students at both public and private four-year colleges and universities. Excluding two-year institutions altogether-at least at first-helps avoid the problem of multiple educational missions. Most students attending four-year institutions intend (eventually) to earn a baccalaureate degree and are not simply engaging in classwork to enhance job skills or to pursue a personal interest. At the same time, baccalaureate-granting institutions share common curricular features at the undergraduate level, including general education and an upper-division major, and all purport to prepare students in similar areas consistent with similar objectives. Moreover, virtually all claim to enhance student abilities in such areas as communication, critical thinking, and higher-order reasoning.

Be administered to freshman- and senior-level students who have attended the institution for at least two terms. We know from research that the experiences of lower-division and upper-division students are quite different at most colleges and that what happens in upper-level courses in a student's major is especially distinctive. Such variations will be captured by sampling students at two points in their academic careers in order to paint a fair picture of an overall collegiate experience. Deliberately sampling students at different levels will also help adjust for the fact that "survivors" have generally had more successful experiences than dropouts at any given institution.

Be administered to adequate samples at participating institutions. To ensure meaningful and credible results, random samples, typically ranging from 450 to 1,000 students based upon total undergraduate enrolment, are drawn from each institution's pool of freshmen/first-year students and seniors. While smaller samples might produce consistent results, sufficient numbers of cases are needed to allow the kinds of disaggregation (e.g., by student level or major) required to make sense of the data and to guide meaningful discussion and improvement on both the local and national level. As a result, the NSSE incorporates "best practices" in its survey administration in order to maximize institutional response rates.

Be flexible. Recognizing that institutions also need tailored information to guide improvement, The College Student Report is designed to accommodate alternative sets of questions especially suited to particular types of institutions-as well as the ability to add questions designed by colleges and universities themselves. A layered data design permits identification of a common core of questions appropriate for universal distribution and broad comparison while also permitting the addition of tailor-made questions posed by consortia. One can imagine a range of different attributes that might be of interest, including attributes related to community involvement, attributes related to the attainment of particular student goals, or "process"measures, such as the number of times students use the library or the ease with which students can design their own major.

Be administered by a credible third-party survey organization. The eventual administering organization for the NSSE, a joint venture between the Indiana University Center for Postsecondary Research, the Indiana University Center for Survey Research, and the National Center for Higher Education Management Systems, is not part of the existing accountability structure of colleges and universities. As such, it is in a position to report results to the public with high credibility and remain free from the direct control of outside stakeholders. A high visibility National Oversight Board composed of educational researchers and public representatives will ensure the effort's independence and objectivity.

Many of the items included on the current version of the NSSE are derived from existing student questionnaires including the College Student Experiences Questionnaire (CSEQ), the Cooperative Institutional Research Program (CIRP) Freshman and follow-up surveys, and student and alumni surveys administered by the University of North Carolina system.

The NSSE instrument went through several drafts and revisions by the Design Team and was reviewed by several groups of potential NSSE users such as representatives of the press including U.S. News and World Report, of accrediting agencies like the Middle States Association, of state higher education oversight agencies, and of higher education constituency organizations including the American Council on Education. Institutional representatives from potential institutional participants were also provided with the opportunity to review and react to The College Student Report.

The Design Team also decided that it was important to collect institutional data in addition to student data. As a result, an institutional data form, to be completed by the administrator designated as the NSSE contact, was developed and additional background data on each participating institution will be assembled from published sources.