Search

This paper reports an inexpensive and efficient alternative method of measuring teaching quality. The method aligns students' in-course expectations for change with their lecturers expectation for change in three process objects: Skills, Understanding, and Attitudes. It is shown, using sample course data, that these in-course alignment indicators predict quality teaching, which is measured after the course using outcomes of students' academic achievements and course enjoyment. Traditional student evaluations of teaching are post-mortem measures that do not provide feedback to help the students in the course. However, a major contribution of the alignment method is that optimum in-course changes can be calculated to minimize the in-course alignment scores for the whole course or for any student subgroup of interest, thus maximizing students' predicted attainments. As the method only takes 5 minutes to administer, it can be introduced at the level of the individual instructor who wishes to keep his or her teaching on track. The alignment method uses a single form and results in one decision point number. It can be used by the administration at the end of courses for comparable promotion and tenure decisions across the institution. Web-based alignment software is now being developed to enable lecturers and administrators from tertiary institutions worldwide to avail themselves of the method. (Contains 9 tables, 5 figures, and 23 references.) (SLD)

This paper demonstrates an alternative method of using students' evaluations of teaching (SETs) that circumvents many of the problems associated with traditional SETs. It shows how course feedback, consisting of eight ratings, can be used to optimize post-course academic attainment. The method is illustrated with data from course feedback at the University of the West Indies. The new method allows feedback to be used to optimize teacher quality during the course for the whole class, for individuals, or for identified subgroups of students within the whole group. The method operationally defines three educational process objectives: Skills, Understandings, and Attitudes. Feedback forms used during the course give data on the lecturers' and students' expectations for change in the emphasis of these objectives. These data allow for calculations of the alignment between lecturers and students expectations for change. The Alignment theory presented is that students' academic success and enjoyment of teaching are maximized when students and their lecturer are working towards the same changes. The theory is revalidated with each course by correlations of alignments with results, which show that in-course alignment predicts post-course academic success. The paper also describes how the data are used during the course to determine the changes that will best align in-course student/lecturer expectations and maximize the post-course academic attainment for the whole group or different student subgroups. (Contains 5 figures, 9 tables, and 23 references.) (SLD)

The research literature on student evaluation of teaching (SET) is filled with criticisms of the process, its applications, and the student feedback questionnaire it uses. SETs are still used, however, because there has seemed to be no economical, valid, and reliable alternative. This paper reports on an alternative alignment process for evaluating quality teaching and learning that fits the requirements of economy, validity, and reliability, and offers additional institutional, faculty, and student benefits. The method measures both the individual student's and the lecturer's expectations for change in three process objectives that underlie quality teaching and learning: skills, understanding, and attitudes. These three objectives and their assessment are operationally defined. Methods for using these three process objectives as vehicles for teaching, learning, and course development are supported by the institution through staff and course development programs and through recognition of their assessment in student grades and in faculty promotion and tenure decisions. The post-course criteria measuring quality teaching and learning are students' high academic attainments and course enjoyment. Teaching techniques for attaining these goals are left as a matter of informed professional choice to the lecturer. The separately measured in-course predictors of these criteria are the alignments of student/lecturer expectations of change in the three objectives. The theory behind this approach is that quality learning results when students and lecturer are working toward the same goals. This paper introduced the method and presents evidence validating the theory and showing that the indicators correlate with the criteria. (Author/SLD)

Continuing education students bring with them expectations from previous teaching that challenge the effectiveness of usual teacher evaluation procedures. This paper presents a validated approach to the evaluation of teaching that identifies potential problems with individuals and minority groups of continuing students. This approach can be used during the course to steer teaching back on course before problems arise. The approach, the Three Ability Framework (3AF), results in a single quantitative measure of the quality of teaching that can be used as an administrative decision point. The 3AF feedback form takes only a few minutes to complete. The teacher explains to the students the three abilities research has identified as essential to effective teaching: (1) technical skills; (2) professional competence; and (3) professional attitudes. Students are asked to rate their assessments of each of these areas and their expectations of how the areas should be fulfilled in the course, and instructors rate themselves in each area. Student and instructor ratings are compared (student change rating is subtracted from lecturer change rating), and the resulting number is makes an evaluative decision point. "Zero" would be a perfect indicator of the alignment between student and instructor expectations that is indicative of effective teaching. The method allows frequent assessments over the duration of the course, permitting adjustments when necessary, and it is particularly suited to the evaluation of courses in continuing education. (Contains 35 references.) (SLD)

This paper presents an alternative to student questionnaires for the evaluation of teaching quality, which is defined as maximizing students' academic attainment and the satisfaction of students and the lecturer with the course. The approach aligns students' and the teacher's expectations for change during the course. Previous research has identified three fundamental abilities that lecturers seek to change during the course: (1) technical skills; (2) professional competence; and (3) professional attitudes. A Three Ability Framework (3AF) has been developed to compare students' assessments of the change in these three abilities with the instructor's perceptions of change. The alignment between these evaluations, expressed as the result of subtracting student-rated change from lecturer-rated change, is a figure that can be used as the effective teaching score. To use the 3AF in practice, lecturers explain the abilities to their students, and then ask for ratings at desired time points during the course. This method promotes a positive teaching and learning culture in direct and indirect ways. (Contains 33 references.) (SLD)

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. This Three Ability Framework (3AF) is explained to the students by the lecturer, who is then rated by the students on each of these abilities. The lecturer rates himself or herself on the same abilities, and the alignment between the self-rating and the students' ratings is used as a measure of the effectiveness of teaching. The indicator of effective teaching is that the students and the instructor are working toward the same changes. This method promotes a positive teaching and learning culture in direct and indirect ways as it encourages teaching that promotes students' critical and evaluative thinking and high standards in technical skills and professional values. (Contains 33 references.) (SLD)

An instrument called the Selected Phrase Empathic Ability Key (SPEAK) was developed to measure empathy. The simply scored projective measure was then validated against physiological measures of reactive empathy. The SPEAK followed the classical instruments and used emotionally neutral line drawings (clip art) as projective stimuli for assessing empathy. The measure presented 113 of these drawings using a balanced analysis of variance design to 48 subjects. The reviewed 29-item instrument was administered to 1,323 mathematics staff and students, aged 15 to 63 years, from 24 tertiary institutions in England. Varimax factor analysis of the empathy scores showed two types of empathy. However, correlations with all 16 personality factors of Cattell's 16 Personality Factor (16PF) instrument were very low, with only two correlations reaching significance. This suggests that both types of empathy processes are probably additions to the 16PF. (Contains 1 figure, 2 tables, and 15 references.) (SLD)

This paper presents the results of research into a technique for the measurement of effective teaching that avoids the problems inherent in student evaluations of teacher effectiveness. The method takes as little as 5 minutes to administer and so can be used in-course by the teacher to track his or her teaching effectiveness. It results in a single number that can be used at the end of a course as an administrative decision point measure. The initial research with 3 classes of teacher training students (n=12, n=12, and n=23) was done in an Australian university and is now being replicated in other universities. The measure operationally defines the three basic abilities of the Three Ability Framework (3AF): technical skills, professional competence, and professional attitudes. The 3AF method assesses the degree of alignment between the changes the students expect in these three abilities and the changes toward which the teacher is working. Correlations between the scores of academic attainment, degree of teacher/student alignment, and course satisfaction indicated that when the students and their teacher were working towards the same proportion and amount of these three abilities, the students had higher academic attainment and greater course satisfaction. The four steps of the application of the method are outlined. (Contains 1 table and 28 references.) (SLD)

This paper describes a new, but tried and tested, paradigm in teacher evaluation: the Three Ability Framework (3AF). Previous research has identified three fundamental abilities that faculty implicitly expect to develop in their courses: technical skills, professional competence, and professional attitudes. The 3AF monitors the alignment of the student's and teacher's expectations of these developments. The process respects the professional freedom of lecturers to be responsible for how they believe their subjects should be taught. Because the process is so efficient, it can be used two or three times during courses to monitor the quality of teaching. This evaluation method also links student assessment to the professional awareness of the teacher, resulting in a single number that is used as an administrative decision point. The 3AF is applied in a five-step process that begins with training provided the lecturer by the institution. Alignment between student ratings and the lecturer's own rating is calculated. It has been found in previous research that the alignment of percentage ratings correlates with academic attainment as measured by examination and coursework and that the alignment of proportions of the three abilities correlates with enjoyment of the course. (Contains 14 references.) (SLD)

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. In practice, lecturers explain these three skill areas to their students and describe how students will be taught and assessed. Students are asked to rate the lecturer on these three areas, and then the lecturer rates himself or herself. The indicator of effective teaching, that the students and the lecturer are working toward the same changes, is measured by the change expected from the students subtracted from the change expected by the lecturer. A "zero" is the score that indicates perfect alignment. This method promotes a positive teaching and learning culture by encouraging teaching and learning that promotes students' critical and evaluative thinking. (Contains 31 references.) (SLD)