Search

This paper reports an inexpensive and efficient alternative method of measuring teaching quality. The method aligns students' in-course expectations for change with their lecturers expectation for change in three process objects: Skills, Understanding, and Attitudes. It is shown, using sample course data, that these in-course alignment indicators predict quality teaching, which is measured after the course using outcomes of students' academic achievements and course enjoyment. Traditional student evaluations of teaching are post-mortem measures that do not provide feedback to help the students in the course. However, a major contribution of the alignment method is that optimum in-course changes can be calculated to minimize the in-course alignment scores for the whole course or for any student subgroup of interest, thus maximizing students' predicted attainments. As the method only takes 5 minutes to administer, it can be introduced at the level of the individual instructor who wishes to keep his or her teaching on track. The alignment method uses a single form and results in one decision point number. It can be used by the administration at the end of courses for comparable promotion and tenure decisions across the institution. Web-based alignment software is now being developed to enable lecturers and administrators from tertiary institutions worldwide to avail themselves of the method. (Contains 9 tables, 5 figures, and 23 references.) (SLD)

This paper demonstrates an alternative method of using students' evaluations of teaching (SETs) that circumvents many of the problems associated with traditional SETs. It shows how course feedback, consisting of eight ratings, can be used to optimize post-course academic attainment. The method is illustrated with data from course feedback at the University of the West Indies. The new method allows feedback to be used to optimize teacher quality during the course for the whole class, for individuals, or for identified subgroups of students within the whole group. The method operationally defines three educational process objectives: Skills, Understandings, and Attitudes. Feedback forms used during the course give data on the lecturers' and students' expectations for change in the emphasis of these objectives. These data allow for calculations of the alignment between lecturers and students expectations for change. The Alignment theory presented is that students' academic success and enjoyment of teaching are maximized when students and their lecturer are working towards the same changes. The theory is revalidated with each course by correlations of alignments with results, which show that in-course alignment predicts post-course academic success. The paper also describes how the data are used during the course to determine the changes that will best align in-course student/lecturer expectations and maximize the post-course academic attainment for the whole group or different student subgroups. (Contains 5 figures, 9 tables, and 23 references.) (SLD)

The research literature on student evaluation of teaching (SET) is filled with criticisms of the process, its applications, and the student feedback questionnaire it uses. SETs are still used, however, because there has seemed to be no economical, valid, and reliable alternative. This paper reports on an alternative alignment process for evaluating quality teaching and learning that fits the requirements of economy, validity, and reliability, and offers additional institutional, faculty, and student benefits. The method measures both the individual student's and the lecturer's expectations for change in three process objectives that underlie quality teaching and learning: skills, understanding, and attitudes. These three objectives and their assessment are operationally defined. Methods for using these three process objectives as vehicles for teaching, learning, and course development are supported by the institution through staff and course development programs and through recognition of their assessment in student grades and in faculty promotion and tenure decisions. The post-course criteria measuring quality teaching and learning are students' high academic attainments and course enjoyment. Teaching techniques for attaining these goals are left as a matter of informed professional choice to the lecturer. The separately measured in-course predictors of these criteria are the alignments of student/lecturer expectations of change in the three objectives. The theory behind this approach is that quality learning results when students and lecturer are working toward the same goals. This paper introduced the method and presents evidence validating the theory and showing that the indicators correlate with the criteria. (Author/SLD)

Economic globalization make it increasingly important for Caribbean professionals to be able to communicate effectively in English, French, and Spanish. Accordingly, the University of Technology in Jamaica is developing French and Spanish courses designed to teach culturally appropriate and successful communication for specific professions. The design of these courses is radically different from the traditional university courses in both subject content and teaching methodology in order to accommodate the specific practical communication demands now placed on professionals in the Caribbean. This paper discusses the crucial importance of introducing specific occupational foreign language courses to equip undergraduate students with the fundamental foreign language skills necessary to function competently in linguistically diverse Caribbean work environments. It explains how linguistic competence is achieved by designing foreign language courses, using discipline-oriented lexical registers, content-based simulated situations, and nonconformist communicative contextual foreign language teaching methodology. Two illustrative examples of occupational French or Spanish course development for hotel and tourism management are described, highlighting the practical relevance of their course contents and teaching methodology to hotel and tourism management. (Contains 25 references.) (Author/KFT)

This paper explores the use of student evaluation of teaching (SET) as part of the quality assurance cycle and suggests an alternative approach to evaluate the provision of teaching in tertiary institutions. SETs usually involve questionnaires that ask students anonymously to rate the quality of teaching on a 4- or 5-point scale. SETs have been used in universities for more than 30 years to assess the quality of teaching, but the use of SET instruments had often been accompanied by counter-productive effects. Using them for proportion and tenure decisions had contributed to the lowering of academic standards. An alignment method of teacher evaluation yields better results. The alignment method considers skills, understanding, and attitudes, and whether the current state of these change attributes is closely aligned with the ideal state. Aligning changes expected by the lecturer with changes expected by the students results in improved teaching, and the evaluation of the alignment serves as an indicator of teacher effectiveness. (SLD)

This paper introduces a new instrument for assessing teacher motivation and uses it to compare the motivations of male and female teacher trainees in Jamaica. The results are then contrasted with the gender specific motivations that researchers have found for teachers in other parts of the world. The instrument is a three factor model based on a 13-item questionnaire that can be easily replicated for the assessment of teacher motivation. The three factors are extrinsic motivation, intrinsic motivation, and altruistic motivation. The items were developed from 15-minute semistructured individual and focus group interviews with 130 teacher trainees and teacher educators in Jamaica. The factor model was tested on 1,444 teacher trainees who represented an island-wide sample of one-third of the inservice and preservice teacher trainees covering all 3 years of teacher training in Jamaica. The model identified a 51% systematic variation in their responses. It was then used to compare the motivations of male and female teacher trainees and to explore how motivations for teaching were related to age and teaching experience. The results show similarities and differences in the gendered motivations of teachers reported from other countries, and they throw some light on the global phenomenon of the higher attrition for male teachers that can be of some use across cultures in policy decisions for teacher recruitment. (Contains 8 tables and 33 references.) (SLD)

Continuing education students bring with them expectations from previous teaching that challenge the effectiveness of usual teacher evaluation procedures. This paper presents a validated approach to the evaluation of teaching that identifies potential problems with individuals and minority groups of continuing students. This approach can be used during the course to steer teaching back on course before problems arise. The approach, the Three Ability Framework (3AF), results in a single quantitative measure of the quality of teaching that can be used as an administrative decision point. The 3AF feedback form takes only a few minutes to complete. The teacher explains to the students the three abilities research has identified as essential to effective teaching: (1) technical skills; (2) professional competence; and (3) professional attitudes. Students are asked to rate their assessments of each of these areas and their expectations of how the areas should be fulfilled in the course, and instructors rate themselves in each area. Student and instructor ratings are compared (student change rating is subtracted from lecturer change rating), and the resulting number is makes an evaluative decision point. "Zero" would be a perfect indicator of the alignment between student and instructor expectations that is indicative of effective teaching. The method allows frequent assessments over the duration of the course, permitting adjustments when necessary, and it is particularly suited to the evaluation of courses in continuing education. (Contains 35 references.) (SLD)

This paper presents an alternative to student questionnaires for the evaluation of teaching quality, which is defined as maximizing students' academic attainment and the satisfaction of students and the lecturer with the course. The approach aligns students' and the teacher's expectations for change during the course. Previous research has identified three fundamental abilities that lecturers seek to change during the course: (1) technical skills; (2) professional competence; and (3) professional attitudes. A Three Ability Framework (3AF) has been developed to compare students' assessments of the change in these three abilities with the instructor's perceptions of change. The alignment between these evaluations, expressed as the result of subtracting student-rated change from lecturer-rated change, is a figure that can be used as the effective teaching score. To use the 3AF in practice, lecturers explain the abilities to their students, and then ask for ratings at desired time points during the course. This method promotes a positive teaching and learning culture in direct and indirect ways. (Contains 33 references.) (SLD)

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. This Three Ability Framework (3AF) is explained to the students by the lecturer, who is then rated by the students on each of these abilities. The lecturer rates himself or herself on the same abilities, and the alignment between the self-rating and the students' ratings is used as a measure of the effectiveness of teaching. The indicator of effective teaching is that the students and the instructor are working toward the same changes. This method promotes a positive teaching and learning culture in direct and indirect ways as it encourages teaching that promotes students' critical and evaluative thinking and high standards in technical skills and professional values. (Contains 33 references.) (SLD)

The accuracy of the mean of two estimates was compared with the accuracy of a single independent estimate from the same subject. A subject was asked to estimate the size of one attribute of a constant stimulus, e.g., the total of a set of numbers. The same subject was also asked to give an estimate for an upper and lower bound on the size of the same attribute of the same stimulus. The experiment was designed to ensure that the single and double estimates were independent and given by the subjects under the same experimental conditions. The experimental design compared the accuracy of estimates of two stimulus attributes using a 3 by 4 randomized block design. Participants were 187 college students competing for $10 prizes with the time severely limited. Only 50 subjects managed to complete the tasks. The frequency with which one estimate was either extremely high or extremely low suggests that the levels of task complexity were too high for the stress level of the timed competition. The results have potential ramifications for methods of collecting judgmental data, but future research should use a task of more appropriate complexity. (Contains 1 figure, 2 tables, and 19 references.) (SLD)

This paper describes the subjectivist teaching paradigm, which has been introduced into a mathematics education course at the University of the South Pacific. Subjectivism integrates such aspects of current teaching philosophy as lifelong learning through student empowered, self-directed learning. Subjectivism, which is student centered, emphasizes the individual's subjective learning experience by adding feelings, interest, and motivation to enhance cognitive learning in constructivist classrooms. It involves engrossing socially constructed classroom activities to enculturate students into the skills and values of the content subject. Two humanitarian aims of subjectivism are subject enculturation and student empowerment. Subjectivism achieves these aims using similar enculturation processes in classroom activities that children experience in learning their own sociocultural processes and values. Enculturation processes include peer pressure, social recognition, compliance with authority, shared experience, establishing role identity, in-group bonding, and out-group competition. Hence, classroom activities focused on the subject content become authentic living experiences. This paper illustrates how these social constructivist activities, called surface purposes, are designed using three affect-structuring techniques (emotional anchors, motivators, and cognitive direction). It describes empowering enculturating activities that have been successfully used with education students and school children. (Contains 45 references.) (SM)

An instrument called the Selected Phrase Empathic Ability Key (SPEAK) was developed to measure empathy. The simply scored projective measure was then validated against physiological measures of reactive empathy. The SPEAK followed the classical instruments and used emotionally neutral line drawings (clip art) as projective stimuli for assessing empathy. The measure presented 113 of these drawings using a balanced analysis of variance design to 48 subjects. The reviewed 29-item instrument was administered to 1,323 mathematics staff and students, aged 15 to 63 years, from 24 tertiary institutions in England. Varimax factor analysis of the empathy scores showed two types of empathy. However, correlations with all 16 personality factors of Cattell's 16 Personality Factor (16PF) instrument were very low, with only two correlations reaching significance. This suggests that both types of empathy processes are probably additions to the 16PF. (Contains 1 figure, 2 tables, and 15 references.) (SLD)

This paper presents the results of research into a technique for the measurement of effective teaching that avoids the problems inherent in student evaluations of teacher effectiveness. The method takes as little as 5 minutes to administer and so can be used in-course by the teacher to track his or her teaching effectiveness. It results in a single number that can be used at the end of a course as an administrative decision point measure. The initial research with 3 classes of teacher training students (n=12, n=12, and n=23) was done in an Australian university and is now being replicated in other universities. The measure operationally defines the three basic abilities of the Three Ability Framework (3AF): technical skills, professional competence, and professional attitudes. The 3AF method assesses the degree of alignment between the changes the students expect in these three abilities and the changes toward which the teacher is working. Correlations between the scores of academic attainment, degree of teacher/student alignment, and course satisfaction indicated that when the students and their teacher were working towards the same proportion and amount of these three abilities, the students had higher academic attainment and greater course satisfaction. The four steps of the application of the method are outlined. (Contains 1 table and 28 references.) (SLD)

This paper reports the results of an experiment on the accuracy of information processing during attention focused arousal under two conditions: single estimation and double estimation. The attention of 187 college students was focused by a task requiring high level competition for a monetary prize ($10) under severely limited time conditions. The task was to give an accurate estimate of how many numbers there were in a set of numbers and to estimate their total accurately. Under the first condition, only one estimate was required, but under the second condition, upper and lower estimates are required. Of the 50 subjects who were able to complete the tasks under the time constraints, a high number reported extremely unbalanced estimates, revealing a lack of consistency between single and mid-double (mean of the double estimates) estimates that was indicative of overarousal for the levels of complexity of the tasks. The mid-double estimate appeared more accurate than the traditional single estimate, but additional research should be done where the press is reduced to match task complexity or the task complexity is reduced to match the level of the press. (SLD)

A method for measuring the contribution an individual makes to group work is described, and its use is supported through a study of 57 university students aged from 20 to 46 years working in 8 groups of 4 to 10 members each. The method recognizes that the most valid sources of information on the contribution of each individual to the group work are the group members. The study relied on peer evaluation of the percentage contribution that each member made to the group work. Group members were also asked for a rationale for each evaluation given. The reliability of the individual's mark was indicated by the variance of the marks the group member received from his or her peers. Additional validity was provided by the qualitative agreement of the rationales. The group work was assessed separately using usual appropriate criteria. An individual's mark was calculated as his or her mean percentage of the group's mark, weighted by the number of people in the group. By separating the measurement of process from the assessment criteria for the group work, the method highlighted the cooperative intragroup dynamics. The method also allowed the identification of consistent patterns of correlations within group scores that indicate a problem common to students with low achievement in group work that is independent of the subject assessed. (Contains 2 tables and 18 references.) (SLD)

Why trainee teachers join the teaching profession in Jamaica was studied. Previous research has suggested that their reasons may be extrinsic (related to characteristics of the job), intrinsic (related to the individual's wishes), or altruistic (related to the social contribution). Some reasons that could be classified into these 3 categories were identified in 130 interviews with students from the Teaching College of the University of the West Indies. Approximately one third of the students at Jamaican colleges of education (n=1,444) were asked to rate their agreement with each of the 19 reasons on a scale from 0 to 9. Results indicate that extrinsic, intrinsic, and altruistic consideration were three distinct motivations teacher trainees had for choosing the profession. Extrinsic motivation was the most important, accounting for 24.2% of the variance, as compared to 14.6% for altruistic motivation, and 8.8% for intrinsic motivation. (Contains 3 tables and 16 references.) (SLD)

This paper explores the issue of why some students in the Caribbean do not do as well as expected in assessed groupwork. The study was based on peer assessment in 2 university courses taken by 57 students who worked in 8 groups (group sizes 4 to 10). Both males and females participated, their ages ranging from 20 to 46 years. The study was designed to focus on fundamental learning problems rather than on problems that might be associated with learning the content of a particular course. This was done by separating the assessment of the final quality of each group's work from the assessment of individual contributions to the work and by duplicating the study in two courses. Findings indicate that a fundamental problem associated with less able students is their lack of discrimination about what the work entails. These findings are robust in that the correlations show that the effect is not just apparent for low achieving students, but that the effect decreases as the ability of the student increases. Findings were the same across groups of different sizes and across content areas. Suggestions are made about how students might reduce this problem. (SLD)

This paper describes a new, but tried and tested, paradigm in teacher evaluation: the Three Ability Framework (3AF). Previous research has identified three fundamental abilities that faculty implicitly expect to develop in their courses: technical skills, professional competence, and professional attitudes. The 3AF monitors the alignment of the student's and teacher's expectations of these developments. The process respects the professional freedom of lecturers to be responsible for how they believe their subjects should be taught. Because the process is so efficient, it can be used two or three times during courses to monitor the quality of teaching. This evaluation method also links student assessment to the professional awareness of the teacher, resulting in a single number that is used as an administrative decision point. The 3AF is applied in a five-step process that begins with training provided the lecturer by the institution. Alignment between student ratings and the lecturer's own rating is calculated. It has been found in previous research that the alignment of percentage ratings correlates with academic attainment as measured by examination and coursework and that the alignment of proportions of the three abilities correlates with enjoyment of the course. (Contains 14 references.) (SLD)

This paper describes subjectivism, a new learning paradigm that incorporates successful pedagogic practices from the past into an affect-structured constructivism. Its two goals are student empowerment and subject enculturation. Student empowerment is achieved by designing learning for success so students can take credit for and feel confidence from this success. Subject enculturation attaches professional attitudes, appreciation, and guiding values to the course content. Examples of enculturation processes include peer pressure, social recognition, compliance with authority, shared experience, establishing role identity, in-group bonding, and out-group competition. Student empowerment develops autonomous and self-directed learning. Surface purpose activities (social constructivist activities) use affect-structuring techniques to enhance student empowerment. Three affect-structuring techniques for designing surface purpose activities include: an emotional anchor, a motivator, and cognitive direction. The paper describes how four activities allowed students at the University of the South Pacific, Fiji, to perfect what they planned to say on a television debate. (Contains 24 references.) (SM)

This paper presents an alternative means of assessing faculty teaching that can be used by administrators. This method has been derived from in-depth faculty and student interviews, and its criterion of quality has been validated with empirical data through computer sensitivity analysis. The assessment process has been successfully tested in clinical teaching trials. Interviews with faculty have identified that their implicit expectations for professional development can be described in terms of technical skills, professional competence, and professional attitudes. In practice, lecturers explain these three skill areas to their students and describe how students will be taught and assessed. Students are asked to rate the lecturer on these three areas, and then the lecturer rates himself or herself. The indicator of effective teaching, that the students and the lecturer are working toward the same changes, is measured by the change expected from the students subtracted from the change expected by the lecturer. A "zero" is the score that indicates perfect alignment. This method promotes a positive teaching and learning culture by encouraging teaching and learning that promotes students' critical and evaluative thinking. (Contains 31 references.) (SLD)

This paper introduces the subjectivist teaching/learning paradigm (STP), which uses students' social and cultural enculturation processes to structure affect-laden constructivist learning experiences focused in the subject area being taught. STP uses powerful learning experiences to promote two goals: enculturation into the subject area and student empowerment. STP techniques for practical teaching mirror the effective processes of enculturation by which students learn the skills, understandings, and attitudes of their own sociocultural groups. STP lessons concentrate real life subjective experiences into a learning area. STP helps students learn to choose what to learn and how to learn it, offering them an increasing variety of learning experiences. It develops student empowerment by continually reinforcing successful self-directed learning through affect structuring, covert directives, and self-cuing coping strategies. Surface purposes begin with three affect structuring techniques (emotional anchor, motivator, and cognitive direction), which utilize the positive subject experiences of affect, self-selection, and motivation. The paper illustrates surface purposes within a workshop at the University of the South Pacific, Fiji, in which students practiced expressing their disapproval of nuclear testing in the Pacific. Six workshop activities are described, highlighting their setting, surface purpose, and hidden pedagogic purpose. (Contains 24 references.) (SM)