Search

Feedback is considered of great relevance for supporting student learning. It is therefore the focus of a significant body of theoretical work and is included in many observation frameworks for measuring teaching quality. However, little is currently known about the extent to which the theoretical and empirical knowledge of feedback from the literature is represented in operationalizations of feedback in observation frameworks. In this exploratory study, we first reviewed the literature and identified nine quality criteria for effective feedback. Using content analysis, we then explored the extent to which 12 widely used observation frameworks for teaching quality reflect these criteria and the similarities and differences in their approaches to capturing feedback quality. Only ten of the 12 frameworks measured feedback. Nine frameworks addressed feedback directly, while one framework only captured feedback indirectly. All frameworks differed in the number of feedback quality criteria they captured, the aspects they focused on for each one, and the detail in which they described them. One criterion (Feed Up) was not captured by any framework. The results show that more clarity is needed about which facets of feedback are integrated into frameworks and why. The study also highlights the importance of finding ways to complement observation frameworks with other measures so that feedback quality is captured in a more comprehensive fashion.

Die Unterrichtsqualitätsforschung hat in den vergangenen Jahrzehnten viel zum Verständnis der Merkmale beitragen, die einen qualitätsvollen Unterricht auszeichnen. Sie hat einflussreiche Modelle hervorgebracht, die eine gemeinsame Grundlage für die Erforschung des Unterrichts bereitstellen. Gleichzeitig bringen gesellschaftliche Veränderungsprozesse eine Neuorientierung des schulischen Unterrichts mit sich, und es lässt sich fragen, ob die Unterrichtsqualitätsforschung mit ihren bisherigen Ansätzen in der Lage ist, in Zukunft zum Gelingen von Unterricht beizutragen. Im Rahmen einer Zukunftswerkstatt hat das Leibniz-Netzwerk Unterrichtsforschung daher einen längerfristig angelegten Prozess gestartet, um sich mit dem Unterricht der Zukunft zu beschäftigen und zu versuchen, drängende Fragen und Handlungsfelder für die Unterrichtsqualitätsforschung zu identifizieren. Der vorliegende Beitrag stellt die Ergebnisse dieses initialen Austauschs dar. Zudem wird ein Einblick in die gegenwärtige Arbeit im Netzwerk gegeben, die aufbauend auf den Ergebnissen der Zukunftswerkstatt eine Weiterentwicklung der Unterrichtsqualitätsforschung anstrebt. English version Over the past decades, research on teaching quality has substantially contributed to the understanding of high quality teaching. It has brought forth influential models that provide a common ground for research on teaching. At the same time, societal change requires a reorientation of teaching in schools. It seems questionable whether the current approaches of teaching quality research can contribute to the success of teaching in the future. Therefore, based on a future workshop method, the Leibniz Network of Research on Teaching Quality has started a longer-term working process to consider future developments of teaching and to identify urgent questions and fields of action for research on teaching quality. The present paper outlines the results of this initial exchange. Furthermore, it gives insight into the current work within the network that aims at an advancement of research on teaching quality based on the results of the workshop., +repphzhbib2023A

Classroom observations are frequently used in teacher evaluation, training, and professional development as well as in research. Ensuring that the observations are reliable and valid is therefore of great importance. Studies have shown however, that they are often neither, even when the raters are trained (Bell et al., 2014; Gitomer et al., 2014; Praetorius & Charalambous, 2018). There is limited systematic and causal evidence about ways of enhancing the reliability and validity of such ratings (Cohen & Goldhaber, 2016; Hill, Charalambous, & Kraft, 2012; Praetorius, 2014). The aim of this project is to systematically investigate aspects of rater training and their effect on the quality (i.e., reliability and validity) of judgments of teaching quality. We aim to generate evidence for how the training of raters can be optimized, taking into account feasibility issues.

ZusammenfassungDie Unterrichtswissenschaft feiert ein Jubiläum – sie erscheint im Jahr 2023 seit 50 Jahren. Dieser Beitrag ist die Einführung für das Jubiläumsheft (Heft 1, 2023). Er stellt zunächst die Gründung der Zeitschrift dar. Anhand der Protokolle der Herausgebersitzungen wird skizziert, wie sich die „Unterrichtswissenschaft“ als Zeitschrift der empirischen Lehr-Lernforschung entwickelt hat und welche Herausforderungen dabei zu bewältigen waren. Abschließend werden die bisherigen Überlegungen der derzeitigen Herausgeber*innen für die Entwicklung der Zeitschrift und der Lehr-Lern-Forschung vorgestellt.

In this project, we investigate teacher judgment accuracy of students' personality profiles based on brief videos and no prior acquaintance between perceivers and students using the Social Accuracy Model as analytical approach (SAM).

On the basis of Bandura’s social cognitive theory, researchers often assume that a teachers’ self-efficacy (TSE) will have a positive effect on teaching quality. However, the available empirical evidence is mixed. Building on previous research into TSE, we examined whether assessing class-/task-specific TSE gives a more accurate indication of the associations between TSE assessments and student-rated teaching quality. The analyses were based on the English sample of the Teaching and Learning International Survey (TALIS) Video Study. Mathematics teachers (N = 86) rated their self-efficacy beliefs using generalized task-specific TSE items and class-/task-specific TSE items. Their students (N = 1,930) rated the quality of teaching in their math class. Multilevel regression analyses revealed stronger associations between student-rated teaching quality and class-/task-specific TSE than generalized task-specific TSE. We discuss possible reasons for these results and outline the potential benefits of using class-specific assessments for future TSE research.

The present study investigated whether the varying referents (i. e., teacher or student referent) of student ratings of established classroom management measures differ in their associations with compositional classroom characteristics and students' math achievement. Re-analysis of a large-scale dataset (PISA 2003) showed that classrooms with a higher proportion of male students, as well as those with lower math performance, exhibited lower scores on classroom management factors referring more to students than the teacher. These were in turn related to lower pre-adjusted math achievement of students. There were no associations with a measure referring to the teacher. Our results indicate that varying referents tap into different aspects of the classroom management process. (DIPF/Orig.) In der vorliegenden Studie wurden die Konsequenzen variierender Referentenbezüge (Lehrkraft vs. Schülerinnen und Schüler) bei etablierten Skalen zur Erfassung der Klassenführung aus Schülersicht untersucht. Die Ergebnisse einer Re-analyse der PISA 2003-Daten zeigten, dass Klassen mit höherem Jungenanteil und niedrigerem mittleren Leistungsniveau niedrigere Werte auf Skalen mit stärkerem Schülerbezug aufwiesen. Diese waren wiederum mit einer geringeren Leistungsentwicklung von Schülerinnen und Schülern im Fach Mathematik assoziiert. Für eine Skala mit Lehrkraftbezug fanden sich hingegen keine Zusammenhänge. Die Ergebnisse legen nahe, dass der Referentenbezug von Skalen für die erfassten Aspekte der Klassenführung entscheidend ist. (DIPF/Orig.)

As part of the social distancing measures for preventing the spread of COVID-19, many university courses were moved online. There is an assumption that online teaching limits opportunities for fostering interpersonal relationships and students’ satisfaction of the basic need for relatedness – reflected by experiencing meaningful interpersonal connections and belonging – which are considered important prerequisites for student motivation and vitality. In educational settings, an important factor affecting students’ relatedness satisfaction is the teachers’ behavior. Although research suggests that relatedness satisfaction may be impaired in online education settings, to date no study has assessed how university lecturers’ relatedness support might be associated with student relatedness satisfaction and therefore, student motivation and vitality. This study tested this mediating relationship using data collected during the early days of the COVID-19 pandemic. The study also investigated whether the relations were moderated by a high affiliation motive which reflects a dispositional wish for positive and warm relationships. The possible importance of the communication channel selected by the lecturers (video chat yes/no) and the format of a class (lecture/seminar) were also investigated. In a sample of N = 337 students, we tested our hypotheses using structural equation model (SEM). Results confirmed mediation, but not moderation. The use of video chat (video call) seems to facilitate the provision of relatedness support but our data did not show that the format of a class was associated with relatedness. Our findings indicate that both teaching behavior and the technical format used to deliver lectures play important roles in student experiences with online classes. The results are discussed in light of other research conducted during the pandemic. (DIPF/Orig.)

This initial report gives an overview of some preliminary results. Please note that due to the explorative nature of some analyses, the significant tests should only be used to aid interpretation. We recommend focusing to the effect sizes presented when interpreting the results (see Wasserstein & Lazar, 2016). The interim report starts by describing the demographic data of our student and university teacher sample. Following that, some general findings on students’ and lecturers’ general life satisfaction and well-being are presented. Well-being is measured by self-reports of perceived stress which is compared to ratings of a norm sample (Fliege et al., 2005). In the paragraph on teaching, students’ and university lecturers’ attitudes towards e-learning are presented. Moreover, an overview on the choice of communication channels is provided along with the type of course (seminar, lecture, etc.). Additionally, the student perceived interaction between students and lecturers during online teaching is discussed. Following this, lecturer behavior concerning the support of students’ basic psychological needs is presented, as well as the teachers’ and students’ motivation and well-being during classes. To conclude the interim report, the descriptive data on perceived support from the university and peers from the perspective of lecturers is presented.

Dieser Beitrag widmet sich Angebots-Nutzungs-Modellen der Wirkweise des Unterrichts. Konkreter hat er zum Ziel, verschiedene Angebots-Nutzungs-Modelle zu vergleichen und durch diesen Vergleich konzeptuelle Unschärfe innerhalb des Ansatzes aufzudecken. Kritisch diskutiert werden sollen Unterschiede hinsichtlich a) dem zugrundeliegenden Verständnis von Unterricht, Angebot und Nutzung, b) Zusammenhängen zwischen Angebot und Nutzung, c) der Bedeutung von Wahrnehmung und Interpretation, d) der Verortung von Angebot und Nutzung im Mehrebenensystem, e) der Bedeutung von Kontexten und f) der aufgeführten Kriterien unterrichtlicher Wirksamkeit. Der Beitrag schließt mit einem Fazit, in dessen Rahmen ein integriertes Angebots-Nutzungs-Modell vorgestellt wird.

Dieser Beitrag widmet sich Angebots-Nutzungs-Modellen der Wirkweise des Unterrichts. Konkreter hat er zum Ziel, verschiedene Angebots-Nutzungs-Modelle zu vergleichen und durch diesen Vergleich konzeptuelle Unschärfe innerhalb des Ansatzes aufzudecken. Kritisch diskutiert werden sollen Unterschiede hinsichtlich a) dem zugrundeliegenden Verständnis von Unterricht, Angebot und Nutzung, b) Zusammenhängen zwischen Angebot und Nutzung, c) der Bedeutung von Wahrnehmung und Interpretation, d) der Verortung von Angebot und Nutzung im Mehrebenensystem, e) der Bedeutung von Kontexten und f) der aufgeführten Kriterien unterrichtlicher Wirksamkeit. Der Beitrag schließt mit einem Fazit, in dessen Rahmen ein integriertes Angebots-Nutzungs-Modell vorgestellt wird. (DIPF/Orig.) This paper discusses so-called 'opportunity-use models of the effects of teaching'. More specifically, it compares different 'opportunity-use-models' with the aim to unveil existing conceptual fuzziness. The following issues will be critically discussed: a) the underlying conception of 'teaching', 'opportunity', and 'use', b) relations between 'opportunity' and 'use', c) the role of perception and interpretation, d) the localization of 'opportunity' and 'use' in a multilevel system, e) the role of the context, and f) the selection of teaching effectiveness criteria. In the conclusion an integrated opportunity-use model is developed. (DIPF/Orig.)

One puzzling finding in education research is that teachers, students, and external observers agree only marginally on their ratings of teaching quality. In this theoretical contribution, we summarize and reappraise previous findings on agreement between different raters of teaching quality. We explain these findings by thoroughly examining the instruments that have been used to measure teaching quality. Building on this, we propose a reference perspective matrix, which should be useful in explaining perspective-specific rating mechanisms behind responses to certain survey or observation items. The reference perspective matrix could thus afford a theoretical foundation for future studies on the assessment of teaching quality. (DIPF/Orig.) Die Urteile von Schüler*innen, Lehrkräften und externen Beobachter*innen zur Unterrichtsqualität stimmen häufig nur in geringem Maße überein. In dem vorliegenden konzeptuellen Beitrag geben die AutorInnen einen Überblick über bisherige empirische Befunde zu Perspektivenvergleichen und versuchen diese Befunde durch eine Analyse der eingesetzten Erhebungsinstrumente zu erklären. Darauf aufbauend skizzieren sie eine Referenten-Perspektiven-Matrix zur Klassifikation existierender Fragebogen- und Beobachtungsitems. Diese kann zur Erklärung der perspektivenspezifischen Mechanismen beitragen, welche der Beantwortung von Items zugrunde liegen. Die vorgestellte Matrix bietet damit auch eine Grundlage für künftige Arbeiten zur Erfassung von Unterrichtsqualität. (DIPF/Orig.)

Sowohl in der Forschung als auch in der schulischen Praxis wird zunehmend auf Schülerfeedback rekurriert, um Auskunft über Unterrichtsqualität zu erhalten. Für die prognostische Validität der auf Klassenebene aggregierten Schülerurteile gibt es insbesondere im Bereich der Sekundarstufe zahlreiche empirische Befunde (z. B. Clausen, 2002). Hinweise auf faktorielle Validität liegen ebenfalls vor, auch für die Primarstufe (Fauth, Decristan, Rieser, Klieme & Büttner, 2014a; Lenske, 2016). Weiterführende Untersuchungen auf Basis von mixed-method-Ansätzen ergaben jedoch, dass es teilweise schwierig ist, die identifizierten Faktoren zu interpretieren, da die dazugehörigen Items im Primarbereich häufig nicht wie intendiert beantwortet werden (Lenske, 2016). Wie Untersuchungen in der Primarstufe zeigen, reicht eine Faktorenstruktur als Validitätshinweis nicht aus und kann für die Unterrichtsentwicklung sogar irreführend sein. Es stellt sich nun die Frage, ob mit dem Übergang in die Sekundarstufe die Schwierigkeiten bezüglich des Verstehens und Beurteilens von Items zur Unterrichtsqualität tatsächlich ausbleiben. Dieser Frage geht der vorliegende Beitrag nach.Im Rahmen einer qualitativen Studie wurden mittels Probing 14 Schüler/innen der Sekundarstufe im Anschluss an eine Unterrichtsstunde aufgefordert, die Gründe für ihr Kreuz auf dem Fragebogen zu dieser Stunde zu erläutern. Die Resultate der Inhaltsanalyse verdeutlichen, dass es zwischen den Unterrichtsqualitätskriterien deutliche Unterschiede bezüglich des Itemverständnisses gibt. Vergleichbar mit den Untersuchungen zur Primarstufe wurden die Items zu Klassenführung vergleichsweise gut verstanden; Items zu Differenzierung und insbesondere zu kognitiver Aktivierung wurden von der Mehrheit der Schüler/innen hingegen nicht im intendierten Sinn verstanden. Both in research and school practice, student feedback is increasingly used to obtain information about the quality of teaching. There are numerous empirical findings on the predictive validity of student questionnaires aggregated on the classroom level (e. g. Clausen, 2002), especially for the secondary school students. There is also evidence for factorial validity, on both primary and secondary school level (Fauth, Decristan, Rieser, Klieme & Büttner, 2014a; Lenske, 2016). However, further investigations based on mixed-method approaches have shown that it might be difficult to properly interpret the identified factors, since the associated items are often not answered as intended, at least on primary school level (Lenske, 2016). These studies at primary school level indicate that the factor structure alone is not sufficient as a validity indicator and can even be misleading for instructional development. It is unclear, though, whether the difficulties in understanding and answering such items disappear with the transition to secondary school. To examine this question in the context of a qualitative study, 14 secondary school students were asked to explain the reasons for their answers on a questionnaire regarding a specific lesson. The results of the content analysis showed that there are clear differences in item understanding depending on the teaching quality dimension. Similarly to the primary school level studies, the items regarding classroom management were interpretedcomparably well; items on differentiation and especially cognitive activation were not understood in theintended sense by most students.

In this paper we elaborate upon the relevance of theories of teaching (quality) in quantitative empirical research on teaching. First we introduce, the quantitative empirical research approach. Then, we present the origin and current status of research with respect to a model – the Three Basic Dimensions of teaching quality that is especially popular in quantitative research on teaching quality in German-speaking countries. Next, we reflect on the extent to which the model fulfills criteria for a good theory, before deriving conclusions for future research that focuses on a process of successive theory building., Zeitschrift für Pädagogik - Beiheft, vol. 66, no. 1

In this paper we elaborate upon the relevance of theories of teaching (quality) in quantitative empirical research on teaching. First we introduce, the quantitative empirical research approach. Then, we present the origin and current status of research with respect to a model - the Three Basic Dimensions of teaching quality - that is especially popular in quantitative research on teaching quality in German-speaking countries. Next, we reflect on the extent to which the model fulfills criteria for a good theory, before deriving conclusions for future research that focuses on a process of successive theory building. (DIPF/Orig.) Der vorliegende Beitrag fokussiert auf die Bedeutung von Theorien zu Unterricht und dessen Qualität für die quantitative empirische Unterrichtsforschung. Dabei wird zunächst der quantitative empirische Forschungsansatz vorgestellt. Anschließend stellen wir die Herkunft und den aktuellen Status eines populären Modells in der deutschsprachigen quantitativen Unterrichtsforschung dar, dem Modell der drei Basisdimensionen von Unterrichtsqualität. Es folgt eine Reflexion in welchem Ausmaß dieses Modell Kriterien guter Theorien erfüllt. Abschließend werden Schlussfolgerungen für zukünftige Forschung mit einem Fokus auf die Entwicklung von Theorien gezogen. (DIPF/Orig.)

In this synthesis paper, the current state of research regarding teaching quality is discussed comparing science education, physical education, and history education, by referring to research on teaching in general. A framework with seven dimensions (Praetorius and Charalambous 2018) was used as a comparison base. This framework contains the dimensions of selecting and introducing the content and subject matter methods, cognitive activation, support of practicing, formative assessment, support of all students’ learning, socio-emotional support, and classroom management. The comparison across subject matters showed that subject-specificity was dependent on the level under consideration: On the dimension level, no adaptations and supplementations were deemed necessary. On the level of sub-dimensions, and especially on the level of observable indicators for these sub-dimensions, this was, however, the case. Challenges for future research are discussed. © 2020, The Author(s)., Im vorliegenden Diskussionsbeitrag wird der Forschungsstand zu Unterrichtsqualität für verschiedene Fachdidaktiken – naturwissenschaftliche Fächer, Sport und Geschichte – vor dem Hintergrund der allgemeinen empirischen Unterrichtsforschung vergleichend diskutiert. Als Vergleichsgrundlage wurde ein Framework mit den sieben Dimensionen Auswahl und Thematisierung von Inhalten und Fachmethoden, kognitive Aktivierung, Unterstützung des Übens, formatives Assessment, Unterstützung des Lernens aller Schülerinnen und Schüler, sozio-emotionale Unterstützung und Klassenführung genutzt (Praetorius und Charalambous 2018). Es zeigte sich, dass die Fachspezifität von der fokussierten Ebene abhing: Während eine Ergänzung des Frameworks auf Ebene der sieben Dimensionen nicht notwendig erschien, waren auf der Ebene der Subdimensionen und insbesondere der diesen Subdimensionen zugeordneten beobachtbaren Indikatoren fachspezifische Anpassungen und Ergänzungen erforderlich. Der Beitrag schließt mit einer Diskussion zentraler Herausforderungen für zukünftige Forschung.

In C. Köhler & A. Naumann (Chairs), Herausforderungen bei der Messung der Wirkungen von Unterricht. 8. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Potsdam (D), März 2020, [Konferenz abgesagt]

The three basic dimensions framework for assessing quality teaching distinguishes the dimensions of cognitive activation, student support, and classroom management. Research from various disciplines suggests, however, that cognitive support is not sufficiently represented in the framework as yet. In the present study, two-level factor analyses based on student ratings of teaching quality in science education (2,659 students, grades 4 and 6) suggest four dimensions of teaching quality, with cognitive support being a separate dimension. Moreover, cognitive support predicted student achievement. The results suggest that including cognitive support as a separate dimension contributes to a more comprehensive, yet parsimonious framework of teaching quality. (DIPF/Orig.) Das Modell der drei Basisdimensionen von Unterrichtsqualität unterscheidet die Dimensionen kognitive Aktivierung, konstruktive Unterstützung und Klassenführung. Forschung aus unterschiedlichen Disziplinen legt nahe, dass kognitive Unterstützung in dem Modell nicht hinreichend berücksichtigt ist. Faktorenanalysen auf der Basis von Schülereinschätzungen zum naturwissenschaftlichen Unterricht (2,659 Schüler*innen, Klasse 4 und 6) weisen in der vorliegenden Studie auf ein vierdimensionales Modell mit kognitiver Unterstützung als separater Dimension hin. Kognitive Unterstützung sagte zudem den Lernerfolg der Schüler*innen voraus. Die Ergebnisse unterstreichen, dass die Ergänzung um diese Dimension zu einem vollständigeren, aber dennoch sparsamen Modell von Unterrichtsqualität beiträgt. (DIPF/Orig.)

In L. Grützmacher & A. Naumann (Chairs), Relationship between heterogeneity and students’ learning gains. EARLI Special Interest Group (SIG) 18, Freiburg (D), September 2020, [Conference postponed]

Observation-based frameworks of instructional quality differ largely in the approach and the purposes of their development, their theoretical underpinnings, the instructional aspects covered, their operationalization and measurement, as well as the existing evidence on reliability and validity. The current paper summarizes and reflects on these differences by considering the 12 frameworks included in this special issue. By comparing the analysis of three focal mathematics lessons through the lens of each framework as presented in the preceding papers, this paper also examines the similarities, differences, and potential complementarities of these frameworks to describe and evaluate mathematics instruction. To do so, a common structure for comparing all frameworks is suggested and applied to the analyses of the three selected lessons. The paper concludes that although significant work has been pursued over the past years in exploring instructional quality through classroom observation frameworks, the field would benefit from establishing agreed-upon standards for understanding and studying instructional quality, as well as from more collaborative work. 50 3 535 553

Zielorientierungen von Lehrkräften stellen wesentliche Komponenten ihrer Motivation dar und sind relevant für ihr Erleben und Verhalten. Zu ihrer Förderung wurde ein Interventionsansatz entwickelt, bei dem auch die Merkmale effektiver Lehrkraftfortbildungen berücksichtigt wurden. Im Sinne einer Doppelstrategie fokussierte er daneben auch direkt motivationsförderliches Unterrichtshandeln (Herstellung einer Lernzielstruktur im Unterricht). An drei Workshop-Tagen wurde Lehrkräften Wissen zu Zielorientierungen vermittelt, dessen Implementierung sie in praktischen Phasen einübten. In diesem Rahmen erhielten sie zweimalig Feedback zu ihrem Unterricht. Zudem wurde ein Tagebuchansatz zur Reflexion von Zielsetzungsprozessen genutzt. Zur Evaluation des Interventionsansatzes wurde er mit Lehrkräften aus bayerischen Mittelschulen umgesetzt. Realisiert wurden eine Trainingsgruppe mit 40 Lehrkräften und eine Kontrollgruppe mit 29 Lehrkräften; daneben wurden die insgesamt 1172 Schülerinnen und Schüler dieser Lehrkräfte einbezogen. Durchgeführt wurden ein Prätest, ein Posttest und ein Follow-up (drei Monate nach dem letzten Workshop-Tag). Die Analysen zeigten einen hohen subjektiven Lernertrag und wahrgenommenen Nutzen seitens der Lehrkräfte sowie eine sehr große Zufriedenheit mit dem Interventionsansatz. In Analysen der Veränderungen über die drei Messzeitpunkte im Vergleich zur Kontrollgruppe waren jedoch keine Effekte auf Lehrkraftziele, wahrgenommene Klassenzielstruktur oder die Motivation der Schüler(innen) nachweisbar. Basierend auf den gewonnenen Befunden werden Konsequenzen für die Konzeption und Evaluation zukünftiger Lehrkrafttrainings diskutiert. Achievement goals of school teachers are relevant for their cognition and behaviour at work. To facilitate an orientation towards favourable achievement goals, we developed an intervention approach based on features of effective professional development courses for in-service teachers. At three workshop days, we focussed on knowledge on achievement goals of school teachers, and practiced its implementation. To this end, teachers twice received feedback in regard to their teaching. Additionally, a diary approach was used to support reflection of personal goal setting. For the purpose of evaluation, we implemented the intervention with teachers from secondary schools. Specifically, 40 teachers participated in a treatment group, and 29 in a control group; including their altogether 1172 students. We conducted a pretest, a posttest, and a follow-up test (three months after the last workshop day). Analyses revealed high subjective learning gains, a strongly perceived usefulness, and high satisfaction of the teachers in regard to the intervention approach. However, no treatment effects could be detected in analyses on teachers’ achievement goals, student rated instructional behaviours (i. e., perceived classroom goal structures), and motivation of the students. Based on these findings, we discuss consequences in regard to the conception and evaluation of prospective teacher trainings.

Paper accepted at the 23th Conference of EARLI‘s JURE (Junior Researchers of EARLI), 9.-11. August 2019, Aachen (DE), The aim of this study was to investigate to what extent teachers’ judgment accuracy of academic performance is specific to the content areas within mathematics. Judgments by 63 teachers regarding student’s academic performance in algebra, geometry and measures, functions, and probabilities of 1369 students were analysed. The results showed that the possibility to generalize from the accuracy in one content area to another is rather limited. Furthermore, the results of the present study indicate the need for further research in order to answer the question whether accurate judgments of students' academic achievement can be assumed to be domain general and thus to be accounted by a single, underlying general ability (e.g. diagnostic competence) or whether they are domain specific and draw on several individual underlying competencies. (Conference abstract)

Beitrag präsentiert an der 7. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), 25. - 27. Februar 2019, Köln (D)

Researchers from different fields have developed different observational instruments to capture instructional quality with a focus on generic versus content-specific dimensions or a combination of both. As this work is fast accumulating, the need to explore synergies and complementarities among existing work on instruction and its quality becomes imperative, given the complexity of instruction and the increasing realization that different frameworks illuminate certain instructional aspects but leave others less visible. This special issue makes a step toward exploring such synergies and complementarities, drawing on the analysis of the same 3 elementary-school lessons by 11 groups using 12 different frameworks. The purpose of the current paper is to provide an up-to-date overview of prior attempts made to work at the intersection of different observational frameworks. The paper also serves as the reference point for the other papers included in the special issue, by defining the goals and research questions driving the explorations presented in each paper, outlining the criteria for selecting the frameworks included in the special issue, describing the sampling approaches for the selected lessons, presenting the content of these lessons, and providing an overview of the structure of each paper. 50 3 355 366

International audience; Metacognition plays an essential role in learning mathematics. However, due to the lack of observational systems for evaluation of metacognition in mathematics instruction, rarely anything is known about how metacognition is practised and fostered when teaching and learning mathematics in class. This paper presents an observational system (a rating system) developed to reliably assess metacognitive activities in mathematics instruction. It also explains the methodology used to evaluate the reliability of ratings achieved with this tool and to investigate the stability of metacognitive-discursive practices between lessons of an individual teacher/class. Despite the high inference of conclusions needed to assess metacognitive-discursive instructional quality in seven dimensions, highly reliable ratings have been achieved for six dimensions. The paper discusses reasons for and consequences of the high reliability. Metacognition has been ascribed an essential role in regulating students' cognitive processes in problem solving as well as in learning mathematics in general, in particular when constructing, organising, systematising, and connecting (pieces of) knowledge (cf. Schraw & Moshman, 1995; Wilson & Clark, 2004). However, hardly anything is known about how metacognition is practised and fostered in mathematics instruction. Assuming that enhancing learners' metacognition is essential for promoting learning, research on metacognition in this area definitely merits future research (cf. Mevarech & Kramarski, 2014; Depaepe et al., 2010). For this kind of research a tool is needed that allows to reliably assess metacognitive practices when teaching and learning mathematics in a class. This paper reports on a research project that aimed at developing such a tool, named the rating system for analysing and assessing the metacognitive-discursive instructional quality (Nowińska, 2016). This tool can be used to first describe metacognition during class discussion, and to second evaluate how metacognitive activities are used to foster understanding in mathematics, in particular by elaborating students' ways of thinking and reasoning, and by discussing them in a coherent and comprehensible way. One important research question underlying our work on developing and evaluating this tool was how stable metacognitive-discursive instructional quality is across lessons of an individual teacher in one class. In addition to advancing our knowledge about the occurrence of metacognitive-discursive instructional quality, investigating its stability allows identifying the number of lessons per teacher/class which would be needed to reliably measure metacognitive-discursive instructional quality. Metacognition is often understood as knowledge about cognition and regulation of cognition (Flavell, 1976; Schraw & Moshmann, 1998). The groundwork for investigating metacognition in the domain of teaching and learning mathematics in a class has been done by Cohors-Fresenborg and

Beiträge zur Lehrerbildung 31 (2013) 2, S. 174-185, Die Einschätzung von Unterricht durch externe Beobachterinnen und Beobachter ist seit jeher konstitutiver Bestandteil des Lehrberufs. In welchem Ausmass solche Beobachtereinschätzungen zuverlässige Aussagen über den Unterricht von Lehrkräften darstellen, wurde bislang jedoch kaum empirisch überprüft. Der vorliegende Beitrag gibt einen Überblick über Studien, die infrage stellen, inwiefern es sich bei Beobachtereinschätzungen um den «Königsweg» zur Erfassung von Unterrichtsqualität handelt. Der Beitrag schliesst mit Folgerungen für die Unterrichtspraxis. Dabei wird unter anderem die Notwendigkeit des Einsatzes mehrerer Beobachterinnen und Beobachter sowie einer Beobachtung von längeren Zeiträumen diskutiert., Assessments of teaching by external observers have always been a constitutive part of the teaching profession. Still, the extent to which these ratings allow reliable statements about the observed teachers’ instruction has rarely been empirically investigated. Against this background, the following article provides an overview of studies which challenge the assumption that observer ratings are the ideal method of measuring instructional quality. It concludes with some implications for instructional practice. Among other things, the discussion touches on the need to include several observers and to cover longer periods of observation.