Search

Increasingly, student assessments of courses are being conducted online as opposed to administered in class. A growing body of research compares response rates and course ratings of courses evaluated online versus on paper. The present study extends this research by comparing student course assessments before and after the University of South Florida made online evaluations mandatory for all courses. This change only directly affected courses taught on-campus, as online courses were already being assessed online. However, we examine the effect of this change on courses taught on-campus and online, because we expect this change in policy to have differential effects. We hypothesise that by making online assessments mandatory for all courses, online assessments went from a novel method of evaluation to the norm; and, therefore, increased response rates for online courses, but had the opposite effect for on-campus courses. We find mixed support for our hypothesis. [ABSTRACT FROM AUTHOR]

It is incumbent upon universities to deliver quality degree programs that produce employable graduates with discipline-specific knowledge and a well-developed set of generic skills known as graduate attributes. To safeguard the achievement of these outcomes, this paper argues for a holistic, longitudinal approach to degree planning known as programmatic assessment. The status of this relatively new phenomenon in the Australian university landscape is examined through an analysis of assessment-related policies from twenty-two of Australia's top-performing universities. Discourse analysis was employed to determine how programmatic assessment is depicted in policy, how key players are positioned in this space, and the discursive practices used to imbue policy with hortatory intent. The results of this analysis are outlined and indicate that policy constructions of a program-level approach to assessment are inconsistent across different universities and very few policies have specific guidelines about how programmatic assessment should be implemented. [ABSTRACT FROM AUTHOR]

Assessment in higher education has focused on the performance of individual students. This focus has been a practical as well as an epistemic one: methods of assessment are constrained by the technology of the day, and in the past they required the completion by individuals under controlled conditions of set-piece academic exercises. Recent advances in learning analytics, drawing upon vast sets of digitally stored student activity data, open new practical and epistemic possibilities for assessment, and carry the potential to transform higher education. It is becoming practicable to assess the individual and collective performance of team members working on complex projects that closely simulate the professional contexts that graduates will encounter. In addition to academic knowledge, this authentic assessment can include a diverse range of personal qualities and dispositions that are key to the computer-supported cooperative working of professionals in the knowledge economy. This paper explores the implications of such opportunities for the purpose and practices of assessment in higher education, as universities adapt their institutional missions to address twenty-first century needs. The paper concludes with a strong recommendation for university leaders to deploy analytics to support and evaluate the collaborative learning of students working in realistic contexts. [ABSTRACT FROM AUTHOR]

Using additional learning materials positively influences student performance. The literature focuses mainly on learning outcomes, whereas the present study investigates whether two intermediate (take-home) tests are a successful tool to increase student use of online formative assessments (OFAs). This study is a between-subjects quasi-experiment, administered in an undergraduate accounting course. Measures are used to capture both the intensity and the timing of OFA use. Results show that more students begin using OFA when intermediate tests are introduced. In addition, students use OFA more intensively and indications are present that students also begin their OFA use earlier. This paper contributes to the higher education literature by illustrating how online take-home tests can prevent procrastination. Its advantage lies in stimulating the use of online formative assessment and providing students with immediate feedback. Results are discussed for both blended and online learning. [ABSTRACT FROM AUTHOR]

A successful first year is of vital importance to academic achievement in higher education. How can we identify those students at risk at a very early stage, i.e. even before they have entered university? This study focuses on three pre-university factors, namely academic self-efficacy, self-regulated learning, in terms of autonomous motivation and time-management, and beyond classroom engagement. 2355 Dutch prospective first-year students in the fields of science, humanities and social sciences filled in our survey. Data on first year academic achievement in terms of grade point average (GPA) was obtained at the end of the first year. Prospective first-year students scored high on all of the variables; we found gender-related differences, favouring female students, as well as discipline-related differences. Time management and autonomous motivation were found to be positive predictors of achievement; beyond classroom engagement was a negative predictor of achievement. This paper contributes to the research on first-year academic achievement and emphasises the importance of providing students with personalised trajectories from the moment they enter university. [ABSTRACT FROM AUTHOR]

An increasingly regulated higher education sector is renewing its attention to those activities referred to as ‘moderation’ in its efforts to ensure that judgements of student achievement are based on appropriate standards. Moderation practices conducted throughout the assessment process can result in purposes identified as equity, justification, accountability and community building. This paper draws on the limited studies of moderation and wider relevant research on judgement, standards and professional learning to test commonly used moderation practices against these identified purposes. The paper concludes with recommendations for maximising the potential of moderation practices to establish and maintain achievement standards. [ABSTRACT FROM PUBLISHER]

Authentic assessment has theorised benefits for student outcomes in higher education. More needs to be done, however, to empirically test relationships between the critical elements of authenticity and student outcomes, particularly in online learning environments. In this paper we examine whether an online simulation-based learning and assessment design, developed using the tools and functionality within the available university online ecosystem, has measurable 'authenticity' in a postgraduate course. In addition, we assess whether the simulation is associated with improvements in student learning behaviours and outcomes. We analyse qualitative data from lecturer and student interviews and student work submitted online, as well as data analytics from the learning management system in both control and experimental conditions. Seven critical elements of authentic assessment were identified including new aspects of authenticity relating to simulation-specific affordances (e.g. emotional content, temporal dimensions and unpredictability). Students watched lecture videos and participated in online discussion more frequently in the online simulation than in the control condition and indicated perceived mastery of, and confidence in, the learning outcomes. These findings make important contributions to authentic assessment theory and have practical implications for simulations built in online ecosystems available in higher education. [ABSTRACT FROM AUTHOR]

The term 'feedforward' is increasingly employed in higher education, and this paper focuses on the way in which it fits into contemporary debates about feedback and its impact on practitioners. Semi-structured interviews were used to investigate the practices academics associate with feedforward and the ways in which their intentions and understandings varied. The term resulted in practices being framed as a process and prompted academics to consider the points in the future when students were expected to deploy information and improve. Three future horizons were identified: the 'within-module' future horizon dominated, while the challenges of 'beyond-module' and 'beyond-programme' horizons became apparent. Written comments, guidance, formative assessment and design practices were associated with feedforward, comprising both transmission and student-focused approaches. It is concluded that the modular system restricts longer-term future horizons and sustainable practices, and more must be done to address the divide between theoretical and practitioners' perspectives. [ABSTRACT FROM AUTHOR]

In today's market-driven educational culture, universities are coming under increasing pressure to justify funding through the disclosure of measurable outcomes in education and research. One educational objective that receives particular attention is critical thinking, regarded as an essential skill in both academic and work environments. The assessment of critical thinking has become a significant enterprise, with a number of standardised tests available for both individuals and organisations. While these tests are based on well-known taxonomies of critical thinking, this paper argues that institutions should be wary of using them as a means to measure educational outcomes. First, they fail to take into account fundamentally contested issues within conceptions of critical thinking. They also have significant weaknesses in terms of validity and reliability. Finally, and most importantly, they provide only a limited assessment of critical thinking, failing to evaluate the skills exercised in real-life academic tasks. A more effective approach to critical thinking testing would be one implemented at a faculty level, with assessments carried out on coursework integral to the curricula of specific academic disciplines. [ABSTRACT FROM AUTHOR]

Since the early 2010s the literature has shifted to view feedback as a process that students do where they make sense of information about work they have done, and use it to improve the quality of their subsequent work. In this view, effective feedback needs to demonstrate effects. However, it is unclear if educators and students share this understanding of feedback. This paper reports a qualitative investigation of what educators and students think the purpose of feedback is, and what they think makes feedback effective. We administered a survey on feedback that was completed by 406 staff and 4514 students from two Australian universities. Inductive thematic analysis was conducted on data from a sample of 323 staff with assessment responsibilities and 400 students. Staff and students largely thought the purpose of feedback was improvement. With respect to what makes feedback effective, staff mostly discussed feedback design matters like timing, modalities and connected tasks. In contrast, students mostly wrote that high-quality feedback comments make feedback effective - especially comments that are usable, detailed, considerate of affect and personalised to the student's own work. This study may assist researchers, educators and academic developers in refocusing their efforts in improving feedback. [ABSTRACT FROM AUTHOR]

The student evaluation of teaching process is generally thought to produce reliable results. The consistency is found within class and instructor averages, while a considerable amount of inconsistency exists with individual student responses. This paper reviews these issues along with a detailed examination of common measures of reliability that are utilised with the instruments. While inter-item consistency of the evaluations has been shown to be high, the agreement between students was shown to be no better than what would be expected by chance, indicating that students do not agree on what they are being asked to evaluate. The reliability measures generated by the student evaluations of teaching are an insufficient foundation for establishing validity. Further, the pattern of reliability indicates that the instruments are generally providing information about students, not instructors. [ABSTRACT FROM AUTHOR]

There is a lively discussion on how to evaluate competence-based higher education in both evaluation and competence research. The instruments used are often limited to course evaluation or specific competences, taking a rather narrow perspective. Furthermore, the instruments often comprise predetermined competences that cannot be adapted to higher education needs. The aim of this paper is to provide a flexible screening instrument into which practitioners/evaluators of study programmes can themselves insert the competences they are interested in, i.e. competences students ought to acquire through their study programme. The present study describes the development of the Competence Screening Questionnaire for Higher Education (CSQ-HE) in different domains of higher education. Results show that the CSQ-HE provides plausible screening information whilst fulfilling evaluation standards. It can be integrated into a higher education institution's internal quality management system to demonstrate a programme's legitimacy and enhance competence-based teaching and learning. Strengths, weaknesses and implications for practice are discussed. [ABSTRACT FROM AUTHOR]

The aims of this paper are to examine and critically evaluate a selection of different technological methods that were specifically chosen for their alignment with, and potential to enhance, extant assessment for learning practice. The underpinning perspectives are that: (a) both formative and summative assessment are valuable opportunities for learning, and (b) using technology may enhance learning in assessment and feedback processes. Drawing on the literature and empirical evidence from a research study in a Scottish university, the advantages and drawbacks of using technology are examined. It is asserted that, by adopting a flexible approach and taking small incremental steps, the use of different types of technology can be beneficial in facilitating effective assessment for learning and feedback in higher education. [ABSTRACT FROM AUTHOR]

Sustaining students in becoming reflective practitioners is considered as a valued outcome of higher education. The paper aims to evaluate the impact of the learning environment conditions inspired by Schön's theory of reflective practicum, by discussing a case study of a master's degree class. The learning environment was designed to sustain reflective practice and meaningful engagement through professional practice simulation, problembased learning and reflective writing. Unlike much of the research into reflective learning, the quality of learning was evaluated by assessing the use of reflective practice in students' weekly journaling, rather than measuring students' satisfaction or perceptions of effectiveness. Two hundred and six journal entries of 23 students were assessed and used in a quantitative analysis based on a linear mixed-effects model. Findings indicated that the reflective practicum has an incremental effect on students' reflective practices and that reflective practice is dynamic and sensitive to specific learning environment conditions. In conclusion, students' co-responsibility of the learning environment allowed by the professional practice simulation appears to foster meaningful and reflective learning. Conversely, conditions that stimulate only cognitive engagement have little impact or even inhibit reflective practices. Practical implications of the use of reflective journal are discussed. [ABSTRACT FROM AUTHOR]

A flexible approach to assessment may promote students’ engagement and academic achievement by allowing them to personalise their learning experience, even in the context of large undergraduate classes. However, studies reporting flexible assessment strategies and their impact are limited. In this paper, I present a feasible and effective approach to flexible assessment, describe choices made by 2016 students in 12 sections of two different courses using this approach, and explore associations between students’ choices and academic achievement. Students decided at the beginning of the term whether they would use the assessment scheme proposed by the instructor or modify it by selecting (from within ranges provided) which assessments they would complete and the value of each in calculating their final grade. Most students (62%) made some change, but many (38%) opted to use the suggested values. Students did not choose to minimise their workload by selecting the fewest assessments possible. Opting out of a large assignment was the most common change, but the majority (69%) completed the assignment and almost all (95%) included quizzes based on readings. Different choices were not associated with notable differences in achievement: midterm score was the most important predictor of performance on a cumulative final examination. [ABSTRACT FROM AUTHOR]

External referencing of assessment and students’ achievement standards is a growing priority area within higher education, which is being pressured by government requirements to evidence outcome attainment. External referencing benefits stakeholders connected to higher education by helping to assure that assessments and standards within courses are appropriate and comparable among institutions. External referencing takes many forms, which have different resourcing requirements, outcomes, and operational strengths and challenges. This paper describes the External Referencing of Standards (ERoS) approach developed, tested and adopted by a university consortium. ERoS draws on the strengths of existing methodologies to produce an evolved model that is effective, efficient, transparent and open, capability building and sustainable. The model enables participants to communicate directly and construct peer relationships, and findings suggest this is a significant design strength. The process facilitates capability building, such that participants garnered insights valuable to enact quality assurance and enhancement of existing courses, and fosters connections that facilitate collaboration and peer-learning. ERoS successfully used open source collaborative tools to review work samples, which can be used to benchmark costed systems. [ABSTRACT FROM AUTHOR]

Using the concepts of criticality, reflectivity and praxis, the paper presents an analysis of our reflections on participants’ responses to the assessment requirements for a course for lecturers on teaching. The context in which the course is being taught has changed considerably in the last few years in terms of the mode of delivery, as well as the number and diversity of participants. Our analysis has generated insights into ways in which the course is not meeting all the learning needs of the participants, nor preparing them adequately to demonstrate, in writing,their learning. Using insights gained, we suggest pedagogic processes and strategies for ensuring that the course focuses on both writing to learn and learning to write; and for assisting participants to acquire the practices to demonstrate their learning in written assessment tasks, using the requisite literacy including criticality, reflectivity and praxis. [ABSTRACT FROM PUBLISHER]

Assessment regulations in higher education, which are important for assuring threshold academic standards, reflect institutional cultures and histories, and are shaped by pragmatic concerns about quality indicators such as retention and progression rates, as well as principles of equity. This paper articulates some of the tensions that confront higher education institutions in the development and revision of regulatory frameworks, and describes a survey of assessment regulations from a sample of UK higher education institutions for the first year of undergraduate study. The survey identifies key variations in regulatory policy and practice that challenge assumptions about comparability of academic standards between higher education institutions. These findings imply that student success and progression may not be a simple reflection of academic attainment, and raise questions about notions of equity. It is intended that this research will contribute to informed discussion in the sector about academic standards and the regulatory frameworks underpinning first-year assessment policy and practice, and thus potentially lead to more transparent and consistent practice. [ABSTRACT FROM PUBLISHER]

This is a systematic review conducted of primary research literature published between 2007 and 2018 on the deployment and effectiveness of data analytics in higher education to improve student outcomes. We took a methodological approach to searching databases; appraising and synthesising results against predefined criteria. We reviewed research on the effectiveness of three differentiated forms of data analytics: learning, academic and learner analytics. Student outcomes are defined as retention, academic performance and engagement. Our results find that three quarters of studies report the use of educational data analytics to be effective in improving student outcomes but their relationship with student outcomes requires further and more robust investigation and assessment. We argue that research must interpret and communicate effectiveness qualitatively, as well as quantitatively, by including the student voice in assessments of impact. [ABSTRACT FROM AUTHOR]

This paper reports on domestic and international students’ perceptions of the influence of group diversity on communication, learning, task performance and assessment grades. The study’s methodology involved quantitative and qualitative analysis of surveys (N = 312), focus group interviews of students (n = 26) and individual staff interviews (N = 7). More domestic (79%) than international (29%) students stated that they preferred working with students from a similar background. While a proportion of the sample of domestic students stated that intercultural group work raised their awareness of interaction styles in other cultures, others noted that the group experience was negatively affected by issues related to language proficiency and awareness of academic requirements. Qualitative analysis reveals that ‘othering’, a process by which members of an in group (Us) distance themselves from an outgroup (Them), was often used as a basis for these students avoiding intercultural group assessment. Finally, there was a clear pattern overall of higher achieving international and domestic students perceiving that their grades for group assignments were below what they would normally receive for individual assignments, and lower achieving students perceiving that their grades were higher for group assignments. [ABSTRACT FROM PUBLISHER]

We examined the reliability of grading master's theses at a New Zealand university, where a variant of the academic journal review system is employed. The overall correlation between the grades recommended by internal and external markers of master's theses in psychology and applied psychology at this university was 0.39, which is similar to that produced between reviewers of submissions to academic journals. We conclude that thesis grading is about as reliable as the determination of the merit of journal submissions, which is usually regarded as rather poor. Applying a more objective rubric to thesis marking might raise the reliability, but such a change might also lower the creative and unique nature of theses, and hence reduce the contribution that an individual thesis can make to its particular discipline. [ABSTRACT FROM AUTHOR]

Screencast feedback is higher in quantity, explicit and engaging and may better enable uptake compared to written feedback. However, most studies deploy screencast feedback as 'transmission' of feedback comments, positioning learners as passive and neglecting the importance of agency and action within the uptake process. This study attempts to overcome this limitation by conceptually positioning and deploying screencast feedback in a way that supports agency by providing technology-mediated opportunities for learners to request feedback and initiate uptake-oriented dialogues with providers. Taking a qualitative case approach, using written data, reflections (N = 14) and surveys (N = 14) to progressively focus interviews with 13 undergraduate advanced writing students in South Korea, three themes were developed. First, screencasts appeared to enhance understanding of feedback, helping learners understand standards and to set and achieve goals. Second, through initial feedback requests and opportunities to seek and clarify feedback, responding to feedback dialogically enabled learners to better understand and enact it, supporting their agency. Finally, the perception that the feedback was supportive and caring encouraged trust and motivation to engage with and use feedback. The findings have several theoretical and practical implications and are especially relevant to higher education practitioners wishing to support agency and uptake with a relational approach. [ABSTRACT FROM AUTHOR]

Authentic assessment aligns higher education with the practices of students' future professions, which are increasingly digitally mediated. However, previous frameworks for authentic assessment appear not to explicitly address how authenticity intersects with a broader digital world. This critical scoping review describes how the digital has been designed into authentic assessment in the higher education literature. Our findings imply that the digital was most often used to enhance assessment design and to develop students' digital skills. Other purposes for designing the digital into assessment were less present. Only eight studies situated the students within the wider context of digital societies, and none of the studies addressed students' critical digital literacies. Thus, while there are pockets of good practice found within the literature, the vast majority of the studies employed the digital as an instrumental tool for garnering efficiencies. We suggest that in order to fit its purpose of preparing students for the digital world, the digital needs to be designed into authentic assessment in meaningful ways. [ABSTRACT FROM AUTHOR]

It has been suggested for many years that students who are able to judge their own performance should do well in academic assessments. Despite the increasing number of empirical studies investigating the effect of self-assessment on academic performance, there has not been a recent synthesis of findings in the higher education context. The current meta-analysis aims to synthesise the effects of self-assessment on academic performance. In particular, it examines the difference between situations in which the process of self-assessment is revealed or observable (explicit) or not revealed or unobservable (implicit). A total of 98 effect sizes from 26 studies either reported a comparison between a group with self-assessment interventions and a control group (n = 20, k = 88) or a pre-post comparison (n = 6, k = 10). The overall effect of such interventions was significant (g =.455). Self-assessment interventions involving explicit feedback from others on students' performance had a significantly larger effect size (g =.664) than those without explicit feedback (g =.213). There were no other significant moderators identified for either the overall effect or the effect of interventions involving explicit feedback. Supplemental data for this article is available online at http://dx.doi.org/10.1080/02602938.2021.2012644. [ABSTRACT FROM AUTHOR]

Despite the benefits swift academic progress holds for many stakeholders, there is scarce literature on how academic progress may be improved by changes to assessment policies. Therefore, we investigated academic progress of first-year students after an alteration of characteristics of the assessment policies in three large course programmes: business administration (n = 2048) changed the stakes; medicine (n = 1630) changed the stakes and performance standard; psychology (n = 1076) changed the stakes, performance standard and resit standard. Results indicate that students' academic progress was sensitive to the characteristics of the assessment policy in all three course programmes. The changes in progress could be explained by differences in performance, as well as by differences in selection for progress by the different policies. Implications are that assessment policies seem effective in shaping student progress, although one size does not fit all. [ABSTRACT FROM AUTHOR]

Effective use of learning analytics systems has been purported to confer various benefits to learners in terms of both attainment and retention. There is, however, little agreement on which data are meaningful or useful. Whilst measures of engagement might correlate with outcomes, thereby retrospectively 'predicting' them, there are fewer studies which attempt to predict using 'live' system data in a face-to-face teaching environment. This study reports an analysis of week by week data from a learning analytics system which monitored 1,602 first year UK undergraduates. Uniquely, although students could view their own data, no formal interventions took place. Results showed that students who obtained the highest end-of-year marks were more likely to be in a higher engagement quintile as early as the first 3–4 weeks, and that early engagement was highly predictive of future engagement. Students who started in a higher engagement quintile, but their engagement decreased, were more likely to have higher marks than those that started in a lower quintile and then increased their engagement. Early measures of engagement are therefore predictive of future behaviour and of future outcomes, a finding which has important implications for universities wishing to improve student outcomes. [ABSTRACT FROM AUTHOR]

Does attendance matter? This question has still not been answered conclusively. In general, studies find positive but mostly weak correlations between attendance and performance. However, due to technological changes in learning, attendance in higher education seems to lose its importance since students do not have to attend class to get access to course material. The question that arises is whether information on purely descriptive attendance is sufficient to prove the positive effects of attendance on performance. This study takes a closer look at the link between attendance and performance, examining classroom engagement dimensions as mediating factors. The results suggest that it does not matter if but rather how students attend class. [ABSTRACT FROM AUTHOR]

The aim of this study was to determine the mediating effect of learning engagement on the relationship between perceived teacher feedback and college students' academic performance, and the moderating effect of assessment characteristics on the relationship between perceived teacher feedback and learning engagement. A sample of 2,458 students in a university in mainland China was studied. Results indicated that perceived teacher feedback had a positive impact on students' academic performance; learning engagement had a mediating effect on the relationship between perceived teacher feedback and students' academic performance; and assessment frequency, difficulty and diversity had moderating effects between perceived teacher feedback and learning engagement. With the three factors of high frequency, difficulty and diversity assessment, perceived teacher feedback was more likely to improve students' learning engagement. In contrast, feedback without these three factors could decrease or have no impact on students' learning engagement. This indicates that teacher feedback indirectly affects college students' academic performance by promoting their learning engagement, and is more effective when the frequency, difficulty or diversity of assessments is high. [ABSTRACT FROM AUTHOR]

In this study, we present a case study involving two self-service dashboards providing feedback on learning and study skills and on academic achievement. These dashboards were offered to first-year university students in several study programmes in Flanders, Belgium. Data for this study were collected using usage tracking (N = 2875) and a survey taken at the beginning of the second year before (N = 484) and after (N = 538) the introduction of the dashboards. We found that early dashboard usage is related to academic achievement later in the academic year and that students' review of the feedback received in the first year improved. Although these results are modest in comparison to how high the bar is sometimes set for learning analytics applications, we argue that low-cost deployments of self-service dashboards are an interesting approach to start building experience with similar tools and to start paving the way for future developments. [ABSTRACT FROM AUTHOR]

This is the introduction to the special edition on learning analytics. [ABSTRACT FROM AUTHOR]

Assessment and learning analytics both collect, analyse and use student data, albeit different types of data and to some extent, for various purposes. Based on the data collected and analysed, learning analytics allow for decisions to be made not only with regard to evaluating progress in achieving learning outcomes but also evaluative judgments about the quality of learning. Learning analytics fall in the nexus between assessment of and for learning. As such it has the potential to deliver value in the form of (1) understanding student learning, (2) analysing learning behaviour (looking to identify not only factors that may indicate risk of failing, but for opportunities to deepen learning), (3) predicting students-at-risk (or identifying where students have specific learning needs), and (4) prescribing elements to be included to ensure not only the effectiveness of teaching, but also of learning. Learning analytics have underlying default positions that may not only skew their impact but also impact negatively on students in realising their potential. We examine a selection of default positions and point to how these positions/assumptions may adversely affect students' chances of success, deepening the understanding of learning. [ABSTRACT FROM AUTHOR]

The field of learning analytics is progressing at a rapid rate. New tools, with ever-increasing number of features and a plethora of datasets that are increasingly utilized demonstrate the evolution and multifaceted nature of the field. In particular, the depth and scope of insight that can be gleaned from analysing related datasets can have a significant, and positive, effect in educational practices. We introduce the concept of degree pictures, a symbolic overview of students' achievement. Degree pictures are small visualizations that depict graphically 16 categories of overall student achievement, over the duration of a higher education course. They offer a quick summary of students' achievement and are intended to initiate appropriate responses, such as teaching and pastoral interventions. This can address the subjective nature of assessment, by providing a method for educators to calibrate their own marking practices by showing an overview of any cohort. We present a prototype implementation of degree pictures, which was evaluated within our School of Computer Science, with favourable results. [ABSTRACT FROM AUTHOR]

The validity of traditional opinion-based student evaluations of teaching (SETs) may be compromised by inattentive responses and low response rates due to evaluation fatigue, and/or by personal response bias. To reduce the impact of evaluation fatigue and personal response bias on SETs, this study explores peer prediction-based SETs as an alternative to opinion-based SETs in a multicultural environment. The results suggest that statistically significant fewer respondents are needed to reach stable average outcomes when peer prediction-based SETs are used than when opinion-based SETs are used. This implies that peer prediction-based SETs could reduce evaluation fatigue, as not all students would need to do each evaluation. The results also report that the peer prediction-based method significantly reduces the bias evident in the opinion-based method, in respect of gender and prior academic performance. However, in respect of the cultural variables, race and home language, bias was identified in the peer prediction-based method, where none was evident in the opinion-based method. These observations, interpreted through the psychology literature on the formulation of perceptions of others, imply that although peer prediction-based SETs may in some instances reduce some personal response bias, it may introduce the perceived bias of others. [ABSTRACT FROM AUTHOR]

Despite the frequently reported association of characteristics of assessment policies with academic performance, the mechanisms through which these policies affect performance are largely unknown. Therefore, the current research investigated performance, motivation and self-regulation for two groups of students following the same statistics course, but under two assessment policies: education and child studies (ECS) students studied under an assessment policy with relatively higher stakes, a higher performance standard and a lower resit standard, compared with Psychology students. Results show similar initial performance, but more use of resits and higher final performance (post-resit) under the ECS policy compared with the psychology policy. In terms of motivation and self-regulation, under the ECS policy significantly higher minimum grade goals, performance self-efficacy, task value, time and study environment management, and test anxiety were observed, but there were no significant differences in aimed grade goals, academic self-efficacy and effort regulation. The relations of motivational and self-regulatory factors with academic performance were similar between both assessment policies. Thus, educators should be keenly aware of how characteristics of assessment policies are related to students' motivation, self-regulation and academic performance. [ABSTRACT FROM AUTHOR]

This article explores the effect university widening participation activities have had in improving access to university of students from rural government-run schools in New South Wales, Australia. An effective effort to evaluate the success of widening participation programs in rural Australia, specifically New South Wales, has not been achieved to date. This article addresses this issue in three distinct ways. First, previous efforts to evaluate widening participation effectiveness are explored and their successes and failings are described. Second, a method to rigorously evaluate university-run outreach programs in Australia's unique socio-cultural context is constructed and explained. Third, this method is employed using a sample of Australian rural schools to evaluate outreach effectiveness. A mixed-method design combining multilevel growth models and in-depth interviews of careers advisors revealed teacher support of university outreach presence in schools did not necessarily translate to an opinion of outreach effectivity in raising university access levels. This was supported by quantitative analyses showing widening participation efforts in rural New South Wales have had little to no effect on the progression of rural students to university between 2010 and 2016. [ABSTRACT FROM AUTHOR]

Recent literature on the use of exemplars in the context of higher education has shown that exemplar-based instruction is implemented in various disciplines; nevertheless, how exemplar-based instruction can be implemented in English-as-a-Second-Language (ESL) writing classrooms in higher education institutions remains under-explored. In this connection, this article reports on a textbook development project which adopts an exemplar-based instruction approach to be used by university English instructors to prepare students for IELTS writing (academic module). The goal of the textbook is to cultivate students' understanding of the assessment standards of the two IELTS writing tasks through the design and use of exemplar-based dialogic and reflective activities. In this article, theoretical underpinnings of the use of exemplars, namely tacit knowledge, assessment as learning and dialogic feedback, will first be discussed in detail. Then, an overview of an ongoing project which aims to develop an exemplar-based IELTS writing textbook will be given. The last section of this article suggests practical strategies for ESL writing teachers who are interested in using exemplars to develop students' understanding of assessment standards. [ABSTRACT FROM AUTHOR]

This study aims to investigate the academic achievement of students who were admitted to a college from three different college entrance channels in Taiwan. The three channels are personal application, admission by examination and placement, and the Star Program. Longitudinal data composed of 1053 students during eight semesters over 4 years from one university were examined. Beyond basic descriptive statistics, linear growth curve modelling was utilized to capture the dynamic extent of the exploratory variables for predicting students' learning performance. The descriptive statistical results showed that, prior to college, students' learning performance followed the order in which the college entrance channels were listed. During their time at college, however, the pattern of ranking of the students' performance was totally reversed. The results from the growth curve modelling indicated that prior knowledge, that is, students' standardized test scores, had a positive influence on initial grades but a negative impact on the slope of students' learning achievement. Different patterns of learning performance of students from the three channels were observed. Implications and suggestions for evaluating the effectiveness of college entrance channels and assessing students' learning performance from a longitudinal perspective are discussed. [ABSTRACT FROM AUTHOR]

Contract cheating occurs when students outsource assessed work. In this study, we asked experienced markers from four disciplines to detect contract cheating in a set of 20 discipline-specific assignments. We then conducted a training workshop to improve their detection accuracy, and afterwards asked them to detect contract cheating in 20 new assignments. We analysed the data in terms of sensitivity (the rate at which markers spotted contract cheating) and specificity (the rate at which markers spotted real student work). Pre-workshop marker sensitivity was 58% and specificity was 83%. Post-workshop marker sensitivity was 82% and specificity was 87%. The increase in sensitivity was statistically significant, but the increase in specificity was not. These results indicate that markers can often detect contract cheating when asked to do so, and that training may be helpful in improving their accuracy. We suggest that markers' suspicions may be crucial in addressing contract cheating. [ABSTRACT FROM AUTHOR]

This article presents an empirical evaluation of the validity and reliability of a peer-assessment activity to improve academic writing competences. Specifically, we explored a large group of psychology undergraduate students with different initial writing skills. Participants (n = 365) produced two different essays, which were evaluated by their course peers and professors using a specific grading instrument (rubric). The validity of the task was demonstrated by a high inter-grader agreement and a strong degree of consistency between the ratings of the peers and professors. Although all students did not improve their writing skills between their first and second essays, the peer-assessment activity enhanced the writing abilities of low but not high-achieving participants. The pedagogical implications of these results are discussed. [ABSTRACT FROM AUTHOR]

The main goal of this study is to show that the association between university entrance score and first-year students' academic performance varies randomly across courses after controlling for students' sociodemographic, schooling trajectory and motivational variables. The sample consists of 2697 first-year students who were enrolled in 54 courses at a Portuguese public university in 2015/16. Multilevel modelling of academic performance suggests that 34% of variability in grade point average is due to differences among courses and that 80% of such variability is explained by the field of study, whether the university is the student's first choice, and the student's gender, age and parents' level of education. In addition, the results corroborate that the university entrance score is the strongest predictor of first-year academic performance. [ABSTRACT FROM AUTHOR]

Assessment is a central feature of student learning in higher education and has a strong influence on the student experience. Accordingly, the appropriate communication of assessment aims is a priority for all higher education institutions. This study proposes an analytical framework for the interpretation and creation of assessments across higher education disciplines. The framework suggests that assessments can be categorised according to rhetorical purposes, formats, modes and group arrangements. Assessments from three degree programmes at one Australian university are analysed using the framework to show its usefulness in classifying and evaluating task components and generating broad images of degrees based on assessment regimes. We draw on these practical applications to explain and compare discipline-specific qualities of each degree, and argue that the framework might enhance the communication of assessment aims to benefit higher education stakeholders. [ABSTRACT FROM AUTHOR]

The number of students studying for university qualifications through the medium of English and for whom English is not their first language has increased significantly in recent years. This, along with efforts to widen access to those under-represented in higher education, has brought into focus the question of what academics see as constituting a 'good' piece of student writing. In this small-scale pilot study, Thurstone's method of paired comparisons was used to establish a scale for ranking six essays in terms of how favourably each was viewed by academic lecturers when compared with every other essay in the set. Kelly's repertory grid technique was subsequently applied to interviews conducted with the same lecturers to establish which traits they associated with the upper and lower end of the scale. Findings suggest that this methodology represents a promising approach to establishing what academic tutors regard as the key elements of good writing and thus provides an indication of what English teachers might fruitfully focus on in their teaching of the skill. [ABSTRACT FROM AUTHOR]

This article focuses on the validity of the new Narrative Reflection Assessment Rubric (NARRA) used to assess students' reflective narratives in higher education. We evaluate its formulation and usefulness from an instructional point of view. To those ends, we engage in both a quantitative and a qualitative analysis, using data from 100+ preservice teachers before and after they attend a Reflective Practice Seminar. We find that there are statistically significant differences in student scores between the pre-seminar test and the post-seminar test, suggesting both that students learned to reflect in the framework of the seminar and that the rubric is a valid and reliable instrument to measure this learning. We also establish five categories about the usefulness of the rubric: description, intrapersonal inquiry, interpersonal inquiry, argumentation and improvement. We conclude that the NARRA is a useful tool for identifying students' current level of reflection in higher education. [ABSTRACT FROM AUTHOR]

The UK National Student Survey (NSS) is a sound and widely used instrument for assessing students' academic experiences. We aimed to translate the NSS for Portuguese students and to validate the instrument in a sample of medical undergraduates. The research team translated and adapted the NSS for Portuguese students (NSS-P). The survey was administered on an online platform to 1,256 final-year students at eight Portuguese medical schools. A total of 329 medical students (69.9% female) replied to the NSS-P, a response rate of 26.2%. Confirmatory factor analysis showed that the original six-factor structure had an adequate fit to the data. Adequate internal consistency was observed for all the subscales. Medium to large correlations were found among all the subscale scores and between the subscale scores and the students' overall satisfaction. Multiple regression showed that the scores on the Teaching, Organization and Management and Personal Development subscales significantly predicted the students' overall satisfaction. Approximately 64% of the students reported being satisfied with the quality of their courses. Significant differences among the medical schools in their NSS-P scores were found. The NSS-P is a valid and reliable measure for assessing medical students' perceptions of academic quality. [ABSTRACT FROM AUTHOR]

Feedback is important for student learning; however, research shows that students can have a number of difficulties when attempting to learn from feedback. Based on an in-depth analysis of undergraduate students' self-reported reflection logs, we present findings about students' experiences with oral and written feedback and how they act upon this feedback when a portfolio is the main assessment and learning tool. Our findings indicate that, within our context, students' overall experiences with receiving feedback are positive. Oral feedback was perceived as particularly valuable to the students while written feedback challenged their understanding. We identified four specific actions the students engaged in to create meaning from the feedback: internal feedback, using oral feedback, initiating dialogue and interacting with peers. The findings indicate that the students take responsibility in the feedback process. The findings are discussed in relation to current perspectives on feedback in higher education. [ABSTRACT FROM AUTHOR]

In spite of the rising tide of metrics in UK higher education, there has been scant attention paid to assessment loads, when evidence demonstrates that heavy demands lead to surface learning. Our study seeks to redress the situation by defining assessment loads and comparing them across research and teaching intensive universities. We clarify the concept of 'assessment load' in response to findings about high volumes of summative assessment on modular degrees. We define assessment load across whole undergraduate degrees, according to four measures: the volume of summative assessment; volume of formative assessment; proportion of examinations to coursework; number of different varieties of assessment. All four factors contribute to the weight of an assessment load, and influence students' approaches to learning. Our research compares programme assessment data from 73 programmes in 14 UK universities, across two institutional categories. Research-intensives have higher summative assessment loads and a greater proportion of examinations; teaching-intensives have higher varieties of assessment. Formative assessment does not differ significantly across both university groups. These findings pose particular challenges for students in different parts of the sector. Our study questions the wisdom that 'more' is always better, proposing that lighter assessment loads may make room for 'slow' and deep learning. [ABSTRACT FROM AUTHOR]

Conventionally, quantitative surveys are used for student evaluations in higher education, but the validity of this practice has been questioned. In the present study, we investigated a qualitative method for evaluating a bachelor's programme. Forty-one first-year students and twenty-eight thirdyear students on a bachelor's programme in Public Administration were asked to answer a single question in relation to their programme: 'What could have been done in order to support your learning?' The students provided rich responses of a depth and breadth not attainable through conventional surveys, making this method worthwhile for educational development at the programme level. The responses focused on students' own learning rather than on judgement of teacher performance. The results indicate that the students underwent a transition from the first to third year, suggesting a shift in needs. For example, first-year students asked for direct interaction, whereas third-year students asked for feedback on accomplishments. Practical applications are discussed. [ABSTRACT FROM AUTHOR]

The objective of the present study was to better understand a relatively under-researched topic, namely, undergraduate students' attitudes towards mistakes and how their attitudes relate to academic achievement. A series of online surveys were administered to a sample of 207 first- and secondyear undergraduate students. Using structural equational modelling, a hypothesised model was proposed to evaluate the following four research questions: (1) What is the effect of students' trust in their instructor on students' achievement goals? (2) What is the relationship between students' achievement goals and their attitudes towards mistakes? (3) Is students' attitudinal behaviour towards mistakes predictive of their perception of feedback? (4) Is students' openness to discussing mistakes (ODM) and attitudinal behaviours predictive of their academic achievement? Both direct and indirect effects were found. Direct effects indicated that students' trust in their instructor predicted achievement goals, students' achievement goals differentially predicted attitudes towards mistakes, students' ODM predicted perceptions of feedback, and students' attitudinal behaviours predicted academic achievement. Students' trust in their instructor and achievement goals had indirect effects on students' achievement. [ABSTRACT FROM AUTHOR]

In the current research, we introduce the team CARE model for supporting team development during post-secondary education. Team CARE is part of a larger suite of assessments at itpmetrics.com. Team CARE is a free, online, survey-based assessment that allows team members to rate their team's health and functioning in four key categories (communicate, adapt, relate and educate), as well as provide written feedback about the team's functioning to add nuance and supplemental context to the numeric scores. Team members completing the assessment receive a report documenting their team's scores on the variables measured. We report on data from student learning teams suggesting that the variables in the team CARE model are reliable, and that they are correlated with team performance outcomes. Students' perceptions of the tool were also examined, and the findings suggest that team CARE is perceived to be valuable, useful and easy to use. Recommendations for practice are detailed, including sample assessment schedules for teams with differing life spans. Implications for future research and implementation are discussed. [ABSTRACT FROM AUTHOR]

Research on peer assessment in higher education has been conducted for decades. However, ways to improve the quality of peer feedback and optimise student' benefits through peer assessment remain a puzzle. The present study aimed to examine the impacts of synchronous discussion between assessors and assessees on writing performance, qualitative feedback quality, meta-cognitive awareness and self-efficacy in web-based peer assessment. A total of 64 undergraduate students participated in the study and were randomly assigned into either the experimental or control group. Participants in the experimental group conducted synchronous discussion after the first round of peer assessment, while students in the control group did not conduct any synchronous discussion. The results revealed that synchronous discussion between the assessors and assessees significantly improved the students' writing performance, especially content writing skills, affective and meta-cognitive feedback quality, meta-cognitive awareness and self-efficacy. Practical implications, limitations and future directions are discussed based on these findings. [ABSTRACT FROM AUTHOR]