Search

Project Based Learning (PBL) is a pedagogical approach that seeks to provide Year 7 pupils with independent and group learning skills to meet both the needs of the Year 7 curriculum as well as support their learning in future stages of their education. It particularly aims to improve their engagement in learning as well as practical literacy skills. This trial evaluates a specific type of PBL known as "Learning through REAL Projects", developed by the Innovation Unit--an independent social enterprise that aims to improve public sector services. A year long pilot project in 2013/2014 with eight schools (763 pupils) established the feasibility of a main trial. It provided the opportunity to develop the intervention and test the research procedures. The results contained in this report relate to a randomised controlled trial that took place between September 2014 and April 2016. Twelve intervention schools (2,101 pupils) and 12 control schools (1,973 pupils) were involved in the trial. REAL projects was delivered by teachers, supported in many instances by teaching assistants (TAs), with further support from senior leadership colleagues. It was delivered for a year in the intervention schools with a relatively large proportion of timetabled teaching (varying between 20% and 50%). In almost all cases it was delivered to mixed-ability Year 7 classes. The intervention used structured cross-subject "REAL Projects" planned by the delivery teachers who were supported by Classroom Coaches from the Innovation Unit. Schools were also supported by Leadership Coaches. REAL Projects are driven by an "essential question" which has significant educational content. The projects encouraged pupils to create an "excellent" product through drafting and redrafting and then to exhibit their work to an "authentic" audience. Visits were made to intervention schools at two time points and consisted of lesson observations, interviews, and focus groups with school leadership, project leads, class teachers, and pupils. Two case studies were also conducted in schools that it was agreed delivered the intervention well. Key conclusions are as follows: (1) Adopting PBL had no clear impact on either literacy (as measured by the Progress in English assessment) or student engagement with school and learning; (2) The impact evaluation indicated that PBL may have had a negative impact on the literacy attainment of pupils entitled to free school meals. However, as no negative impact was found for low-attaining pupils, considerable caution should be applied to this finding; (3) The amount of data lost from the project (schools dropping out and lost to follow-up) particularly from the intervention schools, as well as the adoption of PBL or similar approaches by a number of control group schools, further limits the strength of any impact finding; (4) From our observations and feedback from schools, we found that PBL was considered to be worthwhile and may enhance pupils' skills including oracy, communication, teamwork, and self-directed study skills; and (5) PBL was generally delivered with fidelity but requires substantial management support and organisational change. The Innovation Unit training and support programme for teachers and school leadership was found to be effective in supporting this intervention. Overall, the findings have low security. The trial was designed as a two-armed randomised controlled trial with schools being allocated to intervention or control groups. It was set up as an efficacy trial which aimed to test if the intervention can succeed under ideal conditions. However, 47% of the pupils in the intervention and 16% in the control group were not included in the final analysis. Therefore there were some potentially important differences in characteristics between the intervention and control groups. This undermines the security of the result. The trial results did not find that the PBL programme had an impact either on the pupils' literacy performance (as measured by Progress in English 12 tests), engagement, or attendance. The analysis did find a statistically significant negative impact on students eligible for free school meals (FSM), however no negative impact was found for lower-attaining pupils more generally, which makes it difficult to hypothesise why PBL might negatively impact FSM pupils specifically. This adds to the uncertainty of the finding.

The "Hallé SHINE on Manchester" (HSoM) programme is a Saturday school educational programme designed to increase the reading and maths attainment, as well as engagement with school, of underachieving and disadvantaged pupils at Key Stage 2. Developed in collaboration between the SHINE Trust and Hallé Orchestra, the intervention provides additional school-based literacy and numeracy lessons, based on musical themes, as well as visits to Hallé rehearsals, performances and other theme-based activities. Twenty-five Saturday sessions, each lasting five hours, were planned for the intervention over the course of an academic year, delivered by qualified teachers, teaching assistants, peer mentors, and professional musicians. The evaluation consisted of two randomised controlled trials (RCTs)--a pilot trial and a main trial--and a process evaluation conducted with primary schools in the Manchester area between July 2012 and July 2015. The pilot trial involved 361 Year 5 and 6 pupils in 18 schools; the main trial involved 2,306 Year 4, 5 and 6 pupils in 38 schools. The second year involved both the pilot and main trial running concurrently. The third year involved main trial schools only. Key conclusions include: (1) There is no evidence that the "HSoM" programme had an impact on the reading attainment of children in the trial. This finding was consistent across all age groups, for pupils eligible for free school meals, and for all three years of the trials; (2) There is no evidence that the "HSoM" programme had an impact on the attainment of children in mathematics, or attitudes to reading, maths, music, and school of the children in the trial; (3) Attendance of eligible pupils was often low and considered as a barrier to successful implementation. Reasons for low attendance included pupils' lack of availability to attend the Saturday sessions, variable parental engagement with the programme, and limited time at the beginning of the programme for schools to engage children and parents; and (4) The process evaluation revealed a positive picture of involvement and engagement for those pupils who attended the Saturday school activities. Evaluators observed good working relationships between the teachers and pupils, and positive and purposeful learning environments in lessons. All stakeholders felt pupils were making noticeable improvements in behaviour, confidence, and development of social skills.

"Affordable Online Maths Tuition" is a one-to-one tutoring programme where pupils receive maths tuition over the internet from trained maths graduates in India and Sri Lanka. It is delivered by the organisation Third Space Learning (TSL). Tutors and pupils communicate using video calling and a secure virtual classroom. Before each session, the pupils' normal classroom teachers are able to select lessons from TSL's maths curriculum to target individual learning issues. The intervention was targeted at Year 6 pupils who were working at Key Stage (KS) 2 level 3 or an insecure KS2 level 4, and was delivered over 27 weeks from September 2014 to May 2015 by (TSL) in an initial testing phase, with support from Nesta and Nominet Trust. The impact of the intervention was evaluated using a randomised controlled trial design, involving 64 schools and 600 pupils. Schools were randomised to either receive the intervention or deliver "business as usual" teaching, which might have involved intensive one-to-one support for maths. A process evaluation was undertaken to understand the perceptions of teachers and pupils, assess whether the intervention was delivered as intended, and inform any future development of the intervention. The evaluation should be considered an efficacy trial. Key conclusions include: (1) The impact evaluation found no evidence that the intervention had an impact on Key Stage 2 maths, compared with "business as usual" teaching and support in Year 6; (2) Teachers were largely positive about the online tuition, and reported that it appeared to improve pupils' comprehension, verbal fluency, and confidence in maths; (3) Schools should consider whether their computer network can support the implementation of an online programme. Teachers were positive about the technical support and user experience of the programme, but some experienced technical challenges in the implementation; (4) As the online tuition is a "talking" intervention, it appeared to work better when the pupils were spaced out in larger rooms so that the noise from other sessions was less distracting; and (5) Future research could examine whether the programme has an impact on pupils' comprehension, mathematical capacities, verbal fluency, and confidence in maths, as this was an outcome reported by teachers.

This is a report of a pilot programme of the SHINE in Secondaries Saturday school transition intervention. The programme aims to improve attainment by focusing on literacy and numeracy and revisiting areas where pupils are struggling through a creative curriculum approach and enrichment opportunities. The Saturday programme is run by teachers from the school for 25 weeks throughout Year 7. Schools are responsible for developing their own curriculum of activities to suit the abilities and needs of their pupils following guiding principles specified by SHINE. The programme is designed to run for approximately 60 pupils staffed by four qualified teachers, three teaching assistants and three peer mentors all recruited and employed by the host secondary school. This evaluation was set up as a pilot study with four schools and 613 pupils participating. The study had three aims: (1) To explore the impact of SHINE in Secondaries on literacy, numeracy and attitudes to school and learning outcomes; (2) To explore the organisational implications and lessons for future wider roll-out; and (3) To explore the feasibility of using regression discontinuity design (RDD) as a pilot study. The evaluation showed a very small impact on literacy outcomes across all four schools using the Progress in English and Progress in Maths tests. These findings are limited evidence of promise of the intervention. A larger RDD study or randomised controlled trial would be required to investigate the impact of the intervention and the size of the effect (if any) with confidence. A subgroup analysis of children eligible for free school meals (FSM) was conducted. This showed no evidence of the intervention differentially benefiting pupils eligible for free school meals in any of the schools. These findings should be treated with caution because, due to the small sample size, the study was unable to detect the small effects with confidence. The pilot demonstrated that it was feasible to design, implement and analyse the RDD successfully; and the design could be used again to evaluate similar interventions where randomisation is not possible or desirable. Key conclusions include: (1) Impact of the intervention: Very small effect sizes on the primary and secondary outcomes of literacy and numeracy were detected. As this was an exploratory evaluation design and the number of schools involved in the project was small, this made it unlikely that any impact of SHINE could be detected, should the true estimate of effect be small; (2) Implementation and organisation issues: Feedback from the teachers and pupils suggested that the intervention was perceived to be valuable in providing an opportunity to consolidate literacy and numeracy skills; and (3) Feasibility of evaluation design: The purpose in piloting this design was to explore the feasibility of using this approach when randomly allocating pupils to an intervention group was not possible or desirable. The regression discontinuity design is an appropriate evaluation method for education interventions. However, because it is less powerful than a randomised controlled trial, the number of pupils involved must be two and a half times greater if a small effect is to be detected.

Self-Regulated Strategy Development (SRSD) is a writing process model in which students are encouraged to plan, draft, edit, and revise their writing. In this evaluation 23 primary schools and their Year 6 teachers in the Calderdale area of West Yorkshire were randomly allocated to receive training in the SRSD approach from an external consultant. Twelve schools were allocated to the comparison group and 11 schools to the intervention group. The evaluation was set up as an efficacy trial to test the impact of using SRSD in conjunction with memorable experiences at the transition from primary to secondary school. The study was well conducted with independent randomisation and a moderate sample size. Intention-to-treat analysis was used (i.e., pupils were compared in the groups to which they were originally randomly assigned), blind marking of the test papers was undertaken, and the analysis was adjusted for school randomisation. Key conclusions include: (1) The approach had a strong positive effect on the writing outcomes of low attaining pupils at the transition from primary to secondary school among a sample of pupils in State schools in the West Yorkshire area; (2) The approach had beneficial effects for both free school meals (FSM) and non-FSM pupils; (3) These findings, in combination with existing evidence from the United States and elsewhere, suggest that the Self-Regulated Strategy Development approach has substantial promise as a literacy catch-up approach; (4) A larger effectiveness trial could be commissioned to test the approach on a larger scale and with other age groups; and (5) Teachers were trained in the SRSD approach by the North American developers, but adapted it in some ways for an English context.