Birmingham Council – IDL Numeracy Pilot

Following the success of the IDL Literacy pilot in Summer 2018, IDL worked with the Pupil and School Support service to deliver a pilot with primary and secondary schools in Birmingham to test IDL Numeracy. This was for a period of ten weeks, with a focus on Number, Place Value, Addition and Subtraction. Schools selected pupils who had identified Special Educational Needs (SEN) or who it was felt would benefit from a targeted maths intervention.

14 schools from across Birmingham took place in the pilot between April and July 2019.

Implementation

Schools were required to make the following commitments in order to take part in the pilot:

• Must commit to training and regular timetabled usage of IDL
• Minimum of 10 pupils to take part
• Had to attend a half day training session
• Pupils would use IDL for a minimum 4 x 15-minute sessions per week, for a total of one hour per week
• Schools had a maximum time of 15 weeks with access to the program but were only required to complete 10 weeks of intervention
• Schools needed to use the Birmingham Maths Toolkit to track progress to provide pre and post assessment information
• Computer and headphones were required with internet access. 

Schools were asked to track pupils using the following threads of Birmingham’s Maths Toolkit, focussing on the number element of the descriptors:

• Number and Place Value

• Addition

• Subtraction

Of the fourteen schools which completed the pilot successfully ten were primary and four were secondary.

Assessment

Schools were asked to track pupils using the Birmingham Maths Toolkit. For the full academic year it is considered:

2 Bands progress – the pupil maintains the current gap between them and their peers.

3 Bands progress – ‘accelerated’ progress and the pupil will begin to close the gap between them and their peers.

As the pilot took place over the equivalent of 10 weeks then any progress identified on the threads was considered as accelerated (‘closing the gap’) progress.

Analysis of Results

The results were analysed through the comparison of pupils with different criteria and the number of bands of progress made on the Birmingham Maths Toolkit whilst using IDL. The different criteria included:

• Gender – 57% were girls and 43% boys

• SEN/no SEN – 69% of participants were identified as SEN and 26% as not having SEN needs
• EHCP – 5% of participants were identified as having an Education, Health and Care Plan
• Pupil Premium – 60% of participants were eligible for this additional funding
• Ethnicity – Data regarding ethnic groups was provided for 205 of pupils in the pilot. Of this 34% - White British/English, 11% - Other Pakistani, 5% - White and Black Caribbean, 4% - any other white background, 4% - Bangladeshi, 4% - Any Other Asian background, 4% - Black Caribbean with the remaining 34% spread across another 27 categories.

Results

The results of the pilot saw high numbers of pupils making accelerated progress particularly in the addition thread. There were some differences between the results of different ethnic groups, but the data indicates the program worked for all.

Ethnicity

Some differences to note between ethnicity and performance in the pilot are:

• Pupils whose ethnicity was recorded as other Pakistani fell in the top three positions with the most progress recorded in all the threads
• Pupils whose ethnicity was identified as Black Caribbean and Other Asian Background showed greater levels of progress for two of the three threads assessed. These were Number and Place Value and Addition
• Pupils whose ethnicity was identified as White British or White English made the least amount of progress in all three threads assessed, ranging from 54% to 59%
• Pupils whose ethnicity was recorded as Mixed White and Black Caribbean showed lower levels of progress in the Subtraction thread (25%)
• At least 50% of pupils within the identified ethnic groups made accelerated progress in each thread, excluding those identified as Mixed White and Black Caribbean. 

Starting Points

The results also include the analysis of the progress of pupils with different starting points. However, regardless of the starting point the average accelerated progress in all three threads (Number and Place Value, Addition and Subtraction) that were assessed (Number and Place Value, Addition and Subtraction) is 60%-80%.

SEN/ NOT SEN

Pupils from SEN and NOT SEN groups progressed similarly. That shows that the intervention, when used properly, works for these pupils in the same way. 

Number and Place Value Thread

The results on the Number and Place Value thread were very good as a whole. For the pupils who were assessed pre-intervention as working at bands 1 to 14 on the Birmingham Maths Toolkit, 74% made accelerated progress (1 band or more). Within this, a further 42% of pupils made between 2 to 6 bands of progress within the intervention period.

For those assessed at entry on bands 3 or 5, 100% of pupils made accelerated progress.

Pupils who started at bands 2 or 16 were not as successful, with a lower range of accelerated progress being made. It should be noted that there were smaller numbers of pupils recorded at these bands, so this data is not as reliable.

Addition Thread

For the pupils who were assessed pre-intervention as working at bands 1 or 4 to 14, 64% of pupils made accelerated (1 band or more) progress. Within this a further 31% of pupils made between 2 to 5 bands of progress within the intervention period.

100% of pupils whose starting points were at bands 1 or 4, made accelerated progress, with 94% of pupils making accelerated progress (whose starting point was band 6).

Subtraction Thread

The results for the subtraction thread were also positive, with 55% of pupils making accelerated progress in entry bands 1, and 3 to 15. Within this a further 31% of pupils made between 2 to 5 bands of progress within the intervention period. A significant number of pupils starting at Bands 4 made 80% accelerated progress.

Conclusion

The trial using IDL Numeracy by both primary and secondary schools in Birmingham was constructive and encouraging. Given that pupils had a maximum of 10 weeks using the programme the number of pupils who made accelerated progress was high, particularly when tracked against the Number and Place Value Thread.

The results suggest that this program works well for most of the pupils who are currently working on targets from Birmingham’s Maths Toolkit, and if used over the full academic year IDL Numeracy would work for all pupils and not just those with SEN.