Scientific research has established that there are clear connections between class performance at NAPLAN and class performance in Maths Invaders with very strong links to both attainment level and answer speed as a measure of student confidence.
This finding confirms the argument that, for optimal development in mathematics, students should progress in academic attainment concurrently with increases in confidence – as evidenced by speed. In other words, good mathematicians are both knowledgeable and quick.
This pilot study compared a snapshot of the year level performance of Year 3 and 5 classes in the Maths Invaders Adaptive Learning system from a range of schools as measured from data accrued in Maths Invaders against the Year 3 and Year 5 NAPLAN results for other classes at the same year level. The research showed that:
4.1 There is variation in speed beyond what is accounted for by attainment and in the snapshot, it is clear that year levels at some schools are doing speed much better than others.
4.2 From our data and the research literature it is reasonable to postulate that this difference in speed is due to revision leading to increase computational and recall speed. We have shown in other research that use of Maths Invaders results in an increase in speed.
4.3 It is therefore predicted that students in schools with regular revision through strategies designed to build fluency and agility will be more confident mathematicians with the increase in confidence evidenced by increases in speed.
5. There are only small systematic differences in speed between Year 3 and Year 5
5.1 Speed within the Year 3 classes studied ranged from an average of 11.2 answers per minute to 22 answers per minute yet the average increase in speed between Year 3 and Year 5 was only 1.8 answers per minute (see table).
5.2 The broad variation in average speed between classes indicates that factors apart from mathematical ability are responsible for this variation.
5.3 On average speed improves only slowly with classroom teaching over time and must be gained through revision activity designed to build fluency and agility.
It is recognised in the literature that there is direct link between answer speed and maths confidence and that maths confidence directly affects the culture of the learning environment. It is acknowledged that maths confidence is a broad ranging concept and that speed, although a major component of it, is not all of it. However, measures of maths speed are the best indicator we have for this confidence.
It is also recognised that for satisfactory progress and optimal development in mathematics students should be making gains in attainment level and in confidence (as indicated by speed) throughout their learning trajectory.
For the study anonymised Maths Invaders adaptive learning result data was extracted for Year 3 and Year 5 classes in the period before and after the NAPLAN tests in 2021 and 2022, from the beginning of the school year until August – this range was chosen to include sufficient work to get an accurate snapshot of the students’ adaptive learning level and speed. Pooled class results were included in the study where:
A total of 42 class cohorts were identified over the two years, 22 at Year 3 level, 20 at Year 5 level. These classes came from 34 schools spread across the five Australian mainland states, with 13 public schools and 21 private schools. Maths Invaders attainment level was recorded on the Maths Invaders internal raw underlying scale of 128 levels that are used by the adaptive learning system. To calculate answer speed, answer time was averaged for correct 2-character answers (to control for typing time), measured in milliseconds. This average was then converted to answers per minute.
| Year Level | Classes | Maths Attainment (Maths Invaders) Mean (range) |
Maths Speed (answers/minute) Mean (range) |
Maths Speed (milliseconds) Mean |
|---|---|---|---|---|
| Year 3 | 22 | 28 (11 – 51) | 13.3 (11.2 – 22.0) | 4514 |
| Year 5 | 20 | 39 (20 – 55) | 15.1 (12.9 – 16.9) | 3956 |
Both attainment level and answer speed may be observed to, on average, progress systematically across the 2-year age gap. However, it is readily apparent that there is a great range between the worst-performing and best-performing classes.
NAPLAN results are calculated on a single scale across all years. In both cases the average performance of cohorts using Maths Invaders was well above the national average, on the raw figures – before any of the several socio-economic factors measured for NAPLAN have been applied. However, it is reasonable to use the raw numbers only, on the expectation that socio-economic influences would also apply within Maths Invaders.
| Year Level | Year | National Average | Studied Classes Mean (range) |
|---|---|---|---|
| Year 3 | 2021 | 403 | 416 (348 – 488) |
| Year 3 | 2022 | 400 | - |
| Year 5 | 2021 | 495 | 506 (434 – 564) |
| Year 5 | 2022 | 488 | - |
Maths Invaders attainment level was highly correlated to NAPLAN scores at 0.81. This indicates that the design of content and progression within Maths Invaders is a comprehensive resource for enhancing maths capability. Answer speed was also highly correlated to NAPLAN score at 0.78. Indeed, these two measures were also highly correlated to each other at 0.88. This means that as classes reach higher levels of attainment, they also tend to answer questions more quickly. Taken together, a linear regression model using Maths Invaders data to predict NAPLAN scores showed r2 = 0.82, a high result.
There is a high correlation between the attainment levels in Maths Invaders and NAPLAN results.In isolation, analysing attainment on its own the regression showed that advancing a class by one Maths Invaders attainment level would be predictive of a gain of almost 4 points in NAPLAN. (There are approximately 45 NAPLAN points per year between the recorded NAPLAN results for Years 3 and 5.
Again, in isolation, the regression showed that improving class answer speed by 0.33 answers/minute (a time reduction of only 0.1 seconds) would be predictive of a gain of 8 points in NAPLAN.
Note that due to the high correlation between attainment level and answer speed, these outcomes are not additive. Combining an increase of Maths Invaders attainment of 2 levels with a speed improvement of 0.33 answers/second, was predictive of a NAPLAN gain of 8.4 points which could also be expressed as 8 weeks gain of a 40-week school year.
Importantly, while Maths Invaders attainment level, answer speed and NAPLAN scores are all highly correlated, there is still considerable observed variation in answer speed at class level. This means that most classes have room for improvement – speed improvements which will directly contribute to both attainment level and NAPLAN results. If your class is not yet performing as well as the best classes we have seen – 22 answers/minute – or more, you should start to unlock that potential today with Maths Invaders.
These results show clearly that performance in NAPLAN (and in mathematics generally) is not merely a product of what students know, but how well they know it – as shown, for example, through recall speed.
We have previously made the case, from a very wide range of scholarship, that revision of mathematical knowledge is essential to solidifying learning and for achieving automaticity in recall, which is a vital step for progressing to higher level maths concepts (Read the research). Furthermore, the literature shows that in most cases where students are struggling to keep up with the maths content being taught in class – the cause can be traced back to some earlier concept that has not been sufficiently solidified in the student’s mind.
We have also shown that Maths Invaders is an invaluable tool for improving students’ attainment level (Read the research) and their answer speed (Read the research). Through its Adaptive Learning approach, each student is presented with precisely the material that is most valuable for their revision – the perfect balance of consolidation and extension. Indeed, this is one of very few means by which a struggling student can plausibly regain ground on their peers – short of expensive individual tuition.
Maths Invaders comprehensively covers the mathematics curriculum for simple arithmetic, times tables and number facts. It has content rated up to Year 10 level, but it does not attempt to cover the full curriculum at these higher levels – which goes further and further beyond arithmetic. However, achieving solid results in the basics is no less important for more advanced students – it is the essential foundation.
