Spelling Force Learning Experience Research

During the development of Spelling Force our development team has been able to analyse the accrued responses generated by the answering of over 1,000,000 words. We have used this research to iteratively modify the algorithm that drives the Spelling Force adaptive learning system and to influence the sequencing of the words presented. This process has helped make the students’ Learning Experience more efficient. To illustrate the process a sample of one of these investigations is reproduced below.

The Spelling Force Learning Experience

Dr Lindsay Brash

In earlier work we tried to define the most and least learnable words by comparing success at first attempt with success at all other attempts, but this was confounded by the order the words appeared, apparently via the effect this had on the student’s growing ability to spell and to learn to spell. A more nuanced approach was devised so that the difficulty (assessed across all attempts) and learnability (assessing the difference in success between first attempts and all subsequent attempts) of each word was measured in the context of the other words in the same set. For the first 100 words (10 sets of 10) in Era 1, the table lists the words in the 10% (approximately) most and least difficult, and most and least learnable.

The word set is also listed, demonstrating that these words are spread across the sets rather than being weighted towards early or later presentation. However, the system could still use a little fine-tuning to avoid words being classified as both least difficult and least learnable – this is because of the natural ceiling on spelling success (ie something close to 100%) that means where performance is high at first attempts, there is little room for improvement at subsequent attempts. This can be corrected in a simple refinement. Note that this adjustment will also improve the precision of classification of other words as least learnable.

This process is leading to better understanding of the learning process and therefore has the potential to offer insights which can guide the design of Spelling Force. In designing the word sets in Era 1 (and also the early word sets in later Eras), the early words should avoid high difficulty words, but could usefully include words of moderate difficulty with high learnability. The students’ experience will be less intimidation and greater sense of accomplishment.

Further, we should expect that new spellers, or those new to an era, will take longer to master a word set. Consequently, the criteria for re-levelling could be relaxed in these circumstances. This information will then be fed into the word set re-design indicated by high-difficulty words in earlier work.

In addition, as agreed earlier, we can provide specific feedback on capitalised words (“I”) to raise awareness of capitalisation and the shift key.

The formula for learnability is Learnability = (0.05 + NC – FC)/(1 – 0.99FC)

Where FC = first attempt correct answer rate for the word, standardised within word set.

NC = all other attempt correct answer rate for the word, standardised within word set.

(skips are excluded)

The following table is updated from my previous report.

Note: Moderately Learnable is defined as being in the middle 80% of words for learnability.

Moderately Difficult is defined as being in the middle 80% of words for difficulty.

As a consequence of Difficulty and Learnability assessments, changes are proposed for early Word sets. Note that the changes retain the patternation from the original DEM phonics progression lists as published in the Spelling Force CD edition.

Existing word sets 1-21.

Era set_name Set

1 2-letter words 1.1 1 I a be to go on is no it in

1 2-letter words 1.2 2 an me so he at do or as up am

1 2-letter words 1.3 3 for tug son we my if by us hi of

1 an, un 4 can man ran run sun fun van fan bun gun

1 en, in 5 fin pin hen ten bin win den tin men pen

1 at, it 6 cat sit fat sat hat bit bat hit rat mat

1 it, ot, ut 7 not but got cot hot cut pot lot hut pit

1 et 8 get met pet wet vet bet set jet yet net

1 ad, ed 9 had red bed dad bad sad led mad fed pad

1 id, od, ud 10 did bud cod nod pod hid mud lid rod bid

1 ag, eg, og 11 wag dog jog bag leg log beg fog peg rag

1 ig, ug 12 big wig pig hug bug rug dig dug mug fig

1 ap, up 13 yap sap cup map gap lap tap pup nap cap

1 ip, op 14 lip dip hip mop top pop rip hop sip zip

1 ab, ib, ob, ub 15 cab tab bib rib sob fib nib rob tub rub

1 am, im, em, um 16 mum ram dim hem him sum jam gum ham hum

1 Mixed CVC 17 his has fit fix kid rid dot pat pan job

1 Sight words 1.1 18 the was yes and you she her who saw are

1 Plural s 19 egg gem nut gems eggs hogs nuts hog bill bills

1 Homonyms 1.1 20 bat bill bit can fan jam kid pen saw tap

2 an, in, am, im 21 span spin trim swam skim plan skin tram twin scan

Proposed Word sets 1-21.

Era set_name Set

1 2-letter words 1.1 1 go be on is it in me at do am

1 2-letter words 1.2 2 as we my to an he so by up a

1 2-letter words 1.3 3 no or if us hi of I for pat son

1 an, un 4 run can man ran sun fun van fan bun gun

1 en, in 5 fin hen ten den tin men pen bin win pin

1 at, it 6 cat sit fat sat hat bit bat hit rat mat

1 it, ot, ut 7 not but got cot hot cut pot lot hut pit

1 et 8 get met pet wet vet bet set jet yet net

1 ad, ed 9 had red bed dad bad sad led mad fed pad

1 id, od, ud 10 did bud cod nod pod hid mud lid rod bid

1 ag, eg, og 11 wag dog jog bag leg log beg fog peg rag

1 ig, ug 12 big wig pig hug bug rug dig dug mug fig

1 ap, up 13 sap yap cup map gap lap tap pup nap cap

1 ip, op 14 lip dip hip mop top pop rip hop sip zip

1 ab, ib, ob, ub 15 cab tab bib rib sob fib nib rob tub rub

1 am, im, em, um 16 mum ram dim hem him sum jam gum ham hum

1 Mixed CVC 17 his has fit fix kid rid dot tug pan job

1 Sight words 1.1 18 the was yes and you she her who saw are

1 Plural s 19 egg gem nut gems eggs hogs nuts hog bill bills

1 Homonyms 1.1 20 bat bill bit can fan jam kid pen saw tap

2 an, in, am, im 21 span spin trim swam plan skin tram twin scan skim