Spaced repetition software and TOEIC test proficiency

Reading Time: 3 minutes

Spaced repetition software (SRS) such as Anki and Memrise are touted as effective aids to vocabulary learning especially for self-directed learners. Classroom use has been limited. Bower & Rutson-Griffiths (2016) looked at Japanese learners of English studying a TOEIC wordlist in class time and measured performance on live, non-practice TOEIC tests. A significant correlation was found between SRS repetition and overall TOEIC gains.

Article
Bower, J. V., & Rutson-Griffiths, A. (2016). The relationship between the use of spaced repetition software with a TOEIC word list and TOEIC score gains. Computer Assisted Language Learning, 1-11. DOI: 10.1080/09588221.2016.1222444

Study

  • Participants
    60 Female 1st and 2nd year students, 18-20 years old, in a Japanese University English department.
  • Instruments
    Cooori a (commercial) SRS program. Features include a teacher dashboard, native speaker audio, sentence-building and multiple-choice cards.
    TOEIC wordlist from a corpus of old TOEIC tests, practice TOEIC tests and popular Japanese high school textbooks.
  • Treatment
    688 flashcards each with a Japanese translation and example sentence with recorded audio. Participants learned the Cooori software in the first 2 weeks and received a pdf of the full word list. Both class time and outside class time used and encouraged respectively over a 10 month period. Regular 20 item multiple choice tests similar to reading section in TOEIC given to incentivize students.
  • Measures collected
    Pre test and post test TOEIC given, these were live, commercial versions.
    A repetition of a flashcard was counted as one click of “know” the word or “did not know” the word.
  • Data results
    A significant positive correlation was found between number of SRS repetitions and TOEIC overall score gains (i.e. listening & reading combined). A significant correlation was found for TOEIC listening gains but not for TOEIC reading gains. An effect size calculation showed SRS repetition accounted for approximately 11% of overall TOEIC score gains.

Discussion
The authors claim this is the first study to show a statistically significant relationship between SRS study and score gains on a standardized proficiency test. They admit that the effect sizes are small but note that when compared to other variables such as 270 hours of English classes over 10 months the average SRS study time was 3.9 hours which represents 1.5% of the total English study time. This 1.5% accounts for 11% of TOEIC score gains. They further note the flashcard repetition count included both successful and unsuccessful recall of words. So fast memorizers would produce less SRS repetitions than slower memorizers hence the correlation between SRS use and TOEIC gain score would be greater if only successful recalls were used. A delayed post test would have been ideal to include in the study design. Finally they explain that motivation of the students could be an unaccounted variable in the study.

In my experience of using flashcards personal motivation is a problem! Because I am not on a course of study such as these students my drive is different and I have tried to work round this by using items of personal interest such as TV shows – Learning vocabulary through subs2srs and Anki. Final note – one practical consequence for me of reading this paper is discovering a TOEIC wordlist I was not aware of. Nice.

Mura Nava on GoogleMura Nava on TwitterMura Nava on Wordpress
Mura Nava
Interested in most things language wise. Member of TaWSIG, Teachers as Workers Special Interest Group which promotes discussion and action on working conditions in language teaching. Check us out at http://www.teachersasworkers.org/.

5 thoughts on “Spaced repetition software and TOEIC test proficiency”

  1. This is interesting. I wonder if simply studying vocab through flascards could have a similar effect. I’ve used Anki and found it cumbersome. I’ve used Quizlet, which is not exactly SRS but does have some SRS elements, and I have found it to be much more convenient. Did the participants use Cooori on their phones or on another device?

    Finally, is there a link to the TOEIC list?

Leave a Reply