Running Records are an uninformative waste of teacher time31 Replies
I’ve been doing lots of assessment of my clients’ skills in the following areas lately:
- Receptive and/or expressive language
- Phonological awareness
- Phonological/auditory memory
- Rapid Automatised Naming
- Word and pseudoword reading accuracy and efficiency
These allow me to identify problems in their reading and spelling systems, and work out how significant/severe these problems are, and what to do about them.
I use the robust, evidence-based Simple View of Reading (SVR) to guide my decision-making.
Wherever possible, I use valid, reliable, standardised tests for assessment. However, I once administered a Running Record to a child with selective mutism, because she would talk to me, but not other adults at school (we were working on it). Her class teacher thus asked me to administer the assessment required by the school, which (sad face) used a multi-cueing model of reading and a text level gradient approach to reading assessment.
Running Records, like most widely-used book-levelling systems (LLI, PM, Reading Recovery, etc), focus on language meaning much more than language structure (speech sounds, their spellings and meaningful word parts), though language structure is the main basis of our writing system.
Running Records should be relegated
Running Records are based on the now-discredited multicueing/three cueing model of reading. They’re highly subjective, and take far too long to find out not much. I don’t know why any school is still using them in 2019, let alone why they’re still on my state’s Education Department’s website.
The Running Record I administered years ago didn’t tell me much about the child’s reading skills that I didn’t already know because I knew about her listening, speech, phonemic awareness and word-level reading. I could have almost completed the assessment form without listening to her read, except some of its questions came from Planet Multicueing, and made no sense to me. I did what I could, gave it back to the nice teacher, and heard no more.
The three cueing system and the four humours
Asking a speech pathologist or teacher who understands the current reading science to administer a Running Record is a bit like asking a modern doctor to assess the Four Humours.
The Four Humours were the four bodily fluids of ancient and mediaeval medicine – black bile, yellow bile, phlegm and blood – thought to influence temperament and health. They were thought to require balancing in amount and strength for good health, and surpluses or imbalances in the four humours were thought to affect not only health but also personality.
A modern doctor asked to assess the Four Humours would simply say, “that’s not how the body works, so I can’t do that”. They’d insist on using a scientifically-based model of health, and prefer objective assessment tools developed in accordance with this model: stethoscope, thermometer, blood tests and so on, not rely on their own observations, let alone prescribe the leeches, infusions and vomiting which made sense to doctors using Four Humours thinking.
Everyone now knows that the Four Humours aren’t how the human body works, but many of today’s teachers still don’t know that children do not learn to read words by using meaning, structure and visual cues. The three cueing system/multicueing model of word-level reading is still the basis of Running Records, and these are still widely used in schools.
You can watch someone administering an assessment along Running Record lines here:
First the child is told to read the book silently, and reminded to look carefully at the pictures, as “making connections between the text and illustrations is an important reading strategy”. The video does not say that children should be encouraged to sound out words they don’t instantly recognise, and discouraged from guessing words from pictures. Once words are identified accurately, pictures can be used to support story comprehension and enjoyment (the oral language part of reading), but should not be used as word identification crutches.
Next, the child is asked to summarise the story, and then read it aloud. The teacher ticks each word the child gets right on the scoresheet, telling the child any words he doesn’t know, and writing down the child’s errors. The teacher then asks the child some comprehension questions, some of which require inference and the application of general knowledge. Amusingly, the child in this video got the first, literal, supposedly-easiest question wrong, and the rest right.
The teacher then goes back through the form and codes and counts the child’s errors and self-corrections, and “in terms of whether they used meaning, whether they used structure or whether the child used the visual information system”.
Since reading scientists have shown that context and syntax play almost no part in accurate word identification, and the multi-cueing model on which this coding system is based has been debunked, this is a waste of precious teacher time.
Stanislas Dehaene (who will be a keynote speaker at the Language, Literacy and Learning Conference in Perth in April) gives a nice video explanation of how the brain actually learns to read here, and David Kilpatrick unpacks the psychological and educational research on this here.
The Running Record form then has a section for recording “Reading Behaviours Observed” which starts with a check box for “Concepts about Print established”. Concepts about Print means things like holding the book up the right way, turning the pages in the right direction and looking at the book not licking it. I’ve never met a school-aged child without a severe intellectual disability who doesn’t do these things correctly, so perhaps this is one of those educational hooray-everyone-gets-a-tick-for-something things.
There’s a check box for “recognised high-frequency words in the text”, although the form doesn’t seem to identify which words are the high-frequency ones, clarify how many of them the child must recognise before the box is ticked, or explain whether to count high-frequency words that are sounded out (i.e. identified) rather than being instantly recognised. On Planet Multicueing, they don’t know about orthographic mapping.
There is a check box for “Applied knowledge of letter-sound relationships to accurately decode some words”, but it’s unclear whether this box should be ticked for words the child reads correctly without having to sound them out, or just some (two? five? ten?) words for which the child can be heard saying individual sounds and then blending them (correctly?) into a word.
Further check boxes allow the assessor to subjectively evaluate “Strategies” such as “attending to meaning” and “searching for print details” as well as “fluency”, though as David Kilpatrick explains at 1:07:45 on the clock in the video here, “the lion’s share of your fluency is determined by your sight vocabulary”, and fluency is not really its own separate Thing (Dr Kilpatrick will be speaking at LDA seminars in Australia this coming August, details will be here soon).
What Running Records do not tell us
Running Records do not attempt to assess children’s phonological awareness. Poor phonological awareness is the most common underlying cause of reading problems, so this is quite an oversight. The version in the video above doesn’t even mention phonemes, it just refers to “visual” errors. There also seems to be nowhere on a Running Record for children’s articulation errors, though these are often a sign of phonological processing weakness.
A Running Record is not designed to tell us which phoneme-grapheme correspondences a child knows, although I can tell you from watching the child in the video read that he seems to be able to decode words with “short” vowels and basic consonant spellings with VC, CVC, VCC, CVCC and CCVC structures, knows about past tense -ed, open syllables with “long” vowels like “no”, “he” and “my”, and consonant digraphs “ck”, “sh”, “th”, “ss” and “ff”.
However, I could tell you that kind of thing from watching him read any book written for a child around his age. However, it doesn’t matter which book he reads, I won’t be able to tell you whether he knows all the graphemes and word types he’s been taught, unless I use an assessment specifically designed to sample them all, such as a word reading test.
The child in the video was also able to read some words containing other digraphs: “down”, “please”, “ride”, “bike”, “you”, “your”, “too”, “looked”, “said” and “little”, but further assessment would be needed to find out whether he can read unfamiliar words containing these digraphs. He didn’t know the digraphs “ay” in “today” or “al” in “walk”, but other common digraphs were not sampled.
The child read “back” as “dack”, then changed it to “duck” so he is still reversing some letters, and sometimes guesses words, in this case on the basis of sound not meaning. This suggests some Set for Variability skills, and he was also able to correctly work out the words “path” and “grass” after sounding them out with /a/ as in “cat” (Set for Variability is discussed from 32:22 on the video clock here). I don’t know how to formally assess Set for Variability, but I wish I did.
Running Records tell us nothing about a child’s phonological memory or working memory, though these can have a massive impact on a child’s learning. Running Records also don’t help us understand kids’ oral vocabulary or Rapid Automatised Naming skills, though deficits in these areas are strong predictors of ongoing reading difficulties, more details are here.
Like many young school-aged children, the child in the video says “holded” not “held” and “hisself” not “himself”, so is still learning irregular past tense verbs and reflexive pronouns, but this kind of incidental linguistic information is not captured by the Running Record.
Finally, and most importantly, Running Records don’t more than a subjective comparison of a child’s skills with other children the same age. They don’t and can’t tell us, clearly and objectively, who is falling behind and needs intervention. This still requires teacher judgement, and if the reports I get from parents are anything to go on, many teachers are still condemning children to ongoing reading failure by saying “wait and see“.
There’s a lovely Creative Commons emoticon version of the four different personality types thought to result from too much of each of the Four Humours:
If I wasn’t far too busy I’d make stickers of these saying “and don’t forget the four humours!” and make them available to readers of this blog to stick on all mentions of the Three Cueing System/Multicueing on school noticeboards, until the blasted things are all taken down and shredded. A kind of fun-though-going-on-a-bit-too-long form of BS Bingo, to keep our spirits up.
Tests I use most
Here are the tests I’ve been using a lot lately, with their age ranges, what they assess, and how long each one takes:
|Test||Age range (years:months)||Administration time||Information provided|
|Clinical Evaluation of Language Fundamentals 5 (CELF-4)||5:0 to 21:11||30-45 minutes for core subtests||Listening and speaking (receptive and expressive language)|
|Articulation Survey||3:5 to 7:11||10-15 minutes||Consonant sound production|
|Comprehensive Test of Phonological Processing 2 (CTOPP-2)||4:0 to 24:11||40 minutes||Phonological awareness
Rapid Automatised Naming
|Rapid Automatised Naming/Rapid Alternating Stimulus tests (RAN/RAS)||5:0 to 18:11||5-10 minutes||Rapid Automatised Naming|
|Phonological Awareness Screening Test (PAST)||Preschool – adult||About 10 minutes||Phonological awareness and proficiency|
|Castles and Coltheart 2 (CC2)||6:0 to 11:6||About 10 minutes||Word and pseudoword reading accuracy|
|Test of Word Reading Efficiency 2 (TOWRE-2)||6:0 to 24:11||10 minutes or less||Word and pseudoword reading efficiency|
|Test of Written Spelling – 5 (TWS-5)||6:0 to 18:11||20 minutes||Spelling|
The recommended times on some of these tests are pretty generous. The actual testing part of the TOWRE-2 is two 45-second tasks, so it’s possible to do it in well under 10 minutes with a cooperative older child or adult with good oral language.
These tests give me a rich source of information about how far a client’s reading and spelling-related skills are behind their peers, what intervention they need, and whether they are likely to have long-term problems.
Most of these tests are expensive, clinical ones that can’t be used by classroom teachers, sometimes the test forms cost ~$10 each! However, the CC2 and other MOTIF tests are free and can be used by teachers, and the Articulation Survey is inexpensive and can be used by teachers.
Form A of the PAST and its instructions also seem to be freely available online, though I’d recommend buying the book Equipped for Reading Success (now available in Australia for $99.95 from Silvereye) because it contains all four forms, relevant theory and plenty of finely-graded phonological awareness activities.
I have suggested a few minor edits to the PAST author, Dr David Kilpatrick (did I mention he will be presenting seminars in Australia for LDA in August?), to make this test more relevant to Australian English, where deleting the “l” sound from the word “love” does not result in the word “of”. Fingers and toes crossed he can find time for this, and if anyone has time to go through his One Minute Activities and identify the Americanisms that don’t work in our accent, I’d be happy to share suggested changes to these here, so that people can make their copy of his book more relevant for use with Aussie kids.
My list of assessments that are more consistent with the reading science than Running Records is here, and please let me know if you have a good one that isn’t listed, so I can add it.
Thanks to a nice, frustrated teacher called Therese whose email prompted me to write this post, and to Heidi G for proofreading it.