Running Records are an uninformative waste of teacher time

I’ve been doing lots of assessment of my clients’ skills in the following areas lately:

  • Receptive and/or expressive language
  • Articulation
  • Phonological awareness
  • Phonological/auditory memory
  • Rapid Automatised Naming
  • Word and pseudoword reading accuracy and efficiency
  • Spelling.

These allow me to identify problems in their reading and spelling systems, and work out how significant/severe these problems are, and what to do about them.

I use the robust, evidence-based Simple View of Reading (SVR) to guide my decision-making. A new, plain-English explanation of the SVR by retired US teacher Stephen Parker can be found on Pamela Snow’s blog.

Wherever possible, I use valid, reliable, standardised tests for assessment. However, I once administered a Running Record to a child with selective mutism, because she would talk to me, but not other adults at school (we were working on it). Her class teacher thus asked me to administer the assessment required by the school, which (sad face) used a multi-cueing model of reading and a text level gradient approach to reading assessment.

Running Records, like most widely-used book-levelling systems (LLI, PM, Reading Recovery, etc), focus on language meaning much more than language structure (speech sounds, their spellings and meaningful word parts), though language structure is the main basis of our writing system.

Running Records should be relegated

Running Records are based on the now-discredited multicueing/three cueing model of reading. They’re highly subjective, and take far too long to find out not much. I don’t know why any school is still using them in 2019, let alone why they’re still on my state’s Education Department’s website.

The Running Record I administered years ago didn’t tell me much about the child’s reading skills that I didn’t already know because I knew about her listening, speech, phonemic awareness and word-level reading. I could have almost completed the assessment form without listening to her read, except some of its questions came from Planet Multicueing, and made no sense to me. I did what I could, gave it back to the nice teacher, and heard no more.

The three cueing system and the four humours

Asking a speech pathologist or teacher who understands the current reading science to administer a Running Record is a bit like asking a modern doctor to assess the Four Humours.

The Four Humours were the four bodily fluids of ancient and mediaeval medicine – black bile, yellow bile, phlegm and blood – thought to influence temperament and health. They were thought to require balancing in amount and strength for good health, and surpluses or imbalances in the four humours were thought to affect not only health but also personality.

A modern doctor asked to assess the Four Humours would simply say, “that’s not how the body works, so I can’t do that”. They’d insist on using a scientifically-based model of health, and prefer objective assessment tools developed in accordance with this model: stethoscope, thermometer, blood tests and so on, not rely on their own observations, let alone prescribe the leeches, infusions and vomiting which made sense to doctors using Four Humours thinking.

Everyone now knows that the Four Humours aren’t how the human body works, but many of today’s teachers still don’t know that children do not learn to read words by using meaning, structure and visual cues. The three cueing system/multicueing model of word-level reading is still the basis of Running Records, and these are still widely used in schools.

You can watch someone administering an assessment along Running Record lines here:

First the child is told to read the book silently, and reminded to look carefully at the pictures, as “making connections between the text and illustrations is an important reading strategy”. The video does not say that children should be encouraged to sound out words they don’t instantly recognise, and discouraged from guessing words from pictures. Once words are identified accurately, pictures can be used to support story comprehension and enjoyment (the oral language part of reading), but should not be used as word identification crutches.

Next, the child is asked to summarise the story, and then read it aloud. The teacher ticks each word the child gets right on the scoresheet, telling the child any words he doesn’t know, and writing down the child’s errors. The teacher then asks the child some comprehension questions, some of which require inference and the application of general knowledge. Amusingly, the child in this video got the first, literal, supposedly-easiest question wrong, and the rest right.

The teacher then goes back through the form and codes and counts the child’s errors and self-corrections, and “in terms of whether they used meaning, whether they used structure or whether the child used the visual information system”.

Since reading scientists have shown that context and syntax play almost no part in accurate word identification, and the multi-cueing model on which this coding system is based has been debunked, this is a waste of precious teacher time.

Stanislas Dehaene (who will be a keynote speaker at the Language, Literacy and Learning Conference in Perth in April) gives a nice video explanation of how the brain actually learns to read here, and David Kilpatrick unpacks the psychological and educational research on this here.

The Running Record form then has a section for recording “Reading Behaviours Observed” which starts with a check box for “Concepts about Print established”. Concepts about Print means things like holding the book up the right way, turning the pages in the right direction and looking at the book not licking it. I’ve never met a school-aged child without a severe intellectual disability who doesn’t do these things correctly, so perhaps this is one of those educational hooray-everyone-gets-a-tick-for-something things.

There’s a check box for “recognised high-frequency words in the text”, although the form doesn’t seem to identify which words are the high-frequency ones, clarify how many of them the child must recognise before the box is ticked, or explain whether to count high-frequency words that are sounded out (i.e. identified) rather than being instantly recognised. On Planet Multicueing, they don’t know about orthographic mapping.

There is a check box for “Applied knowledge of letter-sound relationships to accurately decode some words”, but it’s unclear whether this box should be ticked for words the child reads correctly without having to sound them out, or just some (two? five? ten?) words for which the child can be heard saying individual sounds and then blending them (correctly?) into a word.

Further check boxes allow the assessor to subjectively evaluate “Strategies” such as “attending to meaning” and “searching for print details” as well as “fluency”, though as David Kilpatrick explains at 1:07:45 on the clock in the video here, “the lion’s share of your fluency is determined by your sight vocabulary”, and fluency is not really its own separate Thing (Dr Kilpatrick will be speaking at LDA seminars in Australia this coming August, details will be here soon).

What Running Records do not tell us

Running Records do not attempt to assess children’s phonological awareness. Poor phonological awareness is the most common underlying cause of reading problems, so this is quite an oversight. The version in the video above doesn’t even mention phonemes, it just refers to “visual” errors. There also seems to be nowhere on a Running Record for children’s articulation errors, though these are often a sign of phonological processing weakness.

A Running Record is not designed to tell us which phoneme-grapheme correspondences a child knows, although I can tell you from watching the child in the video read that he seems to be able to decode words with “short” vowels and basic consonant spellings with VC, CVC, VCC, CVCC and CCVC structures, knows about past tense -ed, open syllables with “long” vowels like “no”, “he” and “my”, and consonant digraphs “ck”, “sh”, “th”, “ss” and “ff”.

However, I could tell you that kind of thing from watching him read any book written for a child around his age. However, it doesn’t matter which book he reads, I won’t be able to tell you whether he knows all the graphemes and word types he’s been taught, unless I use an assessment specifically designed to sample them all, such as a word reading test.

The child in the video was also able to read some words containing other digraphs: “down”, “please”, “ride”, “bike”, “you”, “your”, “too”, “looked”, “said” and “little”, but further assessment would be needed to find out whether he can read unfamiliar words containing these digraphs. He didn’t know the digraphs “ay” in “today” or “al” in “walk”, but other common digraphs were not sampled.

The child read “back” as “dack”, then changed it to “duck” so he is still reversing some letters, and sometimes guesses words, in this case on the basis of sound not meaning. This suggests some Set for Variability skills, and he was also able to correctly work out the words “path” and “grass” after sounding them out with /a/ as in “cat” (Set for Variability is discussed from 32:22 on the video clock here). I don’t  know how to formally assess Set for Variability, but I wish I did.

Running Records tell us nothing about a child’s phonological memory or working memory, though these can have a massive impact on a child’s learning. Running Records also don’t help us understand kids’ oral vocabulary or Rapid Automatised Naming skills, though deficits in these areas are strong predictors of ongoing reading difficulties, more details are here.

Like many young school-aged children, the child in the video says “holded” not “held” and “hisself” not “himself”, so is still learning irregular past tense verbs and reflexive pronouns, but this kind of incidental linguistic information is not captured by the Running Record.

Finally, and most importantly, Running Records don’t more than a subjective comparison of a child’s skills with other children the same age. They don’t and can’t tell us, clearly and objectively, who is falling behind and needs intervention. This still requires teacher judgement, and if the reports I get from parents are anything to go on, many teachers are still condemning children to ongoing reading failure by saying “wait and see“.

Humour me

There’s a lovely Creative Commons emoticon version of the four different personality types thought to result from too much of each of the Four Humours:

If I wasn’t far too busy I’d make stickers of these saying “and don’t forget the four humours!” and make them available to readers of this blog to stick on all mentions of the Three Cueing System/Multicueing on school noticeboards, until the blasted things are all taken down and shredded. A kind of fun-though-going-on-a-bit-too-long form of BS Bingo, to keep our spirits up.

Tests I use most

Here are the tests I’ve been using a lot lately, with their age ranges, what they assess, and how long each one takes:

Test Age range (years:months) Administration time Information provided
Clinical Evaluation of Language Fundamentals 5 (CELF-4) 5:0 to 21:11 30-45 minutes for core subtests Listening and speaking (receptive and expressive language)
Articulation Survey 3:5 to 7:11 10-15 minutes Consonant sound production
Comprehensive Test of Phonological Processing 2 (CTOPP-2) 4:0 to 24:11 40 minutes

Phonological awareness

Phonological memory

Rapid Automatised Naming

Rapid Automatised Naming/Rapid Alternating Stimulus tests (RAN/RAS) 5:0 to 18:11 5-10 minutes Rapid Automatised Naming
Phonological Awareness Screening Test (PAST) Preschool – adult About 10 minutes Phonological awareness and proficiency
Castles and Coltheart 2 (CC2) 6:0 to 11:6 About 10 minutes Word and pseudoword reading accuracy
Test of Word Reading Efficiency 2 (TOWRE-2)  6:0 to 24:11 10 minutes or less Word and pseudoword reading efficiency
Test of Written Spelling – 5 (TWS-5) 6:0 to 18:11 20 minutes Spelling

The recommended times on some of these tests are pretty generous. The actual testing part of the TOWRE-2 is two 45-second tasks, so it’s possible to do it in well under 10 minutes with a cooperative older child or adult with good oral language.

These tests give me a rich source of information about how far a client’s reading and spelling-related skills are behind their peers, what intervention they need, and whether they are likely to have long-term problems.

Most of these tests are expensive, clinical ones that can’t be used by classroom teachers, sometimes the test forms cost ~$10 each! However, the CC2 and other MOTIF tests are free and can be used by teachers, and the Articulation Survey is inexpensive and can be used by teachers.

Form A of the PAST and its instructions also seem to be freely available online, though I’d recommend buying the book Equipped for Reading Success (now available in Australia for $99.95 from Silvereye) because it contains all four forms, relevant theory and plenty of finely-graded phonological awareness activities.

I have suggested a few minor edits to the PAST author, Dr David Kilpatrick (did I mention he will be presenting seminars in Australia for LDA in August?), to make this test more relevant to Australian English, where deleting the “l” sound from the word “love” does not result in the word “of”. Fingers and toes crossed he can find time for this, and if anyone has time to go through his One Minute Activities and identify the Americanisms that don’t work in our accent, I’d be happy to share suggested changes to these here, so that people can make their copy of his book more relevant for use with Aussie kids.

My list of assessments that are more consistent with the reading science than Running Records is here, and please let me know if you have a good one that isn’t listed, so I can add it.

Thanks to a nice, frustrated teacher called Therese whose email prompted me to write this post, and to Heidi G for proofreading it.

Related Posts Plugin for WordPress, Blogger...

8 thoughts on “Running Records are an uninformative waste of teacher time

  1. Natalie Campbell

    “Guided Reading” needs a whole rethink in Australia. Google this term and you will see why.
    Teachers need an instructional reading lesson format and record sheet to replace the usual guided reading process and running record sheet. We need something that can be built in to current practice and something that can promote reading science within each reading lesson.
    Could Spelfabet create such a template?

    Reply
    1. alison Post author

      Hi Natalie, I agree, there is a whole educational artifice constructed around the incorrect idea that learning to read is just like learning to speak, and it needs to be pulled down and rebuilt around the scientific evidence. Would the Multilit Reinforced Reading program be of any use as a tool to replace Guided Reading? See https://multilit.com/programs/reinforced-reading-program. I am not a teacher so I’m not sure I’m the best person to create a template for teachers, plus I have about a thousand other things on my to-do list at present. All things I neglected today in order to write this blog post! All the best, Alison

      Reply
  2. Jasmine Shannon

    Hi Alison,
    I just love your break down of running records. The assessments you reference though are for speechies. I wonder if teachers are able to use them also? We use the Read Write Inc assessments here in the NT and it is proving to be really successful.

    I do have one bone to pick though…. when you say, “I’ve never met a school-aged child without a severe intellectual disability who doesn’t do these things correctly, so perhaps this is one of those educational hooray-everyone-gets-a-tick-for-something things” I must strongly disagree.

    I work in some of the most remote schools in the NT. Some of these kids come to school never owning or seeing an English text. These kids can speak up to four languages before school and only hear English for the first time when they are at school. Often they only hear English at school and have no other use for it. So I guess what I’m saying is the print concept skills also need to be taught to many kids. It’s not just white middle class kids we are trying to teach to read, but our most marginalised kids of Australia too.

    Good luck and I look forward to your guided reading blog- hint hint-

    Reply
    1. alison Post author

      Jasmine, hi, lovely to hear from you again. You’re right, the assessments I use are mostly for clinicians and special ed teachers, but the MOTIF ones are accessible to all teachers, aren’t they? And teachers can use the PAST, I wish they would, though having an Aussie version would really help with that. I don’t know the RWI assessments but I’m sure they are good ones too. I wish I knew all the good assessments that teachers can use, but maybe I will find out about more of them from the responses to this blog, so I can update my list.

      Thanks for your feedback regarding the indigenous kids you work with who arrive at school speaking four languages but no English. I met kids like that when my sister worked for the Northern Lands Council and I went with her on a trip to Victoria River Downs Station, so I know they exist, and I know they don’t have many books, but are you sure they arrive at school not knowing which way up to hold a book or how to turn the pages? The kids I met certainly knew how to answer the phone in the phone box up next to the big house with its lawn and swimming pool (whereas they lived in windowless tin shacks in red dust, and swam in the creek with the crocs) when my sister rang to talk to the local elder about their land claim. The kids liked telling my sister to “puck off”, and then hanging up, giggling. I just got back from a holiday at Uluru and Alice Springs where I bought a whole lot of great indigenous picture books for my waiting room, and I know Mandy Nayton has been involved in producing teaching materials in indigenous languages, and I have the Honey Ant Readers on my list of things to buy, so I hope literacy-teaching is now more culturally inclusive of indigenous kids. It certainly wasn’t in 1972 when I made friends with an indigenous kid at school (we’re still friends). I’m sure there are some kids who start school without ever having seen a book, but I’ve personally never met one. I’ve also worked with some very disadvantaged kids, I had a part-time job for 15 years at a secondary school next to the Flemington hi rise, where many of the students arrived almost straight from refugee camps, had never been to school till they came to Australia, and lived in public housing with parents or other relatives (if their parents were dead, which was not uncommon) who spoke no English, and the household had few or no books. I am currently working a couple of hours a week seeing 14 kids at Fitzroy PS, and they mostly live in the hi rise, and are almost all of refugee, migrant or indigenous background. I genuinely haven’t yet met a child with reasonable cognitive skills who didn’t know which way to hold a book or how to turn the pages, even the Arabic-speaking ones knew books in English have pages that turn towards the left. I’m sure you have, and I’m sure you’re doing great work with them, but I’m not just talking about white middle class kids. The disadvantaged kids I’ve known have done well with systematic phonics, even some kids who officially had IQ scores that suggested they probably wouldn’t be able to learn to sound out words. I’m sure yours are too.

      I think my Guided Reading blog goes like this: “Guided Reading sucks, why don’t you try Reinforced Reading? https://multilit.com/programs/reinforced-reading-program.” Would that work for you?

      All the very best, Alison.

      Reply
      1. Jasmine Shannon

        Yes, “guided reading sucks” works really well for me actually!

        Thanks for your reply, your experiences are not that different to ours here. But yes, sadly, we get kids come to school for the first time not knowing which way is up on a book. Remember, they have an oral culture, and the only print these kids may possibly have seen before school is at the clinic or brands of food at the shop. There are no books in homes and even some communities don’t have street names, just big painted numbers on their houses.

        Come for a visit sometime. It will take your breath away. I’m off to jiklmingan, ngukarr, minyerri this week. Google them.

        Keep up the good fight! I don’t know how you do it. I lose sleep over one whole language proponent’s tweet these days, I’m completely hopeless.

        J

        Reply
        1. alison Post author

          I’d love to visit Jiklmingan, Ngukarr and Minyerri, and I have far too much to do here right now but if you’re still there next year, maybe. I’m old and these days I don’t care too much what people think of me. And if I can’t stick my head above the parapet and say “the emperor of running records has no clothes” then how can class teachers?

          Reply
  3. Luisa

    Thank you for this excellent blog. I have shared it on facebook in the hope that some of my fellow educators will read it. It is particularly pertinent to me at the moment as my school is insisting on only using RRs and has abandoned its DIBELs assessments and MiniLit intervention. I have been given the Learning Support /EAL/D role as I have been deemed too big of a risk to have my own class this year (I previously taught Yr 1 using a Structured Literacy approach which my principal vehemently disapproves of – she is Balanced Literacy all the way). So far in this role I have been used to give the Yr 1 teachers release in order for them to perform RRs on their students. I have asked to do DIBELs at least on the children who were identified last year so that I can know where their learning deficits are and so I can monitor their progress. Because I knew the answer was likely a no, I went back to an earlier blog of yours which suggested MOTIF assessments. I signed up and was approved the next day. Unfortunately I have been told explicitly not to use any of these assessments, I am to use the RRs in order to decide how I’ll group the children for their Tier 2 intervention and I am to do Guided Reading with them. It turns out that phonological and phonemic assessments are not part of my principal’s “vision”. I am at a total loss of how exactly to deal with this situation.

    Reply
    1. alison Post author

      OH, LUISA, THAT’S AWFUL, I’m so sorry to hear it. I guess since you’re in the system, you need to do as you’re told, despite what you know about the evidence. I just don’t know what to make of Whole Language diehards who currently occupy positions of authority in our system, and don’t accept the scientific evidence. They are education’s answer to climate sceptics, and like climate sceptics they put children’s futures at risk. I’m so happy to work for myself and be able to follow the evidence not someone else’s instructions. If you live near the school, or other schools with a similar approach, maybe you could you set yourself up as an LDA tutor (www.ldaustralia.org/tutor-referral-services)? You’d have plenty of customers, and the freedom to decide for yourself how to teach. All the best, Alison

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *