Arrowsmith

The Arrowsmith program has been going for about 35 years, and had a research program for about 18 years.

In all that time, not one study has appeared in a peer-reviewed journal providing reliable scientific evidence that it works. Not one.

Perhaps the Australian Education Union didn’t realise this when giving Canada’s Barbara Arrowsmith-Young (AKA The Woman Who Changed Her Brain) a platform to promote her program here last week.

Radio National also interviewed her but failed to ask her hard questions, like whether her program has actually been shown to work.

I’ve already written about my concerns about the Arrowsmith program here and here, so won’t repeat myself, or rehash excellent blog posts on this topic by Head of La Trobe University’s Rural Health School Professor Pamela Snow and Oxford Professor of Developmental Neuropsychology Dorothy Bishop. Just read them here and here if you haven’t already.

Head of the School of Psychology at Charles Sturt University, Associate Professor Tim Hannan, wondered in Australasian Science magazine in 2015, “…why has the Catholic Education Office in Sydney, in company with a dozen other schools in Australia and New Zealand, decided to embrace a program with no reliable scientific evidence of its efficacy?” (Vol 36, 9, p41). We are all still scratching our heads.

We now have a new Canadian court decision about whether a government school discriminated against a student by not offering him the Arrowsmith program. It states:

“During this hearing, no scientific basis was demonstrated and among the five experts who testified, no one was able to show empirical evidence supporting the Arrowsmith method.”

Charisma, anecdotes, testimonials. Pay no attention. In fact, if someone offering a program for children with learning difficulties offers you anecdotes and testimonials, run a mile. They are usually a strong indication of a lack of proper evidence for the program.

What matters for children with learning difficulties is whether there is good evidence to support doing what the program does. Science is how we make sure we aren’t just believing what we want to believe.

The best scientific evidence you can get, at the top of the Evidence Pyramid, is a systematic review of all the well-conducted research in an area.

You can click here to read one such systematic review of the research into treatment approaches for children and adolescents with reading disabilities, which says, “The results revealed that phonics instruction is not only the most frequently investigated treatment approach, but also the only approach whose efficacy on reading and spelling performance in children and adolescents with reading disabilities is statistically confirmed.”

No, that doesn’t make phonics a blanket recommendation for all children who are struggling with reading. Some children have unusual profiles, and the evidence pyramid has several levels.

However, “my personal experience” and “I just know” are simply not part of the evidence pyramid, despite what satirists might say.

Related Posts Plugin for WordPress, Blogger...

26 thoughts on “Arrowsmith

  1. K Easter

    Obviously, there should be some independent peer-reviewed research on the program, although I believe that there is some research being undertaken at the moment. Does however, a lack of randomised controlled trials mean that the program doesn’t work?

    I do understand that there are many bogus programs out there that have swindled parents, so I commend those who recommend to parents to check if the remediation they are seeking for their child is likely to help. If programs don’t work, the word does get around and this effects participation rates, the bottom line and negative reviews abound.

    The Arrowsmith program is getting larger and growing every year and positive reviews from schools and parents are everywhere. Even one parent went as far as to try and sue a school board for not offering Arrowsmith to their son. Why is that there is so much vehement support for the program, in spite of experts warning that there is no independent evidence of its efficacy? Can it just be down to schools and parents being hoodwinked by slick marketing campaigns? There must be some reason why half the student population at the Arrowsmith School in Vancouver are Australian students and they stay for the full 3-4 year length of the program when they could drop out and come back after 6 months.

    Schools like Holy Innocents in Sydney underwent a 2 year trial and could have decided not to continue with the program. 2 years is long enough to see through bogus claims. Instead the Catholic Education Office is singing Arrowsmith’s praises.

    The biggest promoters of the Arrowsmith program are the schools that take it on, find that it works well and then recommend it to other schools. https://www.sydneycatholic.org/…/2015/2015112_1129.shtml
    Other bigger promoters are the parents who take it on, find that it helps their children and recommend it to other parents. https://www.facebook.com/PerthParentsAdvocacyforArrowsmith/

    I would suggest that instead of dismissing or even slandering the program from afar, couldn’t the movement in favour of Arrowsmith and the intense curiosity in the program possibly indicate potential value in looking into it more closely? That way, at least one’s criticism will be based on personal experience rather than second hand sources.

    Personally I’m sick of the all the controversy surrounding Arrowsmith and I would just like to know one way or the other. So would many other people in education. Perhaps the experts who point out that the program has not been tested rigorously should offer to carry out the research required instead of just sitting around whinging about the lack of evidence. Just a thought.

    Reply
  2. Karina McLachlain

    Hi Alison,
    I don’t have any direct link with Arrowsmith. However, a former student of mine at a special needs school in London was enrolled by his mother in the Arrowsmith School in Toronto at about the same time that I moved back to Australia. I keep in touch with his parents from time to time because he was a lovely boy and I am interested in his progress.
    I have seen the allegation several times that the lack of negative Arrowsmith reviews by parents and former students is due to a ‘confidentiality agreement’ that parents are forced to sign that prevents them from saying anything negative. I decided to ask my former student’s mother if there was any truth to the rumour, as I know that she is a very frank person and not one who would easily be silenced. If parents are forced to sign such an agreement, obviously that would be a scandal and a huge blight on the school’s reputation.

    Mary Goldblatt gave me her permission to cut and paste her reply into this blog:

    Regarding the comments on a confidentiality clause, it is completely untrue, an
    utter fabrication and predictably offensive to parents.
    It does though highlight an issue those parents of children with learning
    differences particularly too often face, in that we are considered so desperate,
    that we would sign up for rat poison if someone told us it would work! That old
    adage though that ‘need is the mother of invention’ is a better representation
    of the position, certainly ours anyway, rather than ingest failure or limits we
    would prefer to create opportunity with the potential for growth with no rat
    poison included!

    Her son has made good general progress, although he has worked hard for it. This one boy does not constitute a sample size, but at least I know that the programme works for some. I too would love to see some quantitative research so that we all have a better idea of what kinds of difficulties can be helped and which aspects of the programme are the least/most effective (and for whom/which disability).

    Unfortunately, I don’t think that the lack of research is all Arrowsmith’s fault. It is my impression that Linda Segal has tried to discourage researchers from taking an interest in this programme. She carried out a shonky piece of research more than 10 years ago that apparently showed negative results for Arrowsmith when compared to another group given a standard special ed support programme. Without releasing her report for scrutiny, she quoted that her results showed that the Arrowsmith Programme was ‘ineffective’, and at the same time criticised Arrowsmith of not having any publishable data.
    Once her report finally became public, after more than a decade (Segal had given a copy to a parent who forwarded it on to Howard Eaton), it was evident that her research had been carried out with the agenda to shut down the Arrowsmith programme from the school board that was trialling it, regardless of whether it was effective or not.
    The research design had students just beginning the Arrowsmith programme compared to students who had been doing the standard programme for at least a year already. There were a substantial number of dropouts from the standard programme (none from the Arrowmsith), but this skewing of data was not taken into account when analysing the results. Segal’s stats were re-analysed by expert statisticians who found that the results actually favoured Arrowsmith. This is despite the fact that the programme was cut after only 8 months – substantially shorter than its actual length.

    During the decade that she kept the report under wraps, Segal used many platforms to broadcast the apparent poor results of the Arrowsmith Programme and influence a lot of experts in the area of learning difficulties to join in the criticism, instead of developing an interest into looking into the programme and judging its merits without pre-conceived bias. If Segal hadn’t of scuttled the trial that she was researching, there would already have been at least one fully completed review of the programme with a control group, possibly more. As the Eaton Arrowsmith school agreed to participate in the trial/study that Segal scuttled, doesn’t that show that Arrowsmith are willing for their programme to be researched? So, where are the researchers?It rather seems that Segal seems has gone out of her way to discourage the initiation or completion of empirical research in to Arrowsmith. You can’t do this on the one hand and criticise Arrowsmith for not having the research.

    Strangely, none of the experts who joined in with Segal to so vehemently to criticise Arrowsmith, said one critical word when Segal’s subterfuge came to light. In fact, she was given a distinguished researcher’s award by Learning Difficulties Australia, wasn’t she? Learning Difficulties Australia criticise lack of empirical research, but reward shonky research that even a first year Psychology student could tell was biased and poorly designed????????????

    Thanks Mr Easter, I agree that instead of criticising what hasn’t been done, researchers could instead get out there and do it.

    Reply
  3. Pingback: Arrowsmith | The Echo Chamber

  4. Clare

    I am staggered in this world of so much research that universities who fund programs and research then don’t disseminate the information though courses offered to their students! Take spelling for example, we know what works and yet many schools still run the same spelling lists and look, cover, write, check stuff that doesn’t work. It is very worring to have programs in place that just don’t work.

    Reply
  5. Jo-Anne Gross

    Dr. Linda Seigel is one of Canada`s most prominent researchers-she worked frequently with the highly reknown Dr. Keith Stanovich.

    in no way would she ever be involved with quackery or fudged research,that`s a preposterous scenario.

    The reason Arrowsmith gets bought is because the schools give our dyslexic children or L.D. children nothing but empty IEPS to simply justify a diagnosis and DO NOTHING for families and children..it`s educational malpractice-my God-Dyslexia with a phonological base-the norm-can be dramatically improved in 6 months.

    While I know Arrowsmith does not honour the cutting edge research on explicit systematic synthetic phonics or Orton Gillingham it`s SOMETHING for kids who get platitudes.

    Reply
  6. Lara

    For the sake of all children with Dyslexia please please please use Sounds~Write. It is an evidence based linguistic phonic programme designed to be delivered sequentially. It works, I know as I use it daily with students with a moderate to severe diagnosis. These kids don’t need marketing ploys and lesson fillers they need understanding passionate teachers that understand and know phonics themselves.

    Many in the industry should be ashamed of themselves. The evidence is available, speak to the experts, DSF Dyslexia SPELD will soon tell you what works best. They know, it is their passion and are totally committed to students with a learning difficulty. Many mental health problems seen today could very easily be attributed to reading difficulties and constant feelings of failure. Again the evidence is also available regarding such studies.

    Reply
      1. Lois Smith

        My son has recntly attended a 6 week cognitive intensive program with the Arrowsmith Program in Canada. We were never asked to sign an agreement nor asked not to speak about the Arrowsmith program in an unfavourable way. We were also part of an independent research program. I also, am deeply offended, that the author perhaps thinks the parents have somehow been duped or hypnotized into attending these programs. Unfortunately parents are left with little alternative but to try different learning opportunities for their children as the education system is failing them in droves. I have the impression that the author has little true understanding of the needs of the children who attend the Arrowsmith Program. Perhaps instead of only looking to researchers to draw all of your opinions, try and speak to the zombie like parents who follow blindly and stupidly into the crackpot world of Arrowsmith. The 6 weeks that we spent at the Arrowsmith program was the most rewarding 6 weeks of my child’s education and he is now 13 and I am really no further ahead with finding help for him back here in the ‘normal’ education system. He is not dyslexic, he does not have dysgraphia, he is not diagnosed with ADD but he does have a learning deficit and believe me, he works twice as hard at his class work than most of the kids in his class and he receives 1/2 of the reward. Probably not even that. What should I tell my son. We will just wait here and see if they bring out some research before I go off and try something new. In the meantime lets follow the droconian footsteps of the education system that basically hasn’t changed in hundreds of years and still believes ‘rote’ learning is the answer to all. Oh and by the way I’ve tried and spent a fortune on speech therapists, the maths tutors, child psychologists trying to get a ‘diagnosis’ for my son so I can access learning support. Why are you so against trying something different? Do you really believe that all children are round pegs and will fit neatly into your round hole if they just follow the research trail. Speak to the people at the coalface of the problem. The parents, the mad, deflated, frustrated parents. At the end of the day I will do what I think is right for my child. Research of no research. I’m not that afraid

        Reply
        1. alison Post author

          Dear Lois, I’m so sorry to hear you’ve been so let down by the current system that you feel you need to resort to interventions that are expensive and not backed up by peer-reviewed research. As a full-time practising clinician I AM one of the people at the coalface, working with parents and teachers as well as children and some adults with difficulties, for example I have for the last two terms been working at a local school with a lot of kids from public housing, many from refugee and other disadvantaged families, who could barely read or write just six months ago. Their letters were back to front and they left sounds out of words and struggled to decode the simplest books, but on Monday our group of six kids aged 7 and 8 each read a whole chapter book from Level 1 of the Sound Out Chapter books, and completed four pages of worksheets about them targeting spelling, vocabulary and comprehension, in a one hour session. One kid (who used to act up and climb under the table) voluntarily stayed back into her lunch break to finish the last sheet, as she was determined to get the whole lot done. These kids are very proud of themselves, one of them said unprompted during the session “I love reading, I used to hate it but now I love it”. This is of course an anecdote and should not be treated as evidence, but am very proud of using methodologies that ARE based on the peer-reviewed research, and our test scores and in-class observations regarding progress are consistent with the research. I hope your son achieves similar success soon, and assure you that his best chances lie with methodologies that are based on current, scientific models of reading and spelling.

          Reply
          1. Lois Smith

            With respect to your opinion, it is just that, your opinion. I believe brain plasticity is a current scientific model based on sound methodology and, as I said, I have just experienced, for the first time, success with my son and that was through the Arrowsmith Program. I don’t look at his academic record to mark his progress, his deficits affect him in far more ways than a maths or comprehension test is going to show and therein lay the frustration with the Education system. I am looking to improve his mental health and keep his self esteem intact by going back to the grass roots (his brain) and trying to give him a better chance at life, not just schooling. It took him about 3 years to learn to ride a bike because he doens’t understand the ‘push’, ‘pull’ dynamic or how to blow his nose correctly. These deficits go way beyond reading and spelling. He is great reader and speller, in fact he is in the 90th percentile for both but he doesn’t understand what he is reading, beyond the basic, or he doesn’t see patterns and what he learns today, he’s lost tomorrow. These children experience a very wide range of issues and I did not understand how much it affected their lives until I spent 6 weeks speaking to parents of similar children at the Arrowsmith Program. I was not learning this from the ‘brainwashes’ at the Arrowsmith program, I was learning this from the real people at the coalface. The people who live with these children and see them going backwards year after year despite all attempts at peer reviewed, current, acceptable research. I am not doubting for one minute that your program works, but I also don’t understand why you so strongly don’t accept that this program could work simply because its lack of research. I understand its been very difficult to get the program independently researched and also funding to research it and it is obviously a long term research program. Perhaps the critics should enrol for a 6 week clocks program and speak to the people accessing this course and ask them why they are accessing it and believe that it is making a difference to their children. How about you do a 6 week Cognitive Intensive program and I will do the Sound Out Program.

          2. alison Post author

            Hi Lois, sorry to hear your son has such extensive difficulties. There is a lot of neurobabble around at the moment, and a lot of programs that require parents to “believe” in something without much evidence, but saying “brains are plastic” means the same thing as saying “children can learn”, and talking about “brain based learning” is as silly as talking about “lung-based breathing”, “leg-based walking” or “stomach-based digestion”.

            When I spend money and time on professional development, I make sure first that it’s credible and based on proper, peer-reviewed research, or consistent with the models derived therefrom. I don’t spend time or money on a program that has been around for three decades, has been doing research for almost two decades, but still has not a single article published in a peer-reviewed journal providing evidence of its effectiveness. There are too many known-to-be-effective programs available, and that’s where I am focussing my attention. All the best in helping your son with his learning, but don’t use the Sound Out Chapter Books with him if his decoding and spelling are fine, concentrate on his language comprehension, it sounds like he may have a developmental language disorder. Lots of good information about helping kids with oral language difficulties, including comprehension difficulties, is here: https://www.youtube.com/user/RALLIcampaign. All the very best, Alison

  7. Susan Caves

    I really want to join the other Arrowsmith parents here to once again respond to attacks from educators who pat us on the head and ignore what we and many teachers with experience with the Arrowsmith Program are saying. For a start I would refer you to the NSW Parliamentary Inquiry – Students with a disability or special needs in NSW schools. Read the Correspondence dated 9/8/2017 in the Other Documents tab, particularly the Annexures L-N which are 70-plus pages of parent support. It was all ignored in the published report of course. Also, the story about Linda Segal’s report is true. It happened and the data has now been re-analysed.
    Second, I would refer all educators who disapprove of the Arrowsmith Program methods to broaden your academic reading to Journals such as ‘Trends in Neuroscience and Education’ and ‘Learning and Individual Difference’. Read up on topics such as relational frame theory and multiple exemplar training to get an idea of why Arrowsmith is so successful. Your claims that there is no peer-reviewed evidence for Barbara’s methods is becoming rather threadbare – here’s a good one to start with: Hale et al. (2016) Trends in Neuroscience and Education 5; 41-51.
    The cognitive exercises that Barbara Arrowsmith-Young has developed over the past 30 years are now being understood through the developments in neuroscience. Educators actually can’t continue to ignore these findings and expect to be taken seriously by the parents of children with learning disabilities. Just open your minds to something that can make a huge, life-changing difference to educational outcomes for around 10% of kids in every school. We’re not saying every child needs the Arrowsmith Program, but it can literally save the lives of those who do.

    Reply
    1. Lois Smith

      Thankyou Susan, such a well written comment. Please please let the forward thinking educators take note.

      Reply
    2. alison Post author

      Supportive parent comments, no matter how numerous, lengthy or sincere, do not constitute evidence in education. I looked up Arrowsmith in Trends in Neuroscience and Education and found zero articles. The abstract for the article you mention says it is arguing that “early intervention and remediation of skill deficits is the preferred evidence-based practice”, which I don’t imagine anyone disagrees with, The article goes on to argue for “systematic brain-based differentiated instruction” which could mean many things, and the term “brain-based” is an odd one, since all learning happens in the brain, there is no such thing as “elbow-based learning”. The abstract doesn’t mention Arrowsmith, and anyway the article doesn’t seem to be presenting any new evidence, so I am not going to pay the usually large amount of money required to access the entire article.

      Reply
  8. Susan Caves

    Thanks Lois, I share your deep frustration! We also took our daughter to Toronto for intensives in 2015 (when she was in Year 6) and again in 2016. I would agree wholeheartedly with you – the time we spent there was extremely rewarding and really opened our eyes as to what might be possible for our daughter’s future. Now we have her in a full time program here in Australia and she continues to make amazing gains. It’s not easy at all, but she works really hard because she can see and feel for herself the difference that the cognitive exercises are making. I wish you you and your son the best of luck- keep talking, keep pushing. More and more schools here are implementing the Arrowsmith Program every year.

    Reply
  9. Susan Caves

    Thank you Alison for your response and for taking the time to look at the abstract of the article I mentioned. Unfortunately, without reading the full article it is difficult for you to follow the argument, which in very abbreviated terms is, that there is lots of fertile and productive common ground between the two different approaches to children who are not able to access the general education curriculum. These authors use the term ‘brain-based’ to signify an approach to learning which not only accepts but actively addresses the neurological processing deficits underlying the literacy and numeracy skills deficits. They differentiate this from the approach more commonly used by educators which is to try all manner of ways to teach the skill itself without addressing the more basic cognitive background to the problem. Go ahead and fork out a few dollars for the entire article, it’s worth it.
    You are correct, Arrowsmith is not mentioned once in the entire article and it is a review and analysis rather than an empirical paper. If I could I’d attach a copy for you, but I had assumed from your blog that you had access to academic literature. The other journals I mentioned also do not directly reference the Arrowsmith Program, but for those familiar with the kinds of cognitive exercises in the program, the parallels are striking and it seems to me as if Barbara Arrowsmith-Young’s work anticipated the neuroscience by about 30 years.
    I think this article resonated for me because looking back at the efforts of my daughter and those of her teachers and parents before she had the benefit of the Arrowsmith Program, it really was as if we were trying to teach a blind person to see or a deaf person to hear. I am neither a teacher nor a neuroscientist so I am prepared to accept that I have the theory completely wrong, all I ask is that you approach this with an open mind rather than ridiculing something when you are not prepared to engage with it. I’m sure it’s challenging in lots of ways, but you know, not nearly as challenging as being a kid with learning disabilities in our school system.

    Reply
    1. Lois Smith

      Susan I find it hard to understand how parents are not considered to be a part of this solution. Unfortunately some educators appear determined to be part of the problem . I’m all for open mindedness. I wholeheartedly agree with your analogy – its is as though by telling our blind children to open their eyes more so that they can see and as we know, they are trying very hard to open their eyes more. Thankyou again for your thoughtful and well worded response.

      Reply
      1. Susan Caves

        Your earlier comment about difficulty riding a bike and not being able to understand the difference between sniffing and blowing one’s nose made me smile – just the kind of problems our daughter had. She has severe receptive and expressive language deficits and It’s my feeling that Clocks is the most powerful exercise these kids with language problems can do. It seems to unlock their access to logic and allow them to follow reasoning in verbal and written work. But more importantly, it makes a crucial difference in their ‘internal voice’ (their thoughts I guess?) so that they can use language effectively for their own thinking and decision-making. It was when I read Alexander Luria’s book (The Man With the Shattered World) that I understood this. Do you think this is what you have seen with your son?

        Reply
        1. Lois Smith

          Susan, Yes I do agree wholeheartedly although it takes time to see these changes or, in particular, the recognition of these changes in our children. They appear in insignificant ways but then a growing realisation that they would not have been capable of that particular action or reasoning prior. My son found it difficult to break into, follow or understand the gist of conversations (correction, was not able to) but is now able to listen, wait and break into the conversation with relatable input. There are also other changes that I notice but his one in particular sticks in my mind. My son’s profile is defined as significant having a severe diagnosis in the areas of Motor Symbol Sequencing, Memory for Information/Instructions and Symbolic Thinking. This does not provide a good foundation for classroom instruction and comprehension. He is above average in symbol recognition though which helps him to hang on by his fingernails. He actually enjoys maths but has no real understanding of its application or the patterning involved. His spelling and reading are of an acceptable or above level. I can also see very good gains in the ‘inner voice’ that you described however I think that there is still much work to be done so the gain in one area is being outweighed by the cloud in the other areas. I would love to get in touch with you personally to hear about your journey.

          Reply
          1. Susan Caves

            Yes, Alison might prefer us not to hijack her blog. i’m on the Hunter Community Learning Association facebook page if you want to send a request : )

  10. Susan Caves

    Sorry Alison, I didn’t make myself clear when I suggested that you look at the NSW Parliamentary Inquiry. The reason for that suggestion was that it would give you an idea of how many people have the same story – not just Lois and I, but many many families. We are not being sold something that doesn’t work, if we were we would have moved on to try the next thing. We are all concerned that there are still so many children missing out, which is why I respond directly to criticism of the Arrowsmith Program.

    The only people I’ve ever heard or seen complaining about the Arrowsmith Program are actually people who have had no experience with it. In fact it is the ‘experts’ in education who take every opportunity to not only discredit the program but to make personal attacks on the integrity of Barbara Arrowsmith-Young. It’s time to move beyond dismissing parents’ stories as worthless. Perhaps it’s not the kind of evidence you want to see, but our childrens’ experience is the only evidence that counts for us.

    By the way, as I suggested, you should probably search on relational frame theory or multiple exemplar training to reach the kind of articles I’m talking about.

    Reply
    1. jonathan

      Hello Susan,

      I’m an adult who has been dealing with learning disabilities all my life. Do you have any experience with adults that are in the program? If so, can you share me your experiences?

      Thanks

      Reply
  11. Susan Caves

    Hi Jonathan
    I don’t have direct experience, but I know many adults do the Arrowsmith Program and that there are schools who offer the program to adults in Australia. I understand that they see the same kinds of cognitive gains in adults as they do in children and the adults I have spoken to that have done the program say the same thing.

    If I were you , I’d go to the Cognitive Profile at the Arrowsmithschool.org website (here is the direct link to the Questionnaire https://www.brainex.net/protoQuest/User.html). If you go through the questions it will give you a good idea of whether there are areas that the program could help with, and if you can go through the questions with someone you trust and who knows you well that might help too.

    There is lots of detailed information on the Arrowsmith website, and if you go to the Participating Schools page you can find out if there is one offering the adult courses near you. If you phone/email one of those you can get more information and some further guidance about suitability.

    Good luck with it Jonathan, and good on you for persisting with your search for answers.

    Reply
    1. alison Post author

      Can I please ask advocates of the Arrowsmith program not to use my website as a chatroom, and remind all readers that to the best of my knowledge there is no independent, peer-reviewed research evidence which indicates Arrowsmith is effective, which is extremely odd, given how many years it has been around. Struggling readers need and deserve programs backed by robust scientific evidence.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *