Arrowsmith

The Arrowsmith program has been going for about 35 years, and had a research program for about 18 years.

In all that time, not one study has appeared in a peer-reviewed journal providing reliable scientific evidence that it works. Not one.

Perhaps the Australian Education Union didn’t realise this when giving Canada’s Barbara Arrowsmith-Young (AKA The Woman Who Changed Her Brain) a platform to promote her program here last week.

Radio National also interviewed her but failed to ask her hard questions, like whether her program has actually been shown to work.

I’ve already written about my concerns about the Arrowsmith program here and here, so won’t repeat myself, or rehash excellent blog posts on this topic by Head of La Trobe University’s Rural Health School Professor Pamela Snow and Oxford Professor of Developmental Neuropsychology Dorothy Bishop. Just read them here and here if you haven’t already.

Head of the School of Psychology at Charles Sturt University, Associate Professor Tim Hannan, wondered in Australasian Science magazine in 2015, “…why has the Catholic Education Office in Sydney, in company with a dozen other schools in Australia and New Zealand, decided to embrace a program with no reliable scientific evidence of its efficacy?” (Vol 36, 9, p41). We are all still scratching our heads.

We now have a new Canadian court decision about whether a government school discriminated against a student by not offering him the Arrowsmith program. It states:

“During this hearing, no scientific basis was demonstrated and among the five experts who testified, no one was able to show empirical evidence supporting the Arrowsmith method.”

Charisma, anecdotes, testimonials. Pay no attention. In fact, if someone offering a program for children with learning difficulties offers you anecdotes and testimonials, run a mile. They are usually a strong indication of a lack of proper evidence for the program.

What matters for children with learning difficulties is whether there is good evidence to support doing what the program does. Science is how we make sure we aren’t just believing what we want to believe.

The best scientific evidence you can get, at the top of the Evidence Pyramid, is a systematic review of all the well-conducted research in an area.

You can click here to read one such systematic review of the research into treatment approaches for children and adolescents with reading disabilities, which says, “The results revealed that phonics instruction is not only the most frequently investigated treatment approach, but also the only approach whose efficacy on reading and spelling performance in children and adolescents with reading disabilities is statistically confirmed.”

No, that doesn’t make phonics a blanket recommendation for all children who are struggling with reading. Some children have unusual profiles, and the evidence pyramid has several levels.

However, “my personal experience” and “I just know” are simply not part of the evidence pyramid, despite what satirists might say.

Related Posts Plugin for WordPress, Blogger...

14 thoughts on “Arrowsmith

  1. K Easter

    Obviously, there should be some independent peer-reviewed research on the program, although I believe that there is some research being undertaken at the moment. Does however, a lack of randomised controlled trials mean that the program doesn’t work?

    I do understand that there are many bogus programs out there that have swindled parents, so I commend those who recommend to parents to check if the remediation they are seeking for their child is likely to help. If programs don’t work, the word does get around and this effects participation rates, the bottom line and negative reviews abound.

    The Arrowsmith program is getting larger and growing every year and positive reviews from schools and parents are everywhere. Even one parent went as far as to try and sue a school board for not offering Arrowsmith to their son. Why is that there is so much vehement support for the program, in spite of experts warning that there is no independent evidence of its efficacy? Can it just be down to schools and parents being hoodwinked by slick marketing campaigns? There must be some reason why half the student population at the Arrowsmith School in Vancouver are Australian students and they stay for the full 3-4 year length of the program when they could drop out and come back after 6 months.

    Schools like Holy Innocents in Sydney underwent a 2 year trial and could have decided not to continue with the program. 2 years is long enough to see through bogus claims. Instead the Catholic Education Office is singing Arrowsmith’s praises.

    The biggest promoters of the Arrowsmith program are the schools that take it on, find that it works well and then recommend it to other schools. https://www.sydneycatholic.org/…/2015/2015112_1129.shtml
    Other bigger promoters are the parents who take it on, find that it helps their children and recommend it to other parents. https://www.facebook.com/PerthParentsAdvocacyforArrowsmith/

    I would suggest that instead of dismissing or even slandering the program from afar, couldn’t the movement in favour of Arrowsmith and the intense curiosity in the program possibly indicate potential value in looking into it more closely? That way, at least one’s criticism will be based on personal experience rather than second hand sources.

    Personally I’m sick of the all the controversy surrounding Arrowsmith and I would just like to know one way or the other. So would many other people in education. Perhaps the experts who point out that the program has not been tested rigorously should offer to carry out the research required instead of just sitting around whinging about the lack of evidence. Just a thought.

    Reply
    1. alison Post author

      Hi Karen, I am confident that what I wrote about Arrowsmith is factual, and not even remotely slanderous. One concern many experts express about Arrowsmith which I didn’t mention in this blog is that parents signing up for the program are required to sign some kind of document which (I understand) restricts their ability to criticise the program if they are unhappy with it. So lots of people who have done it are critical of it in private but are unable to speak about this publicly, while we do hear from people who are happy with it.

      History is littered with things that lots of people believed which were later proven to be incorrect or harmful, but which were well-marketed by a charismatic personality, a compelling narrative etc. As the Arrowsmith program is not in the public domain, it is necessary to have the cooperation of the program owner in order to research it, and I am not sure that Barbara Arrowsmith-Young is interested in having independent researchers study it. Perhaps you know whether this is the case? And if it’s not, perhaps you could encourage this? I imagine, given the uptake of the program across the country you mention, and the associated expenditure of public funds, that it would be possible to assemble a team of researchers to undertake this.

      Reply
  2. K Easter

    Hi Alison,
    Do you have definite information that parents have to sign away their right to write an honest review or could it relate only to non-disclosure of the commercial elements of the program (so that parents, former teachers and other companies are prevented from duplicating the program or opening their own schools)?
    Yes, the program would require the cooperation of the program owner. There seems to be a lot of speculation about is she or isn’t she. There is only one way to find out if Arrowsmith Young is interested in having independent researchers investigate the efficacy of the program and that’s to contact her and put a (good quality) proposal to her.
    I’m always interested in empirical research, so if I happen to meet Ms Arrowsmith, then I would encourage her to accept any offers to have the program investigated. I’m also happy to hear that it would be possible to get some researchers interested.

    Reply
    1. alison Post author

      I don’t have precise information about the content of the agreement parents must sign when they enrol their child in an Arrowsmith program so perhaps in a strictly legal sense it does not prevent them from reviewing the program, but such agreements can often have the effect of making people fearful of making public comment, I can’t say for sure. I hope that people able to conduct research do contact Ms Arrowsmith-Young and put research proposals to her, I myself am a clinician without the resources or time to do this. Great to hear that you’d encourage this. All the best, Alison

      Reply
  3. Karina McLachlain

    Hi Alison,
    I don’t have any direct link with Arrowsmith. However, a former student of mine at a special needs school in London was enrolled by his mother in the Arrowsmith School in Toronto at about the same time that I moved back to Australia. I keep in touch with his parents from time to time because he was a lovely boy and I am interested in his progress.
    I have seen the allegation several times that the lack of negative Arrowsmith reviews by parents and former students is due to a ‘confidentiality agreement’ that parents are forced to sign that prevents them from saying anything negative. I decided to ask my former student’s mother if there was any truth to the rumour, as I know that she is a very frank person and not one who would easily be silenced. If parents are forced to sign such an agreement, obviously that would be a scandal and a huge blight on the school’s reputation.

    Mary Goldblatt gave me her permission to cut and paste her reply into this blog:

    Regarding the comments on a confidentiality clause, it is completely untrue, an
    utter fabrication and predictably offensive to parents.
    It does though highlight an issue those parents of children with learning
    differences particularly too often face, in that we are considered so desperate,
    that we would sign up for rat poison if someone told us it would work! That old
    adage though that ‘need is the mother of invention’ is a better representation
    of the position, certainly ours anyway, rather than ingest failure or limits we
    would prefer to create opportunity with the potential for growth with no rat
    poison included!

    Her son has made good general progress, although he has worked hard for it. This one boy does not constitute a sample size, but at least I know that the programme works for some. I too would love to see some quantitative research so that we all have a better idea of what kinds of difficulties can be helped and which aspects of the programme are the least/most effective (and for whom/which disability).

    Unfortunately, I don’t think that the lack of research is all Arrowsmith’s fault. It is my impression that Linda Segal has tried to discourage researchers from taking an interest in this programme. She carried out a shonky piece of research more than 10 years ago that apparently showed negative results for Arrowsmith when compared to another group given a standard special ed support programme. Without releasing her report for scrutiny, she quoted that her results showed that the Arrowsmith Programme was ‘ineffective’, and at the same time criticised Arrowsmith of not having any publishable data.
    Once her report finally became public, after more than a decade (Segal had given a copy to a parent who forwarded it on to Howard Eaton), it was evident that her research had been carried out with the agenda to shut down the Arrowsmith programme from the school board that was trialling it, regardless of whether it was effective or not.
    The research design had students just beginning the Arrowsmith programme compared to students who had been doing the standard programme for at least a year already. There were a substantial number of dropouts from the standard programme (none from the Arrowmsith), but this skewing of data was not taken into account when analysing the results. Segal’s stats were re-analysed by expert statisticians who found that the results actually favoured Arrowsmith. This is despite the fact that the programme was cut after only 8 months – substantially shorter than its actual length.

    During the decade that she kept the report under wraps, Segal used many platforms to broadcast the apparent poor results of the Arrowsmith Programme and influence a lot of experts in the area of learning difficulties to join in the criticism, instead of developing an interest into looking into the programme and judging its merits without pre-conceived bias. If Segal hadn’t of scuttled the trial that she was researching, there would already have been at least one fully completed review of the programme with a control group, possibly more. As the Eaton Arrowsmith school agreed to participate in the trial/study that Segal scuttled, doesn’t that show that Arrowsmith are willing for their programme to be researched? So, where are the researchers?It rather seems that Segal seems has gone out of her way to discourage the initiation or completion of empirical research in to Arrowsmith. You can’t do this on the one hand and criticise Arrowsmith for not having the research.

    Strangely, none of the experts who joined in with Segal to so vehemently to criticise Arrowsmith, said one critical word when Segal’s subterfuge came to light. In fact, she was given a distinguished researcher’s award by Learning Difficulties Australia, wasn’t she? Learning Difficulties Australia criticise lack of empirical research, but reward shonky research that even a first year Psychology student could tell was biased and poorly designed????????????

    Thanks Mr Easter, I agree that instead of criticising what hasn’t been done, researchers could instead get out there and do it.

    Reply
  4. Pingback: Arrowsmith | The Echo Chamber

  5. Clare

    I am staggered in this world of so much research that universities who fund programs and research then don’t disseminate the information though courses offered to their students! Take spelling for example, we know what works and yet many schools still run the same spelling lists and look, cover, write, check stuff that doesn’t work. It is very worring to have programs in place that just don’t work.

    Reply
  6. Jo-Anne Gross

    Dr. Linda Seigel is one of Canada`s most prominent researchers-she worked frequently with the highly reknown Dr. Keith Stanovich.

    in no way would she ever be involved with quackery or fudged research,that`s a preposterous scenario.

    The reason Arrowsmith gets bought is because the schools give our dyslexic children or L.D. children nothing but empty IEPS to simply justify a diagnosis and DO NOTHING for families and children..it`s educational malpractice-my God-Dyslexia with a phonological base-the norm-can be dramatically improved in 6 months.

    While I know Arrowsmith does not honour the cutting edge research on explicit systematic synthetic phonics or Orton Gillingham it`s SOMETHING for kids who get platitudes.

    Reply
  7. Lara

    For the sake of all children with Dyslexia please please please use Sounds~Write. It is an evidence based linguistic phonic programme designed to be delivered sequentially. It works, I know as I use it daily with students with a moderate to severe diagnosis. These kids don’t need marketing ploys and lesson fillers they need understanding passionate teachers that understand and know phonics themselves.

    Many in the industry should be ashamed of themselves. The evidence is available, speak to the experts, DSF Dyslexia SPELD will soon tell you what works best. They know, it is their passion and are totally committed to students with a learning difficulty. Many mental health problems seen today could very easily be attributed to reading difficulties and constant feelings of failure. Again the evidence is also available regarding such studies.

    Reply
      1. Lois Smith

        My son has recntly attended a 6 week cognitive intensive program with the Arrowsmith Program in Canada. We were never asked to sign an agreement nor asked not to speak about the Arrowsmith program in an unfavourable way. We were also part of an independent research program. I also, am deeply offended, that the author perhaps thinks the parents have somehow been duped or hypnotized into attending these programs. Unfortunately parents are left with little alternative but to try different learning opportunities for their children as the education system is failing them in droves. I have the impression that the author has little true understanding of the needs of the children who attend the Arrowsmith Program. Perhaps instead of only looking to researchers to draw all of your opinions, try and speak to the zombie like parents who follow blindly and stupidly into the crackpot world of Arrowsmith. The 6 weeks that we spent at the Arrowsmith program was the most rewarding 6 weeks of my child’s education and he is now 13 and I am really no further ahead with finding help for him back here in the ‘normal’ education system. He is not dyslexic, he does not have dysgraphia, he is not diagnosed with ADD but he does have a learning deficit and believe me, he works twice as hard at his class work than most of the kids in his class and he receives 1/2 of the reward. Probably not even that. What should I tell my son. We will just wait here and see if they bring out some research before I go off and try something new. In the meantime lets follow the droconian footsteps of the education system that basically hasn’t changed in hundreds of years and still believes ‘rote’ learning is the answer to all. Oh and by the way I’ve tried and spent a fortune on speech therapists, the maths tutors, child psychologists trying to get a ‘diagnosis’ for my son so I can access learning support. Why are you so against trying something different? Do you really believe that all children are round pegs and will fit neatly into your round hole if they just follow the research trail. Speak to the people at the coalface of the problem. The parents, the mad, deflated, frustrated parents. At the end of the day I will do what I think is right for my child. Research of no research. I’m not that afraid

        Reply
        1. alison Post author

          Dear Lois, I’m so sorry to hear you’ve been so let down by the current system that you feel you need to resort to interventions that are expensive and not backed up by peer-reviewed research. As a full-time practising clinician I AM one of the people at the coalface, working with parents and teachers as well as children and some adults with difficulties, for example I have for the last two terms been working at a local school with a lot of kids from public housing, many from refugee and other disadvantaged families, who could barely read or write just six months ago. Their letters were back to front and they left sounds out of words and struggled to decode the simplest books, but on Monday our group of six kids aged 7 and 8 each read a whole chapter book from Level 1 of the Sound Out Chapter books, and completed four pages of worksheets about them targeting spelling, vocabulary and comprehension, in a one hour session. One kid (who used to act up and climb under the table) voluntarily stayed back into her lunch break to finish the last sheet, as she was determined to get the whole lot done. These kids are very proud of themselves, one of them said unprompted during the session “I love reading, I used to hate it but now I love it”. This is of course an anecdote and should not be treated as evidence, but am very proud of using methodologies that ARE based on the peer-reviewed research, and our test scores and in-class observations regarding progress are consistent with the research. I hope your son achieves similar success soon, and assure you that his best chances lie with methodologies that are based on current, scientific models of reading and spelling.

          Reply
          1. Lois Smith

            With respect to your opinion, it is just that, your opinion. I believe brain plasticity is a current scientific model based on sound methodology and, as I said, I have just experienced, for the first time, success with my son and that was through the Arrowsmith Program. I don’t look at his academic record to mark his progress, his deficits affect him in far more ways than a maths or comprehension test is going to show and therein lay the frustration with the Education system. I am looking to improve his mental health and keep his self esteem intact by going back to the grass roots (his brain) and trying to give him a better chance at life, not just schooling. It took him about 3 years to learn to ride a bike because he doens’t understand the ‘push’, ‘pull’ dynamic or how to blow his nose correctly. These deficits go way beyond reading and spelling. He is great reader and speller, in fact he is in the 90th percentile for both but he doesn’t understand what he is reading, beyond the basic, or he doesn’t see patterns and what he learns today, he’s lost tomorrow. These children experience a very wide range of issues and I did not understand how much it affected their lives until I spent 6 weeks speaking to parents of similar children at the Arrowsmith Program. I was not learning this from the ‘brainwashes’ at the Arrowsmith program, I was learning this from the real people at the coalface. The people who live with these children and see them going backwards year after year despite all attempts at peer reviewed, current, acceptable research. I am not doubting for one minute that your program works, but I also don’t understand why you so strongly don’t accept that this program could work simply because its lack of research. I understand its been very difficult to get the program independently researched and also funding to research it and it is obviously a long term research program. Perhaps the critics should enrol for a 6 week clocks program and speak to the people accessing this course and ask them why they are accessing it and believe that it is making a difference to their children. How about you do a 6 week Cognitive Intensive program and I will do the Sound Out Program.

          2. alison Post author

            Hi Lois, sorry to hear your son has such extensive difficulties. There is a lot of neurobabble around at the moment, and a lot of programs that require parents to “believe” in something without much evidence, but saying “brains are plastic” means the same thing as saying “children can learn”, and talking about “brain based learning” is as silly as talking about “lung-based breathing”, “leg-based walking” or “stomach-based digestion”.

            When I spend money and time on professional development, I make sure first that it’s credible and based on proper, peer-reviewed research, or consistent with the models derived therefrom. I don’t spend time or money on a program that has been around for three decades, has been doing research for almost two decades, but still has not a single article published in a peer-reviewed journal providing evidence of its effectiveness. There are too many known-to-be-effective programs available, and that’s where I am focussing my attention. All the best in helping your son with his learning, but don’t use the Sound Out Chapter Books with him if his decoding and spelling are fine, concentrate on his language comprehension, it sounds like he may have a developmental language disorder. Lots of good information about helping kids with oral language difficulties, including comprehension difficulties, is here: https://www.youtube.com/user/RALLIcampaign. All the very best, Alison

Leave a Reply

Your email address will not be published. Required fields are marked *