Arrowsmith

The Arrowsmith program has been going for about 35 years, and had a research program for about 18 years.

In all that time, not one study has appeared in a peer-reviewed journal providing reliable scientific evidence that it works. Not one.

Perhaps the Australian Education Union didn’t realise this when giving Canada’s Barbara Arrowsmith-Young (AKA The Woman Who Changed Her Brain) a platform to promote her program here last week.

Radio National also interviewed her but failed to ask her hard questions, like whether her program has actually been shown to work.

I’ve already written about my concerns about the Arrowsmith program here and here, so won’t repeat myself, or rehash excellent blog posts on this topic by Head of La Trobe University’s Rural Health School Professor Pamela Snow and Oxford Professor of Developmental Neuropsychology Dorothy Bishop. Just read them here and here if you haven’t already.

Head of the School of Psychology at Charles Sturt University, Associate Professor Tim Hannan, wondered in Australasian Science magazine in 2015, “…why has the Catholic Education Office in Sydney, in company with a dozen other schools in Australia and New Zealand, decided to embrace a program with no reliable scientific evidence of its efficacy?” (Vol 36, 9, p41). We are all still scratching our heads.

We now have a new Canadian court decision about whether a government school discriminated against a student by not offering him the Arrowsmith program. It states:

“During this hearing, no scientific basis was demonstrated and among the five experts who testified, no one was able to show empirical evidence supporting the Arrowsmith method.”

Charisma, anecdotes, testimonials. Pay no attention. In fact, if someone offering a program for children with learning difficulties offers you anecdotes and testimonials, run a mile. They are usually a strong indication of a lack of proper evidence for the program.

What matters for children with learning difficulties is whether there is good evidence to support doing what the program does. Science is how we make sure we aren’t just believing what we want to believe.

The best scientific evidence you can get, at the top of the Evidence Pyramid, is a systematic review of all the well-conducted research in an area.

You can click here to read one such systematic review of the research into treatment approaches for children and adolescents with reading disabilities, which says, “The results revealed that phonics instruction is not only the most frequently investigated treatment approach, but also the only approach whose efficacy on reading and spelling performance in children and adolescents with reading disabilities is statistically confirmed.”

No, that doesn’t make phonics a blanket recommendation for all children who are struggling with reading. Some children have unusual profiles, and the evidence pyramid has several levels.

However, “my personal experience” and “I just know” are simply not part of the evidence pyramid, despite what satirists might say.

Related Posts Plugin for WordPress, Blogger...

6 thoughts on “Arrowsmith

  1. K Easter

    Obviously, there should be some independent peer-reviewed research on the program, although I believe that there is some research being undertaken at the moment. Does however, a lack of randomised controlled trials mean that the program doesn’t work?

    I do understand that there are many bogus programs out there that have swindled parents, so I commend those who recommend to parents to check if the remediation they are seeking for their child is likely to help. If programs don’t work, the word does get around and this effects participation rates, the bottom line and negative reviews abound.

    The Arrowsmith program is getting larger and growing every year and positive reviews from schools and parents are everywhere. Even one parent went as far as to try and sue a school board for not offering Arrowsmith to their son. Why is that there is so much vehement support for the program, in spite of experts warning that there is no independent evidence of its efficacy? Can it just be down to schools and parents being hoodwinked by slick marketing campaigns? There must be some reason why half the student population at the Arrowsmith School in Vancouver are Australian students and they stay for the full 3-4 year length of the program when they could drop out and come back after 6 months.

    Schools like Holy Innocents in Sydney underwent a 2 year trial and could have decided not to continue with the program. 2 years is long enough to see through bogus claims. Instead the Catholic Education Office is singing Arrowsmith’s praises.

    The biggest promoters of the Arrowsmith program are the schools that take it on, find that it works well and then recommend it to other schools. https://www.sydneycatholic.org/…/2015/2015112_1129.shtml
    Other bigger promoters are the parents who take it on, find that it helps their children and recommend it to other parents. https://www.facebook.com/PerthParentsAdvocacyforArrowsmith/

    I would suggest that instead of dismissing or even slandering the program from afar, couldn’t the movement in favour of Arrowsmith and the intense curiosity in the program possibly indicate potential value in looking into it more closely? That way, at least one’s criticism will be based on personal experience rather than second hand sources.

    Personally I’m sick of the all the controversy surrounding Arrowsmith and I would just like to know one way or the other. So would many other people in education. Perhaps the experts who point out that the program has not been tested rigorously should offer to carry out the research required instead of just sitting around whinging about the lack of evidence. Just a thought.

    Reply
    1. alisonalison Post author

      Hi Karen, I am confident that what I wrote about Arrowsmith is factual, and not even remotely slanderous. One concern many experts express about Arrowsmith which I didn’t mention in this blog is that parents signing up for the program are required to sign some kind of document which (I understand) restricts their ability to criticise the program if they are unhappy with it. So lots of people who have done it are critical of it in private but are unable to speak about this publicly, while we do hear from people who are happy with it.

      History is littered with things that lots of people believed which were later proven to be incorrect or harmful, but which were well-marketed by a charismatic personality, a compelling narrative etc. As the Arrowsmith program is not in the public domain, it is necessary to have the cooperation of the program owner in order to research it, and I am not sure that Barbara Arrowsmith-Young is interested in having independent researchers study it. Perhaps you know whether this is the case? And if it’s not, perhaps you could encourage this? I imagine, given the uptake of the program across the country you mention, and the associated expenditure of public funds, that it would be possible to assemble a team of researchers to undertake this.

      Reply
  2. K Easter

    Hi Alison,
    Do you have definite information that parents have to sign away their right to write an honest review or could it relate only to non-disclosure of the commercial elements of the program (so that parents, former teachers and other companies are prevented from duplicating the program or opening their own schools)?
    Yes, the program would require the cooperation of the program owner. There seems to be a lot of speculation about is she or isn’t she. There is only one way to find out if Arrowsmith Young is interested in having independent researchers investigate the efficacy of the program and that’s to contact her and put a (good quality) proposal to her.
    I’m always interested in empirical research, so if I happen to meet Ms Arrowsmith, then I would encourage her to accept any offers to have the program investigated. I’m also happy to hear that it would be possible to get some researchers interested.

    Reply
    1. alisonalison Post author

      I don’t have precise information about the content of the agreement parents must sign when they enrol their child in an Arrowsmith program so perhaps in a strictly legal sense it does not prevent them from reviewing the program, but such agreements can often have the effect of making people fearful of making public comment, I can’t say for sure. I hope that people able to conduct research do contact Ms Arrowsmith-Young and put research proposals to her, I myself am a clinician without the resources or time to do this. Great to hear that you’d encourage this. All the best, Alison

      Reply
  3. Karina McLachlain

    Hi Alison,
    I don’t have any direct link with Arrowsmith. However, a former student of mine at a special needs school in London was enrolled by his mother in the Arrowsmith School in Toronto at about the same time that I moved back to Australia. I keep in touch with his parents from time to time because he was a lovely boy and I am interested in his progress.
    I have seen the allegation several times that the lack of negative Arrowsmith reviews by parents and former students is due to a ‘confidentiality agreement’ that parents are forced to sign that prevents them from saying anything negative. I decided to ask my former student’s mother if there was any truth to the rumour, as I know that she is a very frank person and not one who would easily be silenced. If parents are forced to sign such an agreement, obviously that would be a scandal and a huge blight on the school’s reputation.

    Mary Goldblatt gave me her permission to cut and paste her reply into this blog:

    Regarding the comments on a confidentiality clause, it is completely untrue, an
    utter fabrication and predictably offensive to parents.
    It does though highlight an issue those parents of children with learning
    differences particularly too often face, in that we are considered so desperate,
    that we would sign up for rat poison if someone told us it would work! That old
    adage though that ‘need is the mother of invention’ is a better representation
    of the position, certainly ours anyway, rather than ingest failure or limits we
    would prefer to create opportunity with the potential for growth with no rat
    poison included!

    Her son has made good general progress, although he has worked hard for it. This one boy does not constitute a sample size, but at least I know that the programme works for some. I too would love to see some quantitative research so that we all have a better idea of what kinds of difficulties can be helped and which aspects of the programme are the least/most effective (and for whom/which disability).

    Unfortunately, I don’t think that the lack of research is all Arrowsmith’s fault. It is my impression that Linda Segal has tried to discourage researchers from taking an interest in this programme. She carried out a shonky piece of research more than 10 years ago that apparently showed negative results for Arrowsmith when compared to another group given a standard special ed support programme. Without releasing her report for scrutiny, she quoted that her results showed that the Arrowsmith Programme was ‘ineffective’, and at the same time criticised Arrowsmith of not having any publishable data.
    Once her report finally became public, after more than a decade (Segal had given a copy to a parent who forwarded it on to Howard Eaton), it was evident that her research had been carried out with the agenda to shut down the Arrowsmith programme from the school board that was trialling it, regardless of whether it was effective or not.
    The research design had students just beginning the Arrowsmith programme compared to students who had been doing the standard programme for at least a year already. There were a substantial number of dropouts from the standard programme (none from the Arrowmsith), but this skewing of data was not taken into account when analysing the results. Segal’s stats were re-analysed by expert statisticians who found that the results actually favoured Arrowsmith. This is despite the fact that the programme was cut after only 8 months – substantially shorter than its actual length.

    During the decade that she kept the report under wraps, Segal used many platforms to broadcast the apparent poor results of the Arrowsmith Programme and influence a lot of experts in the area of learning difficulties to join in the criticism, instead of developing an interest into looking into the programme and judging its merits without pre-conceived bias. If Segal hadn’t of scuttled the trial that she was researching, there would already have been at least one fully completed review of the programme with a control group, possibly more. As the Eaton Arrowsmith school agreed to participate in the trial/study that Segal scuttled, doesn’t that show that Arrowsmith are willing for their programme to be researched? So, where are the researchers?It rather seems that Segal seems has gone out of her way to discourage the initiation or completion of empirical research in to Arrowsmith. You can’t do this on the one hand and criticise Arrowsmith for not having the research.

    Strangely, none of the experts who joined in with Segal to so vehemently to criticise Arrowsmith, said one critical word when Segal’s subterfuge came to light. In fact, she was given a distinguished researcher’s award by Learning Difficulties Australia, wasn’t she? Learning Difficulties Australia criticise lack of empirical research, but reward shonky research that even a first year Psychology student could tell was biased and poorly designed????????????

    Thanks Mr Easter, I agree that instead of criticising what hasn’t been done, researchers could instead get out there and do it.

    Reply
  4. Pingback: Arrowsmith | The Echo Chamber

Leave a Reply

Your email address will not be published. Required fields are marked *