After the Sandy Hook massacre, the Obama White House developed a blueprint for protecting our children and our communities by reducing gun violence. As part of that blueprint, recommendations were created for the development of a new initiative, Project Aware, and the development of state initiatives to support individuals ages 16 to 25 at high risk for mental illness. The state initiatives are known today as the Healthy Transitions grants. Both Project Aware and the Healthy Transitions grants fall under the Now is the Time Technical Assistance Center (NITT-TA).
You can read the White House plan in its entirety here. I encourage you to do so. It will both inspire you and make you gut wrenchingly sick at how little progress has been made in dealing with gun violence since the Sandy Hook massacre, and the countless massacres that have taken place since then.
And now, as of March 1, the national evaluation for these two grant programs that were created in response to the horrors of Sandy Hook, have been shut down before critical information about effective strategies for meeting the needs of young adults at high risk for mental illness could be gleaned.
Why? Especially after the recent tragedy at Marjory Stoneman Douglas High School, why would SAMHSA want to end an evaluation focused on identifying strategies for meeting the needs of young adults 16 – 25 who are at high risk for mental illness?
It just makes no sense.
Unfortunately, what feels like an attack on evaluating evidence-based practices that emerge from the close collaboration of communities, providers, families and youth, is on the rise. It was just a mere two months ago that program developers seeking inclusion in the National Registry of Evidence-based Programs and Practices (NREPP), also hosted by SAMHSA, received an email from the contractor responsible for maintaining the NREPP website stating that the "NREPP contract is being terminated for the convenience of the government." Assistant Secretary McCance-Katz was clear in a press release explaining her rational for ending NREPP. In part, she said:
We at SAMHSA should not be encouraging providers to use NREPP to obtain EBPs, given the flawed nature of this system. From my limited review – I have not looked at every listed program or practice – I see EBPs that are entirely irrelevant to some disorders, “evidence” based on review of as few as a single publication that might be quite old and, too often, evidence review from someone’s dissertation.
But let’s get back to the Now is the Time National Evaluation. Why would SAMHSA cancel a national evaluation that was over 3/4 of the way through its study cycle? Apparently, the word given to grantees by federal project officers was that SAMHSA was revising its approach to program evaluation. Specifically, grantees were told that changes would include efforts to:
maximize resources and strengthen capacity to deliver effective evidence-based practices, and
focus on reviewing data collection analysis and translating results to improve program effectiveness in service delivery.
I wish I fully understood what all of that means, but I don't. What I do understand is that there will be inevitable chaos around a sudden decision like this, and for the life of me, I can't understand the management strategy at play here. I mean, just the logistics alone are unsettling. There are six different web surveys going on, 73 laptops being used all across the country to conduct supplemental interviews, which all need to be returned, and the big looming question for countless grantees, "What happens to the data that has been collected to this point?"
Actually, there is a more important and troubling question. With all that has been invested in this national evaluation over the past four years, and with one year to go, why would you stop an evaluation in its tracks like this? The time 1 baseline assessment has been completed, but the more important time 2 assessments still need to be done. Without those time 2 assessments there is no way to assess meaningful change. As well, the longitudinal study, which would really begin to shed some light on what works, is now a thing of the past.
In spite of this chaotic and abrupt halting of national evaluation activities, grantees are being told to soldier on and continue conducting local evaluations. The elephant in the room question is how will SAMHSA analyze the data collected to date, and what about the time 2 data? How will they capture that? And what about the commitment to a longitudinal study which could finally shed some light on effective approaches to meeting the needs of young adults?
Think of the impact this decision is having on youth involved with Healthy Transitions and Project Aware. Approximately 726 young adults have volunteered to participate in the national evaluation. Data collection from 125 youth is already complete. What happens to their input? Ironically, in a stomach-turning way, part of the interview questions with young adults deals with their thoughts and ideas for how to make schools safe.
What type of message is this sending to young people with mental health challenges who have stepped up and are saying, “We want to be a part of helping service programs provide services and supports that truly meet our needs.” After watching the courageous students from Marjory Stoneman Douglas High School speak out forcefully and articulately about the need for change in the midst of their own personal unfathomable trauma and grief, what message are government leaders sending to them and the youth of the nation by abruptly cancelling this evaluation? An evaluation that could provide much needed answers to the questions swirling around about how we can be more preventive in our work with youth and not wait until after a crisis happens.
I am truly speechless…
* * * * * * * * *
President & CEO
Children's Mental Health Network