Note: For those of you reading this post for the first time as a part of Friday Update, 7-10-15, know that the original post was written 5-15-15. We are featuring the original post again, as we have received a number of requests for some of the more popular background posts written about HR 3717 that are still relevant to current discussions about H.R. 2646.
Morning Zen Guest Blog Post ~ Dennis Embry
Three times, I’ve paid my own way to be part of the listening group of diverse people on the issue of assisted outcome patient treatment, or AOT as it is widely known. As a scientist with multiple practices and programs on the National Registry of Evidence-Based Programs and Practices (NREPP), I advised the Treatment Advocacy Center on how best to present their application on AOT to NREPP, as part of my collegial and moral responsibility to improve outcomes and wellbeing of persons with a serious mental illness. As a clinician who has worked with several psychiatric facilities, juvenile courts, and drug courts, I have seen a fair number of court-ordered treatments. I also have the experience as a young adult having to petition the courts for involuntary commitment of my parents. I’ve witnessed the good, the in-between and bad of court-ordered treatments. Recently, I saw one of the best AOT implementation that is an excellent yardstick and baseline for how to do AOT right, which is important because the current proposed legislation, funding and thinking about AOT by its advocates and policy makers is likely to result in a Congressional victory but community failure—all from good intentions. As my Quaker saying goes, “Beware of Good Intentions.”
Treatment Advocacy Center (the people who hold the NREPP title for Assisted Outpatient Treatment or AOT) recommended the AOT program of Butler County as the best in the country. I was delighted to visit Hamilton, Ohio, which is the county seat of Butler County. I believe they are right, based on my observations; it was easy to see why during the visit that this Ohio site might be the beacon for how AOT should be replicated throughout America, recalling the saying: “So goes Ohio, goes the country.”
What tickles me is that the entirety of Ohio is also leading the way nationally in the implementation of arguably the best early universal prevention strategy protecting against the developmental trajectory of serious mental, emotional, and behavioral disorders cited in the 2009 Institute of Medicine Report : The PAX Good Behavior Game (http://bit.ly/NREPP). What was most interesting to me was how much was silently shared in the procedures of the Butler County AOT program and the prevention program in about 300 schools in Ohio, including just prevention sites just starting in Butler County. More about the similarities between AOT and the prevention strategies, later.
Consider the question of what makes AOT work? Is the power of coercion? Is the treatment itself? Something else or in-between?
Unless you’ve worked with very acutely mentally ill people, it’s hard to understand what is happening to them from their perspective. It is much like the famous Shakespearean quote: “In truth, there is madness; and in madness, there is truth.”
For many in acute, serious mental illnesses, there is an utter sense of powerlessness or omnipotent sense of power to mask the deep sense of powerlessness or flip-flopping versions of both, coupled with overwhelming perceived threats from God knows what for the afflicted persons. This is not a good combination of mental states for human wellbeing. An example from my experience with an acute ward illustrates.
A colleague, a wonderful charge nurse, arrived for her duty for the locked psychiatric ward. Upon opening the door, she heard terrible screaming from one of the rooms, and rushed to that room. Upon entering, the patient was repeatedly screaming at the top of his lungs, “You’re hurting me…get off of me.” Five orderlies and security people were holding him down in every way possible that would evoke absolute fear in anyone. The nurse took immediate charge with a firm order for the orderly and security people: “Get off him now.” She immediately knelt and gently touched the terrified man, speaking to him as softly as a mother to a child. “(Name), if had five men trying to hold me down, I’d be terrified too. They are not holding you now. I can’t have you yelling, as it scares the other patients. We can talk about this in the room, or we can walk and talk in the hallway. Which would be best for you?” The man began to breathe more regularly, with less evident panic. Then he said, “Walk and talk.”
So how was this crises calmed?
First, the nurse made a thoughtful appraisal and recognition of what the patient was likely feeling in a caring way (a well documented therapeutic skill); she expressed empathy concretely to the event (another proven therapeutic skill); and she dramatically reduced the possibilities of perceived force and coercion that evoke terror, fear, and counter aggression (yet another well-documented treatment skill).
Second, she offered the patient equivalent choices to give back locus of control to the patient. Third, she reinforced the patient’s actions that evidenced increasing steps toward self-regulation. Most of these small strategies are what my colleagues and I call, “evidence-based kernels” , which are the smallest scientifically proven unit of behavioral influence.
Third, she “de-escalated” the trajectory by herself actually de-escalating at the stimuli and not use anything that likely be perceived as coercion (and yes, this is also well proven but rarely used because of fear of “losing control” by the therapeutic staff who use more control because of their own fear or psychological inflexibility that the patient has an authentic fear or terror from their perspective).
The Evidence-Based Kernels she used are not programs, though potent evidence-based programs inevitably contain evidence-based kernels as active ingredients . Such kernels tend to produce rapid, measurable effects and are easy to learn and use. Often, kernels can be found in successful cultural practices, which have been selected by their consequences as being effective without clear thought of what we would call experimental designs . For example, any reasonably competent teacher, parent or grandparent around the world uses what the nurse did, and that “kernel” is called offering equivalent choice. When the child has to go outside when it is cold, a wise adult will ask but not tell the child something like: “Do you want to wear your red sweater or your green hoodie.” How did the adult “learn” to use equivalent choice? By learning from the consequence: offering choice dramatically improves compliance by the children; whereas telling the child exactly what to put on causes a tantrum. This same principle was hugely evident in every case we saw in Hamilton, by the wonderful Magistrate—supported both by the “prosecutor” and the “defense” counsel.
What makes the Butler County AOT effort work?
Throughout each case that day, the court team worked hard at scaffolding the patients’ sense locus of control in gentle ways. There were no threats, though the excellent staff occasionally voiced their “concerns”, about possible choices or situations. There were Socratic “bracketing” questions designed to elicit the patients’ motivations and understandings to get better. Again, this a classic use of good teaching and therapeutic skills. This was most evident in one particular case, involving a patient who had a history of swallowing sharp objects with some serious medical consequences. The Magistrate asked what might happen if the patient did X or Y, gently, but seriously guiding the patient’s to think through rather than act impulsively. Again, this was in service of creating self-regulation and practiced locus of control by the patient and not threats or coercion.
Another evidence-based kernel we witnessed is something called, “a positive debrief.” When children or adults make a mistake, people in authority often engage in a confrontational drilling of “why did you do that ___(awful thing)?” This confrontational approach is often counter-productive, yielding things like “I don’t know” or reactive anger because of perceived threats. A positive debrief probes, “How did you do that _______ (positive) thing?” The ensuing interaction helps the patient develop a narration of successful problem solving and mindfulness that can be self-reinforcing and recruit social reinforcement from others.
Wearing my scientist-clinician hat, I was heartened by what I saw. Many of the principles and actions were straight out of the literature on drug courts. The judge and staff freely acknowledge that later at dinner, and praised their drug court training, which they applied to the AOT process. Just like the Drug Court experience, one has to select members of the team carefully, and “hanging” judges and court staff probably not apply.
This was the best AOT ostensibly in the land, and I can imagine terrible ones—just as I have seen bad drug courts. Clearly this site does and will likely help the seriously mentally ill people we saw get better.
Almost all of the patients on that day also presented with drug and alcohol problems co-morbid with their mental illness, which complicates the success. It is alcohol and drug problems that predict a much greater risk of violence to self and others, not the major mental illness alone. The Butler County program seemed lite on alcohol and drug treatment strategies documented to increase sustained sobriety among folks with major or severe mental illnesses. What could they use for drug treatment for seriously mentally ill persons who often what some psychiatrists call “anosognosia” (a fancier new name, which used to be called a “lack of insight” in psychodynamic approaches)? Substance abuse specialists traditionally call the same thing, “denial.” Traditional 12-step approaches are not terribly effective among folks with serious mental illness, but a brilliant, simple random contingency management protocol called the “fish bowl” or “prize bowl” for being clean and engaging in recovery related behaviors [5-11] is probably the single most proven drug treatment strategy so far ever studied by the National Institute on Drug Abuse. It is also the most cost effective drug treatment protocol, which can be implemented by non-treatment staff—especially important for underserved or resourced communities.
I noticed that many clients were not appearing with family AND friends, quite unlike Drug Courts. This seems unfortunate, as the National Institutes of Health have funded research on spectacularly effective strategies that reduce relapse or reoccurrence in both addictions and co-occurring mental health problems called Community Reinforcement and Family Training (CRAFT) [12-16], which also has application for special populations. Useful information can be found at, http://archives.drugabuse.gov/TXManuals/CRA/CRA1.html
Some patients also had obvious learning impairments, as well as nutritional and other self-care problems that impact recovery and wellness. Still, there was profound hope in the room. The team clearly understood that the assisted part of outpatient treatment was not about command, control, and coercion. The team all worked to evoke and reinforce self-regulatory skills in the cases we observed, which was moving and assuring. Careful training, not unlike the best Drug Court training, will be required to make this work well across America.
So how much does it cost to run high quality AOT?
As good as Butler County was, there is room to make even this star child of AOT better with other evidence-based kernels that should be standard issue. This is wise to consider, because the New York State implementation of AOT involved a bit more than $160 million in additional treatment funds to achieve their results, because of historic cuts in the state. There is no free lunch here with AOT, as the “T” is the most important part of AOT.
I did mental math during the luncheon with all the community players for the Butler County AOT. Everybody important in Butler Counter was there, and they had put substantial local resources in place for this amazing program. They had a well run inpatient facility at the community hospital, with a locked option. They had incredible step-down programs and housing options. They invested serious funds for the local staffing and funding for the court itself, which had to be at least $400-$500K per year, and I bet it is more because Ohio has high levels of expenditures for behavioral health in communities compared to most. They had housing, psychiatrists, therapists, counselors and a host of services. They were not serving hundreds of patients per month; they were serving six or so cases we saw just that day. In a year, it must be something like 30-50 at best. There must be many more people who could benefit in Butler County than 30-50 people, but that’s probably about all the resources they have.
Put all this into context. Butler County Ohio has population 371,000, about 0.1% of the population of the United States. To do what Butler County Ohio is dong so well would require non-trivial resources, especially since Butler County’s efforts is only covering a percentage of folks who could benefit by AOT. The best financial on the cost of doing a population-level upgrade of systems to support AOT comes from the State of New York, which spent about $8 per man, woman and child in the state. New York has a population of about 19,700,000 people. Using NY data, that would suggest the real cost of covering ALL of the people who might need AOT in Butler County Ohio is about $3 million per year. To do the same every America state, DC, and community would require a total commitment of from the Federal government of conservatively $2.6 billion of NEW money per year. Nowhere are the proponents signing on to provide nearly $3 billion (and likely more) in new mental health treatment money that will be required to rebuild the mental health and substance abuse treatment infrastructure.
The proponents of AOT promote it as THE way to end waiting for beds for serious mentally ill. That is a noble message, but are they really prepared to spend nearly $3 billion new money to reduce the burden of serious mental illness? I would wager a $100 bill that the answer is no; the new money proposed is more like $50 million. It will probably take way more than $3 billion in new money, because most states have gutted their addictions and treatment programs in preference for depending on corrections. Cutting substance abuse treatment and prevention in the name of treating serious mental illness is a logical as cutting off patients’ legs to save their arms.
What else is needed to make AOT work well?
I will skip the notion of “programs” and focus on small units of behavioral influence (evidence-based kernel). These kernels have huge advantages in dissemination because of low cost, ease of training, and rapid results [2, 4, 17-19], as well as expanding the sense of locus of control in the patients’ minds that appears to be critical in long-term positive outcomes in longitudinal studies funded by NIMH . Contrary to popular belief, this AOT star site spent virtually no time at all on psychotropic medication compliance in the court sessions we observed. Rather, the emphasis was all on drawing out and reinforcing the patients’ self-efficacy—with a few fence posts gently noted.
Now, what to add to improve patients’ locus of control and self-regulation for sustained recovery or quality of life in the context of AOT or any other therapeutic “system” to help folks with serious mental illness. The literature on Drug Courts and evidence-based kernels (i.e., the smallest scientifically proven units of behavioral influence) offer rich but not costly opportunities to improve the outcomes and processes we observed. My specific thoughts fit into what my colleagues call and I call, Nurturing Environments  or the “Nurture Effect”: a) Rich reinforcement of prosocial behavior, b) limit exposure to problematic behaviors, c) reduce toxic influences, and d) increase psychological flexibility. Some suggestions follow around each arm of the Nurturing Environments meta-theory to facilitate both locus of control and self-regulation.
Rich reinforcement of pro-social behavior
A cardinal feature of serious mental illness is withdrawal from prosocial interactions with others, or increased anti-social behavior, or yo-yoing between the two. Coercion by other humans has well documented limits in increasing prosocial behavior among humans—even among mentally ill persons [21-28], in part because humans have been the primary predators of humans since the invention of stone weapons. So how do well-run Drug Courts that compel addiction treatment seem to work? Not by throwing addicts into jail till they get clean or submit to “authority”. Rather, well run Drug Courts use a powerful reward system studied by the National Institute on Drug Abuse called the “Fish or Prize Bowl” , a particularly powerful strategy for folks with serious mental illness [29-32]. With the “Fish or Prize Bowl,” individuals draw random rewards for engaging in verifiable therapeutically helpful behaviors including sobriety and treatment compliance. You can read more about how the Fish or Prize Bowl works, and it is far more cost-effective than other strategies. This procedure is on the National Registry of Evidence-Based Programs and Practices, with significant evidence of effectiveness for the seriously mentally ill. The book that details how to do this procedure is available for $40 on Amazon.
Now why is such reinforcement necessary? Shouldn’t people do the right thing without reinforcement, anyway? That is not hard to answer by example. First, the reward centers of the human brain is highly involved in major mental illnesses such a schizophrenia, which why most of the tobacco products are purchased by chronically or severely mentally ill —in case readers have not noticed . Second, using contingency management (direct reinforcement) for tobacco cessation is effective for seriously mentally ill [34, 35] as well as hard drug use (e.g. cocaine, meth) by seriously mentally ill [36-38].
Reduce exposure to problematic behavior
Mental health worsens when children, youth or adults are exposed to problematic human behavior. What might such problematic human behaviors be? Chronic or acute exposure to violence, abuse, aggression, bullying, uncertainty, untrustworthy humans, homelessness, or other actions that signal uncertainty or lack of safety. Such exposure can trigger epigenetic changes designed to deal with the most significant predator of human beings: other humans. While one might assume that effective treatment is just forbidding exposure to such problems, real healing and recovery involves sustained exposure and engagement in positive or nurturing environments [39-41].
Humans do not thrive on psychotropic medications alone: folks with serious mental illness—just like virtually all humans—need a safe bed at night, decent food, good friends, a sense of family, physical activity, a bit of daily fun, a sense of purpose, and sense of personal agency. It was clear that the Magistrate in Hamilton, Ohio, and the teams in her court absolutely understood this human necessity, and they all spent time to assure that these human necessities were being increased for the individuals appearing.
Reduce Toxic Influences
Toxic influences abounded among the court-supervised patients before the Magistrate that day. Their toxic influences can play havoc with the biochemistry and epigenetics of mental and behavioral wellbeing, potentially undermining the effectiveness of the most treasured therapies. One common toxic influence was profound economic stress, lack of safety, and uncertainty. That, in turn, triggers cascades of adaptive chemistries selected during the long course of mammalian evolution. That evolutionary mechanism, in turn, makes people more likely to take an immediate reward such as stealing, sex, drugs, high-fat food, tobacco, alcohol, violence, etc. That evolutionary principle can be bluntly stated: under conditions of chronic perceived uncertainty, take the money and run—whatever the “money” is.
The Magistrate was sensitive to these toxic influences, and so were members of her team. This was a relatively small community of 60,000 people. In Hamilton and Butler County Ohio, people know people who know people—making things more possible. One must not underestimate the human resources present in or projected into that room, assembled by passion, connections and a significant commitment of local money. Such supports might not happen in city of a million people where I live, or in Baltimore where we’ve worked in the desperately violent and impoverished neighborhoods, or amongst very rural or frontier communities we’ve worked with virtually no or few resources nearby like Wyoming, remote communities in Northern Manitoba, or at the bottom of the Grand Canyon. Remember, the State of New York appropriated more than $160 million to sew up the decay in social service and supports infrastructure for court-supervised treatment. With no resources to reduce toxic influences among the patients, AOT is neutered. To my mind, this is major reason why the Cochrane Review finds no consistent results for AOT : it can be assisted (read compulsory) but with a very tiny “t” (treatment), yielding no change. In Hamilton, their connections resulted in real local money and recruited solid talent for a big “T”.
Increase Psychological Flexibility
Serious mental illness is often framed by rigid, tightly defended thinking—not only by the patient but also by the family, community and yes, even by the clinicians treating these disorders. I notice this among family and providers when they land hard accent on the diagnosis, “XYZ HAS ABC psychiatric disorder.” The emphasis on HAS sounds very much like a terminal cancer diagnosis, with a slogan above the entrance of the treatment program that says, “Abandon All Hope By Ye Who Enter Here.”
I was struck by the flexible thinking of the Butler County team; all believed that the people they were working could have better lives, even productive lives that might include the word that some think is impossible—recovery of a good life though it might be different than their previous lives. The scientific literature endorses the notion that context yields very different trajectories that can include “recovery” from a mental illness. The National Institute of Mental Health (NIMH) has funded amazing longitudinal research on the fact that persons with serious mental illness (bipolar, schizophrenia, major depression) can have prolonged periods of recovery [43-49], and interestingly these studies show that the presumptive cornerstone of recover—medication compliance—is not the key predictor . Some randomized control longitudinal studies suggest that specific psychological strategies that help patients develop psychological flexibility about their intrusive thoughts is critical to sustained reductions in reoccurrence of psychosis [50, 51].
Using the health-care bill to learn what makes AOT work is as important as enabling AOT
The discussion about Assisted Outpatient Therapy (AOT) needs to be informed by the actual science. Yes, AOT is now on the National Registry of Evidence-Base Programs and Practices, but with weak scores on quality of research for good reason. I cautioned its proponents about that weakness, based on the independent review by the Cochrane Collaborative of AOT, also called CCT (Compulsory Community Treatment). That review states : “CCT (AOT) was no more likely to result in better service use, social functioning, mental state, or quality of life compared with either standard voluntary or supervised care. However, people receiving CCT were less likely to be victims of crime than those on voluntary care. Further research is indicated into the effects of different types of CCT as these results are based on 3 relatively small trials.”
There is much Kabuki theatre and political posturing regarding AOT, on both sides of the issue. Yes, it can be a useful tool. Is AOT the proven strategy to reduce the horrific burden of serious mental illness in America? Not by current, gold-standard, scientific research. Could it have significant benefits, if the active ingredients for its success are refined and developed? Absolutely, which is why it is important to have a robust, high-quality scientific evaluation that would be true of any disorder or disease that involves life, death, or serious disability.
Some have claimed that it’s AOT or homelessness, death or other horrors. That’s a rhetorical device, not a fact. The truth is there are other evidence-based practices of merit for dealing with the denial or lack of insight (aka anosognosia) common to mental illness and addictions. Consider a few options that may be as good, better and less expensive than AOT:
- Mental Health Courts have far more peer-reviewed experimental, scientific research than Assisted Outpatient Treatment and may be superior in the limited potential comparison studies .
- The “fish bowl” or ‘prize bowl” contingency management approach is extraordinarily effective in getting and completing treatment, even for the most seriously addicted and homeless who often have serious mental illnesses [29-32, 40, 54, 55].
- How can families get family members into treatment who are in complete denial and resistant (i.e., suffering from anosognosia)? Well one strategy with a great deal of evidence is called, Community Reinforcement and Family Training or CRAFT [13-16, 56, 57] that has been highly successful with addictions.
Sensible elected officials or policy makers, advocacy groups, and concerned families should demand that any redirection of significant federal funds to one approach for the treatment of serious mental illness ought to be vigorously evaluated, especially in the context of comparative effectiveness trials of different options that might be superior, more cost-effective, or both as is the case in Assisted Outpatient Treatment—which is the same standard that I am held to when conducting studies to prevent mental, emotional, and behavioral disorders in the first place. This comparative effectiveness evaluation would not be expensive to conduct, nor unethical, since the effectiveness of AOT is not well established by peer-reviewed studies. Specifically, the enabling legislation must include a randomized wait-list control, comparative effective studies to determine what related federal funding and policies have the most benefits for children, youth and adults afflicted by major mental illnesses.
* * * * * * * *
Dennis Embry, President/Senior Scientist at PAXIS Institute – Dennis D. Embry is a prominent prevention scientist in the United States and Canada, trained as clinician and developmental and child psychologist. He is president/senior scientist at PAXIS Institute in Tucson and co-investigator at Johns Hopkins University and the Manitoba Centre for Health Policy. His work and that of colleagues cited in 2009 the Institute of Medicine Report on The Prevention of Mental, Emotional, and Behavioral Disorders Among Young People. Clinically his work has focused on children and adults with serious mental illnesses. He was responsible for drafting of the letter signed by 23 scientists, who collectively represent scores of randomized prevention trials of mental illnesses published in leading scientific journals. In March 2014, his work and the work of several signatories was featured in a Prime-TV special on the Canadian Broadcast Corporation on the prevention of mental illnesses among children—which have become epidemic in North America. Dr. Embry serves on the Children's Mental Health Network Advisory Council.
1. O'Connell, M.E., T. Boat, and K.E. Warner, eds. Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth and Young Adults: Research Advances and Promising Interventions. 2009, Institute of Medicine; National Research Council: Washington, DC. 576.
2. Embry, D.D. and A. Biglan, Evidence-Based Kernels: Fundamental Units of Behavioral Influence. Clinical Child & Family Psychology Review, 2008. 11(3): p. 75-113.
3. Embry, D.D., Community-Based Prevention Using Simple, Low-Cost, Evidence-Based Kernels and Behavior Vaccines. Journal of Community Psychology, 2004. 32(5): p. 575.
4. Wilson, D.S., et al., Evolving the Future: Toward a Science of Intentional Change. Brain and Behavioral Sciences, 2014. 37(4): p. 395-416.
5. Petry, N.M., Contingency management for substance abuse treatment: A guide to implementing evidence-based practice. 2012: New York, NY, US: Routledge/Taylor & Francis Group. xiii, 320.
6. Nunes, E.V., et al., Multisite effectiveness trials of treatments for substance abuse and co-occurring problems: Have we chosen the best designs? Journal of Substance Abuse Treatment, 2010. 38(Suppl 1): p. S97-S112.
7. Petry, N.M. and J.M. Roll, Amount of earnings during prize contingency management treatment is associated with posttreatment abstinence outcomes. Exp Clin Psychopharmacol, 2011.
8. Olmstead, T.A. and N.M. Petry, The cost-effectiveness of prize-based and voucher-based contingency management in a population of cocaine- or opioid-dependent outpatients. Drug and Alcohol Dependence, 2009. 102(1-3): p. 108-115.
9. Petry, N.M., et al., Serious adverse events in randomized psychosocial treatment studies: Safety or arbitrary edicts? Journal of Consulting and Clinical Psychology, 2008. 76(6): p. 1076-1082.
10. Rash, C.J., S.M. Alessi, and N.M. Petry, Contingency management is efficacious for cocaine abusers with prior treatment attempts. Experimental and Clinical Psychopharmacology, 2008. 16(6): p. 547-554.
11. Rash, C.J., S.M. Alessi, and N.M. Petry, Cocaine abusers with and without alcohol dependence respond equally well to contingency management treatments. Experimental and Clinical Psychopharmacology, 2008. 16(4): p. 275-281.
12. Meyers, R.J., et al., Community reinforcement approaches: CRA and CRAFT, in Interventions for addiction: Comprehensive addictive behaviors and disorders (Vol 3), P.M. Miller, et al., Editors. 2013, Elsevier Academic Press: San Diego, CA, US. p. 47-56.
13. Meyers, R.J., H.G. Roozen, and J.E. Smith, The community reinforcement approach: An update of the evidence. Alcohol Research & Health, 2011. 33(4): p. 380-388.
14. Slesnick, N., et al., Treatment outcome for street-living, homeless youth. Addictive Behaviors, 2007. 32(6): p. 1237-1251.
15. Godley, S.H., et al., Adolescent Community Reinforcement Approach (A-CRA), in Substance abuse treatment for youth and adults: Clinician's guide to evidence-base practice, D.W.S.A. Rubin, Editor. 2009, John Wiley & Sons Inc: Hoboken, NJ, US. p. 109-201.
16. Godley, S.H., et al., A large‐scale dissemination and implementation model for evidence‐based treatment and continuing care. Clinical Psychology: Science and Practice, 2011. 18(1): p. 67-83.
17. Embry, D.D., et al. Best Intentions are Not Enough: Techniques for Using Research and Data to Develop New Evidence-Informed Prevention Programs. Emphasizing Evidence-Based Programs for Children and Youth: An Examination of Policy Issues and Practice Dilemmas Across Federal Initiatives., 2013. 26.
18. Biglan, A. and D.D. Embry, A Framework for Intentional Cultural Change. Journal of Contextual Behavioral Science, 2013. 2(3-4).
19. Biglan, A., et al., The critical role of nurturing environments for promoting human well-being. American Psychologist, 2012. 67(4): p. 257-271.
20. Harrow, M. and T.H. Jobe, How frequent is chronic multiyear delusional activity and recovery in schizophrenia: a 20-year multi-follow-up. Schizophr Bull, 2010. 36(1): p. 192-204.
21. Longinaker, N. and M. Terplan, Effect of criminal justice mandate on drug treatment completion in women. The American Journal of Drug and Alcohol Abuse, 2014. 40(3): p. 192-199.
22. Øhlenschlæger, J., et al., Effect of integrated treatment on the use of coercive measures in first-episode schizophrenia-spectrum disorder. A randomized clinical trial. International Journal of Law and Psychiatry, 2008. 31(1): p. 72-76.
23. Phelan, J.C., et al., Effectiveness and outcomes of assisted outpatient treatment in New York State. Psychiatric Services, 2010. 61(2): p. 137-143.
24. Putkonen, A., et al., Cluster-randomized controlled trial of reducing seclusion and restraint in secured care of men with schizophrenia. Psychiatric Services, 2013. 64(9): p. 850-855.
25. Steinert, T., et al., Subjective distress after seclusion or mechanical restraint: One-year follow-up of a randomized controlled study. Psychiatric Services, 2013. 64(10): p. 1012-1017.
26. Theodoridou, A., et al., Therapeutic relationship in the context of perceived coercion in a psychiatric population. Psychiatry Research, 2012. 200(2-3): p. 939-944.
27. Thornicroft, G., et al., Clinical outcomes of Joint Crisis Plans to reduce compulsory treatment for people with psychosis: A randomised controlled trial. The Lancet, 2013. 381(9878): p. 1634-1641.
28. Wykes, T., et al., Working alliance and its relationship to outcomes in a randomized controlled trial (RCT) of antipsychotic medication. BMC Psychiatry, 2013. 13.
29. Tidey, J.W., Using incentives to reduce substance use and other health risk behaviors among people with serious mental illness. Preventive Medicine: An International Journal Devoted to Practice and Theory, 2012. 55(Suppl): p. S54-S60.
30. Angelo, F.N., et al., Predictors of stimulant abuse treatment outcomes in severely mentally ill outpatients. Drug Alcohol Depend, 2013. 131(1-2): p. 162-5.
31. McDonell, M.G., et al., Randomized controlled trial of contingency management for stimulant use in community mental health patients with serious mental illness. The American Journal of Psychiatry, 2013. 170(1): p. 94-101.
32. Weiss, R.D., Contingency management for patients with serious mental illness and stimulant dependence. The American Journal of Psychiatry, 2013. 170(1): p. 6-8.
33. Lasser, K., et al., Smoking and mental illness: A population-based prevalence study. JAMA, 2000. 284(20): p. 2606-10.
34. Roll, J.M., et al., Use of monetary reinforcement to reduce the cigarette smoking of persons with schizophrenia: A feasibility study. Experimental & Clinical Psychopharmacology, 1998. 6(2): p. 157-161.
35. Tidey, J.W., et al., Effects of contingency management and bupropion on cigarette smoking in smokers with schizophrenia. Psychopharmacology (Berl), 2011. 217(2): p. 279-87.
36. Roll, J.M., S.T. Chermack and J.E. Chudzynski, Investigating the use of contingency management in the treatment of cocaine abuse among individuals with schizophrenia: a feasibility study. Psychiatry Res, 2004. 125(1): p. 61-4.
37. Tenhula, W.N., M.E. Bennett, and J.E.S. Kinnaman, Behavioral treatment of substance abuse in schizophrenia. Journal of Clinical Psychology, 2009. 65(8): p. 831-841.
38. Lubman, D.I., J.A. King, and D.J. Castle, Treating comorbid substance use disorders in schizophrenia. Int Rev Psychiatry, 2010. 22(2): p. 191-201.
39. Chien, W.T. and D.R. Thompson, An RCT with three-year follow-up of peer support groups for Chinese families of persons with schizophrenia. Psychiatr Serv, 2013. 64(10): p. 997-1005.
40. Cook, J.A., et al., Results of a randomized controlled trial of mental illness self-management using Wellness Recovery Action Planning. Schizophr Bull, 2012. 38(4): p. 881-91.
41. Glick, I.D., A.H. Stekoll, and S. Hays, The role of the family and improvement in treatment maintenance, adherence, and outcome for schizophrenia. J Clin Psychopharmacol, 2011. 31(1): p. 82-5.
42. Kisely, S.R., L.A. Campbell, and N.J. Preston, Compulsory community and involuntary outpatient treatment for people with severe mental disorders. Cochrane Database Syst Rev, 2011(2): p. Cd004408.
43. Goldberg, J.F. and M. Harrow, A 15‐year prospective follow‐up of bipolar affective disorders: Comparisons with unipolar nonpsychotic depression. 2011, Wiley-Blackwell Publishing Ltd.: United Kingdom. p. 155-163.
44. Goldberg, J.F. and M. Harrow, Consistency of remission and outcome in bipolar and unipolar mood disorders: A 10-year prospective follow-up. 2004, Elsevier Science: Netherlands. p. 123-131.
45. Goldberg, J.F. and M. Harrow, Subjective life satisfaction and objective functional outcome in bipolar and unipolar mood disorders: A longitudinal analysis. 2005, Elsevier Science: Netherlands. p. 79-89.
46. Harrow, M., et al., Ten-year outcome: Patients with schizoaffective disorders, schizophrenia, affective disorders and mood-incongruent psychotic symptoms. 2000, Royal College of Psychiatrists: United Kingdom. p. 421-426.
47. Harrow, M., et al., Do patients with schizophrenia ever show periods of recovery? A 15-year multi-follow-up study. Schizophrenia Bulletin, 2005. 31(3): p. 723-734.
48. Harrow, M., B.G. Hansford, and E.B. Astrachan-Fletcher, Locus of control: Relation to schizophrenia, to recovery, and to depression and psychosis—A 15-year longitudinal study. 2009, Elsevier Science: Netherlands. p. 186-192.
49. Harrow, M. and T.H. Jobe, Does long-term treatment of schizophrenia with antipsychotic medications facilitate recovery? Schizophrenia Bulletin, 2013. 39(5): p. 962-965.
50. Bach, P. and S.C. Hayes, The use of acceptance and commitment therapy to prevent the rehospitalization of psychotic patients: A randomized controlled trial. Journal of Consulting & Clinical Psychology, 2002. 70(5): p. 1129-1139.
51. Bach, P., S.C. Hayes, and R. Gallop, Long-Term Effects of Brief Acceptance and Commitment Therapy for Psychosis. Behav Modif, 2011.
52. Kisely, S.R. and L.A. Campbell, Compulsory community and involuntary outpatient treatment for people with severe mental disorders. Schizophr Bull, 2015. 41(3): p. 542-3.
53. Munetz, M.R., et al., Mental health court and assisted outpatient treatment: Perceived coercion, procedural justice, and program impact. Psychiatric Services, 2014. 65(3): p. 352-358.
54. Sigmon, S.C. and S.T. Higgins, Voucher-based contingent reinforcement of marijuana abstinence among individuals with serious mental illness. Journal of Substance Abuse Treatment, 2006. 30(4): p. 291-295.
55. McDonell, M.G., et al., A pilot study of the accuracy of onsite immunoassay urinalysis of illicit drug use in seriously mentally ill outpatients. Am J Drug Alcohol Abuse, 2011. 37(2): p. 137-40.
56. Manuel, J.K., et al., Community Reinforcement and Family Training: A pilot comparison of group and self-directed delivery. Journal of Substance Abuse Treatment, 2012. 43(1): p. 129-136.
57. Meyers, R.J., M. Villanueva, and J.E. Smith, The Community Reinforcement Approach: History and New Directions. Journal of Cognitive Psychotherapy, 2005. 19(3): p. 247-260.
 Dr. Dennis Embry is a leading prevention scientist focused on mental, emotional, and behavioral disorders. He retired from clinical practice to devote full time energy to prevention and early intervention research, development and dissemination.