Morning Zen Guest Blog Post ~ Dennis D. Embry, PhD 

If the science is solid and scalable, build with care. But what makes good, scalable science? About 70% of Americans can distinguish between astronomy and astrology, but less than half of Americans—even with a graduate degree—know that water boils at a lower temperature at higher altitudes.

What constitutes good the science for the prevention or treatment of psychiatric or behavioral disorders is way more complicated than the temperature of boiling water in the Rocky Mountains. Since Congressman Murphy’s hearings on SAMSHA, we’ve been exposed to people engaging in hot air about what is good science for the prevention, intervention, and treatment of mental-health disorders.

I have a humorous classification of people who muddle in science: clerics who opine their views as received knowledge, bird-watchers who watch and count things, hunters who like to kill other people’s science as trophies, and farmers who grow proven solutions. I’m a farmer by inclination, perhaps so because of growing up in Western Kansas.

These metaphors are crucial to understanding the kind of science we need to tame the epidemic of mental illness, addictions, and violent behavior at a policy level for America. Please indulge some storytelling to understand how really good prevention and treatment science has to be constructed.

My official start in science began at age 13, winning a National Science Foundation junior scholarship grant to test the effects of an artificial ionosphere to protect against genetic mutation of fruit flies, as a randomized lab trial to protect astronauts from cosmic radiation in space. Yep, I’m a nerd. A few years later, I became U.S. Capitol Page for Gerry Ford and Bob Dole. You can see the early blend of science and policy.

In the late 1970’s, I was the assistant director of Early Childhood Institute at the University of Kansas. My job was to conduct the electronic literature searches for about 300 grad students and faculty of three departments: Human Development, Special Education, and the Bureau of Child Research. One day, request to search on preventing the 3rd leading cause of death of preschool-age children in America: being struck by a car playing outside.

Prior studies revealed that “prevention” strategies killed more kids. Good intentions are a chronic problem in prevention, intervention, and treatment: Good intentions often increase morbidity (sickness) and mortality (death), something especially true in the history of prevention and treatment of substance abuse and psychiatric disorders.

Epidemiology often precedes good prevention or treatment. Semmelweis’ and Snow’s epidemiological measures of child-bed fever deaths and cholera infection stand as scientific classics. Thus, I began direct epidemiological observations of pre-school children playing outside in conditions noted for higher morbidity and mortality of pedestrian injuries. It was not, as many thought in high traffic areas. Most such injuries and deaths happened in quiet residential areas, even in cul-de-sacs. That was NOT what all the “experts” thought.

Being trained in applied behavior analysis and child development, I developed a 10-second interval code and reliability procedures. We discovered that impulsive children (epidemiologically at a higher risk for such deaths or injuries) entered the street during play way more than other kids—about ten times per hour.

We also discovered that some kids were reinforced by accidental parental and peer attention for entering the street, such as “playing chicken” with cars. Another amazing discovery was that parents spanking impulsive kids entering the street increased dangerous play. Oh my, that caused a controversy that lives on the Internet. Direct observation of epidemiological context (e.g., Semmelweis and Snow) vital for designing interventions.

Once we understood cause and effect, setting events, the presence or absence of reinforcement for dangerous or safe behaviors, we began testing each strategy methodically in “single-subject” or interrupted time-series designs. Such designs are crucial to understand and suppress sources of behavioral variability, rather than accepting variability as mere chance. Neither animal nor human actions rarely occur as mere chance: there are almost always antecedents, behaviors, and consequences that alter the frequency, duration and/or intensity of behaviors. In humans, that also involves verbal behavior that can change other behaviors in nanoseconds, which can easily be demonstrated by uttering the word “bomb” or “gun” in a crowded place.

To make a long story short, our evolved strategy was built up inductively from direct observation and interrupted-time series experimental designs that worked. Then, we tested the final prevention strategy across the United States with Sesame Street and then nationally in New Zealand. By the bye, I also sharpened my entrepreneurial skills needed to make scientifically proven practices into scalable, real-world prevention, intervention, and treatment practices for whole countries, communities, states/provinces, and consumers. I also learned to measure for adverse effects, since passionate attempts to do good often can cause great harm.

Poor science, smug policies, shouting advocates, prideful bureaucrats, and politicians facing elections can and do cause great, great harm—especially when political status and one’s dinner hang on the policy. One must be willing to embrace the fact that our policies and practices can (and often do) create great harm. That is why I tell my colleagues that any negative finding we might have must be shared.

Throughout my career owning my own scientific companies since 1982, I’ve been asked, prodded and even paid to reduce seemingly intractable, or scary policy problems of human behavior since my dissertation: child maltreatment, parenting problems, youth violence, military family deployments/casualties/terrorism, violent juvenile offenders, binge drinking by women of a child-bearing age, home visiting programs, children’s mental health, youth tobacco use and illegal sales, after-school programs that create delinquency, state or county-wide meth use, youth mental health disparities, and a few other things. It’s been very challenging and gratifying. My colleagues and I have been able to demonstrate population-level changes in several cases, not just academic results. I’ve lived in trenches of science and policy.

Because of my weird history, I got asked to lead a policy effort on how to scale up science for public policy efforts by the office of the Assistant Secretary for Planning and Evaluation in the Department of Health and Human Services. Our ASPE publication carefully outlines how to grow a scientifically valid strategy that can work at population and policy level [1].

All of this is important for SAMSHA’s brave new world of “Policy Playbooks” and the announced unplugging of NREPP. For the record, I am not aware of any staff member at SAMSHA who has ever conducted and published a scientific-valid evaluation of a population level policy to prevent or treat mental, emotional, behavioral, or psychiatric disorders. Policy Playbooks is not high on scientific citations at www.pubmed.gov.

“Policy Playbooks” would need to move the data needle on BIG meaningful indicators that folks concur have high social and economic validity. Here is the big lesson from our ASPE paper: You cannot make a big change (e.g., population-level indicators) unless you can reliably make changes “little changes” of at least 30% or more of the target groups with at least an effect size of .2 or greater. And you damn well need to measure and account for possible negative effects, too. SAMSHA’s proposed “policy playbook(s)” for preventing, intervening—let alone treating—serious mental illness across all 50 states, territories, and Native American communities with measurable reductions incidence, morbidity, and mortality with no prior publically available science makes me queasy.

Congress, families, states, communities, advocates, taxpayers and the media (and the scientific community) need to see the gold-standard experimental science that supports the presumptive policy playbooks that are supposed to prevent, intervene and treat serious mental illnesses “better than the world has ever seen” in every community in America. So, SAMSHA must show America the rigorous science for its touted Policy Playbooks, before it implements and mandates them.

1. Embry DD, Lipsey M, Moore KA, McCallum DF: Best Intentions are Not Enough: Techniques for Using Research and Data to Develop New Evidence-Informed Prevention Programs. In: Emphasizing Evidence-Based Programs for Children and Youth: An Examination of Policy Issues and Practice Dilemmas Across Federal Initiatives. Edited by Office of the Assitant Secretary for Planning and Evaluation USDoHHS. Washington, DC: Office of the Assistant Secretary for Planning and Evaluation (HHS), Washington, DC. Office of Human Services Policy; 2013: 26.

 *  *   *  *  *  *

Dennis Embry, President/Senior Scientist at PAXIS Institute – Dennis D. Embry is a prominent prevention scientist in the United States and Canada, trained as clinician and developmental and child psychologist. He is president/senior scientist at PAXIS Institute in Tucson, Arizona. Dennis Embry serves on the scientific advisory board for the Children’s Mental Health Network and the U.S. Center for Mental Health Services Advisory Council.