Uncategorized

Testing, Testing – Do We Test Interventions Sufficiently?

Are Americans becoming more skeptical of scientific inquiry? Some are, according to the pundits. See, for example, Chystia Freeland’s article in the New York Times, “A Deep Faith in What’s Been Proved,” and Paul Krugman’s article in the same paper, “Republicans Against Science.”

Although there does appear to be a growing skepticism about the value of science to address problems such as global warming, there has long been a neglect of social science when it comes to evaluating programs designed to change people’s behavior in beneficial ways, such as those that try to get kids to avoid drugs and alcohol, teach parenting skills, and prevent adolescent behavior problems. Myriad programs that receive federal and state funding have never been adequately tested to see if they work. When they are tested, they are often found to be ineffective or even to do harm.

Consider the D.A.R.E. drug abuse resistance program, which is used in 75% of school districts in the United States and in more than 40 countries. D.A.R.E. lists among its sponsors the U. S. Drug Enforcement Administration, the U. S. Food and Drug Administration, the U. S. Department of State, all five branches of the U. S. military, and the White House Office of National Drug Control Policy. President Obama, like his predecessors, designated a day in April as National D.A.R.E. Day to commemorate the program.

There is only one problem: D.A.R.E. doesn’t work. Studies have repeatedly shown that kids who take part in the program are no less likely to smoke, drink, or abuse drugs than kids who do not. To their credit, D.A.R.E. officials revamped the program in 2009, and maybe this new version will do some good (it is currently being tested). But doesn’t it seem like putting the cart before the horse to sink millions of dollars into a program and implement it in 75% of our schools before we know whether it works?

Another example is Healthy Families America, which is a home visitation program designed to prevent child abuse in at-risk families. According to the Healthy Families America website, 54% of its funding came from federals sources, 38% from state governments, and 8% from local sources. But sadly, like the D.A.R.E. drug program, there is little evidence that it works. The program was been implement throughout the United States before it was adequately tested, and like D.A.R.E., it now has an institutional momentum that is hard to stop, despite the lack of hard evidence that it does any good.

I can hear my conservative friends now: “This just goes to show that we should slash funding further instead of wasting government money on failed programs.” But we can’t let problems like teen alcoholism and child abuse go unchecked, the human toll would be intolerable (not to mention the financial toll in terms of health care dollars). I have another suggestion: Let’s invest those funds in finding out which programs work and supporting the ones that do.

Programs such as D.A.R.E. and Healthy Families America are based on common-sense notions of what will work that turn out to be wrong. Meanwhile, there is an abundance of basic research in social psychology that examines the underpinnings of human cognition, emotion, and motivation, research that has led to successful interventions to reduce alcohol and drug use, prevent child abuse, narrow the achievement gap in education, and help people in many other ways. We know that these programs work because they have been put to the test by behavioral scientists in well-controlled experimental studies, in which participants were randomly assigned to get the intervention and or to a control condition that did not.

Timothy Wilson is the Sherrell J. Aston professor of psychology at the University of Virginia. Wilson’s book Redirect: The Surprising New Science of Psychological Change was published by Little, Brown in September.  Dr. Wilson is a regular contributor to THCB.

5 replies »

  1. Science is one activity, and the use of science to pursue political power is quite another. Use your study as evidence that you need more of my money, and then you’re shocked that I’m anti-intellectual? No, I’m just anti-your intellect.

    Calling those who mock co-opted science “skeptical of science” is just another tactic. It goes on and on, and the more time the left spends writing articles depicting themselves as the cherubs of the goddess Reason, the less time they have to steal. So it’s all good.

  2. Why does this seem to me, a plead for some consulting dollars?

    Let’s ask this- will the study work?

    There are too many things, with no confirmed result. eg:
    1. Trickle down economics
    2. Preventive care
    3. Illegal drus smugling
    4. TARP
    Still many are done because thats intuitively correct and while it may not have confirmed positive result, negative result would be very evident in case of inaction.

    That said it is still a great idea to put a target for any initiative along with reasons as to why that initiative was formed and under what scenario it should be scrapped. Too many of those exist, because they just manage to create and ecology of ecomic interests around those and put up fancy show of one happy family to justify cost of millions.

    And to begin with how beautiful would it be for author to put target to the study and criteria measure it’s success criteria and how to eliminate it if it doesnt work.

  3. Too rue. The left has a tendency to throw money at causes assuming that some common sense program will work, without proof that it will work. They then fail to monitor for effect. An awful lot of money thrown into job training and education has been wasted. Without early intervention, and improvement in family structure, spending later is mostly wasted. There are also some things we just dont know how to fix, and we should admit it, rather than waste money.

    We should identify what we want to change, determine if we can and how, trail some plans, then monitor as we go along. We need to be willing to cut those that do not live up to expectations. (I think a lot of this comes from trying to generalize small programs into larger ones. A small program may be successful due to a few highly motivated people. This may not translate well into larger systems.)

    Steve

  4. “Are Americans becoming more skeptical of scientific inquiry?”
    ___

    Yes. As we get dumber as a culture, and more prone to demagogic stuff like “Science is just another belief system” — e.g., “Teach The Controversy,” “Fari and Balanced” etc.

    Objectivism Fallacy. And, science has to fight with one hand tied behind its back, given that it axiomatically has to admit to (and account for) the possibility of error. Science is a pedantic pain in the ass to the massive and growing instant gratification, infotainment cohort.

    BTW, see the Sperber/Mercier paper “Why do humans reason?”

    http://www.dan.sperber.fr/wp-content/uploads/2009/10/MercierSperberWhydohumansreason.pdf

    Also, Google “Science Based Medicine”