Creating Change and Promoting Safety for Kids & Families in Utah

Families First is a program within Utah Youth Village, one of the state’s largest local non-profit organizations.  The program is built to meet people in their homes and teach parents, kids, and families skills to support better communication, relationships, and parenting practices.  A specialist works with the family—individually and collectively—to teach skills to stabilize the home so it’s a safe environment and the kids and the family don’t have to be separated. 

How do they do this?  They use a strengths-based approach.  Families First Program Director Wayne Arner explained, “Rather than coming in and saying, ‘you’re not a good parent because you did “blank,”’ we would come in and try to focus on, ‘what is this family doing well?’  Every parent and every family has strengths.  Based on those, knowing the risk factors, we build an individualized plan to take those strengths and teach them new skills so that their parenting practices aren’t going to be abusive.”

And the program is effective.  On the Youth Outcome Questionnaire, a common tool utilized in understanding the effectiveness of protective services programs, a decrease of 13 points in risk of abuse is considered clinically significant.  In most families, after completing the Families First program, risk of abuse goes down 25 points—nearly double clinical significance.  To Mr. Arner, this isn’t surprising.  “These skills work.  That family focus piece is really powerful.”

Mr. Arner thinks the program works for several reasons.  First, the intervention happens in people’s homes.  “It’s one thing to go to a 50-minute session in an office to talk about issues, but then how do people apply those skills in the home?”  He continued, “We are there where and when the problems occur.  By being in the home where troubles occur, people are less defensive and more open to learning.”  Additionally, he says, “Parents want to be good parents.  Kids want to be successful.  They’re trying their best.  If we can show them a better way, that really increases the chances of such a positive outcome.

OK. So Families First seems like a great program. What does this have to do with the College of Social Work?    

Federal legislation requires states implement evidence-based prevention programs in order to receive federal matching funds for their prevention programs and out-of-home placements.  The federal government does this as an incentive for states to implement evidence-based programs so that there will be better outcomes for the children and families that come in contact with the system.  With Families First, their internal data suggests the program does work, and anecdotally, many of clients talk about the positive changes they’ve experienced through the program.  However, in order to be an officially recognized evidence-based program, Families First must undergo rigorous statistical program analysis—something that requires both funding and expertise that few nonprofits have on-hand.  What does the Social Research Institute (SRI) of the College of Social Work do?  Rigorous statistical program analysis.

Though neither Kristen West nor Meghan Broadbent, the SRI staff who worked on the project, started their careers planning to work on prevention services program evaluations, both feel passionately about the work they’re doing. 

“I think a lot of times, agencies and politicians make decisions about programing not based on anything except for their gut feelings or what is popular, instead of basing them on research,” said Ms. West, a senior research analyst.  “I’ve felt very invested to both help to improve the evidence base of policy and programs, and to help organizations know how to improve their programs by understanding what actually works and what doesn’t.” 

Ms. Broadbent, SRI’s interim associate director, has similar feelings.  “Part of what I love about program evaluation is you actually get to engage in rigorous methodology, which historically has been lacking in this area.  A lot of times when people engage in program evaluation, they’re really not considering how to evaluate something appropriately.  You can put in all this time and effort, and all this work and money, and have it fall apart at the end really easily because you didn’t have someone with the methodologist perspective come in and help set up.  No one asked, how are you going to evaluate things?  How are you going to collect data?  When are you going to collect it?  Who are you going to collect it from?  Having my hands in that part is fun.  I love adding to the credibility and evidence base of the process.”

Because both Families First and the Division of Child and Family Services (DCFS) already collected extensive data about the program, the analysis was able to proceed smoothly.  Though this isn’t common nationwide, it’s one of the things that really sets Utah apart as leaders in child welfare.  Ms. Broadbent explained, “We’re able to do research that other states just can’t do.  Often they just don’t have the data or the systems in place to utilize the data.”  Ms. West also enjoyed designing a rigorous analysis with existing data. “In addition to cutting costs and reducing our timeline, the challenge of designing rigorous analysis from existing data really signifies a step forward.  It shows that this is possible.”

Within the study, they compared children and families who had been through the Families First program with children and families who had treatment as usual to see if there was a difference in their long term outcomes looking, specifically, for repeat maltreatment.  What did they find?  Overall, children who had completed the Families First program, showed a 41 percent reduction in the odds of subsequent child maltreatment compared to treatment as usual—a result that maintained statistical significance in every test. 

It was exciting for these research analysts and Mr. Arner to see program outcomes hold up under intense statistical analysis.  After such promising results, the program analysis was handed over to the Prevention Services Clearinghouse for review.  The evaluation has passed several stages and the group remains hopeful the program will receive a rating.  “If the program does pass Clearinghouse review, not only are we able to refer people to the program with confidence, but also other states are able to contract with the Families First program and adopt the model in their own state,” explained Ms. Broadbent.  “This could serve as a valuable step, taking Utah forward and strengthening our position as national leaders in child welfare.”