"One state in particular stood out as leading the field in using evaluation to inform practice. Utah contracted with the University of Utah to conduct a thorough evaluation of its congregate care programs (e.g., outcome measures, qualitative interviews with youth). This evaluation process has the capacity to immediately respond with staff training when the needs of the placed children require it. Much is needed in the development of evidence-informed and evidenced-based treatment programs for children with serious mental and behavioral challenges, and this is a promising evaluation model that can help build evidence for effective practices." - Office of the Administration for Children & Families
Key Research Personnel
- Mathew Davis, Ph.D.
- Mindy Vanderloo, Ph.D.
- Mike Tanana, Ph.D.
- Kristen West, M.P.A.
The goal of the Utah’s Division of Child and Family Services (DCFS) program improvement project is to increase the effectiveness of the residential and in-home services provided to neglected and abused children by implementing a protocol for evaluating and improving these services.
Evaluating and improving services is important to meeting the needs of the children and families involved with DCFS, by ensuring practice is evidence-based. Leading researchers and practitioners across many social service domains agree that although the evidence of what works is becoming quite clear, evidence-based principles and programs are being implemented into routine practice at an exceedingly slow pace (Balas & Boren, 2000; Fixsen, Blasé, Metz, & Van Dyke, 2013). For example, Balas and Boren (2000) estimated that only 14% of social service programs are effective, even after many years of providing care. That being said, Fixsen, Blasé, Timbers, and Wolf (2001) “reported 80% success in about 3 years” when evidence-based programs are implemented as intended, with adequate initial and ongoing training for staff (as cited in Fixsen et al., 2013, p. 220). Fixsen, Naoom, Blasé, Friedman, and Wallace (2005), among others, have shown empirically based evaluation and improvement structures can allow even new programs to rapidly reach a state of expertise and effectiveness.
To this end, the Social Research Institute (SRI) at the University of Utah has developed a system for continuous evaluation and quality improvement using evidence-based principles of evaluation and improvement. This system is being used by DCFS to evaluate and improve both residential and in-home services.
The evaluation is structured into four steps:
Step 1: Check Target Population. Measure whether appropriate children enter the service or program. Read more.
Step 2: Assess Adherence. Assess the level of adherence to evidence-based practices of the service or program. Read more.
Step 3: Measure Changes. Measure changes exhibited by the children after receiving the service or program. Read more.
Step 4: Provide Technical Assistance. Provide technical assistance to improve the quality of the service or program. Read more.