Michigan Resiliency Court Program Evaluation

Background and Purpose: The Michigan Resiliency Court training aims to provide jurists and other child welfare court professionals with the “tools to uplift and improve courtroom culture and communication with children and parents in child welfare proceedings” (Michigan Judiciary, 2024). A pilot evaluation of the Resiliency Court training program was conducted to assess the training’s effectiveness (i.e., increased knowledge and application of training materials) in the early stages of its rollout.

 

Methods: The scope of this program evaluation included pre/post mock scenarios and a pre/post survey. Mock scenarios: Paired Samples t-tests were used to compare the pre- and post-means. Effect sizes and estimate percentile gains based on effect sizes were also calculated. The variables of interest included overall pre- and post-scores for cultivating change talk, softening sustain talk, partnership, empathy, goals, planning, overall positive MI behavior, and overall negative MI behaviors. Survey: The research team developed the pre- and post-survey using the training program modules as guides (i.e., safety, trauma, community in care, Motivational Interviewing). The surveys collected demographic information and included a 7-point Likert scale and open-ended questions. Descriptive analyses, Paired Samples t-tests, and effect sizes were calculated pre-to post-survey. The variables of interest aligned with the training’s modules and included overall pre- and post-scores for the following factors: safety, trauma, HOPE, community in care, and Motivational Interviewing (MI).
Results:Mock scenarios: Seven participants (N = 7) completed both the pre- and post-mock scenarios and were included in data analysis. Of the eight variables of interest, the variable planning indicated statistical significance. On average, pre-test planning scores were 1.1 points lower than the post-test planning scores (95% CI [-1.78, -0.51]), indicating that the participants improved on using MI techniques to elicit how the client will meet the mutually agreed upon goals from pre- to post. However, all eight measured behaviors were found to have at least small effect sizes with most being in the medium to large. Survey: Four (N = 4) participants were able to be matched from pre-to-post survey. Of the five factors assessed, three (safety factor, trauma factor, and MI factor) resulted in statistical significance from the pre-to-post survey. Qualitative findings include participant perspectives on ways courtroom professionals can promote engagement in the courtroom, collaborate with parents, individualize goals and service plans, and strategies used to enhance parental motivation.

Conclusions and Implications: This pilot program evaluation assessed the effectiveness of the Michigan Resiliency Court training program. Results indicate the Resiliency Court training program provided essential and comprehensive content to child welfare court professionals. Furthermore, on all measured outcomes, participants improved in a meaningful direction which bodes well for future training efforts. As this was a pilot program evaluation, a follow-up program evaluation would be beneficial once recommendations have been implemented into the Resiliency Court training. Further evaluation might examine the degree of benefit participants found each aspect of the training curriculum and activities. Further, evaluation might also seek to understand what the training curriculum could have added to be even more effective.