Lessons Learned from Evaluating a Case Management Program

JessicaBroome

Jessica Broome

A client of mine is a social service agency. One of their smallest programs (serving about 40 clients at any given time) offers case management for individuals involved with the criminal justice system. The typical program participant has recently been released from a long prison sentence, is drug-addicted, and suffers from a chronic health condition (most often HIV, but also hepatitis, diabetes, and cancer). The audience is hard to engage, service staff are chronically overworked, and, largely because the number of participants is so small, the quarterly evaluation has sometimes not shown much progress. After almost four years of evaluating this program, I’ve learned a few lessons that I hope can help other evaluators assess their own approaches:

  1. Use clear and relevant objectives to track progress: This may seem obvious, but it has been a bumpy road! The objectives put forth by the program’s funder were broad and, initially, impossible to track. (“Reduce the spread of HIV in served communities” was one request that we simply could not collect data to measure.) Instead of trying to force our program into this rubric, we came up with a few relevant sub- objectives that we COULD track, like education of community members and decreased viral loads among program participants.
  1. Involve service providers in selecting evaluation techniques: The counselors and case managers who are on the “front lines” of client service have been my best resource when it comes to designing data collection mechanisms and deciding which outcomes should be tracked. For example, a counselor pointed out that program participants often have limited cognitive abilities and are just not able to reliably answer questions rating their health. Further, requiring this was posing a large burden on already-overworked staff, who had to collect the data from participants at regular intervals. Instead of a self-assessment of health, we started collecting lab results directly from medical providers, which provided a more objective measure, and reduced the burden on both participants and line staff.
  2. Keep evolving data needs and sources: For several years, I conducted focus groups and client satisfaction surveys with current clients, who consistently gave the program rave reviews. Ultimately, I realized that we weren’t learning anything new from these evaluative approaches; participants who responded to the survey and attended the focus groups were those who came to the program every day. Those who were unhappy with the services they received had stopped coming, or came only rarely. This year, we’re implementing a “track down” program, where all the resource that were used for the focus groups are diverted to locate and interview individuals who have stopped attending the program.

It’s important to point out that these lessons can be applied to almost any program evaluation! This program works with a very specific target audience, but I will take these lessons with me when I work on other evaluations.

Author: Jessica Broome
Jessica’s question to you
What other lessons have you learned from program evaluations?

Please click the link above to view responses or comment.

 

IYE Operationalize and Execute Strategy Storython

 Blogathondate

storythonwritup

Click the table to enlarge

 Storytellers

peopleforblog

 


What and How?   | Privacy Statement  | Join us

 tellyourstoryhere   askquestions storytelling whatsyourstory

videosign  lessonslearned keeneye

Yourstorycouldhelp   shareyourtoons


QandA

Question and Answers

Please click the link (s) to view or respond to questions.

How to operationalize an Effective Culture of Evaluation at the National and Organizational Levels?

Strategy translation and elaboration questions and answers for non-profits and community based organizations

How can home based consultants and small businesses execute their planned strategies in a timely manner?

Critical factors, behaviours and processes that worked – operationalizing a culture of evaluation in organizations.

EVAL Year Logo Evaluation_Tourch_2015_r1 softenedmagatelogogreenstool.v

All rights reserved ─ Copyright © 2015 by Magate Wildhorse ℠

IYE Evaluation Methods & Design Blogathon

Prospect%20Plantation%20Or_at_Prospectp103 Blogathondate comeshareblogth  Click on the table to engage

topictableblog

meetbloggers

 peopleforblog

JessicaBroome

Jessica Broome

Lessons Learned from
Evaluating a Case Management Program


 What and How?   | Privacy Statement  | Join us

 blogherevideosignaskquestionsshareyourtoons

http://dilbert.com/strip

http://dilbert.com/strip


QandA

Please click the link (s) below to view questions, post answers or comments.

Best Lines of Evidence for Evaluating Strategic Plans

Impact of Evaluation Belief vs Actual Practice on Choice of Design

General comments and feedback

EVAL Year LogoEvaluation_Tourch_2015_r1softenedmagatelogogreenstool.v

All rights reserved ─ Copyright © 2015 by Magate Wildhorse ℠