The Columbus Model: Performance Measurement & Evaluation

Icon

Toolkits | October 15, 2010

Files: PDF | 101 KB | 3 pages

Columbus's homeless assistance system is funded, managed, and evaluated by the community's lead Continuum of Care (CoC) agency, the Community Shelter Board (CSB). CSB operates an outcomes-based funding model that uses measurable performance standards to monitor and evaluate agencies and system progress toward community goals. The performance standards reinforce an overall vision and strategy for improving the homeless services system and working toward the eventual elimination of homelessness.

Columbus evaluates 15 system-level performance measures and more than 30 client- and program- level measures. Key performance measures include prevention, length of stay, housing outcomes, cost-efficiency, and recidivism. In addition, CSB evaluates compliance with administrative and program standards. Performance measure and standards are developed with input from CSB board and staff, the CoC Steering Committee, and the broader CoC stakeholder community. Based on this information, the CSB designs goals for each outcome measure, assigns a performance metric to each measure, devises a methodology for tracking each measure across the system, and plans for integration of feedback. CSB also identifies the key players involved in each performance measure, ensuring accountability. For program level measures specifically, CSB works directly with each provider agency to set and establish a work plan for performance goals. These goals are outlined in a Performance Outcome Plan (POP) and included in the agency's contract. To access an overview of the POP, a sample POP, or sample performance-based contract, click here.

Columbus's annual performance evaluation can be broken into three distinct levels. First, CSB assesses and scores; each system and program performance goal as Achieved (Yes/Y), Not Achieved (No/N), or Not Applicable (N/A). An Achieved Goal is defined as 90 percent or better of a numerical goal or within 5 percentage points of a percentage goal. Not Applicable is assigned when a performance goal was not assigned or the program is too new to be measured.

Second, each system and program is assessed as a High, Medium, or Low performer based on overall achievement of system-level performance outcomes and overall program performance outcomes, respectively. A program that experiences repeated difficulty meeting performance goals is considered a "program of concern". Programs rated as "Low" performers or repeatedly identified as a "program of concern" are handled by CSB through a Quality Improvement Intervention Program (QII). The QII is based on quarterly one-on-one dialogues between the CSB and the provider agency to support performance improvement. Funding decisions or merit awards are often based on the rating that each program receives during its annual evaluation.

System/Program Evaluation Ratings

High Achieve at least 75 percent of the measure outcomes and at least one of the successful housing outcomes (either number or percentage outcome)
Medium Achieve at least 50 percent but less than 75 percent of the measured outcomes
Low Achieve less than 50 percent of the measured outcomes

If a system is scored as low performing, CSB works with all the providers in that system collectively to determine the source(s) of or barrier(s) to stronger performance. For example, the introduction of a new program - such as the Homelessness Prevention and Rapid Re-housing Program (HPRP) - can adversely affect system performance as programs go through a learning phase during the first few months of implementation.

name

At the third and final level of evaluation, Columbus systems effectiveness as a whole is monitored and evaluated by quarterly, semi-annual, and annual reviews of aggregated data from providers and the communities various systems. The table below provides a listing of Columbus system- and program- level performance measures. More detailed information concerning definitions and evaluation methodology can be accessed by clicking here.

System Level Performance Measures:
  1. 1. Average length of stay
  2. Cost per household
  3. Cost per successful housing outcome
  4. Employment status at exit
  5. Households served
  6. Housing retention
  7. Housing stability
  8. Interim housing stability
  9. Movement
  10. Negative reason for leaving
  11. Pass program certification
  12. Recidivism
  13. System occupancy rate
  14. Successful housing outcomes
  15. Turnover rate
Program Level Performance measures:
  1. Average family housing collaborative transition time (ATT)
  2. Average length of participation
  3. Average length of (shelter) stay (LOS)
  4. Completed vocational/ other training (%)
  5. Critical access to housing (CAH) households served
  6. Cost per household
  7. Cost per successful housing outcome
  8. Cost per unit
  9. Detox exits
  10. Diversion recidivism
  11. Employment status at exit (# and %)
  12. Exited households to PSH
  13. Households served
  14. Housing affordability at exit
  15. Housing retention
  16. Housing stability
  17. Increase in income from entry to exit
  18. Interim housing stability
  19. Movement
  20. Negative reason for leaving
  21. Other submitted applications
  22. Pass program certification
  23. Program occupancy
  24. Recidivism
  25. Shelter linkage
  26. Submitted SSI/SSDI applications
  27. Successful SSI/SSDI applications
  28. Successful diversion outcome
  29. Successful housing outcome / successful outcome
  30. Turnover rate
  31. Usage of CSB direct client assistance
  32. Usage of other community resources related to housing stability

The Columbus Model: Becoming a Data Driven System
The Columbus Model: Quality Improvement
The Columbus Model: System Overview

Downloadable Tools

  1. Columbus Program Performance & Evaluation Measures List
  2. Columbus Program Performance Measures Definitions and Measurement Methodology
  3. Columbus System Performance Measures Definitions and Measurement Methodology
  4. Columbus Sample System & Program Level Indicator Report
  5. Columbus Performance Outcome Plan (POP) Overview
  6. Columbus Performance Outcome Plan (POP) – Blank Sample
  7. Columbus Performance-based Contracts Samples
  8. Columbus & Community Shelter Board Administrative and Program Standards

Additional Resources

Community Shelter Board Website