Performance Measurement and Management

“I just wanted to alert you to the fact that I’ve nominated you as part of the Administrative Costs Work Group for an organizational team achievement award for your contributions in developing a review protocol and methodology for examining Title IV-E foster care administrative costs and calculating the national error rate. Thanks for your hard work and time you put into this project. The results of your efforts have been received well by OMB, and we look forward to more success as we go through the pilot process.”
—Paul Kirisitz. Children’s Bureau (CB), ACF

Goals. Outcomes. Accountability.

A program may have worthy goals, but how does a client know if its hopes gets translated into concrete results? Performance measurement and management (PMM) is critical for determining whether a program was implemented as planned, the impact of that program, and the extent to which program goals were met. PMM is often used to monitor resource allocation and results obtained. Thus, PMM provides a foundation for making data-driven programmatic decisions.

Drawing on deep knowledge and experience, JBS’s experts help clients implement and sustain PMM. An essential part of that process is assisting government clients in meeting or exceeding the requirements of legislation including:

Government Performance and Results Act (1993)
President’s Management Agenda (2001)
Office of Management and Budget’s (OMB’s) Program Assessment Rating Tool (2002)

A few examples of JBS success stories in this arena include:

Developed and managed a monitoring and evaluation plan for Afghanistan’s educational system that operated in more than 3,000 rural villages in 17 provinces
Evaluated infrastructure, leadership, resource development, and performance outcomes in states that receive substance abuse prevention block grants
Collected data from community-based organizations to evaluate the accomplishments and outcomes of volunteers in AmeriCorps, VISTA, and Senior Corps
Assessed project outcomes and deliverables of USAID Missions with education programs, worldwide

These efforts make sure that every dollar spent truly matters in changing people’s lives.

What We Do & Why We Do It: Spotlight on the State Systems Technical Reviews Project

You wouldn’t think of driving to an unfamiliar location without a map (or these days, a GPS), would you? So why would an organization even consider planning a path for the organization’s future without a similar map, a way to measure where you are against where you want to go?

Performance management involves activities to ensure that goals are consistently being met in an effective and efficient manner. (Carter McNamara, Field Guide to Consulting and Organizational Development, 2006.)

Performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly progress toward preestablished goals. It is typically conducted by program or agency management…Performance measures may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), or the results of those products and services (outcomes). (United States Government Accountability Office, Performance Measurement and Evaluation: Definitions and Relationships, 2005.)

Performance Measurement, Performance Management. Five years ago these phrases would often be met with a blank stare in the substance abuse treatment field. Today they’re mantras, ways in which states routinely substantiate treatment outcomes—or, to use other buzz words, ways states engage in data-driven decision making.

Joyce Dampeer, Ph.D., and the entire Technical Reviews team know that data are indispensable. Increasingly, states that receive funding to provide substance abuse treatment via the Federal Substance Abuse Prevention and Treatment (SAPT) Block Grant program know it as well. Joyce is Project Director for JBS’s State Systems Technical Reviews Project, or Tech Review, which is funded by the Center for Substance Abuse Treatment (CSAT). Approximately every 3 years, a three-member Tech Review team—comprising clinical, data, and fiscal specialists—travels to each state and each territory to review compliance with the 26 requirements of the SAPT Block Grant program. Block Grants provide funding for substance abuse treatment. States typically contract with treatment providers to provide the actual treatment services.

In response to CSAT’s need to have more quantitative measures of Block Grant outcomes, Joyce’s team searched for and discovered a better way to measure states’ performance management readiness and capacity—a matrix developed for the Substance Abuse and Mental Health Services Administration. It’s called the Performance Management Assessment Capacity Matrix. Readiness and capacity are measured in four areas—the capacity of data systems to collect, move, and manipulate data; the organization’s cultural readiness to make the required shift in paradigms; the capacity of key players to use data in planning and policymaking; and provider capacity. Current level of implementation for each capacity is rated as basic, intermediate, advanced, and expert according to the activities the organization is undertaking related to PMM.

Example of States’ Current Level of Performance Management Capacity and the Provider Capacity, which measures the capacity of providers within a system to implement performance management:

Basic level: A provider collects standardized data.
Intermediate level: Management within the provider agency uses data for planning and decision making.
Advanced level: The provider collects performance management data.
Expert level: Potential substance abuse treatment clients use the data collected by the organization to select a program that best meets their needs.

Joyce reports that Tech Review teams have been on target with their ratings for the states—not one state has challenged the teams’ ratings that were developed using the matrix. Rather, the states welcome the objective identification of specific areas in which they could increase their readiness for PMM.

There are other clear benefits of using the matrix to assess PMM.

“We’ve seen a substantial improvement in data systems,” Joyce says of the reviews since measuring the capacity for and implementation of PMM. “States now know and are beginning to truly understand performance measurement and management, not simply from a perspective of requirements but, more importantly, from a vantage point of understanding how this set of tools can benefit service delivery. The quality of substance abuse treatment services and programming is improving. If you don’t improve the quality of the treatment, you’ll never get the desired outcomes for clients, families, and communities.”

And ultimately, THAT is really what this project is all about.