Performance-based funds produce no equity impact – Study

In the early 2010s, as evidence accumulated that the performance-based funding (PBF) programmes established in the 1990s did not materially improve college and university completion rates, some states put in place what’s been dubbed PBF 2.0.

Under the original PBF model, states make bonus payments to colleges and universities on top of their base funding. The PBF 2.0 model adopted by Tennessee and Ohio, the two states examined in the first study to examine the equity impacts of PBF, published this month in the Journal of Higher Education, linked increased funding to benchmark outcomes and includes premiums for improved outcomes for historically under-served communities, such as older adults, low-income students and racially or ethnically minoritised students.

A widening gap

In “Incentivising Equity? The effects of performance-based funding on race-based gaps in college completion” (“Incentivising Equity”), Professor Monnica Chan, who teaches in the college of education and human development at the University of Massachusetts Boston, and her co-authors show that, far from closing the gap between white graduation rates and blacks and Latinx students, PBF 2.0 has widened it.

Chan and Professor Zachary Mable, who is research professor of education and economics at Georgetown University, Washington, DC, and Preeya Pandya Mbekeani, a post-doctoral research associate at the Annenberg Institute at Brown University, Providence, Rhode Island, examined Tennessee and Ohio, states with long-standing PBF policies that allocate between 80% and 100% of their funding through these policies. They asked two questions:

• Did the adoption of PBF 2.0 policies in these states impact the number of (junior college) certificates and (four-year) degrees awarded to under-represented racially minoritised (URM) students?

• Did the adoption reduce completion gaps between URM and non-URM students?

According to Mbekeani, performance-based policies represent differences in the way in which higher education institutions have traditionally been funded.

Mbekeani said: “They are based on certain student outcomes, such as how many students graduate or persist through college, rather than purely on enrolment. In the United States, there have been efforts to account for equity by including special bonuses or premiums for the completion outcomes of historically under-served groups – students from low-income families, students who identify as black or Latinx.

“There have been no other studies that have looked at the equity implications of these policies and whether they’ve been effective in narrowing completion gaps, not whether they boost enrolment, but whether gaps actually narrow.”

“These [PBF 2.0] policies were ineffective in reducing or closing completion gaps in the two states we studied.”

Synthetic model

“Incentivising Equity” does not directly compare the completion rates by URM students in Tennessee and Ohio. Rather, each state is examined through the lens of a “‘synthetic’ counterfactual” model covering the years 2004 to 2015.

The models were based on college and university completion data and covariates for both URMs and Non-URMs from 16 states that did not use the PBF 2.0 funding model. These states include California, Arkansas, New Hampshire, Vermont and West Virginia.

To ensure that the models are not affected by changes in a college’s status from being a two-year to a four-year institution, the research team examined 1,250 out of a possible 1,695 colleges and universities in the creation of the synthetic model.

The PBF 2.0 incentives established by the governments of Ohio and Tennessee were substantial. Ohio, for example, which had allocated 80%-100% of its funding on the basis of student progression and completion, began placing greater weight on completion (which for the four-year bachelor degree is extended to six years).

While premiums varied over the period and for different schools, over the period the research team studied (2009-15), four-year institutions were eligible to receive between 30% and 40% increased funding based on their URMs’ performance. Students who fit into more than one category – for example, they were older URMs – brought their institution increased bonus payments.

Tennessee provided additional funds to colleges and universities which otherwise would have suffered from the shift away from enrolment-based state grants. Tennessee did not, as Ohio did, identify black and Latinx students.

However, “adult and low-income college students in Tennessee are more likely to be students of colour”, which allowed the team to “explore whether the equity provisions of PBF 2.0 have spillover effects for other historically disadvantaged groups that are not explicitly prioritised”.

Financial incentives as key levers

“States are interested in holding higher education institutions accountable for student performance. Under PBF, financial incentives and losses are a key lever for doing so.

“The additional premia for URM students are aligned with this approach, namely institutions can be financially incentivised to achieve certain outcomes – in this case for particular students.

“Our research showed, even with additional incentives for marginalised students, that PBF funding did not improve the success rate of black and Latinx college students relative to white students,” Mbekeani told University World News.

Over the years studied, despite the fact that one year saw a 7% increase of URMs completing college, inequality in outcomes accelerated. This is because in other years, the number of graduating URMs was either stable or declined by 2%-5%.

Ohio also saw differential impacts on bachelor degree production, the study showed. Plotted on a logarithmic chart, non-URM students increased about by 15% between 2009 and 2015. After increasing at a faster rate than the synthetic model for 2010 and 2011, the number of degrees conferred on URM students dropped precipitously in 2012.

In total, over the course of the years studied, non-URM graduating students increased by 10%-15% per year while URM students decreased in relation to what the model predicted.

Comprehensive supports

At the end of our interview, I asked Mbekeani if she thought that PBF was, simply, a flawed mechanism to improve graduation outcomes. After a moment’s thought, she ended what she had told me was her first media interview as if she were a seasoned pro.

“The evidence of performance-based funding being effective and improving outcomes meaningfully for URM students is not there. And, without robustly funding public institutions so they can provide comprehensive supports for URM students, it will be a challenge for performance-based funding policies to remediate gaps we identified in this paper.”