The Effect of the Child Support Performance and Incentive Act of 1998 on Rewarded and Unrewarded Performance Goals

Child Support Incentives As Performance Management

The CSPIA of 1998 is just one of the large number of policies aimed at improving public sector performance. As such, the impact of the policy falls within a larger literature on the impact of performance measurement/management on public sector performance. The modern incarnation of these policies is often given provenance in the 1970s, though there is evidence for use of performance information as early as 1903 (Williams, 2003).

At the federal level, there have been numerous policies designed at improving the performance of entirely federal agencies, including the Government Performance and Results Act (GPRA), passed by congress in 1993, followed by the Office of Management and Budget's Program Assessment Rating Tool (PART) in 2002, and the modernization of GPRA in 2010. While GPRA and PART were government-wide attempts at managing performance, there is a longer history of performance incentives in the federal departments and agencies, such as the Department of Labor's Job Training and Partnership Act of 1982. Performance rewards in child support began in 1975 and predate many other performance systems, but for whatever reason, the impact of performance rewards on child support outcomes have been largely left unexplored in the performance management literature.

The rise in the popularity of performance management in the last few decades is owed largely to its intuitive appeal—to be effective stewards of public resources, we ought to manage public agencies by measuring, then managing, public sector performance, a common refrain among early advocates (Wholey & Hatry, 1992, for example).12 However, a significant body of research has challenged performance management from a variety of perspectives, including the implicit (or explicit) values of performance management (Radin, 2006), subjectivity in interpreting performance information (Moynihan, 2008), unintended consequences (Heckman, Heinrich, & Smith, 2002; Heinrich & Marschke, 2010), and the less successful application of performance management in the public sector compared to the private sector (Hvidman & Andersen, 2014).

Performance management has been popularly defined as a “system that generates performance information through strategic planning and performance measurement routines and that connects this information to decision venues, where, ideally, the information influences a range of possible decisions” (Moynihan, 2008, p. 5). CSPIA generates performance information in the form of OCSE annual reports and, importantly, in the form of performance rewards. States and localities “[connect] this information to decision venues” in numerous ways. For example, Gardiner et al. (2003) note, “Interviews with local office supervisors indicated that many of them track the performance of individual workers on a monthly basis for all relevant indicators. One state, for instance, uses a ‘traffic light’ system to rank offices by each incentive measure: Red indicates below the minimum standard for payment; Yellow indicates below state goals but above the minimum; and Green indicates above the state goal. The officials hope this will create friendly competition among the local offices.” New York City even modeled its performance management after the “PerformanceStat” movement, holding monthly meetings to justify programmatic trends (Doar, Smith, & Dinan, 2013). However, states are likely heterogeneous in their willingness and ability to connect performance information to decision venues.

Implementation of CSPIA suggests a number of implications from the performance management literature. One would hypothesize that CSPIA ought to improve performance in the four new child support areas. This is both the global, theoretical intent of all performance incentive systems and the specific intent of this law. Unconditional improvements in child support performance after CSPIA would support this hypothesis, as does an earlier analysis of CSPIA (Huang & Edwards, 2009) that is discussed in greater detail below. However, recent findings of a meta-analysis find no effect or a small positive effect of performance management on performance (Gerrish, 2016). It is still a very open question in this literature whether performance management is empirically correlated with organizational performance.

The impact of CSPIA on the retained performance measure, the cost-effectiveness ratio, is ambiguous. Total collections (the numerator of the ratio) appear in two other performance measures. Thus, increasing current or arrears collections will have an indirect positive effect on cost-effectiveness. Administrative expenditures (the denominator) may increase if states increase expenditures to improve in the four newly rewarded measures. This would have a negative effect on the cost-effectiveness ratio. Which of these effects dominates is unclear.

There are a number of reasons to suspect that child support performance would not increase after CSPIA. These include a decline in the intrinsic (Deci, 1971; Deci, Koestner, & Ryan, 1999) or public service (Perry & Wise, 1990) motivations of state officials after more child support goals were financially remunerated. Additionally, CSPIA may not have an effect on child support outcomes because it is an exercise in performance measurement rather than performance management. Moynihan (2008), Sanger (2013), and Behn (2003), among many others, suggest that how organizations use performance information is inextricably linked to their outcomes. For instance, the use of performance leadership teams and regular performance accountability meetings is an important practice advocated by proponents of CompStat- and CitiStat-like performance management systems (Behn, 2014; Smith & Bratton, 2001). A case study of CitiStat systems found that they varied in a number of important ways, such as the type of staff in charge of the program, the data used, and the structure of leadership meetings (Behn, 2006). Moreover, a recent meta-analysis found that practices such as performance benchmarking were positively associated with performance (Gerrish, 2016). In other words, organizations may claim to be managing performance, but if the extent of their efforts is publishing key performance indicators on a website, we ought to be dubious about claims to performance improvements.

Unfortunately, there is little systematic information about how all 50 states and local governments managed child support performance information over the time period examined in this analysis. However, there is substantial anecdotal evidence surrounding child support performance rewards both before and after CSPIA to suggest that they were highly salient and were the focus of significant bureaucratic attention (Doar, Smith, & Dinan, 2013; Gardiner et al., 2003; Solomon-Fears, 2013). While the examples given above suggest that some states and localities were using some elements of performance management routines, it of course does not suggest all units were using performance information or using performance information effectively. Thus, the possible finding of a null result in this paper does not suggest that performance management is ineffective, simply that the particular system examined here may have been deficient and should be balanced against the broader literature in this area.

In addition to the expectation of positive impacts on the new performance measures, there is the potential for CSPIA to induce dysfunctional behaviors. Performance management systems are well known to create such unintended consequences, typically described as perverse, strategic, or gaming behaviors (Courty & Marschke, 2003, 2004, 2008; Heckman, Heinrich, & Smith, 2002; Heinrich & Marschke, 2010).13 It is essential to search for these behaviors as they can adversely impact performance systems. If present, these strategic behaviors can offset performance gains or reveal that supposed performance gains are illusory. Based on the design and structure of performance incentives under CSPIA, three potential strategic behaviors seem particularly applicable here. First is the pure manipulation of data on child support outcomes by states, driven by the high-powered financial incentives. Second is the timing of reporting to maximize incentives payments (or avoid penalties) in a given year. Third is multitasking behavior in which states focus on the rewarded performance measures to the detriment of goals not rewarded.

Pure data manipulation would have a dramatic effect on reported performance and on this analysis. Data manipulation is also not uncommon in performance systems; Heinrich (2007) cites a number of examples in policy fields ranging from education to health care. Although audits were performed at least once every three years prior to CSPIA, they were focused on administrative procedures rather than on the calculation of the cost-effectiveness performance measure. After CSPIA, every jurisdiction is audited for data reliability each year. The audit is conducted by identifying the universe of child support cases, from which 150 or more are selected for a data reliability audit. If a jurisdiction does not pass the data reliability audit, the state can receive zero payments for that measure and are subject to additional federal financial penalties (or receive a warning and have a corrective year). Given the severe financial penalties for lack of reliable data after CSPIA, it seems unlikely that data were manipulated, even though some states did fail audits after CSPIA.14 Prior to CSPIA there was no incentive to manipulate the child support outcomes that were later measured under CSPIA. There was so little incentive, in fact, that a few states appear to have incorrectly reported some measures in some years by making mathematical errors. States might have had an incentive to manipulate cost-effectiveness (either through collections or administrative expenditures). If this were the case, one might expect to see a discontinuity, either in the mean, median, or in the number of outliers when examining state cost-effectiveness ratios, but no such discontinuity appears in the data (box-and-whisker plots available upon request).

It is also possible that states delayed reporting of child support outcomes to maximize performance rewards. In this vein, Courty and Marschke (2004) examine the timing of graduation in the Job Training and Partnership Act, finding that managers delayed the timing of graduates. However, unlike the gaming behavior found in Courty and Marschke, delaying the reporting of child support outcomes to the next fiscal year does not appear to be a salient problem. In the case examined by Courty and Marschke, performance rewards were all-or-nothing at the cutoff. As mentioned previously, hypothetical performance payments are scaled between the 80 and 50 percent performance ceilings and floors. States above the 80 percent cutoff (states achieving at least 80 percent performance and thus obtaining 100 percent of their hypothetical performance payment) could increase performance in the next year if they delay reporting. However, once states are above 80 percent reliably year to year, delaying reporting provides no benefit. If states are delaying the timing of child support reporting, one might expect performance rewards to bunch just above the cutoffs. This does not appear to be the case. Of the 2,550 state-year performance measure observations post-CSPIA, just 76 observations are within two percentage points above the 80 percent cutoff (3 percent of observations). For states around the 50 percent cutoff (if states achieve less than the 50 percent cutoff the applicable performance reward is 0 percent), delaying performance reporting until the next fiscal year might prevent the state from being severely penalized. However, just 20 observations are within two percentage points above the 50 percent cutoff (0.8 percent of observations).15 Last, histograms of state performance found in Figure 1 suggest no bunching around the ceilings and floors; timing of reporting is not a salient problem.

Unlike data manipulation and timing, however, multitasking behavior may be a concern. Multitasking is derived from a principal-agent model in Holmstrom and Milgrom (1991) in which individuals expend effort on rewarded activities and ignore or otherwise slack on unrewarded activities (or two dimensions of the same activity, like volume and quality).16 In the simplest form of their model, the firm (the principal) rewards its employee (the agent) for just one of two outcomes either due to the principal's preference or perhaps because the second outcome is unobservable. Employees then choose which activities to devote effort toward.

Multitasking behaviors after CSPIA would likely originate from increased attention to the newly rewarded child support outcomes at the expense of other goals. For example, because all five performance measures can be increased by working cases that are entirely contained within the home state, there is likely a stronger incentive (and perhaps pressure) to work these cases. On the other hand, working interstate child support cases may only contribute to a small number of performance measures (e.g., current support and the numerator of cost-effectiveness). Thus, interstate cases may be more costly and may take staff effort away from other cases. As a result, I would expect interstate collections to decline as a result of the adoption of the new performance measures.

It is an open question whether CSPIA would have created additional multitasking responses beyond those that may have existed previously. On one hand, child support services, such as medical support orders, do not contribute to total collections (the numerator of the cost-effectiveness ratio) and so, in theory, would have been underproduced both before and after CSPIA. On the other hand, anecdotal evidence suggests that some states partially ignored the cost-effectiveness ratio to focus on the child support goals they deemed more vital (Doar, Smith, & Dinan, 2013). Therefore, explicitly rewarding four new performance measures may have crowded out other child support goals, such as those considered by the Incentive Funding Workgroup: medical support orders, collections sent to other states (interstate collections), welfare cost avoidance, payment processing performance, and customer satisfaction. Ex ante, I expect new multitasking behaviors to harm the child support outcomes that were considered for performance measures but rejected.

In summary, I hypothesize that the four performance measures added by CSPIA will increase after the policy's implementation, the impact of CSPIA on cost-effectiveness will be ambiguous, and the alternative child support outcomes will decline due to a multitasking response.

Previous Analyses of CSPIA

There are two previous studies of CSPIA. Gardiner et al. (2003) prepared a report for the OCSE, as required by CSPIA. The researchers evaluated the implementation of CSPIA using a mix of data analysis and stakeholder interviews. Their analysis, using data from 2000 to 2002, found that states continued to improve in child support outcomes, even after controlling for socioeconomic factors. In interviews, they found “near-universal” support for the new incentives system among state and local child support staff, with strong support for the new performance measures as measuring the core services of the child support system.

Huang and Edwards (2009) used 1999 as the pre-CSPIA control and regressed state CSPIA performance scores on an index of CSE “capacity” as the key independent variable.17 The CSE index is roughly equivalent to a continuous quasi-treatment variable, in which a higher score indicated a higher capacity for effective policy implementation. Huang and Edward's index used three dimensions: state child support legislation, expenditures on the enforcement, and implementation ability.18 They report that a one-unit increase in the CSE index was associated with a 0.39-unit increase in the child support performance index (both indexes were standardized; units are standard deviations).

Huang and Edwards also note that states improved child support performance in all five measures. The overall collection rate for all states improved by 29 percent between 1999 and 2004 (41 to 53 percent). Similarly, paternity increased by 25 percent, order establishment by 18 percent, arrearage collection by 8 percent, and cost-effectiveness by 13 percent. They conclude that their results provide “evidence that the improvements of state child support performance and enforcement were substantial over the 1999–2004 period” (Huang & Edwards, 2009, p. 248).

The analyses in Huang and Edwards (2009) and Gardiner et al. (2003) differ from this analysis in three key ways. First, neither previous analysis considers the unintended consequences or gaming behaviors that may have resulted from a change in the incentives structure. Second, Huang and Edwards employ a CSE index that accounts for states’ ability to implement CSPIA, creating a continuous quasi-treatment variable. In contrast, this analysis uses an interrupted time series design that recognizes that all states were treated beginning in the same year. Perhaps most important, data from Huang and Edwards begin in 1999, after the reporting changes embedded in CSPIA. Similarly, Gardiner et al. (2003) use only three years of data after CSPIA. Both analyses are unable to capture underlying trends in child support outcomes prior to the passage of CSPIA. In contrast, this article extends child support outcomes back to 1990, when possible, to capture these trends. This data collection is discussed in the following section.

CSPIA Performance Measures

CSPIA performance measures reward states for the stepwise process of enforcing child support: paternity establishment, order establishment, and collection of current and past-due support, as well as cost-effectiveness (total collections divided by administrative expenditures). While none of these performance measures are in mechanical opposition to one another, there may be trade-offs between cost-effectiveness and some of the other measures, particularly if new initiatives come at a higher cost than their incremental increase in collections. Some measures, such as paternity establishment and arrears performance, demonstrated data discontinuities and were not used. More detailed information about these measures can be found in the data Appendix.24

Child Support Outcomes Considered but Rejected

As mentioned above, the Incentives Fund Work Group considered five other child support outcomes for rewards but ultimately rejected them (Solomon-Fears, 2013). These alternative goals include medical support orders, interstate collections, welfare cost avoidance, payment processing performance, and customer satisfaction. This analysis was able to construct measures for medical support orders and interstate collections. The others were either never formally measured or not consistently reported by OCSE before and after the policy change. They also cannot be reconstructed from CPS-CSS. More detail can be found in the Appendix. As mentioned above, the decline in these alternative child support goals is suggestive of a multitasking response to explicit performance rewards.

Control Variables

Control variables are included in regression models for three main reasons: first, to control for the size of the child support caseload; second, to control for the ability of noncustodial parents to pay child support; and third, to control for important changes concurrent with welfare reform.

The percentage of total births out of wedlock was collected from the National Center for Health Statistics and controls for the potential size of the child support caseload and child support administrative burden. This variable is likely to be positively associated with collections per capita (a larger base for collections) but potentially negatively associated with current collections for the same reason.

Two variables control for the likelihood both to pay and to seek orders—the poverty rate, collected from the U.S. Census Bureau Current Population Survey, and the state unemployment rate collected from the Bureau of Labor Statistics (BLS). Per capita personal income captures positive (or negative) changes in disposable income at the state level and is collected from the Bureau of Economic Analysis (BEA). It is adjusted to constant 2000 dollars using the consumer price index. These three variables should help to explain variation in collection amounts between states as well as during particular economic conditions (the late 1990s vs. the late 2000s, for example).

Finally, this paper employs three controls for changes in welfare policy and the welfare caseload. These controls are important because PRWORA had both direct and indirect impacts on the child supports system. Directly, PRWORA contained language that changed federal matching funds for paternity establishment and a requirement that states create a statewide automated data processing and information retrieval system by October 1997. The size of welfare caseloads indirectly impacts child support through two mechanisms. First, work requirements and lifetime limits may have induced some custodial parents to seek orders rather than cash assistance (the cost of seeking child support now being less than seeking cash assistance), which would positively impact both cases and collections. Second, PRWORA would have also had a negative impact on individuals who were induced to cooperate with state officials to receive assistance. This indirect effect is likely to be small because this requirement also exists in many states for other social programs, including Medicaid, state child health insurance, food stamps, and child care (Roberts, 2005).

Three variables are used to control for changes in welfare and welfare caseloads. First, a variable captures the month and year states fully implemented TANF (Crouse, 1999). This variable is measured as the percentage of the year in which states implemented TANF. Second, two variables capture the size and changes in the welfare caseload as reported by the Health and Human Services Administration for Children and Families (HHS ACF). HHS ACF reports on the size of the Aid to Families with Dependent Children (AFDC) caseload (Administration for Children and Families, 1998) and the size of the TANF caseload (Administration for Children and Families, 2014). After 2000, states started operating their own programs separate from TANF for reasons related to TANF work requirements—the controls used here include Separate State Program participants. I transform these data into the second and third variables. One is the average monthly number of TANF recipients per 1,000 residents in each state to measure the absolute size of the welfare caseload. The other variable codes for the percentage change in the average monthly number of TANF recipients by state from the previous year. This second variable helps to control for the rapid changes that occur at the outset of welfare reform.

Many of these control variables are also used by related empirical work in this area, including Huang and Edwards (2009; discussed previously) and Tapogna et al. (2004). Tapogna et al. (2004) examined potential “regression-controls” to adjust state CSPIA performance measures for socioeconomic factors in a report prepared for OCSE. The logic is that the residual variation is more likely to be attributed to state efforts and might result in more equitable performance payments between states. Regression controls were never adopted by OCSE, but the report provides a useful list of covariates that have been shown to change with child support outcomes. Two variables included in Tapogna et al. (2004) but excluded here are state expenditures per case and full-time equivalent (FTE) employees per case. These covariates likely overcontrol; the policy effect of CSPIA would be interpreted as “holding expenditures and FTE employees constant,” which makes little sense as states and localities have control over both staffing levels and expenditures.

Descriptive Figures and Statistics

Descriptive statistics of the dependent variables and covariates can be found in Table 4. Figure 2 graphically illustrates national trends in child support outcomes. The top panel of Figure 2 contains rewarded goals (plus the components of the cost-effectiveness ratio), while the bottom panel shows trends in the goals considered but not rewarded. Every CSE outcome is rescaled to one in 1990 to provide a useful comparison among variables of different scales.29 The vertical line indicates 66 percent implementation of CSPIA.

Figure 2 speaks to the broader trends in CSE in the past two decades. As the figure suggests, significant improvements in national child support outcomes have occurred during this period. However, it also illustrates a significant “jump” in child support outcomes did not occur around the passage of CSPIA—total collections, for example, increase rapidly in the 1990s then increase at a slower rate in the post-CSPIA period. The large gains in the 1990s might be explained by two pieces of legislation in the 1980s—the Child Support Amendments of 1984 and the Family Support Act (FSA) of 1988. Both acts made a number of changes to child support processes that may have improved collections over this period. For example, one of the important provisions in both pieces of legislation relates to immediate income withholding. The 1984 law required that states enact delinquency-based income withholding. In other words, when noncustodial parents were sufficiently behind in payments, child support payment could be withheld from pay stubs much like federal income, social security, and Medicare. FSA changed the delinquency-based remedy to immediate withholding and made it presumptive unless an exception was made. One might imagine that this policy, and others like it, took time to wind through the sprawling child support caseload in the 1990s, before slowing as more child support cases had collections immediately withheld.30 Since about the year 2000, gains have been more modest, with an increase in total collections and a decline in collections sent to other states. These patterns are largely reflected in the quantitative analysis. Another trend to note is that administrative expenditures for CSE nearly doubled in the 1990s but then declined slightly through the 2000s. This may impact performance in other areas as state CSE programs attempt to do more with less.

Panel Interrupted Time Series

The results of the regression models for the seven child support outcomes are presented in Table 5. Every model uses a panel interrupted time series design with state fixed effects and state-specific quadratic trends. Results of the baseline specification are contained in Table 5. Table 5 is organized as follows: The leftmost two columns contain dependent variables for two of the four newly rewarded CSPIA performance measures. The middle three columns contain cost-effectiveness and its base components, total collections and administrative expenditures. The rightmost two columns are the goals that were considered but are not rewarded by CSPIA. The primary independent variable of interest is the top coefficient, post.

The effect of CSPIA on the two newly rewarded performance measures examined in this paper, order establishment and current support performance, have opposing signs but neither are statistically different from zero. Practically speaking, both effects are small as sample means are 50 and 55 percent, respectively. Recalling above, the interpretation of these coefficients is the average increase in child support in the post-CSPIA era. These small results (one positive, one negative but both indistinguishable from zero) suggest that state CSE agencies may have difficulty improving child support outcomes beyond the background trend. The hypothesis that CSPIA may improve the measured performance goals finds little support.

The rightmost columns indicate that one of the two child support outcomes considered but rejected by the Incentive Funding Workgroup declined in the post-CSPIA period. The other was not statistically different from zero. Interstate collections (collections sent to other states) declined by about 75 cents per capita, a meaningful decline (as mean per capita interstate collections are $4.76). This result is statistically significant at the 0.01 level. However, orders that stipulate (and provide) medical support were not statistically different from zero. The hypothesis that CSPIA may have induced a multitasking behavior finds support in the negative impact on interstate collections.

Cost-effectiveness also increased slightly in the baseline specification—44 cents more collected per dollar spent. This is a modest policy effect given a mean sample cost-effectiveness ratio of $4.84. Both a slight increase in total collections (36 cents per capita) and a significant decline in administrative expenditures ($1.54 per capita) drive this result. Among the trio both the effect of CSPIA on cost-effectiveness and administrative expenditures reject the null hypothesis test at the 5 percent level. Although the theoretical sign of cost-effectiveness was ambiguous, the results presented here suggest that state and local CSE agencies increased total collections for less expenditures after CSPIA, holding socioeconomic factors constant.

Using the literature above, I hypothesized two results. First, the intent of the system is that the newly rewarded performance measure would show positive results, although empirical evidence is mixed on whether performance systems actually deliver such results. Second, based on the gaming literature, the alternative measures might decline. I did not have a strong ex ante expectation about cost-effectiveness and its component parts because this performance measure continued from the previous incentives regime. While the child support outcomes increased unconditionally, conditional on other variables and the trend prior to CSPIA, the two new performance measures did not change substantially in the decade after CSPIA. These results suggest that CSPIA had little effect on child support outcomes, contrary to the findings of Huang and Edwards (2009) and the naïve examination presented at the outset of this paper.

Lack of positive results is fairly common in the research on performance management; the public performance management literature suggests that many performance systems do not improve organizational performance (Gerrish, 2016). As mentioned above, there are a number of reasons to explain this result. First, state organizations may not use performance information effectively or financial rewards reduce the intrinsic or public service motivation of state child support officials. Which of these hypotheses (among others) dominates cannot be reconciled with the available data. Additionally, state introspection on in-state cases has apparently hampered interstate cooperation, resulting in a decline in interstate collections. However, based on the evidence, CSPIA more than compensated for this decline with an increase in total collections, of which interstate collections comprise about 5 to 6 percent of total collection. Declines in interstate collections remind performance management researchers to examine potential unintended consequences in performance management systems—unwittingly or not, performance rewards may create situations in which individuals or organizations ignore ancillary goals. Because there are so many ways to game performance systems (cream-skimming, parking, timing of reporting, or the multitasking behavior found here), researchers should be aware of the variety of possible unintended consequences when evaluating performance systems.

Limitations

There are three main limitations to these results. First is the lack of a comparison group, without which it is difficult to assess counterfactual child support outcomes. For example, if state child support performance plateaued absent CSPIA, the post-CSPIA indicator would underestimate the true effect of the policy change. This would also be the case if policymakers implemented CSPIA against the backdrop of concerns about future performance declines. While controls for the time trend and other factors attempt to adjust for these unknown factors, the results are limited by the data.

The second limitation is that this analysis is unable to assess the effect of CSPIA on paternity establishment, arrearage collections, welfare cost avoidance, payment processing performance, and customer satisfaction, owing to a lack of consistent reporting of child support outcomes, painting an incomplete picture of how states responded to CSPIA. This problem could be rectified by OCSE maintaining consistent reporting of some child support outcomes after the policy change for at least a few years after any future reporting changes, as recommended in at least one performance measurement handbook (Hatry, 2006, for example).

A final limitation to these results is that they are a black-box approach to examining the effect of the performance management system. CSPIA mandates annual reporting of state performance but does not stipulate how (or whether) operational data are used on an intrayear basis to support performance routines (nor evaluate whether those routines are successful). Previous research describes the importance of distinguishing performance management from merely measurement; the latter would not be expected to improve organizational performance due to the lack of management routines (Moynihan, 2008). However, because this analysis lacks data on state-level management practices, it was not possible to examine how state and local governments used performance information. Anecdotal evidence suggests that the financial incentives did cause states to focus attention on the performance measures (Doar, Smith, & Dinan, 2013; Gardiner et al., 2003; Solomon-Fears, 2013), though perhaps only on a year-over-year basis.