Is Diet Quality Improving? Distributional Changes in the United States, 1989–2008

Definitions

Consider two distributions of HEI scores with cumulative distribution functions (CDFs) F_A(z) and F_B(z), for a population of interest in two distinct time periods or, alternatively, for two mutually exclusive subpopulations within a single time period. We say that distribution B first-order stochastically dominates distribution A if

with strict inequality for some z. In other words, no matter where the threshold for “healthy” is set, a greater share of the population characterized by distribution B have a “healthy” diet. This relationship is illustrated in the left panel of figure 1.

Distributional studies of well-being often look to higher orders of stochastic dominance, notably second-order stochastic dominance (SOSD). Whereas first-order stochastically dominance (FOSD) counts the number of individuals falling below a given healthy diet threshold (which would, in turn, determine the headcount ratio), SOSD captures the depth or severity of inadequate diets. SOSD is sensitive to the extent to which diets fall in the lower tails of the distribution.

To formally define SOSD, let , and likewise for B, so that FOSD of B over A can be written as . F_B will second-order stochastically dominates F_A if

with a strict inequality for some value of z. This relationship is illustrated in the right panel of figure 1, which shows that the CDFs cross, thereby ruling out FOSD over the entire range of HEI. The integrated difference between F_A and F_B, shown in the subpanel, is strictly positive, and thus F_B second-order stochastically dominates F_A. More generally, dominance at order s of B over A is then defined as where,

with a strict inequality for some value of z.

Stochastic dominance maps into social welfare under fairly standard assumptions about the utility derived from a healthy diet (Deaton 1997). For example, if B first-order stochastically dominates A, then for any social welfare function W defined on the distribution of diet quality F(z) such that where U is any monotonically nondecreasing utility function of z (U^′≥0), it must be true that social welfare derived from distribution B will be at least as high as the welfare derived from A. We can extend the mapping of social welfare to SOSD by requiring U to be nondecreasing and concave in z (U^′≥0,U^′′≤0). Note that because dominance of order s implies dominance of order s+1, it follows that the latter is a less stringent condition. Thus, welfare implications are the strongest in the first-order case. Finally, we also make the standard assumption of anonymity so that each individual is weighted equally in the social welfare function.

Estimation

A useful expression for in empirical analyses is (Davidson and Duclos 2000):

(1)

Integrating the empirical analogue of (1) by parts leads to a natural estimator of

(2)

where we account for complex survey design (e.g., CFSII and NHANES) by letting θ_i be an individual's sample weight, is the population size in distribution j (with corresponding sample size n_j), and I(⋅) is the indicator function. The first-order case leads to the empirical CDF

(3)

and the statistic for the second-order case follows directly.

Inference

We are interested in testing the hypothesis that the distribution of dietary quality in one time period dominates the distribution in another time period. For example, allowing distribution F_B to be the more recent time period, the null hypothesis of an increase in dietary quality at order s∈{1,2} is,

where the positive superscript denotes the hypothesis of dietary improvement, whereas in testing the null hypothesis that dietary quality has decreased (denoted by ), the signs would be reversed. One could also posit a null of equality, but notice that rejection of both and implies rejecting equality.

 Bishop, Formby, and Thistle (1989) propose a multiple testing procedure by hypothesizing dominance in both directions—that is, testing the null of and versus their respective alternatives and drawing inferences from the combined acceptance/rejection. A variety of approaches to drawing inferences based on the Bishop, Formby, and Thistle (1989) procedure have been proposed, such as multiple comparison tests (Anderson 1996; Davidson and Duclos 2000) or Kolmogorov–Smirnov-type tests (Barrett and Donald 2003; Bennett 2010; Linton, Massoumi, and Whang 2005; McFadden 1989). Multiple comparison approaches are based on arbitrarily chosen ordinates, which can lead to test inconsistency (Barrett and Donald 2003; Davidson and Duclos 2000). Therefore, in this study we use a Kolmogorov–Smirnov-type statistic that compares all objects within the support of the two distributions.

Let Z be defined as the union of the supports of A and B. Define the following functionals for each order s

(4)

(5)

Notice that that the null hypotheses can be rewritten in terms of these functionals. That is, the null of increased diet quality (F_B dominating F_A) at order s is simply , and similarly for decreased diet quality (F_A dominating F_B) using . When the distributions are mutually independent, Kolmogorov–Smirnov-type tests based on are consistent (McFadden 1989).⁹ Test statistics are calculated using

(6)

Because there are infinitely many F_A(z) satisfying the null such that F_B(z)≤F_A(z), the limiting null distribution is not uniquely defined and depends on the underlying unknown distributions of F_A and F_B. We follow Barrett and Donald (2003) and use the least favorable configuration to construct the null distribution. The least favorable configuration is the point in the null distribution that is least favorable to the alternative hypothesis (i.e., F_A=F_B). As a result, the test is conservative; rejection of the null under the least favorable configuration implies rejection at any point in the null distribution. We construct a bootstrap distribution of to simulate the p values.

We use a recentering bootstrap approach, which has been shown to perform well against alternative methods (see Barrett and Donald (2003); Linton, Maasoumi, and Whang (2005)). Let be defined as above from (2) but computed on a random bootstrap sample drawn with replacement from distribution j.¹⁰ The statistic is recentered by the observed values so that we have . We can then define recentered bootstrap functionals by replacing > with in (4) and (5). The recentered bootstrap t statistics are

(7)

We approximate p values from the distribution of bootstrapped test statistics by

(8)

The p values allow for a test of stochastic dominance at order s based on the rule “reject if ” where α represents the conventional levels of statistical significance. Thus under the Bishop, Formby, and Thistle (1989) procedure, rejection of the null in favor of coupled with a failure to reject is viewed as statistical evidence in favor of F_B dominated F_A at order s.

Robustness Check for FOSD

To determine the stochastic rankings of two distributions, we must distinguish between four possible true states of nature: the distributions are equal, A lies above B, A lies below B, or the curves cross. The Bishop, Formby, and Thistle (1989) procedure described above distinguishes between these four states by conducting two one-sided tests. The result is lower power in detecting a crossing of the CDFs, which could lead to overclassification of dominance (Dardanoni and Forcian 1999; Gastwirth and Nayak 1999). This is at least partially due to the fact that rejection of or by itself is consistent with both FOSD and a crossing—hence, the use of two one-sided tests to rule out the crossing under the Bishop, Formby, and Thistle (1989) procedure.

A second drawback to the Bishop, Formby, and Thistle (1989) procedure is how the total error probability α is apportioned to each one-sided test (Dardanoni and Forcian 1999). As is typical with standard hypothesis testing, the one-sided critical value c(α) is based on ensuring that the probability of committing a type I error (i.e., rejecting , , or both when they are true) is less than the nominal level α. But as noted by Dardanoni and Forcian (1999), the Bishop, Formby, and Thistle (1989) procedure does not allow one to control how the total error probability α is allocated to each classification (equality, dominance in one direction, dominance in the opposite direction, or a crossing).

 Bennett (2013) improves on the Bishop, Formby, and Thistle (1989) procedure by writing it as a two-stage test that allows one to test for a crossing while giving the researcher flexibility in allocating the total error rate to each stage. Let α and β denote a pair of prespecified significance levels for the first and second stage, respectively. The first stage is to posit a null of equality (F_A=F_B) and determine rejection or acceptance based on the critical value a(α). If we accept the null, then we infer that the distributions are indistinguishable. Upon rejection, however, the second stage determines the state of nature among the three alternatives (A dominates B, B dominates A, or they cross) using the critical value b(α,β). This allows β to be the portion of the total error probability α allocated to a crossing (i.e., αβ) and the remaining α(1−β) is split evenly between dominance in either direction.

 Bennett (2013) tabulates asymptotic critical values of a(α) and b(α,β) for frequently used significant levels. In the applications below, we wish to calculate the asymptotic p values. To do so, we need to preset the total error rate α, the level at which we are controlling for falsely rejecting equality. We use two levels of significance (10% and 1%) so that the second stage is robust to our choice of α. The associated a(α) critical values are a(0.1)=1.2239 and a(0.01)=1.6277 (see table 1 in Bennett 2013).

The maximum of the one-sided test statistics found in (6) is used in the first stage, and the minimum is used in the second stage. To simplify notation, let these statistics be and , respectively. We are interested in the distribution of K_min conditional on rejecting equality. In other words, if K_min is large enough (i.e., larger than the second stage critical value b(α,β)) conditional on K_max>a(α), then we reject the null in favor of a crossing. Asymptotically, as shown in proposition 2.6 in Bennett (2013), if F_A=F_B and b<a then

(9)

where¹¹

(10)

(11)

(12)

The two-stage p values (denoted ) are calculated from (9) where we use two levels of α. Thus, a value below conventional levels of significance is evidence that the distributions cross. Put differently, larger p values are consistent with the null hypothesis that the distributions do not cross.

Between Periods

Figure 2 shows the empirical CDFs for the U.S. adult population in each period. Distributions shift systematically to the right over time, in other words toward a more nutritious diet. Because the shifts are relatively small, in this and subsequent figures, we present the estimated difference between the earliest period (1989–91) and the latest period (2005–08) in a subpanel. The area under the difference curve in the subpanel is equal to the area between the distributions. We can see the twenty-year improvement was positive and pointwise statistically significant for the empirically relevant range of HEI scores.

From the first three rows of table 3, we see that in comparing the period 1989–91 to all subsequent periods, the null of decreasing dietary quality is strongly rejected and in no case do we reject the null of an increase in diet quality. In comparing the periods 1994–96 and 2001–04, we are unable to order the distributions in either the first- or second-order case. We do find strong evidence that the period 2005–08 first-order stochastically dominates the period 1994–96, but the results are fairly weak with regards to an ordering of the periods 2005–08 and 2001–04.

Some care is required in interpreting the last two columns of table 3 because they report results from Bennet's two-stage test. As noted, these p values are for the null hypothesis that the CDFs do not cross, as determined by both K_max and K_max being statistically large. Loosely speaking, these can be interpreted as the (conditional) probability of rejecting the hypothesis of no crossing. Thus, a value below conventional levels of significance can be interpreted as evidence that the distributions cross. Bennet's two-stage test supports the main findings in that there is no statistical evidence that the 1989–91 distribution crosses any of the later years.

Between Income Groups

We now turn our attention to direct comparisons of individuals above and below 185% of the poverty guideline. As noted, we choose 185% of the poverty line as our cutoff because it is an upper limit on the threshold for many federal nutrition assistance programs.¹² Panel (a) of figure 3 presents the empirical CDFs and the difference between the periods 1989–91 and 2005–08 for low-income individuals; panel (b) likewise for higher-income individuals. Table 4 presents results from statistical tests of dominance by income group. For both groups, we find strong evidence that the distribution of dietary quality in the period 2005–08 first-order stochastically dominates the distribution in the earliest period, with no evidence of a crossing.¹³

Results support the observation in table 2 that a significant portion of dietary improvement among low-income individuals occurred over the period 2001–08. For example, in comparing the period 1989–91 to the period 1994–96, we find no evidence of a partial ordering according to the bootstrap results, and the two-stage test confirms this by finding significant evidence of a crossing. In comparing the period 1989–91 to the period 2001–04, again the bootstrap results are silent on the ordering, as is the asymptotic test, indicating no dominance at orders 1 or 2. However, in comparing the most recent time period 2005–08 to any of the earlier distributions, all tests show a statistically significant, first-order improvement in dietary quality, with no evidence of a crossing.

Comparing distributions among higher-income individuals, we can see that the period 1989–91 is first-order dominated by each subsequent period, with no evidence of a crossing. Comparing the period 1994–96 to the period 2001–04, the bootstrap results indicate a weak rejection of , which could lead one to infer a partial ordering. However, when consulting the asymptotic two-stage test, we find significant evidence of a crossing, thereby ruling out first-order dominance. We do see that the period 2005–08 first-order stochastically dominates the period 1994–96, but we cannot rank the two most recent time periods.

We can compare the total twenty year improvements in each income group by examining subpanels (a) and (b) in figure 3. We see that low-income individuals experienced relatively smaller increases over the bottom tail of the relevant range of HEI compared with their higher-income counterparts. We can more formally investigate this finding by taking the difference (between above and below 185% of the poverty line) in the differences (between the earlier and later periods). Figure 4 superimposes the subfigures in panels (a) and (b) of figure 3 in the top panel and then plots the difference between the two in the bottom panel—that is, in the subpanel of figure 4 we plot:

(13)

As shown in figure 4, considering lower levels of dietary quality below a HEI of 45, we find higher-income individuals experienced a greater improvement over the period 1989–2008 than low-income individuals. Whereas at higher levels of the HEI distribution, low-income individuals experienced greater increases in dietary quality. In other words, we find some evidence that within the poor dietary quality population, low-income individuals experienced less improvement over the 20-year period than higher-income individuals.

Rate and Location of Change

Given the differential gains in dietary quality noted above, we now investigate when in time and where in the distribution of dietary quality these improvements took place. For consistency and cross-sample/population comparisons, we focus on fixed portions of the distribution of dietary quality. An obvious choice is to use quartiles, which are all roughly segmented by HEI scores of 40, 50, and 60.¹⁴ Table 5 measures the amount of dietary improvement occurring in a particular quartile between two time periods as the percentage of total improvement . That is, we measure the area bounded by the two empirical CDFs within each quartile range of the HEI scores. For example, the percentage of improvement in the United States over the 20-year period that occurred in the bottom quartile (<40) between 1989–91 and 1994–96 was 2.8%. The last column of table 5 measures the overall improvements over the period 1989–2008 within each quartile of the distribution of dietary quality.

For the U.S. adult population, improvements below the median (HEI <50) occurred steadily over the period 1989–2008. In the upper range of dietary quality (HEI above 50), however, virtually all of the gains occurred over the periods 1989–96 and 2001–08. Overall, there were slightly higher gains in the upper quartiles compared to the lower quartiles for the U.S. population.

Comparing the between-period improvements by income group, we see that 71.5% of the total improvement in the diets of the low-income population occurred more recently over the period 2001–08. This is in contrast to the higher-income population, which saw the majority of their improvements occurring over the period 1989–2001 (77.1%). Improvements in the lower quartiles for the higher-income population have been relatively steady over the 20-year period, whereas most of the improvement in low-income diets within the lower quartiles occurred more recently over the period 1994–2008. In other words, at the lower end of the distribution of dietary quality, low-income individuals have seen comparatively limited or lagging improvements.

Table 5 emphasizes the reasons for targeting the most vulnerable group at risk of poor diets—the low-income, low–dietary quality population. This is best seen by examining the last column of table 5, which measures the total gains over the 20-year period within each quartile. The higher-income population has had almost proportional gains across all levels of HEI, whereas the low-income population has seen less improvement in the lower quartiles of diet quality.

Food Reformulation

The composition of the food supply has changed considerably over the last twenty years in response to changes in policy, regulation, technology, and consumer tastes. For example, Vesper et al. 2012 found that levels of transfats in the population declined after new labeling requirements were put in place in 2003. We now investigate how much of the improvement in dietary quality can be attributed to changes in food composition.

To identify foods and food mixtures that have undergone food reformulation (e.g., changes in the type of fat used in processed foods), we use the USDA Food and Nutrient Database for Dietary Studies (FNDDS). FNDDS consists of a series of databases updated every two years in conjunction with the continuous waves of NHANES to reflect the current state of food formulation and packaging. We combine the FNDDS to cover the period 1994–2008. We briefly explain the method here, with more details in the online supplementary appendix.

To construct the distribution of dietary quality in the period 1989–91 as if food were formulated in the period 2005–2008, we first identify all foods coded as reformulated in the 1994–2008 FNDDS. We then replace the nutrient values for these food items in the 1989–91 CSFII with the reformulated values found in the FDNNS. We also replace the MPED values of the 1989–91 reformulated foods with their 2005–08 values. We then construct a new HEI score based on updated nutrient and MPED values for each respondent in the 1989–91 sample. Figure 5 displays the results from the reformulation counterfactual, as well as results from the next section.

The distribution of HEI that accounts for reformulation lies everywhere to the right of the original 1989–91 distribution over the relevant range of the HEI. The implication is that, holding food choices constant, changes in food composition could be a contributing factor to dietary improvement. In figure 5, the indicated shaded area represents the change in the empirical CDFs attributed to reformulation. The ratio of this area to the total area provides a scalar measure of change. Here, improvements attributed to reformulation represent about 10.1% of the total difference between the period 1989–91 and the period 2005–08.

An important caveat is that this exercise captures partial equilibrium effects, and some care must be taken interpreting these results. Our counterfactual analysis cannot account for the fact that individuals in the period 1989–91 might have chosen different foods had their foods been formulated as they were in the period 2005–08. Nevertheless, it shows how food reformulation, all else equal, can play an important role in changing dietary quality.

Demographic Changes

The United States of 2005–08 is an older, more diverse, and better educated country than the United States of 1989–91. To the extent that these factors are correlated with healthy eating, they may explain some of the improvements in dietary quality. Table 6 illustrates demographic changes using data from our sample and from the U.S. Census. There is a clear decrease in the population aged 30–44 years and a concomitant rise in the population aged 45–64 years. The decrease in the non-Hispanic white population has come from an increase in the Hispanic and other race/ethnicity groups. Finally, the overall educational attainment in the population has also increased.

To investigate the effect of evolving population characteristics, we construct counterfactual distributions of HEI scores following an approach proposed by DiNardo, Fortin, and Lemieux. We ask, “What would the distribution of HEI scores look like had the demographic landscape of 2005–08 prevailed in 1989–91?” We focus on age, race/ethnicity, and educational attainment, all of which have been found to be correlated with diet healthfulness (Popkin, Siega-Riz, and Haines 1996). The intuition is to adjust each individual's sampling weight in the base period 1989–91 conditional on a set of demographics such that it captures the relative probability that the individual would be represented in the more recent 2005–08 sample.

To briefly describe the DiNardo, Fortin, and Lemieux 1996 methodology, let each individual observation be a vector (y, h, t), where y is HEI, h is vector of demographic characteristics, and t is time. Thus, all individuals belong to the joint distribution F(y,h,t). The static joint distribution of HEI and demographics in time t is F(y,h|t). The density of HEI at any point in time f_t(y) can be written as the integral of the HEI density conditional on a set of demographics f(y|h,t_y) at a specific date t_y, over the distribution of demographics F(h|t_h) at date t_h

(14)

where Ω_h is the domain of individual demographics. Therefore, our question posed earlier can be written with the above notation as the density of HEI scores in 1989–91 had the 2005–08 demographic landscape prevailed: f(y;t_y=89,t_h=08). This density is written as

(15)

where ψ(h) is a reweighting function defined as ψ(h)=dF(h|t_h=08)/dF(h|t_h=89). Applying Bayes's rule to the function, we can rewrite ψ(h) as

(16)

To obtain an estimate , notice the conditional probabilities Pr(t_h=t|h) can be estimated using a probit model by pooling the data and estimating the probability an individual is observed in time t conditional on a set of characteristics. Because we only compare two dates, the unconditional probabilities Pr(t_h=t) are simply the weighted sums of individuals in period t_h over the weighted sums of individuals in both periods. Because we are interested in applying the above methodology to tests of stochastic dominance, we replace an individual's sampling weight θ_i with in equation (2).

Although long-run demographic changes such as sex, age, and race/ethnicity are plausibly exogenous, the claim that education is uncorrelated with omitted factors that affect diet quality is less plausible. However, we are interested in how changes in the distribution of education affects changes in diet quality, rather than how education affects diet quality. In other words, the conditional independence assumption E[ε|h]=0 is unnecessary for our decompositional analysis. Rather, we only need the weaker assumption of ignorability (also called unconfoundedness or selection on observables) to compute the aggregate compositional effects of all demographics. Ignorability asserts that the correlation between education (or any variable in h) and the error term is the same in both periods.¹⁶

Because of the aggregate decompositional nature of the DiNardo, Fortin, and Lemieux methodology (as opposed to a Oaxaca-style decomposition), the reweighting function ψ(h) does not distinguish between individual variables in the vector h. In the interests of transparency, we construct the counterfactual distributions in two stages. First, we construct a counterfactual distribution accounting for purely demographic changes (sex, age, and race/ethnicity) and denote this reweighting function by ψ_h. We then construct a counterfactual distribution accounting for changes in demographics and changes in education levels, denoted by ψ_h,e.¹⁷ We investigate the effects of the ordering herein.

Figure 5 decomposes the change in the distribution of HEI into four main parts: improvements attributed to reformulation (as shown in the previous section); additional improvements attributed to changes in demographics, with and without education; and, finally, the residual change. As noted above, 10.1% of total improvement can be attributed to changes in food composition. Here we find that roughly equal proportions of the total improvement in HEI scores can be attributed to changes in sex, age, and race/ethnicity (26.6%) and education (26.7%) over the twenty-year period.¹⁸ This leaves 36.6% of the improvement unexplained by reformulation and demographics (i.e., the residual improvement). The residual improvement encompasses many competing factors, such as changes in tastes, relative food prices, scientific discovery, and attitudes toward food in general.

As above, care must be taken in interpreting these results. One important limitation of the partial equilibrium nature of the counterfactual analysis is that food choices in the counterfactual population would not affect the set of foods made available by food manufacturers. Although this assumption is economically unappealing, the exercise provides insight into the effects of changing demographics on diet quality via clear and tractable analytical techniques.

Counterfactuals by Income Group

The counterfactual analyses above suggest that an important part of the improvement in dietary quality can be attributed to changes in food composition and demographics. Given that improvements occurred at different rates for different parts of the HEI distribution for lower-income versus higher-income individuals, we now ask whether changes in food composition and demographics account for differing amounts of improvement by income group. Results are presented in figure 6.

Changes in food composition account for a substantially larger percentage of the dietary improvement for lower-income individuals (19.6%) compared with their higher-income counterparts (6.4%). This is consistent with the observation that low-income individuals eat more processed foods (Drewnowski and Barratt-Fornell 2004), where much of the reformulation is occurring. Changes in sex, age, and race/ethnicity account for a similar share of the improvement for low-income (25.3%) versus higher-income individuals (26.8%). For low-income individuals, changes in educational attainment account for half that of higher-income individuals (13.5% versus 27.0%). The remaining residual share of the twenty-year improvement is larger within the low-income population (41.6%) than the higher-income population (39.8%). This suggests that further research into the determinants of diet quality of low-income individuals may be warranted.

Robustness

The order in which we construct counterfactual distributions using the DiNardo, Fortin, and Lemieux (1996) approach can influence the results. To investigate the robustness of our findings to ordering, we estimate the model using an alternative ordering for each of the three population groups of interest (total population, low-income, higher-income). Note that because reformulation is not estimated, but rather derived from data, it does not matter in which order it is considered. Furthermore, the total aggregate effect (ψ_h,e) remains the same as well. For example, in either case, all demographics account for 53.3% of the total improvement within the U.S. population.

Table 7 provides estimates for the original order as presented above, as well as an alternative ordering where we first consider educational attainment ψ_e and then use ψ_h,e as before. The result places bounds on the magnitude for each set of demographics. For example, the effect of education ranges between 15.6% and 26.7% for the total population, 5.0% and 13.5% for the low-income group, and 16.0% and 27.0% for the higher income group. Although point estimates change, relative comparisons remain substantively the same—changes in education appear to account for a larger share of the improvement for the higher-income group relative to the lower-income group. We note that the bounds are relatively large, and credibly point-identifying each effect remains a task for future work.