A Simulation Study of Quantitative Risk Assessment for Bivariate Continuous Outcomes

2.1. Dose-Response Modeling

We assume F and H (or some monotonic transformation of them) are distributed bivariate normal with means (μ_F, μ_H), variances (σ²_F, σ²_H), and bivariate correlation parameter ρ:

Dose-response models may be specified for all parameters. The means μ_F and μ_H may be modeled as functions of dose and other covariates. For dose-response and adverse event data in particular, responses at different doses are likely to have changes in scale and, for bivariate data, changes in correlation. Thus, it may be appropriate to model the variances σ²_F and σ²_H and bivariate correlation ρ as functions of dose, choices that a likelihood model easily accommodates. We choose a log link for positivity on the variance estimation and a Fisher's Z transformation for the correlation parameter to restrict the parameters between −1 and 1. Thus, we may express the dose-response parameters as:

where are all N×p_a, N×p_b, N×p_e, N×p_f, N×p_g matrices of covariates corresponding to parameter vectors {a, b, e, f, g}, respectively. In controlled dose-response studies, often will be simply {[1 d], …, [1 d]}, where d is the N× 1 vector of doses or some transformation of the dose metric.

The method of maximum likelihood allows straightforward estimation of parameters. The score equations are obtained by taking the derivatives of the log likelihood of the above bivariate distribution. The final regression parameter estimates are easily obtained using a quasi-Newton algorithm. We used the nlminb function in Splus for this purpose.

2.2. BMD Calculations

While quantitative risk assessment for bivariate data could proceed by finding BMDs for each univariate outcome separately using the above model, a joint BMD may be preferred. Topical literature has emphasized the importance of considering multiple outcomes when assessing risk (Ryan, 1992). A BMD calculated from joint risk may encompass greater overall sensitivity than computing BMDs from each individual outcome alone. One method of joint risk assessment for two outcomes, as described in Regan and Catalano (1999a, 1999b), is to consider separate fixed cutpoints for adversity for each of the two responses, and to define the probability of an adverse outcome using those cutpoints according to the methods just described.

We combine elements of their approach and the percentile regression method of Yu (2002) for the bivariate continuous responses of hindlimb and forelimb grip strengths. To establish a definition of adversity, we propose letting c_F and c_H be cutoffs for forelimb grip strength and hindlimb grip strength, respectively, where c_F and c_H are variable (estimated) rather than fixed. That is, these cutoffs are not determined by the investigator, but by the correlation in unexposed subjects and the selected value of u, as one will see later in this section. More specifically, we find the values c_F and c_H that define the lower u× 100% (for example, the lower 5%) of the modeled bivariate distribution in the control group, such that P(F < c_F or H < c_H) =u (Fig. 1), with the added constraint of:

(i.e., equal marginal probabilities). This constraint provides equal weighting of F and H in the risk assessment. Without the constraint, many combinations of (c_F, c_H) satisfy u=P(F < c_F or H < c_H); a high value for c_F and a low value for c_H, or vice-versa, may produce the same u and be arbitrarily (or differentially) weighted.

The determination of c was best kept as a direct analogy to the univariate case. In general, the probability of the union of two events is greater than that of each marginal event; the corresponding adjustments to the cutoffs to adjust the overall probability of adversity result in the definition of c. The notion of a percentile in a bivariate distribution usually refers to contours, which contain a certain percentage of the probability under the bivariate normal surface. However, the lower probabilities defined as such are not consistent with the definitions of adversity in the univariate setting; adverse forelimb strength and adverse hindlimb strength in our example (for example “low” forelimb strength and “low” hindlimb strength) are lower tail values in the respective univariate distributions. For a bivariate distribution, the same or similar values are located on the extreme right and bottom sides of the distribution, as indicated by the shaded regions in Fig. 1.

Given our definition of c, the probability of an adverse event at a given dose d_i is therefore

(1)

and is expressed conveniently as . For the control group at d= 0, the probability of an adverse outcome reduces to 2Φ(c) −Φ₂(c). Note that c is dependent only upon ρ(0), the correlation in unexposed subjects, and the selected value of u. In our example, P(d) is the probability of either low forelimb strength or low hindlimb strength at dose d.

The risk function, r(d), may be written as r(d) =P(d) −P(0) for additional risk, or for extra risk. The is the dose satisfying , where P(0) =u.

Fig. 1 illustrates a bivariate dose response. Each oval represents a contour of a three-dimensional bivariate normal surface at the respective mean response and correlation ρ. The mean responses decrease with dose while the correlation increases with dose, as shown by the “narrowing” of each contour with increasing dose. Shaded areas generally represent the region of adversity defined by the cutoffs c_F and c_H; note that the region of adversity includes points beyond the contour's boundary, as adversity is defined by the cutoff values (dotted lines). For a 5% background risk and a 5% additional risk, for example, the BMD is the dose where this area of adversity equals 0.10.

We present the case for two positively correlated outcomes where adversity is defined by decreasing values. When adversity is defined by increasing levels—for example, lead levels in blood—the region(s) of adversity and calculation(s) for risk would differ accordingly.

2.3. Lower Limit Calculations

A 95% lower bound on the BMD based on asymptotic normality using the delta method is , where and β={a, b, e, f, g} is the vector of dose-response parameters. Because the BMD_q is obtained numerically, may be more conveniently obtained as . Since contains the estimated parameters used to calculate the cutoffs c_F and c_H, the BMDL calculation accounts for the variability of the background percentage. This feature is an advantage over the alternative of using a fixed cutoff or estimating the cutpoint but not taking the estimate into account in the BMDL, as in previous methods.

Crump and Howe (1985) point out that confidence limits based on the delta method may possess certain drawbacks such as negative values or failure to maintain nominal coverage. An alternative, more attractive approach is to construct a confidence interval based on the asymptotic distribution of the likelihood-ratio statistic (Chen & Kodell, 1989). Using this method, a BMDL is determined by finding alternative estimates of the parameters that satisfy r(d) =q, minimizing the BMD, and are subject to the constraint of . The method is implemented by modifying a Nelder-Mead simplex algorithm to incorporate the constraint. Although more computationally intensive, the advantages of the likelihood-ratio method make its use preferable to the delta method. The confidence limits are invariant under parameter transformations and maintain better nominal coverage. Its limitation is that it requires a likelihood. While one could use GEE methods, for example, for estimation if a likelihood were not present or did not have justification, calculations for a corresponding, non-delta method BMDL are not straightforward.

3.1. Parathion Data

Our example comes from a study evaluating the effects of parathion in rats sponsored by the World Health Organization and the International Programme on Chemical Safety (Moser, Tilson, & MacPhail, 1997). Parathion, a neurotoxic agent used as an insecticide, inhibits cholinesterase, an enzyme that is necessary for proper functioning of the nervous system in humans. For this example, we focus on the neuromuscular outcomes of forelimb and hindlimb grip strength.

As for typical neurotoxicity studies examining acute effects, the measurements for this study were taken at baseline, time of expected peak effect, one day, and one week. We are interested primarily in the responses at the time of peak effect. The data are pooled from seven different labs conducting the same experimental protocol. All laboratories had a single control group and four dosed groups, where each lab separately determined the maximum tolerated dose (MTD) from which it chose the three other lower doses. Thus, the dosed groups differed between labs. To control for intraanimal and interlab variability, we consider percent change in grip strength from baseline by subtracting the response at baseline from the response at time of peak effect, dividing by the baseline measurement, and multiplying by 100. Table I displays a summary of grip strength changes from baseline by dose group.

3.2. Dose-Response Modeling

The means for both types of grip strength changes generally decrease with dose; that is, as dose increases, grip strengths at peak times become weaker than those at baseline (Table I). To demonstrate the flexibility of the model and to account for possible heteroscedasticity, we allow the variances to vary linearly with dose. We choose to model the following dose-response trends:

where d_i is the dose administered to animal i. The parameter estimates, listed in Table II, are consistent with the raw data seen in Table I. Fitted means for changes in forelimb grip percent change decrease with dose (12% and −29% in controls and highest dose), and those for hindlimb grip percent change also decrease (3% and −26% in controls and highest dose). The fitted standard deviations for changes in forelimb grip percent change increase slightly with dose (20% to 34% in controls to highest dose), while those for changes in hindlimb grip change do not significantly vary with dose (20% to 21% in control and highest dose groups). The bivariate correlation ρ is positive, as expected, and stays relatively constant (0.10 to 0.20 in controls and highest dose groups).

Table II. Parameter Estimates for the Parathion Data, Percent Changes from Baseline for Forelimb and Hindlimb Grip Strength Outcomes

Model Parameter	Estimate	Standard Error	Z
Forelimb grip strength mean (μF)
Intercept	12.3	1.59	7.7
Dose	−0.041	0.64	−6.4
Hindlimb grip strength mean (μH)
Intercept	2.73	1.46	1.9
Dose	−2.75	0.50	−5.5
Forelimb grip strength SD (ln (σF))
Intercept	6.02	0.099	60.9
Dose	0.101	0.031	3.2
Hindlimb grip strength SD (ln (σH))
Intercept	5.96	0.104	57.6
Dose	0.014	0.035	0.4
Bivariate correlation
Intercept	0.194	0.140	1.4
Dose	0.021	0.045	0.5

3.3. Quantitative Risk Assessment

We used our methods to compute bivariate BMDs for 5% and 10% additional risks at different background rates of 1%, 5%, and 10%. BMDLs were computed using the delta method and the likelihood-ratio approach (Table III).

To define the joint probability of an adverse effect (large negative change in forelimb or hindlimb grip strength), we specify a background rate from which to derive cutoffs. For example, an overall background rate of 5% defines the cutoffs for percent changes in forelimb and hindlimb grip strength as −27 and −36, respectively, where the area below these cutoffs in the bivariate distribution equals 5%. Those values in the area defined by percent changes in forelimb grip strength that are less than −27 or percent changes in hindlimb grip strength less than −36 are adverse. The corresponding value of is −1.95, with a marginal probability of 0.026 for each outcome; the estimated correlation in controls is approximately 0.1. The cutoffs for each outcome vary if we choose different background rates. For example, at 1%, the cutoffs are (−40, −48) with and marginal probabilities of 0.0051; at 10%, the cutoffs are (−21, −29) with and marginal probabilities of 0.052.

While the region of adversity is 100u% of the modeled bivariate surface, the actual percentage of points in the area of adversity is dependent on the fit of the model. A rough goodness-of-fit test may be implemented by finding the percentage of points lying in the modeled region of adversity. For our example, using a nominal 5% background rate, 6.8% of the actual points lie in the area of adversity as defined by our model. For a 10% background rate, 12.2% of the points lie in the adverse area; for a 1% background rate, none of the points lie in the adverse area. The simulations and discussion further address issues of appropriate background rates.

Continuing with our illustration using a 5% additional risk (r(d) = 0.05), we may estimate a BMD₀₅. We specified the risk of an adverse effect in the control group to be 5%, which we denote as P_u(0) = 0.05. The dose satisfying the equation r(d) =P_v(d) −P_u(0) is P_v(d) = 0.10. As reported in Column 9 of Table III, this dose is the . The corresponding BMDL ₀₅ values based on the delta method (dm) and likelihood-ratio (lr) methods are equal, 1.18 mg/kg.

We wish to compare the bivariate results to the univariate cases by computing separate BMDs and 95% BMDLs for changes in forelimb and hindlimb strengths. To do so, we fit linear mean percentile regression dose-response models for both outcomes, allowing the variances for each outcome to vary linearly by dose to account for possible heterogeneity.

As for the bivariate case, we computed univariate BMDs for 5% and 10% additional risks, at background rates of 1%, 5%, and 10% for both outcomes. Fig. 2 depicts an additional risk BMD₀₅ with background percentile of 5%. For changes in forelimb grip strength, the is 1.34 mg/kg with BMDL ₀₅(dm) = 1.05 mg/kg and BMDL ₀₅(lr) = 1.06. The results for changes in hindlimb grip strength are with BMDL ₀₅(dm) = 1.56 mg/kg and BMDL ₀₅(lr) = 1.70 mg/kg (Table III).

The bivariate fall between the two univariate but are closest to the more conservative univariate (in this case, the forelimb grip strength change ). This result is consistent with the explanation in Yu and Catalano (2005) for a bivariate BMD calculated with a jointly defined background rate as described.

5.1. Linear Normal Results

Table IV presents summary statistics for estimated BMD and BMDL distributions of generated normal linear data and linear fit for true ρ= 0. Corresponding results for ρ= 0.35 and dose-varying ρ appear in Tables V and VI, respectively.

In general, the distributions are affected in similar ways by increasing sample size and background rate: they become tighter over all independent and correlated simulation scenarios, the mean and median of the distributions grow closer to the true BMD, and the true BMD itself becomes smaller and closer to zero. For example, at ρ= 0.35, at background rates of 1%, 5%, and 10%, the true BMDs are 14.32, 6.67, and 4.47, respectively.

These general trends are similar for ρ= 0 and for dose-varying ρ. In particular, distributions at the 1% background rate tend to be wider than those at the 5% and 10% background rates. The high variability is apparent when viewing density plots of the distributions (Fig. 4), where a vertical line indicates the true BMD for a specific scenario. An interesting feature on plots 4d and 4g, where the background rate is 1% and n_i= 10, is a small “spike” at the maximum dose of 30. The reason for the spike, which indicates greater spread, is that our algorithm for computing the BMD limits the to the given range of doses in the study. If the estimated BMD numerically exceeds the last dose, the algorithm sets it equal to the maximum dose; a consequence is that the means of the simulated distributions are less meaningful. This “bumping up” against the maximum dose disappears with increasing sample size and background percentile because both factors decrease the variability of the distribution.

As expected, increasing the bivariate correlation results in slightly larger BMDs. A heuristic argument is that less correlated outcomes are “more independent” and result in lower, more conservative BMDs to protect against both outcomes.

Fig. 5 displays boxplots of the estimated BMD, BMDL(dm), and BMDL(lr) distributions for two scenarios. A horizontal line represents the value of the true BMD. Both graphs are for dose-varying ρ and 10% background rate, but the sample sizes per group differ. The results are typical for the different scenarios in the simulation. The BMDL(dm) boxplot for n_i= 10 in Fig. 5a displays the known tendency of BMDLs computed using the delta method to produce negative values, while the BMDL(lr) distribution remains within the range of doses. As expected, the problem is less severe as the sample size increases above 10 animals per dose (Fig. 5b). However, it indicates that in general the delta method may not be as reliable as the likelihood-based method for finding a BMDL. In general, BMDLs based on the delta method are more variable and also have poor coverage.

Coverage probabilities indicate that the BMDL(dm) is extremely conservative, especially at the smaller background rates of 1% and 5%. Across all scenarios, the coverage probabilities for BMDL(dm) are wider than those for BMDL(lr). With increasing sample size, the BMDL(dm) grows closer to a 95% coverage probability, but the BMDL(lr) universally has coverage closer to the nominal 95%. In addition, the empirical 5% of the distribution is closer to the median of the BMDL distribution with increasing sample size and background rate (Tables IV–VI). The differences for BMDL(dm) are generally higher while corresponding differences of BMDL(lr) are fairly consistent.

The differences, especially over the three background rates, further suggest that the delta method is less adequate than the likelihood-based method for calculating the BMDL. The information also indicates that both higher sample sizes and background rates greater than 1% are more reliable for calculating the BMDL and that the BMDL(lr) is preferable over the BMDL(dm). Such results are especially interesting given that the data follow a normal distribution.

5.2. Quadratic Normal Results

Since dose-response trends often possess some degree of curvature, we examine the properties of the BMD for a quadratic dose-response trend. We generated quadratic data and looked at two main cases: one in which the model fit is quadratic and the other in which the model fit is misspecified as linear.

To examine the properties of the BMD for a true quadratic trend and quadratic fit, we generated bivariate normal observations about means corresponding to a quadratic dose-response model with intercept (2, 2), dose coefficients (−0.078, −0.078), and quadratic coefficients (0.0013, 0.0013). As stated above, this quadratic trend was chosen to be similar to the linear trend. The exception is in the last dose group, where the response chosen provides an obvious departure from linearity (Fig. 3). Tables VII–IX summarize statistics for estimated BMD and BMDL distributions of generated normal quadratic data and quadratic fit.

The general trends for the effects of sample size and background rate on the BMD are similar to those from the normal linear results. The distributions are more variable at lower sample sizes and background rates and become tighter as both of these factors increase. Generally, the variability is higher for each scenario than for the corresponding linear trend scenario. The results are likely due to the sampling and estimation variability from the addition of a quadratic term. Similar to the linear trend results, the 1% background rate is inadequate for BMD estimation. The BMDs at the 1% background rate are highly variable for all specified values of ρ, with noticeable spikes at the maximum dose of 30, even for n_i = 50.

The quadratic simulation results suggest that the delta method is inadequate for calculating the BMDL, based on the large percentage of negative values for the BMDL(dm) for all sample sizes at the 1% background rate and for n_i= 10 and n_i= 20 at the 5% and 10% background rates. Some of the means and medians for the BMDL(dm) distributions in Tables VII–IX are negative. As a result, the differences between the empirical 5% of the distribution and the median of the BMDL(dm) distribution are much larger than those for the linear results. Coverage probabilities are close to 1 and do not grow closer to 95% with increasing sample size. As for the linear case, coverage probabilities are extremely wide for the BMDL(dm), whereas those for BMDL(lr) are much more consistent and closer to 95%; the BMDL(lr) is the preferable choice for a lower bound on the .

Because most dose-response trends tend to have some curvature, it is valuable to examine how assumed quadratic data behave under different models. To assess possible model misspecification, we first modeled the quadratic data under a linear assumption. Fig. 6 displays distributions at true ρ= 0.35, with a horizontal line at the true BMD. The results suggest that fitting a linear trend to the generated quadratic data is not recommended because the resulting are clearly biased away from the true BMD. The mean and median tend to be higher than the true BMD.

We also examined how the curvature in the quadratic trend affects the BMD by carefully choosing sampling distributions for the linear and quadratic dose-response trends. For the quadratic trend, the choice of means for the first three doses were similar to those for the linear trend. The main difference is the last dose, where the mean response clearly defines the curvature of the quadratic dose-response trend. To see how the curvature affects the BMD, we removed the last dose of the quadratic trend and sampled only from means of the first three dose groups defined for the quadratic trend. These data are then fit with an assumed linear trend. These distributions, also shown in Fig. 6, show that there is a gain in reducing bias; the BMD estimates are considerably closer to the true BMD from generated quadratic data and quadratic fit than the mean or median resulting from fitting a linear trend to generated quadratic data. These results show that the curvature of a nonlinear trend may greatly affect the BMD calculation and should be incorporated if possible. Fortunately, such trends are easily incorporated into dose-response models.

5.3. Sensitivity to Distributional Shape

The small samples typical of dose-response data may result in skewed, rather than symmetric, distributions of responses within a dose group. A way to examine the behavior of skewed distributions is to generate data from a bivariate lognormal distribution and analyze them under a normal distribution assumption.

To evaluate the sensitivity of our method to changes in distributional shape, we kept the same dose-response trends and bivariate correlations but generated a lognormal distribution about each mean. We chose means at (5, 5) to avoid distributions whose observations were primarily distributed along the x and y axes and variance parameters (2, 2) to provide a clear skewness to the distribution. The lognormal data were treated as normal for modeling and subsequent BMD calculation. Fig. 7 displays some of the simulation results for ρ= 0.35.

The true BMD in this case is one for lognormally generated data calculated using a lognormal assumption. The simulations show that the resulting , calculated under a normal assumption, are clearly biased away from the true BMD. A few of the distributions at the 5% and 10% background rates do not even cover the true BMD for those scenarios.

Misspecifying a distribution should be avoided if possible. Nonetheless, under true lognormality, a simple fix is available. Skewed data are common in bioassays, and researchers often will use a transformation of the data to make them conform to a normal distribution. In our case, taking the log of the observations corrects the problem, as seen in Fig. 8.