Some uses of orthogonal polynomials in statistical inference
Abstract
Every random variable (rv) X (or random vector) with finite moments generates a set of orthogonal polynomials, which can be used to obtain properties related to the distribution of X. This technique has been used in statistical inference, mainly connected to the exponential family of distributions. In this paper a review of some of its more relevant uses is provided. The first one deals with properties of expansions in terms of orthogonal polynomials for the Uniformly Minimum Variance Unbiased Estimator of a given parametric function, when sampling from a distribution in the Natural Exponential Family of distributions with Quadratic Variance Function. The second one compares two relevant methods, based on expansions in Laguerre polynomials, existing in the literature to approximate the distribution of linear combinations of independent chi-square variables.
1 INTRODUCTION
Every random variable (rv) X (or random vector) with finite moments generates a set of orthogonal polynomials (OPS), which can be used to obtain properties related to the distribution of X. This technique has been used in statistical inference, mainly connected to the exponential family of distributions, as it can be seen for instance in Abbey and David,1 Morris,2, 3 López-Blázquez,4 and Barranco-Chamorro and Moreno-Rebollo.5
Other uses in more general settings can be found in Voinov and Nikulin,6 in nonregular distributions in Barranco-Chamorro et al.,7 and in Bayesian statistics in Pommeret.8
More recent results include among others: applications in correspondence analysis Beh,9 D'Ambra et al.,10 and Beh and Lombard,11 birth and death processes Guillemin and Pinchon,12 goodness-of-fit tests for parametric regression models Bar-Hen and Daudin,13 inference in the exponential distribution based on k-sample doubly type-II censored data Sanjel and Balakrishnan,14 Gibbs sampling Diaconis et al.,15 multiple correspondence analysis for ordinal-scale variables Lombardo and Beh,16, 17 reweighted smooth tests of goodness of fit De Boeck et al.,18 study of dependence between ordinal-nominal categorical variables in Lombardo et al.,19 canonical correlations for Dirichlet measures in Griffiths and Spano,20 to adjust the hyperbolic secant and logistic distributions to analyze financial asset returns in Bagnato et al.,21 a general method of calculus in Withers and Nadarajah.22
All these references illustrate the potential interest of this methodology. In this paper we focus on two specific points of univariate and one-parameter case. These are properties of the Uniformly Minimum Variance Unbiased Estimator (UMVUE) of a given parametric function, when sampling from the natural exponential family of distributions with quadratic variance function, and approximations of distributions by using OPS, specifically, in the case in which we have a linear combination of independent chi-square variables. The aim of this paper is twofold. On the one hand, to provide a guide of use of these methods in Statistical Inference. On the other hand, to popularize these techniques. The classical systems of OPS are available in the current statistical software, such as Mathematica, so these methodologies are widely applicable.
2 UNIVARIATE AND ONE-PARAMETER CASE
That is, the expansion given in (1) is convergent in L2- sense.
- Quite often, is such that the OPS associated is well known. For instance, in the normal case we have the Hermite polynomials, for the gamma distribution the (generalized) Laguerre polynomials, for the Poisson distribution the Charlier polynomials.24
- To get manageable expressions for the coefficients .
Whenever that previous aims are fulfilled, expansions in terms of OPS can be used to propose approximations to T(x) and study features related to the quality of these approximation, as follows.
Lower and upper bounds for (5) can be proposed if we have closed expressions for and .
3 NATURAL EXPONENTIAL FAMILY WITH QUADRATIC VARIANCE FUNCTION
In this section we highlight some uses of expansions in terms of OPS when sampling from distributions in the one-parameter Natural Exponential Family with Quadratic Variance Function (NEF-QVF; see Morris2, 3).
The natural parameter space is the largest open set for which is finite, . It will be assumed that is nonempty.
(8) is the variance function.
Definition 1. (NEF-QVF(, ))NEF-QVF(, , refers to NEF of distributions whose variance function is, at most, a quadratic function of the mean
To the NEF-QVF belong the 6 one-parameter families listed in Table 1 (and linear functions of them). Details can be seen in Morris.2, 3
NEF-QVF distributions | |
---|---|
Normal , known, | (constant variance function) |
Poisson , | (linear variance function) |
Gamma , r known, , | (quadratic variance function) |
Binomial B(r, p), r known, , | (quadratic variance function) |
Negative binomial NB(r, p) , r known, , | (quadratic variance function) |
Generalized hyperbolic secant GHS, r known, , | (quadratic variance function) |
3.1 Results when there exists a sufficient and complete statistic
In this subsection we highlight those applications in which we have a sufficient and complete statistic for the parameter and the distributions are parameterized in terms of the mean, .
Additionally, if the srs, under consideration, is from a NEF-QVF(), then follows a NEF-QVF().
Lemma 1.The polynomials defined in (12) verify the following properties:
- (i)
pk, n is a polynomial in of degree k with leading term .
- (ii)
Orthogonality relation
()is the Kronecker delta, for j ≥ 1, and v2 is the coefficient of in (9). - (iii)
For any positive integers 1 ≤ m ≤ n
()
The OPS in the NEF-QVF are well known. They are listed in Table 2.
NEF-QVF distributions | |
---|---|
Normal , known | Hermite |
Poisson | Charlier |
Gamma , r known | (generalized) Laguerre |
Binomial B(r, p), r known | Kratchouk |
Negative binomial NB(r, p) , r known | Meixner of first kind |
Generalized hyperbolic secant GHS, r known | Pollaczek |
4 UMVUE IN THE NEF-QVF
- The variance of Tn(Sn).
- Lower bounds for the variance of Tn(Sn).
-
The effect of an observation on Tn(Sn).
Two robustness measures to assess the effect of a fixed observation, x ∈ support(X1), on Tn(Sn) were proposed in Barranco-Chamorro and Moreno-Rebollo.5 The measures are the conditional bias and the asymptotic mean sensitivity curve (AMSC) for Tn(Sn). They are based on the following relationship where we consider the conditional expectation of the UMVUE given an observation x, for simplicity it is assumed that X1 = x
()It was proven that the conditional bias and the AMSC for Tn(Sn) depend on the parametric function under consideration, h, evaluated at the true and unknown value of the parameter, . -
The limit distribution of Tn(Sn). In the NEF-QVF the limit behavior of the UMVUE of , Tn(Sn), depends on the order of the first nonzero derivative of h at the true and unknown value of the parameter . Specifically, let us denote by . Then
- (a)
If k0 = 1 then
()with . - (b)
For k0 > 1 the limit behavior depends on the OPS of degree k0. For instance, if k0 = 2 then
()
- (a)
-
Comparisons with the MLE of , , can be carried out by considering the expansion of in terms of the OPS and comparing coefficients in both expansions.
Additional details of these results and other uses can be seen in Abbey and David,1 López-Blázquez and Castaño-Martínez,4 and Barranco-Chamorro et al.,25 among others.
5 DISTRIBUTION OF A LINEAR COMBINATION OF INDEPENDENT CHI-SQUARE VARIABLES
The pdf and cdf of Qn can be obtained by using expansions in terms of Laguerre polynomials. In this section we briefly describe two of the most relevant methods existing in the literature to reach this aim: the method proposed in Castaño-Martínez and López-Blázquez28, 29 and the method proposed in Ha and Provost.30
5.1 Method 1
This method is based on the inverse Laplace transform and the property of the uniqueness of the UMVUE in the Gamma distribution. It was proposed in Castaño-Martínez and López-Blázquez.28, 29
Equations (21) and (22) were applied, as follows, in Castaño-Martínez and López-Blázquez,28 taking T(x) = f(x) the pdf of a quadratic form.
As summary, it can be said that the key aspects of this proposal are: Laguerre expansions for the pdf (and cdf) of a sum of weighted central independent chi-square variables are given. The formulae depend on certain parameters. Appropriate choices of them give well-known expressions in the literature. Some new expressions were also obtained. Upper bounds for the truncation errors of these expressions can be proposed. Castaño-Martínez and López-Blázquez28 include some examples with numerical results, which show that their upper bounds can be sharper than others previously proposed in literature.
5.2 Method 2
In this subsection, we will focus on the method proposed in Ha and Provost30 to approximate the distribution of linear combinations of independent chi-square r.v.'s. These authors proposed a moment-matching method, which is based in a gamma pdf adjusted by a linear combination of Laguerre polynomials. Next we briefly recall the main points we need to reproduce their methodology.
Recall that with this notation and .
Other basic properties, which must be taken into account, are:
5.2.1 Moment-matching method by using Laguerre polynomials.
Note that given in (28) is, basically, the scaled version of the gamma distribution considered in Step 0, that is, (28) is almost the pdf of and .
5.2.2 Some useful properties of (generalized) Laguerre polynomials.
To reproduce and become familiar with this method, some identities satisfied by the generalized Laguerre polynomials will be useful. Additional details can be seen in Shao et al.24, 33, 34
Next the generalized Laguerre polynomials are introduced and their most relevant properties for our purposes are listed.
The generalized Laguerre polynomial of degree with respect to the weight function with x > 0 and , will be denoted as .
Ha and Provost30 proposed two (equivalent) expressions to approximate the pdf of the quadratic form introduced in (26).
is the Laguerre approximant, which they propose to approximate the pdf of the quadratic form, .
In (31), we have that , (weight function of generalized Laguerre polynomials ). The coefficients are obtained from moments of , and the coefficients of the Laguerre polynomials of degrees: 0, 1, … , d.
As for bounds for the truncation error by using these approximations, although Ha and Provost provided bounds for the truncation error for the pdf's and cdf's. They said that their bounds are not very tight because their error bounds depend on the moments of the distribution being approximated.
As for the degree of the polynomial approximant, Ha and Provost30 propose to try with several values of d until they obtain close values (for instance for quantiles). To assess the performance of their approximations, they compare their proposals to the exact distribution (if it is possible), or to the simulated distribution.
5.2.3 Illustration
So, the first approximation is by using the distribution with and in (34).
(with ).
Note that , therefore (35) is a kind of correction to the initially proposed in which we impose that the d = 6 first noncentral moments of Z and the Laguerre approximant, Yd, agree. The explicit expression of Laguerre approximant for d = 6 in terms of Laguerre polynomials will follow straightforward.
The multiple plot for the pdf of the initial Gamma (black), and the approximants for d = 6 (red), and d = 14 (blue) are provided in Figure 1.

5.2.4 Comparison to Moschopoulos technique
One of the reviewers suggests to carry out a comparison of the results previously presented with the ones in Moschopoulos.35 Moschopoulos considers Z = Y1 + ⋯ + Yn with independent, and the shape and scale parameters, respectively. By inverting the moment generating function of Z, he obtained a single gamma series for the pdf and cdf of Z. Moschopoulos method is implemented in R package coga (Hu et al.36, 37). In Table 3, some quantiles are given for the illustration previously presented by using Ha and Provost (d = 6 and d = 14) and Moschopoulos methods. Results in Table 3 suggest that in order to apply Ha and Provost method several values of d must be tried until obtain a fixed accuracy. For this illustration, Ha and Provost30 proposed d = 14.
0.01 | 0.05 | 0.10 | 0.50 | |
---|---|---|---|---|
Ha and Provost (d = 6) | 1.92384 | 4.63033 | 6.83939 | 20.3014 |
Ha and Provost (d = 14) | 2.51869 | 5.04397 | 7.03708 | 20.0027 |
coga | 2.57955 | 5.04195 | 7.00921 | 20.04002 |
0.90 | 0.95 | 0.99 | ||
Ha and Provost (d = 6) | 49.0916 | 62.5418 | 91.4214 | |
Ha and Provost (d = 14) | 49.3561 | 61.8384 | 90.9503 | |
coga | 49.41845 | 61.89996 | 90.87084 |
- -
Ha and Provost approach is based on an initial gamma pdf adjusted by a linear combination of Laguerre polynomials (by using a moment-matching method). The parameters of the initial gamma pdf are taken in such a way that its expectation and variance agree with the expectation and variance of Z.
- -
On the other hand, Moschopoulos method is based on a single gamma series based on pdf's of gamma distributions with scale parameter and different shape parameters. The coefficients are computed by simple recursive relations.
These appreciations suggest that the performance of these methods may vary with the values of the parameters, and , i = 1, … , n. In this sense, it is worth mentioning that Hu et al.37 point out that Moschopoulos method computation is demanding when the variability of the scale parameters is large and the shape parameters are small.
6 CONCLUSIONS
The aim of this paper has been to illustrate certain uses of classical OPS's in Statistical Inference. Quite often, the density associated to the problem we are dealing with allows us to consider classical OPS's whose properties are well known. For the univariate and one-parameter case, the general method is described in Section 2.
On the one hand, Sections 3 and 4 are devoted to the NEF-QVF. To this family belong distributions such as the normal, Poisson, gamma, binomial, negative binomial and generalized secant hyperbolic. A pletora of results are listed in Section 4. All of them are based on the fact that given a srs, X1, … , Xn, of a NEF-QVF, the statistic is distributed as a NEF-QVF and the properties of the OPS's associated to the density of Sn. We highlight that the method is general and the OPS's we are dealing with are classical and well known (see Table 2).
On the other hand, Section 5 is devoted to different methods to approximate the distribution of linear combinations of independent chi-square variables (or equivalently, gamma distributions). They are based on generalized Laguerre polynomials. Get to know both methodologies can be useful to obtain properties in both fields of statistics, since the gamma distribution is one of the models included in the NEF-QVF, and the generalized Laguerre polynomials one of the OPS associated, (see Table 2). In our opinion, these two branches of statistics share interesting uses and properties as it has been illustrated throughout the paper.
ACKNOWLEDGEMENTS
Authors would like to thank the anonymous referees for the careful reading of the paper. We really appreciate their helpful suggestions which contribute to improve its presentation.
Biographies
Inmaculada Barranco-Chamorro. Inmaculada Barranco-Chamorro Ph.D. is Associate Professor at the Department of Statistics and OR, Faculty of Mathematics, University of Seville (SPAIN). Her doctoral thesis at the University of Sevilla, Spain, was entitled “Estimación parametrica en distribuciones no regulares” (Parametric estimation in nonregular distributions). Her major research interests are Statistical Inference, Distribution Theory, Estimation, Influence Analysis and Data Analysis in general.
Christos Grentzelos. Christos Grentzelos holds a Master's degree in Applied Mathematical Sciences from the National Technical University of Athens, Greece. His thesis was entitled “Statistical Techniques to Identify and Handle Outliers in Multivariate Data”, supervised by C.Caroni and I.Barranco-Chamorro. This work was carried out at the University of Seville under an Inter-Institutional Agreement of Higher Education Student and Staff Mobility between both universities. His current interests lie in multivariate data analysis and data mining.