Abstract
One sense of “computer-intensive” statistics is just statistical methodology that makes use of a large amount of computer time. (Examples include the bootstrap, jackknife, smoothing, image analysis, and many uses of the EM algorithm.) However, the term is usually used for methods that go beyond the minimum of calculations needed for an illuminating analysis, for example, by replacing analytic approximations by computational ones, or requiring numeric optimization or integration over high-dimensional spaces. We introduce the subject with a very simple yet useful example, and then consider some of the areas in which computer-intensive methods are used, to give a flavor of current research.
References
- 1 Aarts, E. & Korst, J. (1989). Simulated Annealing and Boltzmann Machines. Wiley, New York.
- 2
Almond, R. G.
(1995).
Graphical Belief Modeling.
Chapman & Hall,
London.
10.1007/978-1-4899-7106-7 Google Scholar
- 3 Asimov, D. (1985). The grand tour: a tool for viewing multidimensional data, SIAM Journal on Scientific and Statistical Computing 6, 128–143.
- 4 Barnard, G. (1963). Contribution to the discussion of Bartlett's paper, Journal of the Royal Statistical Society, Series B 25, 294.
- 5
Bates, D. M. &
Watts, D. G.
(1988).
Nonlinear Regression Analysis and its Applications.
Wiley,
New York.
10.1002/9780470316757 Google Scholar
- 6 Becker, R. A., Chambers, J. M. & Wilks, A. R. (1988). The NEW S Language. Chapman & Hall, New York.
- 7
Becker, R. A.,
Cleveland, W. S. &
Shyu, M. -J.
(1996).
The visual design and control of Trellis display,
Journal of Computational and Graphical Statistics
5,
123–155.
10.2307/1390777 Google Scholar
- 8 Besag, J. (1986). The statistical analysis of dirty pictures (with discussion), Journal of the Royal Statistical Society, Series B 48, 259–302.
- 9 Besag, J., Green, P., Higdon, D. & Mengersen, K. (1995). Bayesian computation and stochastic systems (with discussion), Statistical Science 10, 3–66.
- 10
Bishop, C. M.
(1995).
Neural Networks for Pattern Recognition.
Clarendon Press,
Oxford.
10.1093/oso/9780198538493.001.0001 Google Scholar
- 11 Breiman, L. (1996). Bagging predictors, Machine Learning 24, 123–140.
- 12 Breiman, L. (1996). The heuristics of instability in model selection, Annals of Statistics 24, 2350–2383.
- 13 Breiman, L., Friedman, J. H., Olshen, R. A. & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth and Brooks/Cole, Monterey.
- 14
Buja, A.,
Cook, D. &
Swayne, D. F.
(1996).
Interactive high-dimensional data visualization,
Journal of Computational and Graphical Statistics
5,
78–99.
10.2307/1390754 Google Scholar
- 15 Cantoni, O. (1992). Rough large deviation estimates for simulated annealing: application to exponential schedules, Annals of Probability 20, 1109–1146.
- 16 J. M. Chambers & T. J. Hastie, eds. (1992). Statistical Models in S. Wadsworth and Brooks/Cole, Pacific Grove.
- 17
Cook, D.,
Buja, A. &
Cabrera, J.
(1993).
Projection pursuit indices based on orthonormal function expansions,
Journal of Computational and Graphical Statistics
2,
225–250.
10.2307/1390644 Google Scholar
- 18
Cook, D.,
Buja, A.,
Cabrera, J. &
Hurley, C.
(1995).
Grand tour and projection pursuit,
Journal of Computational and Graphical Statistics
4,
155–172.
10.2307/1390844 Google Scholar
- 19
Cook, R. D. &
Weisberg, S.
(1994).
An Introduction to Regression Graphics.
Wiley,
New York.
10.1002/9780470316863 Google Scholar
- 20 de Leeuw, J. Archive of XLispStat software. Use WWW or anonymous ftp to www.stat.ucla.edu.
- 21 Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion), Journal of the Royal Statistical Society, Series B 57, 45–97.
- 22 Drucker, H., Cortes, C., Jaeckel, L. D., LeCun, Y. & Vapnik, V. (1994). Boosting and other ensemble methods, Neural Computation 6, 1289–1301.
- 23 Evans, M. & Swartz, T. (1995). Methods for approximating integrals in statistics with special emphasis on Bayesian integration problems, Statistical Science 10, 254–272.
- 24
Fang, K. -T. &
Wang, Y.
(1994).
Number-theoretic Methods in Statistics.
Chapman & Hall,
London.
10.1007/978-1-4899-3095-8 Google Scholar
- 25 Friedman, J. H. (1987). Exploratory projection pursuit, Journal of the American Statistical Association 82, 249–266.
- 26 Friedman, J. H. & Tukey, J. W. (1974). A projection pursuit algorithm for exploratory data analysis, IEEE Transactions on Computers 23, 881–890.
- 27 Fung, R. & Del Favarro, B. (1995). Applying Bayesian networks to information retrival, Communications of the ACM 38, 42–48, 57.
- 28 Gelfand, A. E. & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities, Journal of the American Statistical Association 85, 398–409.
- 29 Gelfand, A. E., Hills, S. E., Racine-Poon, A. & Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling, Journal of the American Statistical Association 85, 972–985.
- 30 Geman, S. & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741.
- 31 George, E. I. & McCulloch, R. E. (1993). Variable selection via Gibbs sampling, Journal of the American Statistical Association 88, 881–889.
- 32
George, E. I. &
McCulloch, R. E.
(1996).
Stochastic search variable selection, in
Markov Chain Monte Carlo in Practice,
W. R. Gilks,
S. Richardson &
D. J. Spiegelhalter, eds.
Chapman & Hall,
London,
pp. 203–214.
10.1007/978-1-4899-4485-6_12 Google Scholar
- 33
Geyer, C.
(1992).
Practical Markov chain Monte Carlo (with discussion),
Statistical Science
7,
473–511.
10.1214/ss/1177011137 Google Scholar
- 34
Geyer, C. J.
(1996).
Estimation and optimization of functions, in
Markov Chain Monte Carlo in Practice,
W. R. Gilks,
S. Richardson &
D. J. Spiegelhalter, eds.
Chapman & Hall,
London,
pp. 241–258.
10.1007/978-1-4899-4485-6_14 Google Scholar
- 35 W. R. Gilks, S. Richardson & D. J. Spiegelhalter, eds. (1996). Markov Chain Monte Carlo in Practice. Chapman & Hall, London.
- 36 Heckerman, D. & Wellman, M. P. (1995). Bayesian networks, Communications of the ACM 38, 26–30.
- 37 Heckerman, D., Breese, J. S. & Rommelse, K. (1995). Decision-theoretic troubleshooting, Communications of the ACM 38, 49–57.
- 38 Huber, P. J. (1985). Projection pursuit (with discussion), Annals of Statistics 13, 435–525.
- 39 Jones, M. C. & Sibson, R. (1987). What is projection pursuit? (with discussion), Journal of the Royal Statistical Society, Series A 150, 1–36.
- 40 Jordan, M. I. & Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm, Neural Computation 6, 181–214.
- 41 Kirkpatrick, S., Gellat, C. D., Jr & Vecchi, M. P. (1983). Optimization by simulated annealing, Science 220, 671–680.
- 42
Kruskal, J. B.
(1969).
Toward a practical method which helps uncover the structure of a set of multivariate observations by finding the linear transformation which optimizes a new “index of condensation”, in
Statistical Computation,
R. C. Milton &
J. A. Nelder, eds.
Academic Press,
New York,
pp. 427–440.
10.1016/B978-0-12-498150-8.50024-0 Google Scholar
- 43 Kruskal, J. B. (1972). Linear transformation of multivariate data to reveal clustering, in Multidimensional Scaling: Theory and Application in the Behavioural Sciences, R. N. Shephard, A. K. Romney & S. K. Nerlove, eds. Seminar Press, New York, pp. 179–191.
- 44 Madigan, D. & Raftery, A. E. (1994). Model selection and accounting for model uncertainty in graphical models using Occam's window, Journal of the American Statistical Association 89, 1535–1546.
- 45 MathSoft Inc. S-PLUS. Data Analysis Products Division, MathSoft Inc., Seattle.
- 46 Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A. & Teller, E. (1953). Equations of state calculations by fast computing machines, Journal of Chemical Physics 21, 1087–1091.
- 47 Meyer, M. J. statlib. On-line archive of data and computer software. Use WWW, anonymous ftp, or Gopher to lib.stat.cmu.edu.
- 48 Moulton, B. R. (1991). A Bayesian-approach to regression selection and estimation with application to a price-index for radio services, Journal of Econometrics 49, 169–193.
- 49 Neapolitan, E. (1990). Probabilistic Reasoning in Expert Systems. Theory and Algorithms. Wiley, New York.
- 50
Niederreiter, H.
(1992).
Random Number Generation and Quasi-Monte Carlo Methods.
SIAM,
Philadelphia.
10.1137/1.9781611970081 Google Scholar
- 51 Pearl, J. (1987). Evidential reasoning using stochastic simulation of causal models, Artificial Intelligence 32, 245–257.
- 52 Pearl, J. (1988). Probabilistic Inference in Intelligent Systems. Networks of Plausible Inference. Morgan Kaufmann, San Mateo.
- 53 Pentinnen, A. (1984). Modelling Interaction in Spatial Point Patterns: Parameter Estimation in the Maximum Likelihood Method. Jyväskylä Studies in Computer Science, Economics and Statistics, Vol. 7. Jyväskylän Yliopisto, Jyväskyä.
- 54
Philips, D. B. &
Smith, A. F. M.
(1996).
Bayesian model comparison via jump diffusions, in
Markov Chain Monte Carlo in Practice,
W. R. Gilks,
S. Richardson &
D. J. Spiegelhalter, eds.
Chapman & Hall,
London,
pp. 215–239.
10.1007/978-1-4899-4485-6_13 Google Scholar
- 55 Pincus, M. (1968). A closed form solution of certain programming problems, Operations Research 16, 690–694.
- 56 Pincus, M. (1970). A Monte Carlo method for the approximate solution of certain types of constrained optimization problems, Operations Research 18, 1225–1228.
- 57 Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo.
- 58
Raftery, A. E.
(1996).
Hypothesis testing and model selection, in
Markov Chain Monte Carlo in Practice,
W. R. Gilks,
S. Richardson &
D. J. Spiegelhalter, eds.
Chapman & Hall,
London,
pp. 163–187.
10.1007/978-1-4899-4485-6_10 Google Scholar
- 59 Ripley, B. D. (1977). Modelling spatial patterns (with discussion), Journal of the Royal Statistical Society, Series B 39, 172–212.
- 60 Ripley, B. D. (1987). Stochastic Simulation. Wiley, New York.
- 61
Ripley, B. D.
(1988).
Statistical Inference for Spatial Processes.
Cambridge University Press,
Cambridge.
10.1017/CBO9780511624131 Google Scholar
- 62
Ripley, B. D.
(1993).
Statistical aspects of neural networks, in
Networks and Chaos–Statistical and Probabilistic Aspects,
O. E. Barndorff-Nielsen,
J. L. Jensen, &
W. S. Kendall, eds.
Chapman & Hall,
London,
pp. 40–123.
10.1007/978-1-4899-3099-6_2 Google Scholar
- 63 Ripley, B. D. (1994). Neural networks and related methods for classification (with discussion), Journal of the Royal Statistical Society, Series B 56, 409–456.
- 64
Ripley, B. D.
(1996).
Pattern Recognition and Neural Networks.
Cambridge University Press,
Cambridge.
10.1017/CBO9780511812651 Google Scholar
- 65 Ripley, B. D. & Kirkland, M. D. (1990). Iterative simulation methods, Journal of Computational and Applied Mathematics 31, 165–172.
- 66 Shaw, J. E. H. (1988). A quasirandom approach to integration in Bayesian statistics, Annals of Statistics 16, 895–914.
- 67 Spanier, J. & Maize, E. H. (1994). Quasi-random methods for estimating integrals using relatively small samples, SIAM Review 36, 18–44.
- 68 Spiegelhalter, D. J., Dawid, A. P., Lauritzen, S. L. & Cowell, R. G. (1993). Bayesian analysis in expert systems (with discussion), Statistical Science 8, 219–283.
- 69 Spiegelhalter, D. J., Thomas, A., Best, N. G. & Gilks, W. R. BUGS. Bayesian inference Using Gibbs Sampling. Version 0.5. MRC Biostatistics Unit, Cambridge, UK. Available from URL http://www.mrc-bsu.cam.ac.uk or by anonymous ftp from ftp.mrc-bsu.cam.ac.uk.
- 70 Stewart, L. (1987). Hierarchical Bayesian analysis using Monte Carlo integration: computing posterior distributions when there are many possible models, Statistician 36, 211–219.
- 71 Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions (with discussion), Journal of the Royal Statistical Society, Series B 36, 111–147.
- 72 Swayne, D. F., Cook, D. & Buja, A. XGobi. Use WWW, anonymous ftp, or Gopher to lib.stat.cmu.edu.
- 73 Swayne, D. F., Cook, D. & Buja, A. (1991). XGobi: interactive dynamic graphics in the X window system with a link to S, in Proceedings of the ASA Section on Statistical Graphics. American Statistical Association, Alexandria, pp. 1–8.
- 74 Tierney, L. (1990). LISP-STAT. Wiley, New York.
- 75 Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion), Annals of Statistics 22, 1701–1762.
- 76
Venables, W. N. &
Ripley, B. D.
(1997).
Modern Applied Statistics with S-Plus,
2nd Ed.
Springer-Verlag,
New York.
10.1007/978-1-4757-2719-7 Google Scholar
- 77 Wolpert, D. H. (1992). Stacked generalization, Neural Networks 5, 241–259.