Philosophical roots of model validation: Two paradigms
Yaman Barlas
Associate Professor
Systems Analysis Department, Miami University, Oxford, OH 45056, U.S.A.
systems analysis at Miami University, Oxford, Ohio. He received his Ph.D. degree (1985) in industrial and systems engineering from Georgia Institute of Technology. Dr. Barlas is a member of the Institute of Management Sciences and a founding member of the System Dynamics Society. His research interests include validation and application of system dynamics to ecological systems.
Search for more papers by this authorStanley Carpenter
Associate Professor of Philosophy of Technology
Georgia Institute of Technology and a registered professional electrical engineer. He is a joint author of A Guide Book of Technology Assessment and Impact Analysis. Dr. Carpenter is a founding member and secretary of the International Society for Philosophy of Technology. His research interests lie in the area of epistemologies of technological action.
Search for more papers by this authorYaman Barlas
Associate Professor
Systems Analysis Department, Miami University, Oxford, OH 45056, U.S.A.
systems analysis at Miami University, Oxford, Ohio. He received his Ph.D. degree (1985) in industrial and systems engineering from Georgia Institute of Technology. Dr. Barlas is a member of the Institute of Management Sciences and a founding member of the System Dynamics Society. His research interests include validation and application of system dynamics to ecological systems.
Search for more papers by this authorStanley Carpenter
Associate Professor of Philosophy of Technology
Georgia Institute of Technology and a registered professional electrical engineer. He is a joint author of A Guide Book of Technology Assessment and Impact Analysis. Dr. Carpenter is a founding member and secretary of the International Society for Philosophy of Technology. His research interests lie in the area of epistemologies of technological action.
Search for more papers by this authorAbstract
System dynamics models, as causal models, are much like scientific theories. Hence, in evaluating such models, we assume certain norms of scientific inquiry. Most critics hold that the system dynamics approach does not employ formal, objective, quantitative model validation tests. This article argues that this type of criticism presupposes the traditional logical empiricist philosophy of science, which assumes that knowledge is an objective representation of reality and that theory justification can be an objective, formal process. According to the more recent relativist philosophy of science, knowledge is relative to a given society, epoch, and scientific world view. Theory justification is therefore a semiformal, relative social process. We show that relativist philosophy is consistent with the system dynamics paradigm and discuss the practical implications of the two philosophies of science for system dynamics modelers and their critics.
References
- Andersen, D. F. 1980. How Differences in Analytic Paradigms Can Lead to Differences in Policy Conclusion. In Elements of the System Dynamics Method, ed. J. Randers. Cambridge, Mass.: MIT Press.
- Ansoff, H. I., and D. P. Slevin. 1968. An Appreciation of Industrial Dynamics. Management Science 14: 383–397.
- Balci, O., and R. G. Sargent. 1984. A Bibliography on the Credibility Assessment and Validation of Simulation and Mathematical Models. Simuletter 15 (3): 15–27.
- Barlas, Y. 1985. Validation of System Dynamics Models with a Sequential Procedure Involving Multiple Quantitative Methods. Ph.D. Dissertation, Georgia Institute of Technology, Atlanta, GA 30332.
- Barlas, Y. 1989. Multiple Tests for Validation of System Dynamics Type of Simulation Models. European Journal of Operations Research 42 (1): 59–87.
- Bell, J. A., and M. F. Bell. 1980. System Dynamics and Scientific Method. In Elements of the System Dynamics Method, ed. J. Randers. Cambridge, Mass.: MIT Press.
- Bell, J. A., and P. M. Senge. 1980. Methods for Enhancing Refutability in System Dynamics Modeling. In System Dynamics, ed. A. A. Legasto, J. W. Forrester, and J. M. Lyneis, 61–73. TIMS Studies in the Management Sciences. Vol. 14. New York: North-Holland.
- Cyert, R. M., and E. Grunberg. 1963. Assumption, Prediction, and Explanation in Economics. In A Behavioral Theory of the Firm, ed. R. M. Cyert and J. G. March, 298–311. Englewood Cliffs, N.J.: Prentice-Hall.
- Descartes, R. [1641] 1931. Meditations on First Philosophy. In The Philosophical Works of René Descartes, trans. E. S. Haldane and G. R. T. Ross. Cambridge: Cambridge University Press.
- Dewey, J. 1929. The Quest for Certainty: A Study of the Relation of Knowledge and Action. New York: Minton, Balch.
- Dhrymes, P. J., et al. 1972. Criteria for Evaluation of Econometric Models. Annals of Economic and Social Measurement 1: 291–324.
- Forrester, J. W. 1961. Industrial Dynamics. Cambridge, Mass.: MIT Press.
- Forrester, J. W. 1968. A Response to Ansoff and Slevin. Management Science 14: 601–618.
- Forrester, J. W. 1973. Confidence in Models of Social Behavior with Emphasis on System Dynamics Models. System Dynamics Group Working Paper D-1967. Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02139.
- Forrester, J. W. 1980. Information Sources for Modeling the National Economy. Journal of the American Statistical Association 75: 555–574.
- Forrester, J. W., G. W. Low, and N. J. Mass. 1974. The Debate on World Dynamics: A Response to Nordhaus. Policy Sciences 5: 169–190.
- Forrester, J. W., and P. M. Senge. 1980. Tests for Building Confidence in System Dynamics Models. In System Dynamics, ed. A. A. Legasto, J. W. Forrester, and J. M. Lyneis, 201–228. TIMS Studies in the Management Sciences. Vol. 14. New York: North-Holland.
- House, P. W., and J. McLeod. 1977. Large-Scale Models for Policy Evaluation. New York: Wiley.
- Kant, I. [1781] 1933. Critique of Pure Reason, trans. N. K. Smith. London: St. Martins Press.
- Kuhn, T. [1962] 1970. 2d ed. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
- Lakatos, I. 1979. Falsification and the Methodology of Scientific Research Programmes. In Criticisms and the Growth of Knowledge, ed. I. Lakatos and A. Musgrave. Cambridge: Cambridge University Press.
- Locke, J. [1690] 1894. An Essay Concerning Human Understanding, ed. A. C. Fraser. Oxford: Clarendon Press.
- Meadows, D. H. 1980. The Unavoidable A Priori. In Elements of the System Dynamics Method, ed. J. Randers, 23–57. Cambridge, Mass.: MIT Press.
- Mitroff, I. 1969. Fundamental Issues in the Simulation of Human Behavior. Management Science 15: 635–649.
-
Naylor, T. H., and
J. M. Finger.
1968.
Verification of Computer Simulation Models.
Management Science
14:
92–101.
10.1287/mnsc.14.2.B92 Google Scholar
- Nordhaus, W. D. 1973. World Dynamics: Measurement Without Data. Economic Journal 83: 1156–1183.
-
Popper, K.
1959.
The Logic of Scientific Discovery.
New York: Basic Books.
10.1063/1.3060577 Google Scholar
- Quine, W. V. 1953. Two Dogmas of Empiricism. In From a Logical Point of View. Cambridge, Mass.: Harvard University Press.
- Radzicki, M. J. 1988. Institutional Dynamics: An Extension of the Institutionalist Approach to Socioeconomic Analysis. Journal of Economic Issues 22: 633–666.
- Radzicki, M. J. 1990. Methodologia Oeconomiae et Systematis Dynamis. System Dynamics Review 6 (2): 123–147.
- Richardson, G. P., and A. L. Pugh III. 1981. Introduction to System Dynamics Modeling with DYNAMO. Cambridge, Mass.: MIT Press.
- Rorty, R. 1979. Philosophy and the Mirror of Nature. Princeton, N.J.: Princeton University Press.
- Russell, B. [1914] 1949. Our Knowledge of the External World. London: Allen and Unwin.
- Sellars, W. 1963. Empiricism and the Philosophy of Mind. In Science, Perception and Reality. New York: Humanities Press.
- Sterman, J. D. 1984. Appropriate Summary Statistics for Evaluating the Historical Fit of System Dynamics Models. Dynamica 10 (winter): 51–66.
- Sterman, J. D. 1985. The Growth of Knowledge: Testing a Theory of Scientific Revolutions with a Formal Model. Technological Forecasting and Social Change 28: 93–122.
- Toulmin, S. 1977. From Form to Function: Philosophy and History of Science in the 1950s and Now. Daedalus 106: 143–162.
- Wittgenstein, L. 1922. Tractatus Logico-Philosophicus, trans. C. K. Ogden. London: Routledge.
- Wright, R. D., and G. Shahin. 1980. A Bibliography of Simulation Evaluation. In Proceedings of the 11th Annual Pittsburgh Conference, 541–546.
- Zellner, A. 1980. Comment on Forrester's “Information Sources for Modeling the National Economy.” Journal of the American Statistical Association 75: 567–569.
