Validating pragmatic reuse tasks by leveraging existing test suites
Soha Makady
Department of Computer Science, University of Calgary, Calgary, Canada
Search for more papers by this authorCorresponding Author
Robert J. Walker
Department of Computer Science, University of Calgary, Calgary, Canada
Correspondence to: Robert J. Walker, Department of Computer Science, University of Calgary, 2500 University Drive NW, Calgary, Alberta, T2N 1N4, Canada.
E-mail: [email protected]
Search for more papers by this authorSoha Makady
Department of Computer Science, University of Calgary, Calgary, Canada
Search for more papers by this authorCorresponding Author
Robert J. Walker
Department of Computer Science, University of Calgary, Calgary, Canada
Correspondence to: Robert J. Walker, Department of Computer Science, University of Calgary, 2500 University Drive NW, Calgary, Alberta, T2N 1N4, Canada.
E-mail: [email protected]
Search for more papers by this authorSUMMARY
Traditional industrial practice often involves the ad hoc reuse of source code that was not designed for that reuse. Such pragmatic reuse tasks play an important role in disciplined software development. Pragmatic reuse has been seen as problematic due to a lack of systematic support, and an inability to validate that the reused code continues to operate correctly within the target system. Although recent work has successfully systematized support for pragmatic reuse tasks, the issue of validation remains unaddressed. In this paper, we present a novel approach and tool to semi-automatically reuse and transform relevant portions of the test suite associated with pragmatically reused code, as a means to validate that the relevant constraints from the originating system continue to hold, while minimizing the burden on the developer. We conduct a formal experiment with experienced developers, to compare the application of our approach versus the use of a standard IDE (the ‘manual approach’). We find that, relative to the manual approach, our approach: reduces task completion time; improves instruction coverage by the reused test cases; and improves the correctness of the reused test cases. Copyright © 2012 John Wiley & Sons, Ltd.
REFERENCES
- 1 Mcilroy MD. Mass-produced software components. In Software Engineering, Report on a conference sponsored by the NATO Science Committee, Garmisch, Germany, 7th to 11th October 1968, P Naur, B Randell (eds), Scientific Affairs Division. NATO: Brussels, Belgium, 1969; 138–155.
- 2 Mohagheghi P, Conradi R, Killi OM, Schwarz H. An empirical study of software reuse vs. defect-density and stability. Proceedings of the ACM/IEEE International Conference on Software Engineering, Edinburgh, Scotland, United Kingdom, 2004; 282–292.
- 3
Tracz W. Where does reuse start? SIGSOFT Software Engineering Notes 1990; 15(2): 42–46.
10.1145/382296.382702 Google Scholar
- 4 Gaffney JE Jr, Cruickshank RD. A general economics model of software reuse. Proceedings of the ACM/IEEE International Conference on Software Engineering, Melbourne, Australia, 1992; 327–337.
- 5 van Gurp J, Bosch J. Design erosion: problems and causes. Journal of Systems and Software 2002; 61(2): 105–119.
- 6 Biggerstaff TJ. The library scaling problem and the limits of concrete component reuse. Proceedings of the International Conference on Software Reuse, Rio de Janeiro, Brazil, 1994; 102–109.
- 7 Cordy JR. Comprehending reality: practical barriers to industrial adoption of software maintenance automation. Proceedings of the IEEE International Workshop on Program Comprehension, Portland, Oregon, United States, 2003; 196–205.
- 8 Garlan D, Allen R, Ockerbloom J. Architectural mismatch: why reuse is so hard. IEEE Software 1995; 12(6): 17–26.
- 9 Lions JL, Lübeck L, Fauquembergue JL, Kahn G, Kubbat W, Levedag S, Mazzini L, Merle D, O'Halloran C. Ariane 5: Flight 501 Failure. Technical Report, Ariane 501 Inquiry Board, 1996.
- 10 Lange BM, Moher TG. Some strategies of reuse in an object-oriented programming environment. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Austin, Texas, United States, 1989; 69–73.
- 11 Garnett ES, Mariani JA. Software reclamation. IEE Software Engineering Journal 1990; 5(3): 185–191.
- 12 Krueger CW. Software reuse. ACM Computing Surveys 1992; 24(2): 131–183.
- 13
Rosson MB,
Carroll JM. The reuse of uses in Smalltalk programming. ACM Transactions on Computer–Human Interaction 1996; 3(3): 219–253.
10.1145/234526.234530 Google Scholar
- 14 Selby RW. Enabling reuse-based software development of large-scale systems. IEEE Transactions on Software Engineering 2005; 31(6): 495–510.
- 15 Holmes R, Walker RJ. Supporting the investigation and planning of pragmatic reuse tasks. Proceedings of the ACM/IEEE International Conference on Software Engineering, Minneapolis, Minnesota, United States, 2007; 447–457.
- 16 Jansen S, Brinkkemper S, Hunink I, Demir C. Pragmatic and opportunistic reuse in innovative start-up companies. IEEE Software 2008; 25(6): 42–49.
- 17 Brandt J, Guo PJ, Lewenstein J, Dontcheva M, Klemmer SR. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Boston, Massachusetts, United States, 2009; 1589–1598.
- 18 Kim M, Sazawal V, Notkin D, Murphy G. An empirical study of code clone genealogies. Proceedings of the European Software Engineering Conference held jointly with the ACM SIGSOFT International Symposium on the Foundations of Software Engineering, Dubrovnik, Croatia, 2005; 187–196.
- 19 Kapser C, Godfrey MW. ‘Cloning considered harmful’ considered harmful: patterns of cloning in software. Empirical Software Engineering 2008; 13(6): 645–692.
- 20 Frakes WB, Fox CJ. Sixteen questions about software reuse. Communications of the ACM 1995; 38(6): 75–88.
- 21 Sen A. The role of opportunism in the software design reuse process. IEEE Transactions on Software Engineering 1997; 23(7): 418–436.
- 22 Morisio M, Ezran M, Tully C. Success and failure factors in software reuse. IEEE Transactions on Software Engineering 2002; 28(4): 340–357.
- 23 Ravichandran T, Rothenberger MA. Software reuse strategies and component markets. Communications of the ACM 2003; 46(8): 109–114.
- 24 Torkar R, Mankefors S. A survey on testing and reuse. Proceedings of the IEEE International Conference on Software-Science, Technology, and Engineering, Herzelia, Israel, 2003; 164–173.
- 25 Frakes WB, Kang K. Software reuse research: status and future. IEEE Transactions on Software Engineering 2005; 31(7): 529–536.
- 26 Holmes R, Walker RJ. Task-specific source code dependency investigation. Proceedings of the IEEE International Workshop on Visualizing Software for Understanding and Analysis, Banff, Alberta, Canada, 2007; 100–107.
- 27 Holmes R, Walker RJ. Lightweight, semi-automated enactment of pragmatic-reuse plans. Proceedings of the International Conference on Software Reuse, Beijing, China, 2008; 330–342.
- 28 Holmes R. Pragmatic software reuse. PhD Thesis, University of Calgary, November 2008.
- 29 Holmes R, Ratchford T, Robillard MP, Walker RJ. Automatically recommending triage decisions for pragmatic reuse tasks. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, Auckland, New Zealand, 2009; 397–408.
- 30 Holmes R, Walker RJ. Systematizing pragmatic software reuse. ACM Transactions on Software Engineering and Methodology 2011; 21(4). In press.
- 31 National Institute of Standards & Technology. The economic impacts of inadequate infrastructure for software testing. Planning rep. 02-3, Bureau Drive, Stop 1070, Gaithersburg, Maryland, 20899-1070, United States, May 2002.
- 32 Littman D, Pinto J, Letovsky S, Soloway E. Mental models and software maintenance. Proceedings of the Workshop on Empirical Studies of Programmers, Washington, District of Columbia, United States, 1986; 80–98.
- 33 Koenemann J, Robertson SP. Expert problem solving strategies for program comprehension. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, New Orleans, Louisiana, United States, 1991; 125–130.
- 34 Burkhardt JM, Détienne F. An empirical study of software reuse by experts in object-oriented design. Proceedings of the IFIP TC13 International Conference on Human–Computer Interaction, Lillehammer, Norway, 1995; 133–138.
- 35 Weisenberg GM. The Psychology of Computer Programming. Dorset House: New York, New York, United States, 1971.
- 36 Saff D, Artzi S, Perkins JH, Ernst MD. Automatic test factoring for Java. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, Long Beach, California, United States, 2005; 114–123.
- 37 Orso A, Kennedy B. Selective capture and replay of program executions. Proceedings of the International Workshop on Dynamic Analysis, St. Louis, Missouri, United States, 2005; 1–7.
- 38 Elbaum S, Chin HN, Dwyer MB, Jorde M. Carving and replaying differential unit test cases from system test cases. IEEE Transactions on Software Engineering 2009; 35(1): 29–45.
- 39 Kumar P, Baar T. Using AOP for discovering and defining executable test cases. In Proceedings of the International Conference on Perspectives in Systems Informatics, Vol. 5947, Lecture Notes in Computer Science: Novosibirsk, Russia, 2010.
- 40 Cormen TH, Leiserson CE, Rivest RL, Stein C. Introduction to Algorithms, 3rd edn. MIT Press: Cambridge, Massachusetts, United States, 2009.
- 41 Cottrell R, Walker RJ, Denzinger J. Semi-automating small-scale source code reuse via structural correspondence. ACM SIGSOFT. Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering, Atlanta, Georgia, United States, 2008; 214–225.
- 42 Prieto-Díaz R. Status report: software reusability. IEEE Software 1993; 10(3): 61–66.
- 43 Hunt A, Thomas D. The Pragmatic Programmer: From Journeyman to Master, Section 2.7: The Evils of Duplication. Addison-Wesley: Boston, Massachusetts, United States, 1999.
- 44 Standish TA. An essay on software reuse. IEEE Transactions on Software Engineering 1984; 10(5): 494–497.
- 45 Brooks FJ, Fred P. No silver bullet: essence and accidents of software engineering. Computer 1987; 20(4): 10–19.
- 46 Poulin JS, Caruso JM, Hancock DR. The business case for software reuse. IBM Systems Journal 1993; 32(4): 567–594.
- 47 Boehm B. Managing software productivity and reuse. Computer 1999; 32(9): 111–113.
- 48 Parnas DL. Software aging, Proceedings of the ACM/IEEE International Conference on Software Engineering, Sorrento, Italy, 1994; 279–287.
- 49 Basili VR, Selby RW, Hutchens DH. Experimentation in software engineering. IEEE Transactions on Software Engineering 1986; 12(7): 733–743.
- 50 Weyuker EJ. Testing component-based software: a cautionary tale. IEEE Software 1998; 15(5): 54–59.
- 51 Lazzarini Lemos OA, Bajracharya SK, Ossher J, Morla RS, Masiero PC, Baldi P, Lopes CV. CodeGenie: using test-cases to search and reuse source code. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, Atlanta, Georgia, United States, 2007; 525–526.
- 52 Hummel O, Janjic W, Atkinson C. Code Conjurer: pulling reusable software out of thin air. IEEE Software 2008; 25(5): 45–52.
- 53 Linstead E, Bajracharya S, Ngo T, Rigor P, Lopes C, Baldi P. Sourcerer: mining and searching internet-scale software repositories. Data Mining Knowledge Discovery 2009; 18: 300–336.
- 54 Reiss SP. Semantics-based code search. Proceedings of the ACM/IEEE International Conference on Software Engineering, Vancouver, British Columbia, Canada, 2009; 243–253.
- 55 Gallardo-Valencia RE, Elliott Sim S. Internet-scale code search. Proceedings of the ICSE Workshop on Search-Driven Development: Users, Infrastructure, Tools, and Evaluation, Vancouver, British Columbia, Canada, 2009; 49–52.
- 56 Janjic W, Hummel O, Atkinson C. More archetypal usage scenarios for software search engines. Proceedings of the ICSE Workshop on Search-Driven Development: Users, Infrastructure, Tools, and Evaluation, Cape Town, South Africa, 2010; 21–24.
- 57 Binder RV. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley: Boston, Massachusetts, United States, 1999.
- 58 Wang Y, Patel D, King G, Court I, Staples G, Ross M, Fayad M. On built-in test reuse in object-oriented framework design. ACM ACM Computing Surveys 2000; 32(1es): 7/1–7/6.
- 59 Al Dallal J, Sorenson P. Reusing class-based test cases for testing object-oriented framework interface classes. Journal of Software Maintenance and Evolution: Research and Practice 2005; 17(3): 169–196.
- 60 Grieskamp W. Multi-paradigmatic model-based testing. In Formal Approaches to Software Testing and Runtime Verification, Vol. 4262, Lecture Notes in Computer Science, Seattle, Washington, United States, 2006; 1–19.
- 61 Fisher M, Jin D, Rothermel G, Burnett M. Test reuse in the spreadsheet paradigm. Proceedings of the International Symposium on Software Reliability Engineering, Annapolis, Maryland, United States, 2002; 257–268.
- 62 Rothermel G, Elbaum S, Malishevsky AG, Kallakuri P, Qiu X. On test suite composition and cost-effective regression testing. ACM Transactions on Software Engineering and Methodology 2004; 13(3): 277–331.