Empirical Studies of Test Case Prioritization in a JUnit Testing Environment
H. Do, G. Rothermel, and A. Kinneer
Proceedings of the International Symposium on Software Reliability Engineering
November 2004, pages 113-124

Abstract

Test case prioritization provides a way to run test cases with the highest priority earliest. Numerous empirical studies have shown that prioritization can improve a test suite's rate of fault detection, but whether the results of these studies generalize to other language paradigms is an open question because the studies have all focused on a single procedural language, C, and on only a few specific types of test suites. In particular, Java and the JUnit testing framework are being used extensively in practice, and the effectiveness of prioritization techniques on Java systems tested under JUnit has not been investigated. We have therefore designed and performed a controlled experiment examining whether test case prioritization can be effective on Java programs tested under JUnit, and comparing results to those achieved in earlier studies. Our analyses show that test case prioritization can significantly improve the rate of fault detection of JUnit test suites, but also reveal differences with respect to previous studies that can be related to the language and testing paradigm.