My research goal is to develop automated transformation processes that take functional tests as input and generate tests validating non-functional requirements.

By far we have targeted two problems in this area, namely, performance validation and testing of resource-driven exceptional Behaviors. In doing so, we explored and harnessed many program analysis techniques, including symbolic execution, mocking, and specification mining from program traces.

My work is supported by NSF award CCF-0915526 and by AFOSR award #9550-09-1-0687.

Load tests aim to validate whether system performance is acceptable under peak conditions. Existing test generation techniques induce load by increasing the size or rate of the input. Ignoring the particular input values, however, may lead to test suites that grossly mischaracterize a system’s performance. To address this limitation we introduce a mixed symbolic execution based approach that is unique in how it 1) favors program paths associated with a performance measure of interest, 2) operates in an iterative-deepening beam-search fashion to discard paths that are unlikely to lead to high-load tests, and 3) generates a test suite of a given size and level of diversity. An assessment of the approach shows it generates test suites that induce program response times and memory consumption several times worse than the compared alternatives, it scales to large and complex inputs, and it exposes a diversity of resource consuming program behavior.

We developed a compositional approach CompSLG that can automatically generate load tests for complex programs. By compositional we mean that CompSLG uses existing symbolic execution based techniques to analyze the performance of each system component in isolation, summarizes the results of those analyses, and then uses those summaries to perform a global analysis across components and generate load tests for the whole system. In doing so, we achieve gains in both efficiency and scalability by avoiding search on the whole program path space and facing path explosion problem, and by reusing previously computed summaries to save on repetitive computations. In its current form, CompSLG can be applied to any system that is structured in the form of a software pipeline (such as a unix pipeline or an XML pipeline). A study of CompSLG against the non-compositional approach SLG and Random shows that it can induce more loads across different Unix pipelines and an XML pipeline, and has an order of magnitude efficiency gain over the non-compositional counterpart.

Validating code handling exceptional behavior is difficult, particularly when dealing with external resources that may be noisy and unreliable, as it requires: 1) the systematic exploration of the space of exceptions that may be thrown by the external resources, and 2) the setup of the context to trigger specific patterns of exceptions. In this work we present an approach that addresses those difficulties by performing an exhaustive amplification of the space of exceptional behavior associated with an external resource that is exercised by a test suite. Each amplification attempts to expose a program exception handling construct to new behavior by mocking the external resource so that it returns normally or throws an exception following a predefined pattern. Our assessment of the approach indicates that it can be fully automated, is powerful enough to detect 65% of the bug reports of this kind, and is precise enough that 77% of the detected anomalies correspond to faults fixed by the developers.

I am the maintainer of the Carving-Replay toolset developed by people at Esquare Lab. It can assist developers by automatically carving various levels of unit tests from a system test suite. The tool has great impact on the community and has produced several papers (FSE06, ASE08, TSE09). If you are interested, please contact me.

Version 2.0. © 2012. HTML Style by Si Huang. Last updated by Pingyu Zhang on Feb. 15, 2012.