Abstract
Developers of a software's graphical user interface (GUI) often fail
to document the interface specifications. Without these,
models used for automated test generation and execution remain
imperfect and incomplete.
This leads to unexpected behavior that creates unrecoverable
situations for test harnesses, and missed coverage.
In this paper, we present AutoInSpec, a technique to infer an important class
of
specifications, temporal and
state-based invariants between GUI events that have been incorrectly
modeled. Unlike existing specification mining approaches that require full
execution traces, or
source code, and that mine all invariants, we simplify the problem.
We guide AutoInSpec with coverage criteria and use a previously developed
repair
framework that builds coverage-adequate test suites, removing
unexecutable sub-sequences from consideration. These failing
sub-sequences are input to a logic-based inference engine, armed with known
invariant templates, to obtain the missing specifications.
We validate AutoInSpec on a set of well studied GUI
applications.
We thank B. Nguyen for help with the replayer infrastructure and
X. Yuan for sharing her experimental artifacts.
This work was partially supported by the US National Science Foundation
under grants CCF-0747009, CNS-1205472, CNS-0855139, CNS-1205501 and
CNS-0855055, the Air Force Office of Scientific Research through award FA9550-10-1-0406, and
the Office of Naval Research under grant N00014-05-1-0421.
Experimental Subjects for RQ2 and RQ3.
For each application,
we provide a link to the application, an XML mapping of events, the initial
covering array of test sequences (before the GA). Events are represented as integers (see the XML
mapping), and the missing pairs of events after the genetic algorithm completes.
Prolog Output
This data is the output from running our prolog programs. We break out the
results by the type of invariant. The first column represents the invariants
for RQ1 (known invariants) while the second column represents the ivnariants
found on the real applications (RQ2 and RQ3).