Coverage-Directed Differential Testing of JVM Implementations
Java virtual machine (JVM) is a core technology, whose reliability is critical. Testing JVM implementations requires painstaking effort in designing test classfiles (*.class) along with their test oracles. An alternative is to employ binary fuzzing to differentially test JVMs by blindly mutating seeding classfiles and executing the resulting mutants on different JVMs for revealing inconsistent behaviors. However, this blind approach is not cost effective in practice because (1) most of the mutants are invalid and redundant, and (2) the discovered JVM discrepancies, if any, mostly only concern compatibility issues, rather than actual defects.
This paper tackles this challenge by introducing classfuzz, a coverage-directed fuzzing approach that focuses on representative classfiles for differential JVM testing. Our core insight is to (1) mutate seeding classfiles using a set of predefined mutation operators and employ Markov Chain Monte Carlo (MCMC) sampling to guide mutator selection, and (2) execute the mutants on a reference JVM implementation and use coverage uniqueness as a discipline for accepting representative ones. The accepted classfiles are used as inputs to differentially test JVMs and find defects.
We have implemented classfuzz and conducted an extensive evaluation of it against existing fuzz testing algorithms. Our evaluation results show that classfuzz can enhance the ratio of discrepancy-triggering classfiles from 1.7% to 11.9%. We have also reported 62 defect-indicative discrepancies, along with the test classfiles, to JVM developers. A number of our reported issues have already been confirmed as JVM defects, and some even match recent clarifications and changes to the JVM specification, Java SE 8 Edition.
Wed 15 JunDisplayed time zone: Tijuana, Baja California change
13:30 - 15:00
|Coverage-Directed Differential Testing of JVM Implementations|
Yuting Chen Shanghai Jiao Tong University, Ting Su East China Normal University, Chengnian Sun University of California at Davis, USA, Zhendong Su University of California, Davis, Jianjun Zhao Shanghai Jiao Tong UniversityMedia Attached
|Exposing Errors Related to Weak Memory in GPU Applications|
Research PapersPre-print Media Attached
|Lightweight Computation Tree Tracing for Lazy Functional Languages|
Research PapersMedia Attached