MUBench (pronounced "Moo Bench") is an automated benchmark for API-misuse detectors, based on the MUBench benchmarking dataset. If you encounter any problems using MUBench, please report them to us. If you have any questions, please contact Sven Amann.
CI Status:
- Sven Amann (Project Lead)
- Sarah Nadi
- Hoan Anh Nguyen
- Tien N. Nguyen
- Mattis Kämmerer
- Jonas Schlitzer
- 'MUBench: A Benchmark for API-Misuse Detectors' (MSR '16 Data Showcase)
- 'A Systematic Evaluation of Static API-Misuse Detectors' (TSE '18)
We provide instructions to reproduce the MUBench experiments presented in the above publications.
With MUBench, you may run different API-misuse detectors in a number of experiments to determine their precision and recall. To this end, MUBench provides a curated dataset of real-world projects and known misuses. In each experiment run, the respective detector emits findings which you need to review manually. To this end, MUBench publishes (a subset of) the findings to a review website. After you completed your reviews, the site automatically computes experiment statistics.
- Setup a review (web)site to publish detector findings to
- Setup the experiment pipeline to run experiments
- Run experiments, using
./mubench run - Publish detector findings to your review site, using
./mubench publish
We want MUBench to grow, so please be welcome to
All software provided in this repository is subject to the CRAPL license.
The detectors included in MuBench are subject to the licensing of their respective creators. See the information in the detectors' folders.
The projects referenced in the MuBench dataset are subject to their respective licenses.
The project artwork is subject to the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0).