auto-fuzz: umbrella issue for new features

[DavidKorczynski]

A few to start off with:

• add ability to avoid having to re-run fuzz-introspector each time we want to test new auto-generation improvements
• add ability to extract merge all fuzzers into a single project and assess code coverage, filter based on which are "not needed" as they dont contribute, and avoiding duplicates.
• add ability to e.g. compare the code coverage with existing oss-fuzz reports (accessible via https://oss-fuzz-introspector.storage.googleapis.com/index.html)

[DavidKorczynski]

Add a feature for running Python type inference on the code under analysis and then using that type inference to create valid types: we could, e.g. use https://github.com/google/pytype

[DavidKorczynski]

Experiment with how we can integrate LLM-style autogeneration. From looking at the ChatGPT code generation we should be able to make it so we take as input a lot of auto-generated fuzzers from ML-based frameworks.

[DavidKorczynski]

I think soon we should start to look at how we can update the heuristics to work with C/C++ and also use the same approach as we do in python/java: go through oss-fuzz fuzzers and abstract them into heuristics

[DavidKorczynski]

Experiment with how we can integrate LLM-style autogeneration. From looking at the ChatGPT code generation we should be able to make it so we take as input a lot of auto-generated fuzzers from ML-based frameworks.

I think in general we should start to align auto-fuzz to be able to utilise existing auto-gen solutions, for example by focusing on infrastructure to evaluate the quality of a large set of projects as well as focusing on making suggested fuzzers easily build against OSS-Fuzz projects. There seems to be a move towards auto-generation by many research groups, and it would be great to be able to utilise each of these. For example https://github.com/Samsung/UTopia is a valid project that would be great to utilise.

[occia]

Hi guys, I recently released a gpt-based fuzz driver generator. An evaluation on 86 APIs of 30 OSS-Fuzz C projects shows its potential practicality: it can generate correct fuzz drivers for 55 (64%) APIs fully automatically and 23 more APIs (91%) with manually configured semantic correctness checkers for filtering out the generated drivers. Here lists some example drivers generated by the tool and the technical detail behind this tool.

Currently it can directly run on APIs of dozens of OSS-Fuzz C projects. I'm still improving this tool and I have several planning on improvement. Please let me know if you are interested in it. Any suggestion or discussion is welcomed.

[ligurio]

More tools for fuzzing targets generation in ¹.

Footnotes

1. https://github.com/ligurio/sqa-wiki/wiki/fuzzing-targets-generation ↩