Is it possible to build yolov8 with GGML?

[Alexbeast-CN]

I found an example of yolov3-tiny running in ggml. Here is the link.

So, I'm curious is that possible to build yolov8 with ggml. Also, super curious about the performance in ggml compared with onnx.

[pderrenger]

@Alexbeast-CN building YOLOv8 with GGML (Gerganov's General Matrix Library) is theoretically possible, as GGML is designed to be a lightweight C library for neural networks and can run models like YOLOv3-tiny, as you've mentioned. However, YOLOv8 is a more complex and recent model compared to YOLOv3-tiny, so the process might be more involved and require additional work to adapt the model to GGML's capabilities.

Regarding performance comparison with ONNX, it would depend on various factors such as the specific hardware used, the level of optimization in the GGML implementation, and how well the model operations are supported. ONNX is a widely adopted format with strong support for optimization and acceleration on different platforms, so it might offer better performance out of the box, especially when paired with dedicated inference engines like ONNX Runtime.

If you're interested in experimenting with YOLOv8 and GGML, you would need to export the model to a format that GGML can work with, and then implement the necessary layers and operations in GGML. Keep in mind that this could be a non-trivial task requiring a deep understanding of both YOLOv8's architecture and GGML's library.

For more information on exporting YOLOv8 models and the supported operations, you can refer to our documentation on the Export mode. 😊🚀