camera.dart package for camera preview, instead of UltralyticsYoloCameraPreview()

[darwinOne]

Hello,
Can I use the camera.dart package for realtime detection, instead of UltralyticsYoloCameraPreview()?

[pderrenger]

Hello,

Yes, you can use the camera.dart package for real-time detection instead of UltralyticsYoloCameraPreview(). The camera.dart package is a versatile option for handling camera functionalities in Flutter applications, and it can be integrated with YOLO models for real-time object detection.

Here’s a basic outline of how you can achieve this:

1. Set up the camera.dart package:
   First, add the camera package to your pubspec.yaml file:

   dependencies:
     camera: ^0.9.4+5

2. Initialize the camera:
   Initialize the camera and create a controller to manage the camera feed.

   import 'package:camera/camera.dart';
   import 'package:flutter/material.dart';
   
   class CameraScreen extends StatefulWidget {
     @darwinOne
     _CameraScreenState createState() => _CameraScreenState();
   }
   
   class _CameraScreenState extends State<CameraScreen> {
     CameraController? _controller;
     List<CameraDescription>? cameras;
   
     @override
     void initState() {
       super.initState();
       _initializeCamera();
     }
   
     Future<void> _initializeCamera() async {
       cameras = await availableCameras();
       _controller = CameraController(cameras![0], ResolutionPreset.high);
       await _controller!.initialize();
       setState(() {});
     }
   
     @override
     Widget build(BuildContext context) {
       if (_controller == null || !_controller!.value.isInitialized) {
         return Center(child: CircularProgressIndicator());
       }
       return CameraPreview(_controller!);
     }
   }

3. Integrate YOLO model for real-time detection:
   You can use TensorFlow Lite or another suitable library to run your YOLO model on the camera feed. Here’s a simplified example using TensorFlow Lite:

   import 'package:tflite/tflite.dart';
   
   Future<void> loadModel() async {
     await Tflite.loadModel(
       model: "assets/yolov4.tflite",
       labels: "assets/labels.txt",
     );
   }
   
   Future<void> runModelOnFrame(CameraImage image) async {
     var recognitions = await Tflite.detectObjectOnFrame(
       bytesList: image.planes.map((plane) {
         return plane.bytes;
       }).toList(),
       model: "YOLO",
       imageHeight: image.height,
       imageWidth: image.width,
       imageMean: 127.5,
       imageStd: 127.5,
       numResultsPerClass: 1,
       threshold: 0.4,
     );
     // Process recognitions
   }

4. Process the camera feed:
   Capture frames from the camera and pass them to the YOLO model for detection.

   @override
   void initState() {
     super.initState();
     _initializeCamera();
     loadModel();
   }
   
   Future<void> _initializeCamera() async {
     cameras = await availableCameras();
     _controller = CameraController(cameras![0], ResolutionPreset.high);
     await _controller!.initialize();
     _controller!.startImageStream((CameraImage image) {
       runModelOnFrame(image);
     });
     setState(() {});
   }

This setup allows you to use the camera.dart package for capturing camera frames and running YOLO model inference on those frames for real-time object detection.

If you encounter any issues, please ensure you are using the latest versions of the packages. Feel free to reach out if you have further questions or run into any challenges. Happy coding! 😊

[darwinOne]

Thankyou for the response. But what I mean is still using the ultralytics yolo object detector (not the tflite package) using camera.dart. final model = LocalYoloModel( id: id, task: Task.detect /* or Task.classify */, format: Format.tflite /* or Format.coreml*/, modelPath: modelPath, metadataPath: metadataPath, ); final objectDetector = ObjectDetector(model: model); await objectDetector.loadModel(); Is this possible to do?

…

On Thu, 18 Jul 2024, 4:28 am Paula Derrenger, ***@***.***> wrote: Hello, Yes, you can use the camera.dart package for real-time detection instead of UltralyticsYoloCameraPreview(). The camera.dart package is a versatile option for handling camera functionalities in Flutter applications, and it can be integrated with YOLO models for real-time object detection. Here’s a basic outline of how you can achieve this: 1. *Set up the camera.dart package:* First, add the camera package to your pubspec.yaml file: dependencies: camera: ^0.9.4+5 2. *Initialize the camera:* Initialize the camera and create a controller to manage the camera feed. import 'package:camera/camera.dart';import 'package:flutter/material.dart'; class CameraScreen extends StatefulWidget { @darwinOne _CameraScreenState createState() => _CameraScreenState(); } class _CameraScreenState extends State<CameraScreen> { CameraController? _controller; List<CameraDescription>? cameras; @OverRide void initState() { super.initState(); _initializeCamera(); } Future<void> _initializeCamera() async { cameras = await availableCameras(); _controller = CameraController(cameras![0], ResolutionPreset.high); await _controller!.initialize(); setState(() {}); } @OverRide Widget build(BuildContext context) { if (_controller == null || !_controller!.value.isInitialized) { return Center(child: CircularProgressIndicator()); } return CameraPreview(_controller!); } } 3. *Integrate YOLO model for real-time detection:* You can use TensorFlow Lite or another suitable library to run your YOLO model on the camera feed. Here’s a simplified example using TensorFlow Lite: import 'package:tflite/tflite.dart'; Future<void> loadModel() async { await Tflite.loadModel( model: "assets/yolov4.tflite", labels: "assets/labels.txt", ); } Future<void> runModelOnFrame(CameraImage image) async { var recognitions = await Tflite.detectObjectOnFrame( bytesList: image.planes.map((plane) { return plane.bytes; }).toList(), model: "YOLO", imageHeight: image.height, imageWidth: image.width, imageMean: 127.5, imageStd: 127.5, numResultsPerClass: 1, threshold: 0.4, ); // Process recognitions } 4. *Process the camera feed:* Capture frames from the camera and pass them to the YOLO model for detection. @overridevoid initState() { super.initState(); _initializeCamera(); loadModel(); } Future<void> _initializeCamera() async { cameras = await availableCameras(); _controller = CameraController(cameras![0], ResolutionPreset.high); await _controller!.initialize(); _controller!.startImageStream((CameraImage image) { runModelOnFrame(image); }); setState(() {}); } This setup allows you to use the camera.dart package for capturing camera frames and running YOLO model inference on those frames for real-time object detection. If you encounter any issues, please ensure you are using the latest versions of the packages. Feel free to reach out if you have further questions or run into any challenges. Happy coding! 😊 — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZOP3OO246CZQZDVXNUJ4PLZM3OXJAVCNFSM6AAAAABLAVPLEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZUGM2TCNJQGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nuprakash7]

@darwinOne any updates on this? Did you try using only the camera plugin?

[darwinOne]

@nuprakash7 I haven't found the solution yet.