Extracting the labels to make detection to voice

[hafidhhusna]

Hey i got a problem with extracting labels in ultralytics_yolo flutter package in order to make detection to voice app, do you have any idea on how to extract it so that i can pass it to flutter_tts package to make the detection to voice?

[glenn-jocher]

@hafidhhusna hi there!

Thanks for reaching out with your question. It sounds like an interesting project you're working on! To help you extract labels from the Ultralytics YOLO model and pass them to the flutter_tts package, you can follow these steps:

1. Run the YOLO model and get predictions:
   Ensure you have the latest version of the Ultralytics YOLO package. If you haven't already, you can install or update it using:

   pip install -U ultralytics

2. Extract labels from the predictions:
   After running the model, you can extract the labels from the predictions. Here's a basic example in Python:

   from ultralytics import YOLO
   
   # Load the model
   model = YOLO('yolov8n.pt')
   
   # Perform inference
   results = model('path/to/your/image.jpg')
   
   # Extract labels
   labels = [result['name'] for result in results[0].boxes]
   print(labels)

3. Pass labels to Flutter:
   You can then pass these labels to your Flutter app. One way to do this is by using a REST API or a platform channel to communicate between Python and Flutter.

4. Use flutter_tts to convert text to speech:
   Once you have the labels in your Flutter app, you can use the flutter_tts package to convert them to speech. Here’s a basic example:

   import 'package:flutter_tts/flutter_tts.dart';
   
   FlutterTts flutterTts = FlutterTts();
   
   Future _speak(String text) async {
     await flutterTts.setLanguage("en-US");
     await flutterTts.setPitch(1.0);
     await flutterTts.speak(text);
   }
   
   // Example usage
   _speak("Detected label: ${labels[0]}");

By following these steps, you should be able to extract labels from the YOLO model and use flutter_tts to convert them to voice. If you encounter any issues, please ensure you are using the latest versions of the packages and feel free to share more details so we can assist you further.

Good luck with your project! 😊

[fahadismail95]

@hafidhhusna were you able to get the correct bounding boxes from the detectedObject ? for my case they are always at an offset. can you help @glenn-jocher @hafidhhusna

[hafidhhusna]

Hi @fahadismail95 , my bounding boxes works fine. if you're want to know more about my works on this, you can check at my repository https://github.com/hafidhhusna/BILI-Flutter-Based-Sign-Language-Translator-With-Deep-Learning-Yolov8