guides/hyperparameter-tuning/

[giscus[bot]]

guides/hyperparameter-tuning/

Dive into hyperparameter tuning in Ultralytics YOLO models. Learn how to optimize performance using the Tuner class and genetic evolution.

https://docs.ultralytics.com/guides/hyperparameter-tuning/

[T-ani]

from ultralytics import YOLO

Initialize the YOLO model

model = YOLO('yolov8n.pt')

Tune hyperparameters on COCO8 for 30 epochs

model.tune(data='coco8.yaml', epochs=30, iterations=300, optimizer='AdamW', plots=False, save=False, val=False)

in this portion of code I have facing the following error:

local variable 'ckpt_file' referenced before assignment

[pderrenger]

Hey there! 👋 It looks like you've encountered an issue with hyperparameter tuning related to a 'local variable ckpt_file referenced before assignment' error. This error might occur if the checkpoint file isn't correctly set or saved in the tuning process.

Could you try the following tweak and let me know if it helps? Add the save=True and val=True flags to your .tune() call to ensure that the model checkpoints and validation are enabled, which might be necessary for certain configurations:

model.tune(data='coco8.yaml', epochs=30, iterations=300, optimizer='AdamW', plots=False, save=True, val=True)

Setting save=True ensures that checkpoints are saved during the tuning process, which might prevent the ckpt_file error by properly assigning the checkpoint file variable.

If the issue persists or you have more details/logs, feel free to share! We're here to help. 😊

[T-ani]

Thank you. But I am still getting the same error. I am running this code in kaggle.

[pderrenger]

Hey there! Thanks for reaching out and giving it a try. Running into errors can be frustrating, but we're here to help. 🛠️ Since you're working on Kaggle and still facing the issue even after following the guidance, there might be some Kaggle-specific quirks we need to address.

An initial suggestion would be to make sure that the Kaggle environment has the necessary permissions and that all dependencies are correctly installed. Sometimes, environments like Kaggle have read-only filesystems, which could potentially interfere with saving checkpoints.

As a quick workaround, could you try setting save=True and ensuring that your Kaggle notebook has internet access enabled? This can sometimes help resolve file access issues. Example:

model.tune(data='coco8.yaml', epochs=30, iterations=300, optimizer='AdamW', plots=False, save=True, val=True)

If this doesn't resolve the issue, it may help to share a bit more detail about the error message and at what point in the process it occurs. This can provide more context for a tailored solution. Let's get this sorted! 🌟

[Kewb1]

Hello, comp sci student here! I just wanted to ask about how to improve the accuracy. I have it trained a custom dataset involving plants, and after training, I used the tune method cause the accuracy after training wasn't enough. I used best.pt from the tune folder for the chosen model but the results were the same before tuning. Is there anything else I need to be doing/changing to see results?

[glenn-jocher]

Hey there! Thanks for diving into hyperparameter tuning with YOLO models. 😊

If tuning didn’t improve the metrics as expected, consider these steps:

1. More Iterations: Sometimes, a few rounds aren’t enough. Increasing the iterations might bring better results.
2. Metric Choice: Ensure you're focusing on the right performance metrics for your dataset.
3. Hyperparameter Range: Widen the range of hyperparameters being explored. It’s possible that the optimal settings lie outside the current search space.

As an example, adjust your tuning like so:

model.tune(data='your_dataset.yaml', epochs=50, iterations=500, ...)

Remember, tuning is an iterative process. Don't hesitate to experiment with settings outside your initial attempts. Happy training! 🌱

[illink7]

Thanks for sharing this information) It helps me a lot

[glenn-jocher]

Hey! 🌱 Glad to hear the info was helpful! If tuning didn't boost the accuracy as hoped, consider checking these quick tips:

• Review the Ultralytics YOLO Hyperparameter Tuning Guide again to ensure you've covered all bases.
• Sometimes, fine-tuning with different epochs, batch size, or even starting with slightly different initial hyperparameters can yield better results.
• Assess if your dataset might need augmentation or cleaning for better model training.
• Lastly, logging and reviewing each tuning iteration's performance can sometimes reveal insights into what adjustments might be beneficial.

Keep experimenting, and you might uncover the tweak that propels your model's performance! 🚀

[illink7]

What optimizers I can use beside AdamW?

[pderrenger]

@illink7 in addition to AdamW, YOLOv8 supports various optimizers including SGD, Adam, and RMSprop. You can easily switch optimizers by specifying the optimizer argument in your training command. For example:

yolo detect train data=coco128.yaml model=yolov8n.pt epochs=100 optimizer=SGD

This flexibility allows you to experiment with different optimizers to find the one that best suits your dataset and training setup. Happy experimenting! 🚀

[illink7]

Hello, I got stuck with some issue related with GPU workflow.
Here is my device after some code
torch.cuda.get_device_properties(0).name
Result: 'NVIDIA GeForce RTX 3060 Laptop GPU'
But when I start training or even tuning my model in results field I recieve -
"WARNING training failure for hyperparameter tuning iteration 1"
But procces doesn't stop and continue
When I checked the results after 200 iterations - there weren't any changes in fiting, graphics were the same.
Could you help me to solve my problem, please.

[glenn-jocher]

Hello! 👋 It looks like you're encountering a warning during hyperparameter tuning with your NVIDIA GeForce RTX 3060 Laptop GPU. This could be due to several factors, ranging from hardware limitations to specific configurations in your tuning setup.

Firstly, ensure your GPU drivers and CUDA are up to date for the best compatibility with PyTorch. Additionally, monitoring GPU usage during the tuning process might give some insight into potential bottlenecks or issues.

For the tuning process itself, consider starting with a smaller number of iterations and closely monitoring the output for any specific warnings or errors that might provide clues. For instance:

from ultralytics import YOLO

# Initialize the model
model = YOLO('yolov8n.pt')

# Start tuning with a reduced number of iterations for initial troubleshooting
model.tune(data='coco8.yaml', epochs=5, iterations=50)

If the problem persists, adjusting the hyperparameters manually or simplifying the tuning process might help isolate the issue. Our Hyperparameter Tuning Guide offers more detailed steps and explanations that could provide further insights.

Without any changes over 200 iterations, it's possible there might be an issue with how the tuning is set up or with the GPU's ability to process the tasks efficiently. Checking the training logs for any repeated warnings or errors could also point to the underlying issue.

If the problem continues, please feel free to share more details about your setup, including the specific commands or code you're using for hyperparameter tuning, and any additional log information you think might be relevant. This could help diagnose the issue more accurately. 🛠️

Hope this helps! Let's get it sorted.

[gabrieladvent]

hello I'm trying to find the best hyperparameter using gridsearch, but I'm having trouble comparing each mAP. Is there a solution or reference related to using gridsearch to find the best hyperparameter?

[glenn-jocher]

@gabrieladvent hello! It sounds like you're interested in hyperparameter tuning with grid search. While grid search is a common approach, Ultralytics YOLO typically utilizes genetic algorithms for hyperparameter tuning, focusing on mutation to explore the hyperparameter space efficiently.

For detailed guidance on utilizing our tuning strategies, please refer to our Hyperparameter Tuning Guide. This guide provides an overview of the tuning process and explains how to use the model.tune() method effectively.

Here's a quick example of how you might start tuning the hyperparameters:

from ultralytics import YOLO

# Initialize the YOLO model
model = YOLO('yolov8n.pt')

# Tune hyperparameters on a dataset
model.tune(data='coco8.yaml', epochs=30, iterations=300, optimizer='AdamW', plots=False, save=False, val=False)

I hope this helps! If you have any more questions or need further clarification, feel free to ask. Happy tuning! 🚀

[gabrieladvent]

Hello @glenn-jocher , I appreciate your response. For example, I dont know the combination of parameters that can give good results, so I want to try to combine several parameters and find the best parameters using grid search.

param_grid = {
    'imgsz': [240, 256, 416],
    'batch': [16, 32, 64],
    'momentum': [0.937, 0.8],
    'optimizer' : ['adam', 'adamW'],
}

param_combinations = list(product(*param_grid.values()))

from the resulting combinations, I will run training with those combinations, and will take the parameters that produce the best mAPs.
Is this possible in yolov8? If so, how can I take the best mAP value as a comparison? Thank you.

[glenn-jocher]

Hello! It's great to see your initiative in experimenting with hyperparameters to enhance your model's performance. While Ultralytics YOLO largely employs genetic algorithms for hyperparameter tuning due to their effectiveness over grid search approaches in exploring the high-dimensional space, your idea of using a grid search can still be tested!

To implement your approach in YOLOv8, you could manually iterate over each parameter combination as you've described, training the model for each and logging the mAP scores. Here’s a simplified way to do this using Python:

from ultralytics import YOLO
import itertools

param_grid = {
    'imgsz': [240, 256, 416],
    'batch': [16, 32, 64],
    'momentum': [0.937, 0.8],
    'optimizer': ['adam', 'adamW'],
}

# Generate combinations
param_combinations = list(itertools.product(*param_grid.values()))

best_map = 0
best_params = {}
for combination in param_combinations:
    params = dict(zip(param_grid.keys(), combination))
    model = YOLO('yolov8n.pt')
    results = model.train(data='coco128.yaml', epochs=50, **params)
    if results['val/mAP_0.5'] > best_map:
        best_map = results['val/mAP_0.5']
        best_params = params

print("Best mAP: {:.4f} with Parameters: {}".format(best_map, best_params))

Here, params will unpack to the respective arguments expected by the model.train() method, dynamically adjusting each time based on your grid. At the end of each iteration, it checks if the current mAP is higher than all previously recorded ones, and if so, updates the best parameters.

This approach, while straightforward, can be computationally intensive due to the high number of configurations being tested — consider adjusting the range and granularity based on your available computational resources.

[gabrieladvent]

Hello @glenn-jocher, I'm very grateful for your help. It helped me solve a problem I was facing

[glenn-jocher]

Hello! I'm thrilled to hear that the guidance was helpful for you! 🎉 If you're diving into hyperparameter tuning using grid search and want to compare mAP scores, you can manually log the mAP for each parameter combination during your experiments. Here's a simple way to capture and compare these values:

results_dict = {}
for params in param_combinations:
    # Set the model with current parameters
    model = YOLO('yolov8n.pt')
    results = model.train(data='your_data.yaml', epochs=30, **params)
    
    # Store mAP and parameters
    results_dict[params] = results['val/mAP_0.5']

# Find the parameters with the best mAP
best_params = max(results_dict, key=results_dict.get)
print(f"Best parameters: {best_params} with mAP: {results_dict[best_params]}")

This approach will help you systematically evaluate and compare the performance of different hyperparameter settings. If you have any more questions or need further assistance, feel free to ask. Happy tuning! 🚀

[arjunchandra2]

Hello, I recently tuned the yolov8n.pt model on a custom dataset and obtained the following results:

# 35/100 iterations complete ✅ (14957.02s)
# Results saved to runs/detect/tune
# Best fitness=0.2068 observed at iteration 11
# Best fitness metrics are {'metrics/precision(B)': 0.54959, 'metrics/recall(B)': 0.48238, 'metrics/mAP50(B)': 0.46052, 'metrics/mAP50-95(B)': 0.17861, 'val/box_loss': 2.23222, 'val/cls_loss': 1.53813, 'val/dfl_loss': 1.7074, 'fitness': 0.2068}
# Best fitness model is runs/detect/train11

So far so good. However, I wanted to replicate these results, which I believe is possible since the args.yaml file in runs/detect/train11 specifies seed: 0 and deterministic: true. So I use the following code to train the model myself:

from ultralytics import YOLO

d_path = "/projectnb/npbssmic/ac25/Defect_Training/data.yaml"

model = YOLO("yolov8n.pt")

results = model.train(data = d_path,
                      device = [3]
                      )

In my data.yaml file I have copied over all of the arguments from the args.yaml file in runs/detect/train11 so I believe this should work to replicate the training results. However, after running training several times, I am not able to obtain the same metrics as the tuning script (mAP50, precision, recall). The only thing that I changed between data.yaml and args.yaml is that in data.yaml I removed the line val: false as this was causing a FileNotFoundError during training and I believe validating results after every epoch should not change actual results? Please let me know what I am doing wrong in trying to replicate training results from model tuning.

[arjunchandra2]

I have also tried specifying all of the model arguments directly in the python script rather than putting them in the data.yaml file:

from ultralytics import YOLO

d_path = "/projectnb/npbssmic/ac25/Defect_Training/data.yaml"

model = YOLO("yolov8n.pt")

results = model.train(
    mode = 'train',
    model = 'yolov8n.pt',
    data = d_path,
    epochs = 30,
    patience = 50,
    batch = 16,
    imgsz = 640,
    save = False,
    save_period = -1,
    cache = False,
    device = [3],
    workers = 8,
    project = None,
    name = None,
    exist_ok = False,
    pretrained = True,
    optimizer= 'AdamW',
    verbose = True,
      ...)

These results looked closer to tuning results but still do not replicate exactly. Is there a better way to pass all of these parameters to the model - it seemed that the model was not recognizing some parameters in the data.yaml file so I moved them into the python script? I am also now running into an issue where the model is not saving any results. For example, the following is printed to the console Logging results to runs/detect/train18 but no such directory is created and I cannot find any training results anywhere?

[glenn-jocher]

Hi there! It looks like you're on the right track with replicating your training results. The key difference might be the val: false line you removed. Validation during training can indeed affect the results due to the additional computations and potential early stopping mechanisms.

To ensure consistency, try running your training with the exact same validation settings as in the args.yaml. Here's a quick example:

from ultralytics import YOLO

d_path = "/projectnb/npbssmic/ac25/Defect_Training/data.yaml"

model = YOLO("yolov8n.pt")

results = model.train(data=d_path,
                      device=[3],
                      val=False  # Ensure this matches your original tuning settings
                      )

If the issue persists, double-check other hyperparameters and ensure they are identical. For more details, you can refer to the Hyperparameter Tuning Guide. Good luck! 🚀

[nasserdr]

I have a question about hyperparameter tuning:
Should I get the same results if I run 500 iterations compared to if I run 250 iteration + 250 iterations (where the starting model in the second 250 iterations is the 250th model obtained at the end of the first run)?

I illusrate this with two pieces of code:

Run 500 iteration at once is like:

model = YOLO("yolov8m.yaml")
model.tune(data='data.yaml', epochs=50, iterations=500, batch = 32, imgsz=1024, optimizer='AdamW', plots=True, save=True, val=True, device=[0, 1, 2, 3])

Run 250 iterations then 25 iterations

model = YOLO("yolov8m.yaml")
model.tune(data='data.yaml', epochs=50, iterations=250, batch = 32, imgsz=1024, optimizer='AdamW', plots=True, save=True, val=True, device=[0, 1, 2, 3])
best_model = "path/weights/best.pt"
best_hyp = "path/runs/detect/tune/best_hyperparameters.yaml"
model = YOLO(best_model, best_hyp) # load the best model and hyperparameters
model.tune(data='data.yaml', epochs=50, iterations=250, batch = 32, imgsz=1024, optimizer='AdamW', plots=True, save=True, val=True, device=[0, 1, 2, 3])

So, should I get the same results in scenario one and scenario two?

[glenn-jocher]

Hi @nasserdr,

Thank you for your detailed question and for providing the sample hyperparameters. It looks like you're on the right track, but there's a small detail that might be causing the discrepancy you're seeing.

When you load the model with the best weights and hyperparameters, you should ensure that the hyperparameters from the best_hyperparameters.yaml file are correctly applied to the training process. Here's how you can do it:

Loading Best Hyperparameters

Ensure that the hyperparameters are explicitly loaded and applied to the model before starting the training. Here's an updated approach:

from ultralytics import YOLO

# Load the best model and hyperparameters
best_model = "../models/tune2/train403/weights/best.pt"
best_hyp = "../models/tune2/best_hyperparameters.yaml"
model = YOLO(best_model)

# Load the best hyperparameters
model.overrides = best_hyp

# Train the model with the best hyperparameters
model.train(data='rumex.yaml', epochs=100, batch=8, imgsz=2048, device=[0, 1, 2, 3])

Verify Hyperparameters

After loading the hyperparameters, you can verify them by printing the model's hyperparameters before starting the training:

print(model.args)

This will help ensure that the correct hyperparameters are being used.

Additional Considerations

1. Hyperparameter File Format: Ensure that the best_hyperparameters.yaml file is correctly formatted and accessible.
2. Model Overrides: The model.overrides attribute should correctly apply the hyperparameters to the model. If this doesn't work as expected, you might need to manually set each hyperparameter.

Example of Manual Setting

If the automatic loading doesn't work, you can manually set the hyperparameters:

model.args.lr0 = 0.00497
model.args.lrf = 0.00903
model.args.momentum = 0.89773
model.args.weight_decay = 0.0004
model.args.warmup_epochs = 3.25108
model.args.warmup_momentum = 0.67061
model.args.box = 5.47119
model.args.cls = 0.64491
model.args.dfl = 1.05417
model.args.hsv_h = 0.01864
model.args.hsv_s = 0.37932
model.args.hsv_v = 0.47054
model.args.degrees = 0.0
model.args.translate = 0.10692
model.args.scale = 0.72295
model.args.shear = 0.0

# Train the model
model.train(data='rumex.yaml', epochs=100, batch=8, imgsz=2048, device=[0, 1, 2, 3])

By ensuring the correct hyperparameters are applied, you should see the expected behavior during training. If the issue persists, please provide a minimum reproducible example so we can further investigate. You can refer to our Minimum Reproducible Example Guide for more details.

Feel free to reach out if you have any more questions or need further assistance. Happy training! 🚀

[nasserdr]

Both methods do not work. Neither feeding a hyperparameter file using model.overrides = best_hyp
nor manually using model.args.. The same error in both cases is: AttributeError: 'YOLO' object has no attribute 'args'.

An alternative I found is by giving the hyperparameters as a value for the cfg argument.
model.train(data='data.yaml', epochs=300, batch = 4, imgsz=1024, device =[0, 1, 2, 3], cfg = best_hyp)

However, it would be interesting approcah to set them manually. This is especially necessary if I want to explore a specific subset of hyperparameters and see how they affect the results. Any idea how I can make it work?

[glenn-jocher]

Hi @nasserdr,

Thank you for your detailed follow-up. I appreciate your efforts in trying different approaches to set the hyperparameters. Let's address the issue you're encountering.

Issue with model.args Attribute

It seems like the model.args attribute is not available, which is causing the AttributeError. This might be due to changes in the API or how the model's attributes are managed internally.

Alternative Approach: Using cfg Argument

Your alternative approach using the cfg argument is indeed a valid method to set hyperparameters. This method ensures that the hyperparameters from the best_hyperparameters.yaml file are correctly applied during training:

model.train(data='data.yaml', epochs=300, batch=4, imgsz=1024, device=[0, 1, 2, 3], cfg=best_hyp)

Manually Setting Hyperparameters

If you want to manually set specific hyperparameters to explore their effects, you can do so by directly modifying the model's configuration dictionary before training. Here's how you can achieve this:

from ultralytics import YOLO

# Load the best model
best_model = "../models/tune2/train403/weights/best.pt"
model = YOLO(best_model)

# Manually set hyperparameters
model.overrides = {
    'lr0': 0.00497,
    'lrf': 0.00903,
    'momentum': 0.89773,
    'weight_decay': 0.0004,
    'warmup_epochs': 3.25108,
    'warmup_momentum': 0.67061,
    'box': 5.47119,
    'cls': 0.64491,
    'dfl': 1.05417,
    'hsv_h': 0.01864,
    'hsv_s': 0.37932,
    'hsv_v': 0.47054,
    'degrees': 0.0,
    'translate': 0.10692,
    'scale': 0.72295,
    'shear': 0.0
}

# Train the model
model.train(data='rumex.yaml', epochs=100, batch=8, imgsz=2048, device=[0, 1, 2, 3])

Verify Hyperparameters

To ensure the correct hyperparameters are being used, you can print the model's configuration before starting the training:

print(model.overrides)

This will help you verify that the hyperparameters are correctly set.

Conclusion

Using the cfg argument is a straightforward and effective way to apply hyperparameters from a file. For manual adjustments, directly modifying the model.overrides dictionary is a flexible approach. If you continue to face issues, please ensure you are using the latest version of the Ultralytics YOLO package and provide a minimum reproducible example if the problem persists.

For more detailed guidance on hyperparameter tuning, you can refer to our Hyperparameter Tuning Guide.

Feel free to reach out if you have any more questions or need further assistance. Happy training! 🚀

[Ggstuddu]

Hello @glenn-jocher，
I'm new in AI,I want to continue tunning my parameters based on the ones I got last time, so I refer to the above comment and use the following code:

best_model = "D:/code/python_code/ultralytics-8.1.0/runs/detect/tune7/weights/best.pt"
best_hyp = ("D:/code/python_code/ultralytics-8.1.0/runs/detect/tune7/best_hyperparameters.yaml")
model = YOLO(best_model,best_hyp) # load the best model and hyperparameters
model.tune(data="D:/code/python_code/ultralytics-8.1.0/ultralytics/cfg/datasets/Fridge-Dents.yaml",batch=8,epochs=50,iterations=200,imgsz=480,optimizer="AdamW", plots=False, save=False, val=False)

But it turns out that I can't load the parameters that I tuned earlier. The tune method continues to iteratively optimize the parameters using its default parameters. Did I do something wrong?

Thanks a lot.

[glenn-jocher]

It seems like the tuned hyperparameters aren't being applied. Ensure that the best_hyperparameters.yaml file is correctly formatted and accessible. You can also try manually setting the hyperparameters by updating the model's configuration before tuning. If issues persist, please check the file paths and format.

[Quanzan04]

Hi, I'm new in AI, and I want to fine tuned my yolov10 model. I have already trained it on my own data set. Now I want to test to find the best hyperparameter, but I don't know how to write it.
If i want to test multiple lr, momentum, batch_size and epoche, how do i need to write it ?
Thanks a lot

[glenn-jocher]

Hi there! Welcome to the world of AI and YOLO models! 🎉

To fine-tune your YOLOv10 model and test various hyperparameters like learning rate, momentum, batch size, and epochs, you can use the model.tune() method provided by Ultralytics. This method leverages genetic algorithms to optimize hyperparameters iteratively.

Here's a concise example to get you started:

from ultralytics import YOLO

# Initialize your YOLOv10 model
model = YOLO("yolov10.pt")

# Tune hyperparameters on your dataset
model.tune(
    data="your_dataset.yaml",  # Replace with your dataset configuration file
    epochs=30,                 # Number of epochs for each tuning iteration
    iterations=300,            # Total number of tuning iterations
    optimizer="AdamW",         # Optimizer to use
    plots=False,               # Disable plotting for faster tuning
    save=False,                # Disable saving intermediate models
    val=False                  # Disable validation except on the final epoch
)

Key Steps:

1. Initialize Hyperparameters: Start with a reasonable set of initial hyperparameters.
2. Mutate Hyperparameters: Use the _mutate method to produce new sets of hyperparameters.
3. Train Model: Train the model using the mutated hyperparameters.
4. Evaluate Model: Assess the model's performance using metrics like AP50 or F1-score.
5. Log Results: Keep track of both performance metrics and hyperparameters.
6. Repeat: Iterate until you reach the desired performance or the set number of iterations.

For more detailed guidance, you can refer to our Hyperparameter Tuning Guide.

Feel free to ask if you have any more questions. Happy tuning! 🚀

[Quanzan04]

Thank you very much, there is still one thing I don't really understand, how does this line of code shows that i want to fine tuned the learning rate for example ?
Thanks for the answer !

[glenn-jocher]

Hi there!

Great question! When you use the model.tune() method, it automatically handles the fine-tuning of various hyperparameters, including the learning rate. The tuning process involves iteratively adjusting these hyperparameters to find the optimal values that improve your model's performance.

Here's a more detailed example to illustrate how you can fine-tune the learning rate, momentum, batch size, and epochs:

from ultralytics import YOLO

# Initialize your YOLOv10 model
model = YOLO("yolov10.pt")

# Tune hyperparameters on your dataset
model.tune(
    data="your_dataset.yaml",  # Replace with your dataset configuration file
    epochs=30,                 # Number of epochs for each tuning iteration
    iterations=300,            # Total number of tuning iterations
    optimizer="AdamW",         # Optimizer to use
    plots=False,               # Disable plotting for faster tuning
    save=False,                # Disable saving intermediate models
    val=False                  # Disable validation except on the final epoch
)

In this example, the model.tune() method will explore different values for the learning rate (lr0), momentum, batch size, and other hyperparameters. The genetic algorithm will mutate these values to find the best combination that maximizes your model's performance.

To see the specific hyperparameters being tuned and their ranges, you can refer to the Hyperparameter Tuning Guide. This guide provides a comprehensive overview of the hyperparameters and the tuning process.

If you encounter any issues or need further assistance, please provide a reproducible example so we can better understand and address your concerns. You can find more information on creating a reproducible example here.

Happy tuning! 🚀

[Quanzan04]

Hello, it helped a lot, does the same thing exits for yolov9 ?
Thanks

[glenn-jocher]

Hello!

I'm glad to hear that the previous information was helpful! Yes, the same hyperparameter tuning functionality exists for YOLOv9 as well. You can use the model.tune() method to optimize hyperparameters for YOLOv9 just like you would for YOLOv8.

Here's a quick example to get you started with tuning your YOLOv9 model:

from ultralytics import YOLO

# Initialize your YOLOv9 model
model = YOLO("yolov9.pt")

# Tune hyperparameters on your dataset
model.tune(
    data="your_dataset.yaml",  # Replace with your dataset configuration file
    epochs=30,                 # Number of epochs for each tuning iteration
    iterations=300,            # Total number of tuning iterations
    optimizer="AdamW",         # Optimizer to use
    plots=False,               # Disable plotting for faster tuning
    save=False,                # Disable saving intermediate models
    val=False                  # Disable validation except on the final epoch
)

This will help you find the best hyperparameters for your YOLOv9 model. If you need more details on hyperparameter tuning, you can refer to the Hyperparameter Tuning Guide.

Feel free to reach out if you have any more questions. Happy tuning! 🚀

[KamNgadi]

Hi there,

I'm encountering the following error message:
TypeError Traceback (most recent call last)
in
20 epochs=70, **params)
21 # Store mAP and parameters
---> 22 results_dict[params] = results['val/mAP_0.5']
23
24 # Find the parameters with the best mAP

TypeError: 'SegmentMetrics' object is not subscriptable
This error occurs during a grid search with the following code:
import itertools

param_grid = {
'imgsz': [240, 256, 416],
'batch': [16, 32, 64],
'momentum': [0.937, 0.8],
'optimizer': ['SGD', 'Adam', 'AdamW', 'NAdam', 'RAdam', 'RMSProp'],
}

param_combinations = list(itertools.product(*param_grid.values()))
results_dict = {}
for params in param_combinations:
# Set the model with current parameters
params = dict(zip(param_grid.keys(), params))
model = YOLO('yolov8n.pt')
results = model.train(data='mydata.yaml', epochs=70, **params)

# Store mAP and parameters
results_dict[params] = results['val/mAP_0.5']

Find the parameters with the best mAP

best_params = max(results_dict, key=results_dict.get)
print(f"Best parameters: {best_params} with mAP: {results_dict[best_params]}")
Could anyone help me resolve this issue?

[pderrenger]

@KamNgadi hi there,

Thank you for reaching out! The error you're encountering, TypeError: 'SegmentMetrics' object is not subscriptable, suggests that the results object returned by the model.train() method is not being accessed correctly.

In YOLOv8, the results object is a custom object that contains various attributes and methods to access training metrics. To access the validation mAP, you should use the appropriate attribute or method provided by the results object.

Here's how you can modify your code to correctly access the validation mAP:

import itertools
from ultralytics import YOLO

param_grid = {
    'imgsz': [240, 256, 416],
    'batch': [16, 32, 64],
    'momentum': [0.937, 0.8],
    'optimizer': ['SGD', 'Adam', 'AdamW', 'NAdam', 'RAdam', 'RMSProp'],
}

param_combinations = list(itertools.product(*param_grid.values()))
results_dict = {}

for params in param_combinations:
    # Set the model with current parameters
    params = dict(zip(param_grid.keys(), params))
    model = YOLO('yolov8n.pt')
    results = model.train(data='mydata.yaml', epochs=70, **params)
    
    # Store mAP and parameters
    results_dict[params] = results.metrics.box.map50  # Accessing mAP50 correctly

# Find the parameters with the best mAP
best_params = max(results_dict, key=results_dict.get)
print(f"Best parameters: {best_params} with mAP: {results_dict[best_params]}")

In this example, results.metrics.box.map50 correctly accesses the mAP50 value from the results object.

Additionally, please ensure that you are using the latest version of the Ultralytics YOLO package to avoid any compatibility issues. You can update the package using:

pip install --upgrade ultralytics

For more detailed information on hyperparameter tuning, you can refer to our Hyperparameter Tuning Guide.

If the issue persists, please provide a minimum reproducible example as outlined in our Minimum Reproducible Example Guide to help us diagnose the problem more effectively.

[MarioPasc]

Only tuning a set of Hyperparameters

My name is Mario, and I am currently conducting research on atherosclerosis detection in coronary angiography medical images using YOLOv8. Due to specific requirements of my project, I have developed a custom data augmentation class and therefore, I am not utilizing any of the YOLOv8 augmentation parameters.

I would like to inquire if there is a method by which I can selectively fine-tune a specific set of hyperparameters using the Genetic Algorithm (GA). For instance, I wish to optimize parameters such as lr0, lrf, and imgsz, while ensuring that other hyperparameters, including perspective, flipud, and fliplr, remain fixed at 0 throughout the tuning process.

Your assistance and guidance on this matter would be greatly appreciated!

[pderrenger]

@MarioPasc hi Mario,

You can indeed fine-tune specific hyperparameters while keeping others fixed in Ultralytics YOLO. To achieve this, you can set the hyperparameters you want to remain constant to their desired values (e.g., perspective=0, flipud=0, fliplr=0) and then use the model.tune() method to optimize only the hyperparameters of interest, such as lr0, lrf, and imgsz. This approach ensures that the genetic algorithm focuses on mutating only the specified hyperparameters.

For more detailed guidance, you can refer to the Hyperparameter Tuning Guide.

Best of luck with your research!

[junkiiifkg]

can ı use this for yolov4

[pderrenger]

@junkiiifkg yes, you can use Ultralytics for YOLOv4. While the hyperparameter tuning guide primarily focuses on YOLOv8, the principles and methods can be adapted for YOLOv4. Ensure you have the latest versions of the Ultralytics package and YOLOv4 model for compatibility. For detailed steps on hyperparameter tuning, you can refer to the Ultralytics documentation. If you encounter any issues, please check if they persist with the latest updates.

[junkiiifkg]

how can ı use this for yolov4 and can ı use with C++

[pderrenger]

To use YOLOv4 with Ultralytics, you can follow similar steps as for other YOLO models. However, note that YOLOv4 may require specific configurations and dependencies. For C++ usage, you would need to export the model to a format compatible with C++ frameworks, such as ONNX or TensorRT. Ensure you have the latest versions of the Ultralytics package and YOLOv4 model for compatibility. For detailed steps, refer to the Ultralytics documentation. If you encounter any issues, please check if they persist with the latest updates.

[alifazeli00]

How can I choose the best parameters for my custom model?

Which parameters impact the model's performance?

Is using Optuna a good option for finding the best parameter values?

[glenn-jocher]

@alifazeli00 to choose the best parameters for your custom model, focus on key hyperparameters like learning rate, batch size, and number of epochs. These significantly impact performance. Using Optuna for hyperparameter optimization is a good option. For more details, please refer to our Hyperparameter Tuning Guide.

[alifazeli00]

How much should the required parameters be adjusted? Is it correct that each dataset needs specific parameter values to achieve the best accuracy and predictions?

[glenn-jocher]

Each dataset may require specific parameter adjustments to achieve optimal accuracy. Key parameters like learning rate, batch size, and epochs significantly impact performance. Using tools like Optuna for hyperparameter optimization can be effective. For more guidance, check our Hyperparameter Tuning Guide.

[codingxperience]

Greetings, All
I am currently tuning hyperparameters on a custom dataset. Although the best weights are saved and used for validation after the tuning, I encounter the following error during iterations after the validation runs successfully and displays the results:
/opt/conda/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice. return _methods._mean(a, axis=axis, dtype=dtype, /opt/conda/lib/python3.10/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount) Traceback (most recent call last): File "/opt/conda/bin/yolo", line 8, in <module> sys.exit(entrypoint()) File "/opt/conda/lib/python3.10/site-packages/ultralytics/cfg/__init__.py", line 830, in entrypoint getattr(model, mode)(**overrides) File "/opt/conda/lib/python3.10/site-packages/ultralytics/engine/model.py", line 815, in train self.trainer.train() File "/opt/conda/lib/python3.10/site-packages/ultralytics/engine/trainer.py", line 207, in train self._do_train(world_size) File "/opt/conda/lib/python3.10/site-packages/ultralytics/engine/trainer.py", line 476, in _do_train self.run_callbacks("on_train_end") File "/opt/conda/lib/python3.10/site-packages/ultralytics/engine/trainer.py", line 168, in run_callbacks callback(self) File "/opt/conda/lib/python3.10/site-packages/ultralytics/utils/callbacks/wb.py", line 142, in on_train_end _plot_curve( File "/opt/conda/lib/python3.10/site-packages/ultralytics/utils/callbacks/wb.py", line 87, in _plot_curve y_log = np.interp(x_new, x, np.mean(y, axis=0)).round(3).tolist() File "/opt/conda/lib/python3.10/site-packages/numpy/lib/function_base.py", line 1599, in interp return interp_func(x, xp, fp, left, right) ValueError: object of too small depth for desired array WARNING ❌️ training failure for hyperparameter tuning iteration 2
Here is the setup I am using:

model.tune( data=os.path.join(WORKING_DIR, 'data.yaml'), epochs=50, imgsz=1024, iterations=300, optimizer="AdamW", plots=False, save=False, val=False )
Would you be able to provide any guidance or advice on how to resolve this issue? I greatly appreciate your time and assistance.

[glenn-jocher]

@codingxperience it seems like the error might be related to empty arrays during the plotting phase. Please ensure your dataset is correctly formatted and that there are no missing labels or images. Also, try updating to the latest version of the Ultralytics package to see if the issue persists. If the problem continues, consider checking the data preprocessing steps for any anomalies. For more detailed guidance, you can refer to the Hyperparameter Tuning Guide.

[codingxperience]

Well, the datasets contains negative samples; there are empty labels for a negative, could that be the reason? Thanks

[glenn-jocher]

Negative samples with empty labels could indeed cause issues during hyperparameter tuning. Ensure your dataset is correctly formatted, and consider removing or properly handling these samples. If the problem persists, try updating to the latest version of the Ultralytics package.

[arjunchandra2]

Hello. I would like to use YOLOv8 hyperparameter tuning but also I want to optimize the copy-paste augmentation. I noticed by default for hyperparameter tuning the copy-paste augmentation is set to zero across all iterations and not explored. How can I enable this?

[glenn-jocher]

@arjunchandra2 to enable copy-paste augmentation during hyperparameter tuning, you can manually set the copy_paste parameter to a non-zero value in your initial hyperparameters configuration. This will allow the tuning process to explore variations of this augmentation. For more details, please refer to the Ultralytics Hyperparameter Tuning Guide.

[Chebil-Ilef]

hello, I am having trouble running the fine tuning code below

"""
from ultralytics import YOLO

Initialize the YOLO model

model = YOLO("yolov8n.pt")

Tune hyperparameters on COCO8 for 30 epochs

model.tune(data="GlobalWheat2020.yaml", epochs=1, iterations=4, optimizer="AdamW", plots=True, save=True, val=True)
"""

paths are correct for both model and data
yet I get

�[34m�[1mTuner: �[0mInitialized Tuner instance with 'tune_dir=C:\Users\msi\runs\detect\tune'
�[34m�[1mTuner: �[0m💡 Learn about tuning at https://docs.ultralytics.com/guides/hyperparameter-tuning
�[34m�[1mTuner: �[0mStarting iteration 1/4 with hyperparameters: {'lr0': 0.01, 'lrf': 0.01, 'momentum': 0.937, 'weight_decay': 0.0005, 'warmup_epochs': 3.0, 'warmup_momentum': 0.8, 'box': 7.5, 'cls': 0.5, 'dfl': 1.5, 'hsv_h': 0.015, 'hsv_s': 0.7, 'hsv_v': 0.4, 'degrees': 0.0, 'translate': 0.1, 'scale': 0.5, 'shear': 0.0, 'perspective': 0.0, 'flipud': 0.0, 'fliplr': 0.5, 'bgr': 0.0, 'mosaic': 1.0, 'mixup': 0.0, 'copy_paste': 0.0}
WARNING ❌️ training failure for hyperparameter tuning iteration 1
[WinError 2] The system cannot find the file specified
Saved C:\Users\msi\runs\detect\tune\tune_scatter_plots.png
Saved C:\Users\msi\runs\detect\tune\tune_fitness.png

�[34m�[1mTuner: �[0m1/4 iterations complete ✅ (2.54s)
�[34m�[1mTuner: �[0mResults saved to �[1mC:\Users\msi\runs\detect\tune�[0m
�[34m�[1mTuner: �[0mBest fitness=0.0 observed at iteration 1
�[34m�[1mTuner: �[0mBest fitness metrics are {}
�[34m�[1mTuner: �[0mBest fitness model is C:\Users\msi\runs\detect\train
�[34m�[1mTuner: �[0mBest fitness hyperparameters are printed below.

Printing '�[1m�[30mC:\Users\msi\runs\detect\tune\best_hyperparameters.yaml�[0m'
........

it's like the fine tuning is not doing the training at all and there is no folder train, why is that ? how to solve this ?

thank you

[glenn-jocher]

It seems like there might be an issue with the file paths or permissions. Please ensure that all paths are correctly specified and accessible. If the problem persists, try running the script with administrative privileges or check for any missing dependencies.

[Chebil-Ilef]

thank you for your reply yet I want to note that paths are correct and when I do model.train(..) instead of model.tune(..) the training runs correctly without any errors ..

This is a snapshot of the code:

and this is a snapshot of how tuning is saving files (yet returning errors and its not actually running correctly , params are not the actual best params and training is not being done folder train doesnt not exist)

ERROR:

WARNING ❌️ training failure for hyperparameter tuning iteration 1
[WinError 2] The system cannot find the file specified
Saved C:\Users\msi\runs\detect\tune\tune_scatter_plots.png
Saved C:\Users\msi\runs\detect\tune\tune_fitness.png

�[34m�[1mTuner: �[0m1/10 iterations complete ✅ (2.67s)
�[34m�[1mTuner: �[0mResults saved to �[1mC:\Users\msi\runs\detect\tune�[0m
�[34m�[1mTuner: �[0mBest fitness=0.0 observed at iteration 1
�[34m�[1mTuner: �[0mBest fitness metrics are {}
�[34m�[1mTuner: �[0mBest fitness model is C:\Users\msi\runs\detect\train
�[34m�[1mTuner: �[0mBest fitness hyperparameters are printed below.

[Chebil-Ilef]

NOTE:
training doesn't run correctly for me unless I wrap it inside if name=="main" like below:

if name=="main":
from ultralytics import YOLO
model = YOLO("yolo11n.pt")
results = model.train(data="coco8.yaml", epochs=100, imgsz=640)

so I figured that's what's making the tuning fail too so I tried running the code of model.tune(..) in jupyter and it actually worked

[glenn-jocher]

It seems like the issue might be related to how the script is executed. Wrapping the code in if __name__ == "__main__": is often necessary to prevent certain multiprocessing issues, especially on Windows. Since it worked in Jupyter, try running the script in a standalone Python environment with this structure. If the issue persists, please ensure all dependencies are up to date.

[vodatvan01]

I'm encountering an error when running the following code:

from ultralytics import YOLO

model = YOLO("yolo8s.pt")
model.tune(data="mydata.yaml", epochs=30, iterations=50, optimizer="AdamW", plots=False, save=False, val=False)

The error is:
SyntaxError: 'copy_paste_mode' is not a valid YOLO argument. Similar arguments are i.e. ['copy_paste=0.0'].
I have updated to the latest version of ultralytics, but the error still persists. Any help would be appreciated!

[glenn-jocher]

It seems like the error might be due to an incorrect argument. Please ensure that all arguments passed to model.tune() are valid and correctly formatted. If the issue persists, double-check the latest documentation for any updates or changes in argument names.

[isabinimam]

Hey there, i want to ask. What a calculation or formula that hyperparameter tuning use for get the fitness score when we using model.tune? is there a spesific formula?

[glenn-jocher]

@isabinimam the fitness score used during model.tune in Ultralytics YOLO is based on key metrics such as AP50, precision, recall, and loss values, aggregated into a single score for model optimization. For further details, you can refer to the Hyperparameter Tuning Guide. Let us know if you have additional questions!

[isabinimam]

Alright, thank for the answer. ive been training my model using model.tune. I want to do it again continuing from my last progress. Where should i put best_hyperparameter.yaml declaration? is it in data : ?

[glenn-jocher]

@isabinimam to continue tuning from your last progress, you can load the hyperparameters from best_hyperparameters.yaml by explicitly defining them in your training script. Instead of specifying it under data:, you would manually load this YAML file and pass the hyperparameters to your training method like so:

from ultralytics import YOLO

# Initialize your model
model = YOLO('yolo11n.pt')

# Load best hyperparameters
model.overrides.update('path/to/best_hyperparameters.yaml')

# Start tuning
model.tune(data="coco8.yaml", ...remaining parameters...)

Ensure the YAML file path is correctly set, and let me know if further clarification is needed! For more details, refer to the Hyperparameter Tuning Guide.

[isabinimam]

Ah, that really nice. Thank you for your help!

[glenn-jocher]

You're very welcome! I'm glad I could assist. If you have any more questions about Ultralytics YOLO or need further guidance, feel free to ask. Happy tuning!

[isabinimam]

Hey there. What is the exact formula for fitness score in model.tune function? Where i can find it?

I want to compare my grid search method performance with model.tune performance using fitness score on model.tune as based. What the exact formula for this function fitness score?

[glenn-jocher]

@isabinimam the fitness score in the model.tune() function is typically a composite metric, such as fitness = mAP50-95 - (box_loss + cls_loss + dfl_loss), optimized during hyperparameter evolution. Specific details can be found in the train.py or tuner.py codebase. For more information, please review the Tuner class source code.

[isabinimam]

Hey thanks, but i still can't find the exact function. But i've found this function for yolov5. is this calculation still being used in yolov11?

def fitness(x): 
    # Returns fitness (for use with results.txt or evolve.txt) 
    w = [0.0, 0.0, 0.1, 0.9]  # weights for [P, R, [email protected], [email protected]:0.95] 
    return (x[:, :4] * w).sum(1)

[glenn-jocher]

@isabinimam the fitness function for YOLO11 has evolved slightly and is not identical to the YOLOv5 implementation you shared. For YOLO11, fitness is typically calculated as a composite of metrics such as mAP50-95 minus weighted losses (box_loss, cls_loss, etc.). You can reference the exact implementation in the tuner.py file here: ultralytics/engine/tuner.py. Let us know if you need further clarification!