Fine tuning/transfer learning in YOLOV4

[DrMRizk]

I trained a network targeting a custom dataset including C classes. I need to train the network with additional images in order to increase the accuracy. I do not need to increase the number of classes.
I aim to re-train the network on edge devices taking into consideration low computational resources and power consumption.

I prepared the dataset including several images of all classes.
Then, I train the network using the custom training command:
./darknet detector train data_file cfg_file yolo4.conv.137 -dont_show -map
for sure the data_file includes the paths to the images/lables for training datset and validation dataset in addition to the path to save the new weights file and the classes names
also, the cfg_file is edited according to the number of classes by changing the max_batches and filters for layers the comes before the yolo layers

After finishing the training on custom dataset, additional images are prepared.

I need to launch the retraining process to enhance the accuracy;
using the following command does nothing:
./darknet detector train data_file cfg_file custom_weights_file -dont_show -map
even when adding stopbackward=1
where custom_weights_file is the output of my initial custom training

I checked the discussions on this Repository:

I guess I am dealing with issue of fine tuning as I am using pre-trained weights file from same task. The following is suggested:

1- use partial to copy the weights of the selected layers from previous obtained weights file.
2- train starting from a pre-trained weight

in my case, I have to use the following command:
./darknet partial cfg_file selected_weights_file weights_file 137
to select the copy the weights of the first 137 layers from the obtained custom_weights_file from my initial custom training.

then I have to use the weight file including the copied weights (selected_weights_file) to launch the retraining process. I shall use this command:

./darknet detector train data_file cfg_file selected_weights_file

a- is up listed procedure correct?
b- shall I use only the new images (not used before in training) or I have to use all images (add the new images to the old image set) in the re-traing process
c- if the new images belong to the same class. would I have degradation because of over-fitting?
d- what is the minimal number of new images should be added?
e- shall I add stopbackward=1 to the cfg_file and what is its affect in my case?

@AlexeyAB
@WongKinYiu

additional image

[AlexeyAB]

Use flag -clear at the end of command

[DrMRizk]

@AlexeyAB
thank you for your reply.
so the command will be:
./darknet detector train data_file cfg_file selected_weights_file -clear ;
or I shall start from the obtained weights file without selecting the weights of the first 137 layers:
./darknet detector train data_file cfg_file custom_weights_file -clear ;

Shall I add the stopbackward=1 to the cfg_file?

thank you in advance

[rishishounak]

@DrMRizk can you tell whether we can start training with just 100 images Would be helpful if you resolve my doubt the training does not seem to start for yolo v4 tiny model in my case.

[eyebies]

It depends on how many boxes you have per image, the lr you use, the complexity and variation in the dataset ! It's rule of thumb that we use 2k per class, but sometimes the network may take long even to overfit for a small set of images.

…

On Tue, Sep 14, 2021, 10:29 rishishounak ***@***.***> wrote: @DrMRizk <https://github.com/DrMRizk> can you tell whether we can start training with just 100 images Would be helpful if you resolve my doubt — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7869 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAX4O367PU3KMOX2NFLZMOTUB3JE3ANCNFSM475I7HTQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[HosseinMohammadii]

@DrMRizk
Hello. Did you finally find out what should you do for training again?
I have question like all of your question. a to e.

[HosseinMohammadii]

@AlexeyAB
Hi. Thanks for your work but It's being 2 days that I'm reading issues and still I'm confused.
I think it worth to publish or add to README an article about this subject because it is asked a lot.
I know some times it becomes hard and painful to explain every thing to people that are not skilled as you are but it worth.
I mention some of issues I read and saved since yesterday and I'm not understood yet completely what should I do.
Thanks for your attention and patient.

The difference between Transfer learning and Fine tuning -
Which layer to freeze for Transfer learning in yolov3 tiny
How to Pretrain a dataset and do Transfer Learning on Darknet?
Setting the param stopbackward=1 means fine-tuning ?

[AlexeyAB]

@DrMRizk @HosseinMohammadii

a- is up listed procedure correct?

In general - yes. Just if you train for the same number of classes, you can train using ./darknet detector train data_file cfg_file yolo_custom.weights -clear instead of partial weights ./darknet detector train data_file cfg_file yolo_custom.137
Where -clear flag just resets the iteration counter inside weights-file, so you can train it again.

b- shall I use only the new images (not used before in training) or I have to use all images (add the new images to the old image set) in the re-traing process

You should use new+old images for finetuning/transfer learning if you want to detect new+old objects, otherwise the network will forget old objects.

c- if the new images belong to the same class. would I have degradation because of over-fitting?

No.

d- what is the minimal number of new images should be added?

The more the better, there is no specific amount. In total, preferably not less than 200 - 2000 images for old+new

e- shall I add stopbackward=1 to the cfg_file and what is its affect in my case?

stopbackward=1 is for freezing layers before this keyword, this is only needed to speed up training and reduce GPU-memory consumption.

[levipereira]

@AlexeyAB makes sense add new feature to priorize only theses new with more iteration to avoid keep retraining already trained images?
Prioritize means theses new will get a little more iteration than others trained images.
#8152

[HosseinMohammadii]

@AlexeyAB
Thank you very very much for your responsibility and quick reply.