I want to overfit yolov4, how would I do that?

[an3252sv]

I want yolov4 to perfectly learn the training set but completely fail on images not from the training set, i.e very bad overfitting. The reason is that my data set consist of images that have been extracted from a video, and I would like to extract more images from this video without getting any duplicates. So if I can train yolov4 to overfit on my current images, I can use the model on the new images and see if it recognizes them. If it does, then the image was a duplicate and I can discard it.

I have edited the cfg-file to remove augmentations, specifically saturation=0, exposure=0, hue=0, mosaic=0, jitter=0, random=0. Are there any other parameters in the cfg-file that would be beneficial for me to edit? Apart from editing the cfg-file, are there any other changes I could do?

Info about my data set: 8000 images, 700 of those have no object in them, 7 classes in total, approximately 2000 bounding boxes per class except for one which has 10000 bounding boxes.

[stephanecharette]

It is extremely unlikely that you'll overfit so much that you can use that to detect if new frames are duplicates of ones already in your dataset.

You'd have better luck using software like DarkMark, which identifies duplicates as part of the "review" functionality. It looks for duplicates by comparing the images once they've been resized to the network dimensions. https://www.ccoderun.ca/darkmark/Summary.html#DarkMarkReview

[an3252sv]

Thanks for your answer, I will look into DarkMark.

I set batch-normalization=0 in the cfg-file together with my other changes which resulted in 60 mAP after 50.000 iterations. So I will not use this approach, but out of curiosity, what other cfg-paramaters do you think I should change to achieve over-fitting.

[Tony904]

I would just write a simple python program with opencv that does an element-wise comparison of each frame against all the images you are checking for duplicates of.

[an3252sv]

Yes, that was my first approach, but I have over 20 videos, and each video is 2 hours long.

I set batch-normalization=0 in the cfg-file together with my other changes which resulted in 60 mAP after 50.000 iterations. So I will not use this approach, but out of curiosity, what other cfg-paramaters do you think I should change to achieve over-fitting.