Getting rid of duplicate emails across mboxes

[alaflaquiere]

I just discovered GYB recently, and it has proven very helpful! So many thanks to the team.

I do have a question regarding duplicate emails and how to get rid of them.

In summary, I have an older Google Takeout folder that I want to upload to a new gmail account. Looking at the content of the mbox files, I realized that many emails actually have duplicates. For instance, the same exact email can be found in the Archived folder, the Open folder and the Important folder. I would like to get rid of those 1) to save some space, as these mbox are quite large, and 2) to speed up the future upload to the mailbox.

I checked the GYB wiki, but I didn't find a way to get rid of such duplicates (for instance, by specifying that only the emails in the Archive folder should be kept).
The closest action that I could find was purge-labels, but my understanding is that it will remove all the labels in the gmail mailbox, and not directly operate on the mbox files. And as a side effect, it would also get rid of all my labels, which is not necessarily something I want. :)

So, is there a way to do what I want with GYB?

Thank you for your feedback!

[jay0lee]

GYB makes no attempt to de-duplicate emails at any time. However Gmail servers do. If you try to restore the exact same message twice you'll be left with only one copy. This is easy enough to test by running multiple restores with the --noresume argument appended.

[alaflaquiere]

Thank you for the lightspeed answer!

Does that mean that under the hood Gmail will only store one email --applying different labels (folders) to it-- and will only create duplicates during the creation of the Takeout?

In any case, I understood that GYB doesn't do this kind of operation, so I think I'll try to implement a script myself to do it.

[Antonio615-cmyk]

Removing duplicate emails in MBOX files can be challenging sometimes, but MailsClick MBOX Duplicate Remover software makes it easy. This software helps the user get rid of duplicate emails without any technical knowledge. Using it, you not only remove unnecessary emails but also reduce the size of your mailbox, making your email management system more organized and faster. The simple interface and fast scanning process of the software make it the ideal solution for everyone. Now free your MBOX files from duplicate emails and enjoy better performance.