forked from imapsync/imapsync
-
Notifications
You must be signed in to change notification settings - Fork 0
/
TODO
1375 lines (996 loc) · 47.2 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/bin/cat
# $Id: TODO,v 1.257 2021/07/05 20:19:31 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
===========================================================================
TODO file for imapsync
----------------------
https://imapsync.lamiral.info/TODO
SUGGESTED 2021_05_29 on github https://github.com/imapsync/imapsync/issues/78
ripmime https://github.com/inflex/ripMIME
altermime http://pldaniels.com/altermime/
SUGGESTED 2021_05_29 by Gilles
Add --regexbody --regexheader, similar to --regexmess
Hum hum, time goes by:
SUGGESTED 2016_06_09 by Gilles and David Carter
In order to avoid useless headaches from --regexmess, add
--regexheader
--regexbody
SUGGESTED 2021_01_26 by Gilles and Martin L. R. @duelhost.dk
I am trying to move some e-mail using imapsync but the domain I'm moving
to have the Danish "ø"-character which it doesn't seem like imapsync can handle.
Make a functional test of this scenario.
SUGGESTED 2021_01_26 by Gilles and Mike
https://github.com/imapsync/imapsync/pull/266
Maybe Gmail has changed and that currently "All Mail" really contains all emails
with all the labels well set to do a good Gmail to Gmail sync but I have to
investigate on that.
SUGGESTED 2020_12_11 by Gilles
Look at XAPPLEPUSHSERVICE for iCloud imap service Apple (Is it Dovecot masquerade?)
SUGGESTED 2020_12_11 by Gilles
Check my Office365 account (outlook.com) and play with the configuration
to still allow imap with imapsync. It already works but it is an old account.
It looks like new O365 accounts are less likely to allow basic LOGIN imap
with a password.
SUGGESTED 2020_12_11 byTimo Smit
Allow the syntax --host1 "example.com/Sent"
to sync only the Sent folder.
SUGGESTED 2020_11_30 by Gilles
Add a FAQ.Invalid_Argument.txt
This error is a low level connexion issue, either ipv6 or dns resolution of
the name.
SUGGESTED 2020_11_24 by Gilles
On Centos 8 SELinux is blocking imapsync online /X by default.
Dig into this to find the commands that will allow only the online /X service.
Instead of:
sestatus
setenforce 0
sestatus
SUGGESTED 2020_11_21 by Mike Alexander https://github.com/mtalexander and Wittmer, Christian https://github.com/computersalat
Make a "make github" target in Makefile and add an automatic
git tag 1.977 # (not 1.977 of course, the actual release number)
https://github.com/imapsync/imapsync/issues/156
https://github.com/imapsync/imapsync/issues/251
SUGGESTED 2020_11_01 bilogic
How to uninstalled cpanm modules installed by cpanm and installed by
perl -Mlazy imapsync
https://github.com/imapsync/imapsync/issues/211#issuecomment-719889417
SUGGESTED 2020_11_04 by Tim O (and others before)
Add an option to sync newest emails first.
SUGGESTED 2020_10_15 by Alexander Perlis
Bugfix. With --delete1 duplicate messages on host1 are deleted only
one at a time, one at each run. Imapsync should delete them all on first pass.
Add --delete1duplicates
SUGGESTED 2020_09_22 by Jon Ward
--idatefromreceivedheader
As I thought about how this option works, I wondered if it makes more sense to
use the date from the final RECEIVED header rather than the DATE header?
This seems like a better solution since it avoids the case of wrong or
spoofed times in the DATE header.
Also, I would guess that in most cases the final RECEIVED header is for
the closest server (to me) and is probably the most correct date/time.
SUGGESTED 2020_09_17 by Peter Franken
I found a bug:
If I use the parameters "--subfolder1 Sent/2017 --subfolder2 Sent/2017",
the mails from "Sent/2017" on Host 1 are transferred to "Sent/2017/INBOX" on Host 2.
==============================================================================
OAUTH2 requests
Readings
https://developers.google.com/identity/protocols/oauth2
https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/
https://unix.stackexchange.com/questions/495691/how-to-provide-oauth2-client-id-and-client-secret-to-an-open-source-bash-scr
https://stackoverflow.com/questions/64150126/best-practice-for-open-source-applications-using-imap-with-oauth2-regarding-cred
SUGGESTED 05/12/2020 by David Carter
https://massivescale.com/microsoft-v2-endpoint-primer/
A big complication is https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth :
"Get an access token
... you can select an appropriate flow from the following list and follow the corresponding steps to call the underlying identity platform REST APIs and retrieve an access token.
OAuth2 authorization code flow
OAuth2 Device authorization grant flow
OAuth access to IMAP, POP, SMTP AUTH protocols via OAuth2 client credentials grant flow is not supported."
SUGGESTED 2020_10_15 by Alexander Perlis
https://gitlab.com/muttmua/mutt/-/blob/master/contrib/mutt_oauth2.py.README
https://gitlab.com/muttmua/mutt/-/blob/master/contrib/mutt_oauth2.py
SUGGESTED 2020_09_10 by Joe Rizzo
I can follow these instructions and authenticate to imap.
Is there a way I can use the access token generated by this process to authenticate with imapsync?
https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough
$ python2.7 oauth2.py --test_imap_authentication --access_token=XXXX [email protected]
35:05.38 > CCPK1 AUTHENTICATE XOAUTH2
35:05.39 < +
35:05.39 write literal size 276
35:05.46 < * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584
35:05.46 < CCPK1 OK [email protected] authenticated (Success)
SUGGESTED 2021_01_07 by many people
Add OAUTH2 individual to Gmail and Office365
http://vdirsyncer.pimutils.org/en/stable/config.html#google
Office365:
https://techcommunity.microsoft.com/t5/exchange-team-blog/announcing-oauth-2-0-support-for-imap-and-smtp-auth-protocols-in/ba-p/1330432
https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow
https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-device-code
SUGGESTED on 31 Jan 2017 by tibbsbrookside
Try this code XOAUTH2
https://github.com/imapsync/imapsync/issues/87#issuecomment-276298841
End of OAUTH2 requests
==============================================================================
SUGGESTED 2020_04_16 by Gilles and tom@mklab
Reinvestigate the memory issue for huge folder with
326691 messages in them. It sounds that imapsync
take 2.5 Ko memory per message.
Trace memory consumption by message.
SUGGESTED 2020_04_23 by Gabriel
Folders Size human readable
une option pour avoir la taille des dossiers individuels en MB,
lors de l'utilisation de --justfoldersizes ?
SUGGESTED 2020_03_02 by https://github.com/gpothier
https://github.com/imapsync/imapsync/issues/27#issuecomment-593425945
Bugfix. Do fail when --minage or --maxage are employed with --search
Or make them be consistent, ie, do an AND between them.
Or at least document the bad current behavior which is that
--search overrides any --minage or --maxage
SUGGESTED 2020_01_03 by Gilles
Bugfix. When running ./imapsync --tests --testslive
Host Nb folders are 0, which is false.
SUGGESTED 2020_01_03 by Gilles
https://lamiral.info/~gilles/imapsync/cover_db/imapsync.html
is result of ./imapsync --tests --testslive
Increase the subroutine coverage to 100% by
removing unused ones or add a test call for the remaining.
SUGGESTED 2019_12_26 by Gilles
For /X Look at
https://en.wikipedia.org/wiki/ModSecurity
SUGGESTED 2019_12_26 by Gilles
Look at
https://metacpan.org/pod/Test::Warnings
SUGGESTED 2019_12_26 by https://github.com/masbaehr
Add the "throttle" word in the README about
--maxmessagespersecond --maxbytespersecond
See https://github.com/imapsync/imapsync/issues/204
SUGGESTED 2016_07_12 by Fronik
With --automap apply the mapping to subfolders of mapped folders.
If
Sent => Envoyes
then
Sent.Foo => Envoyes.Foo
Sent.Foo.Bar => Envoyes.Foo.bar
SUGGESTED 2019_10_22 by Gilles
Add the download link in imapsync itself, with --releasecheck
SUGGESTED 2019_10_20 by Olav Seyfarth
One suggestion/request regarding INSTALL.OnlineUI.txt: Please add a
typical NGINX setup since NGINX is used very often now and requires a
quite different setup: one has to set up a fcgi service.
SUGGESTED 2019_10_02 by Joel from Blue Net Box
Fix this issue:
imapsync_bin_Darwin cant be opened because its integrity cannot be verified.
This software needs to be updated. Contact the developer for more information.
MacOS 10.15 is the new macOS Catalina (in the last beta phase)
This error message happens in a shell window.
Same imapsync_bin_Darwin in macOS 10.14 works fine.
SUGGESTED 2019_10_02 by Gilles
Look at Traefik to automatic ssl services (webserver).
RE_SUGGESTED 2019_09_26 by Andre Adrian (rotlaus)
Allow @ in subfolder names #195
https://github.com/imapsync/imapsync/issues/195
Allow also & (imap_utf7 character)
Or add a --nosanetizesubfolder option.
SUGGESTED 2019_07_01 by Jeffrey Thatplayer
https://github.com/imapsync/imapsync/issues/181
Make sanitizing an option for --subfolder1 --subfolder2
Two options? --nosanitize1 --nosanitize2
Reflecting more I wonder if sanitizing for --subfolder1 is just stupid.
It's not, it's good from the usage point of view, same string to point out
the destination backup/restore folder
SUGGESTED 2019_09_23 by Celko Michal.
My proposal is to create a status file.
it would be a command line option, ex.
imapsync ... --status-file /var/www/hosting/tmp/xyz.json
When I start imapsync process with this option, it would generate
and update this status file every few seconds, so any application
can access it and see what's going on with this process without
parsing logs or keeping up with the log format when it changes.
Example of such status file would be:
In good progress
{
"status": "in progress",
"progress": "245/5000",
"message_rate":"7.06 msgs/s",
"bandwidth_rate":"2.801 KiB/s",
"errors_count": "0",
"logfile": "LOG_imapsync/[email protected][email protected]",
"pid": "123456",
"exit_value": ""
}
On complete example*
{
"status": "finished",
"progress": "5000/5000",
"logfile": "LOG_imapsync/[email protected][email protected]",
"pid": ""
"exit_value": "0"
}
SUGGESTED 2019_07_30 by Gilles
Only since imapsync 1.774 the release check is done with
\r\n to end lines, instead of the not RFC compliant \n.
If I upgrade Apache on imapsync.lamiral.info
ii apache2-mpm-worker 2.2.22-13+deb7u7 amd64 Apache HTTP Server - high speed threaded model
then any imapsync prior to 1.774 will fail to GET the VERSION file.
Release check life with names:
PeerAddr "linux-france.org" in release 1.350
"Host: www.linux-france.org\n\n" in release 1.350 to 1.466
PeerAddr "imapsync.lamiral.info" in release 1.374 and upper
"Host: www.linux-france.org\n\n" in release 1.464 but not after
"Host: ks.lamiral.info\n\n" in release 1.467 and upper
"Host: ks.lamiral.info\r\n\r\n" in release 1.836 and upper
Solution to test: Use the directive
HttpProtocolOptions Unsafe # available from 2.2.32 or 2.4.24
http://httpd.apache.org/docs/2.4/mod/core.html#httpprotocoloptions
ks5 should be ok: Server Version: Apache/2.4.33 (FreeBSD)
SUGGESTED 2019_06_18 by Konrad Wawryn
Add NTLMv2 support
It shouldn't be that hard to support NTLMv2 because
the underlying NTLM module can do it already.
Delegate all the stuff to underlying Mail-IMAPClient.
SUGGESTED 2019_06_05 by Gilles
Code equivalent W/tools/fix_email_for_exchange.py
inside imapsync
SUGGESTED 2019_04_10 by Gilles
With signals show also their numbers
https://perldoc.perl.org/Config.html
SUGGESTED 2019_04_03 by Katja Wolf and Gilles
Put a summary like the one in the log file
in the source and the destination mailbox.
SUGGESTED 2019_03_19 by Gilles
In the final errors listing, add lines
* Host1 banner
* Host2 banner
* Command line parameters
I'm a little fade up to ask them
in order to understand the context
SUGGESTED 2019_03_04 by Gilles
Write a FAQ about deciphering with imapsync.
Test it!
If the encryption is made using gpg
then the decryption via --pipemess could be just the command:
imapsync ... --pipemess 'gpg --decrypt --no-batch --no-tty --override-session-key 9:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
where you previously ran:
gpg --decrypt --no-batch --show-session-key some_rfc822_filecrypted.txt
in order to get the session-key "9:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
SUGGESTED 2019_02_28 by Gilles
Default tmpdir could become ./TMP_imapsync
for --usecache if there is no cache in the
historical tmpdir File::Spec->tmpdir( )
Why? Some systems like Windows do change
their tmpdir upon reboot.
Some Unixes do remove their /tmp upon reboot.
SUGGESTED 2018_09_11 by Gilles
Use Smart::Comments in long loop
Use Smart::Comments in debug/verbose mode
SUGGESTED 2018_09_11 by Gilles
Take a look at
Test::NoWarnings
Test::Warn
SUGGESTED 2018_05_08 by James B. Byrne
Sync also the quota with SETQUOTA
https://tools.ietf.org/html/rfc2087
SUGGESTED 2017_12_19 by Gilles
In foldersizes, add info about the oldest message found: INTERNALDATE EPOCH DAYS_FROM_NOW
See draft in sub info_date_from_uid
SUGGESTED 2017_08_31 by Gilles
Makefile. Add a "make docker" goal that build, test and publish the
docker package on ks3.
SUGGESTED 2017_08_22 by Ismael Baena
when hmailserver hierarchy delimiter is \
When imapsync asks hmailserver about namespace, hmailserver returns \\ instead of \.
After that, imapsync fails when accessing any folder. Comparing hmailserver logs between
imapsync and outlook accessing the account, I see outlook quoting every path
while imapsync dont, and outlook doesnt fail.
I wrote a message to the hmailserver forum,
https://www.hmailserver.com/forum/viewtopic.php?f=7&t=31503
SUGGESTED 2017_05_04 by Gilles
Verif ipv6 connection, seems to fail now
./imapsync --host1 2603:1026:200:52::2 --host2 2603:1026:200:52::2 --justconnect
...
Host1: connecting on host1 [2603:1026:200:52::2] port [143]
Host1: Can not open imap connection on [2603:1026:200:52::2]: Unable to connect to 2603:1026:200:52::2: Invalid argument
SUGGESTED 2017_05_04 via github
Review the "make install" in Makefile
SUGGESTED 2017_04_26 by Franco Fassio
--usecache with trailing dots foo... in folder name breaks on NTFS.
SUGGESTED 2016_11_16 by Flávio Zarur Lucarelli LucaNet Sistemas
Getting subjects of messages for duplicates and print them.
Dates and sizes are easier printable because imapsync gets them each time.
SUGGESTED 2016_10_24 by Gilles
Verify regex --delete2foldersbutnot and --delete2foldersonly are ok
before anything.
SUGGESTED 2016_10_07 by Gilles
Write a FAQ file for packagers,
mostly how to get the dependency Perl modules complete.
SUGGESTED 2016_10_07 by Gilles
Write a FAQ file about all errors and explain what they mean.
Maybe propose an action to solve them in the code,
or even solve them directly in the execution.
SUGGESTED 2016_10_06 by Gilles
Document or fix behavior (do a AND),
curently "--search" makes "--maxage" and "--minage" ignored.
SUGGESTED 2016_09_16 by Gilles
Add a check of all --regextrans2 before going further,
like what is done with --regexmess
SUGGESTED 2016_08_25 by David Carter
It would be helpful if imapsync reported Message-Id:/From:/Subject
as well as folder name and message UID when it skips a message.
SUGGESTED 2016_09_02 by Gilles & David Carter
Exchange Online errors that need a relogin:
"BAD User is authenticated but not connected."
"error while reading data from server: Connection reset by peer"
"timeout waiting 1200s for data from server"
SUGGESTED 2016_08_31 by Gilles
Rename $expungeaftereach $expungeaftereach1 or $expunge1aftereach
SUGGESTED 2016_08_19 by Gilles
Go back to SSL_VERIFY_PEER but include
SSL_ca_file inside imapsync or near.
SUGGESTED 2016_07_29 by Gilles from CSS3 Coursera course
Take a look at
https://modernizr.com/
SUGGESTED 2016_07_29 by Gilles
Move website to HTML5
https://about.validator.nu/
SUGGESTED 2016_07_07 by Jean-Dominique Delyon.
Add a way to know easily which account transfers went wrong.
Générer un fichier des comptes qui ont rencontré des problèmes
et afficher le contenu à la fin de la boucle sur les comptes.
SUGGESTED 2016_06_29 by Gilles
Clarify system flags in RFC
Example to add:
If \Forwarded is not in PERMANENTFLAGS but \* is, then transform
\Forwarded to Forwarded. And by default.
SUGGESTED 2016_06_13 by David Carter
--pipemess could also treat stderr
` $command < $input_tmpfile > $output_tmpfile 2> $error_tmpfile `
SUGGESTED 2016_06_22 by Gilles
Add --quiet mode
Make --quiet mode being switched by a signal
SUGGESTED 2016_06_12 by Gilles
Fix many perlcritic, all?
DONE * 60 perlcritic Double-sigil dereference (Severity 2)
DONE * 627 perlcritic violations of CodeLayout::ProhibitTrailingWhitespace. (Severity 1).
DONE * 458 perlcritic violations of ValuesAndExpressions::ProhibitInterpolationOfLiterals. "Useless interpolation of literal string"
DONE * 420 perlcritic violations of CodeLayout::ProhibitParensWithBuiltins. "Builtin function called with parentheses". (Severity: 1).
SUGGESTED 2016_06_09 by David Carter
Add --pipemess-on-fail
If, and only if, the target server responses with NO or BAD then have another go
with a sanitized version of the message.
SUGGESTED 2016_06_01 by M. Beaubien
Add flag --noheaderwarnings or better name
no header by parse_headers so taking whole header with BODY.PEEK[HEADER]
no header found so adding our own [Message-Id: <151648@imapsync>]
SUGGESTED 2016_04_17 by Gilles
Add a --passfile to allow user=>password style file.
WANTED 2016_02_10
Add stats about "Messages found crossduplicate on host1"
WANTED 2016_01_28 Stephen Sookdeo
Able to send email without attachment (in case size is the problem) and maybe note in main body that attachment was removed...
See http://unix.stackexchange.com/questions/174707/remove-delete-attachments-from-email-server-imap
http://unix.stackexchange.com/a/174726/89270
imapsync ... --pipemess 'remove_attachments --sizemin 10MB --addnote "Attachment was removed" '
WANTED 2016_03_06
Add a real test to authenticate with a ssl certificate.
WANTED 2016_03_08
Check if --ssl1 and --tls1 are used together and warn abort and abort in that case.
Same for --ssl2 and --tls2
Start a wiki for imapsync.
Write a Windows tutorial, TUTORIAL_Windows.html.
Write a good practices migration tips document, GOOD_PRACTICES.html
Add uidexpunge with --delete if possible (like with --delete2).
Add the possibility to specify a unity with --maxsize and --minsize
Example --maxsize 10MB --minsize 10KB
B
KB
MB
GB
TB
WANTED 2015_12_16 Gilles Lamiral
Add "df -i" with usecache and abort if the number of messages
to transfer will exhaust empty inodes used by the cache.
Looks like module Filesys::DfPortable will help for Unix and Win32.
==== Similar skip/error messages ====
WANTED 2016_01_28 Stephen Sookdeo
List all emails that gives errors so you know exactly which. List with detailed info.
Then ability to perform action on these specific emails like delete or ignore per email.
WANTED 2015_06_02 Karen F Bath.
Add skipped messages in the final dump.
Print the list of messages not copied and why (duplicates or void header).
I would like to request if you could add additional errors to the bottom,
as we find that things like MaxLineLength and maxsize limit are classed
as skipped messages and in my opinion are errors; as the email message
is not transferred but this is not logged at the bottom.
We have our own scan script which we run on all log files at the end
and copy the users logs into subfolders that have issues.
I've attached a list of things we search for.
"Error", "Output"
NO Mailbox already exists, "Folder"
NO LOGIN failed, "NoLogin"
Could not create folder, "Linux"
Error sending, "Linux"
Write failed 'no error caught', "Linux"
line length exceeds maxlinelength, "Linux"
BAD Command Argument Error. 11, "Linux"
Failure: can not open, "Linux"
No not connected, "Linux"
Write failed 'Connection reset by peer', "Linux"
could not be fetched:, "Linux"
NO Mailbox does not exist,"Error"
exceeds maxsize limit 20971520 bytes,"Error"
error while reading data from server: Connection reset by peer (4x),"Error"
Folder [Inbox] already exists,"Error"
WANTED Add a well described problem for each problem detected
and counted in error counter statistics.
==== End Similar skip/error messages ====
WANTED 2015_05_18
I'd like to be able to print the messages subjects, on some logs:
For instance:
msg INBOX.Archived/95551 marked \Deleted on host2 [GyDD6WpsFEtyBzNFnv]
msg [Gmail]/All Mail/281079 copying to INBOX.Archived
It would be great to be able to print the email subject, in order to debug or to get more useful information.
WANTED 2015_04_25
Add duplicates test option.
WANTED 2015_03_06
Dealing with Content-Type Message/Partial
Extract the components of the partial messages and construct them
with reformime as one message which can then be transferred. (Larry Moore)
See also uudeview http://www.fpx.de/fp/Software/UUDeview/
(Larry said uudeview is weird on Partial issue, too old maybe)
Apply --disarmreadreceipts only to UNSEEN messages.
Add an exit value when exiting because of --exitwhenover
The goal is to know easily why to restart later.
Write a Mail::imapsync package and use it.
One day, when I have really nothing better to do, evaluate:
http://www.rackspace.com/apps/email_hosting/migrations
http://www.yippiemove.com/
http://www.migrationwiz.com/
http://www.microsoft.com/download/en/details.aspx?id=1329 "Microsoft Transporter Suite"
Add --mark-as-deleted1 --mark-as-deleted2 as
aliases for --noexpunge1 --delete1 and --noexpunge2 --delete2
Fix bug found by Pavel Stano on 01/06/2012 (june) imapsync never stop login
when login fails with a "* BYE Temp error" from server.
Consider /var/tmp/ instead of /tmp (/tmp is destoyed
on some Unix at reboot)
Fix long path over than 260 character on Win32 with --usecache
Fix inode crunching with --usecache
Think about Digest::SHA or Digest::SHA::PurePerl.
Think about a file database like DBM instead.
Look at https://en.wikipedia.org/wiki/Radix_tree (David M. advice)
Look at leveldb (Simon Th actux advice)
Look at http://fallabs.com/kyotocabinet/ (JYB advice)
Also Redis (Ludovic Danigo advice)
I am writing now to suggest that imapsync provide info containing the parameters
with which a message is determined to be a duplicate, so that if I wanted to confirm
they are in fact duplicates by looking at the messages themselves, enough info would
be available to easily locate them. One or two duplicates per folder are easy to
find, but I had one large folder with quite a few, and in this case it would have
been a big help.
Explain that users can win time/bandwidth by using --expunge
Fix "\Forwarded" flag bug in courier.
Does \lalala can be forbidden (courier does a
"16 NO Error in IMAP command received by server"
with
* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited
Add sync imap keywords. Sync Gmail labels to imap keyword
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503159
http://www.linux-france.org/prj/imapsync_list/msg00022.html
http://mail.google.com/support/bin/answer.py?hl=en&answer=77657
http://mail.google.com/support/bin/answer.py?answer=78892
Add and option to sync to & from files.
Add an --aclregextrans2 flag.
"Today we discovered, that Cyrus and Dovecot use different characters for
their ACLs. Syncing ACLs vom Cyrus to Dovecot (at least 1.2) doesn't
work. Cyrus uses c and d, Dovecot uses k and x instead."
Peer Heinlein.
Add more information about skipped messages.
Add Rick Romero patch with
--quiet No output at all
--showstats intended for use with --quiet
Take a look at Mail::IMAPTalk
Simon Bertrang said "way better performance, less problems, easier to use and no
issues so far". Sounds good!
imapsync doesn't report well. It should says "I had
to sync 123 messages but I could transfer only 99 messages"
Maybe count messages not transferred because they're dupplicate.
Fix bug "not possible to use space in the imap password"
Add kerberos authentification
Fix bug with folder names starting with an asterisk: *Archiv
Sending: 220 SELECT *Archiv/Support/foobar
Read: 220 BAD Protokollfehler: "Invalid folder specified in SELECT command"
Add LITERAL+ [RFC 2088] support to imapsync.
Make a m4 file for lsm.imapsync and generate it at each public release.
Add --verbose from Kjetil jumbo patch.
Read the IMAP RFC http://www.faqs.org/rfcs/rfc3501.html
Read:
http://www.washington.edu/imap/documentation/commndmt.txt.html
Add cyrus link about INBOX. namespace
http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html
===========================================================================
Now the TODO done! (or not)
===========================================================================
DONE 2021/04/30 revision 2.111 by Gilles
SUGGESTED 2021_04_22 by popular request
Add an option --syncduplicates that syncs duplicates.
DONE 2019/12/16 revision 1.974 by Gilles
Use Debug_fh to set where go the --debugimap outputs.
SUGGESTED 2017_09_04 by Gilles
STDOUT + logfile instead of STDERR with Mail::IMAPClient output.
Should be easy using $imap->Debug_fh( $fileHandle ) ;
DONE 2019_12_11 by Gilles
SUGGESTED 2019_11_28 by Gilles
Find a place to add this advice:
https://github.com/imapsync/imapsync/issues/201#issuecomment-559500077
Place: https://imapsync.lamiral.info/FAQ.d/FAQ.Docker.txt
DONE 2019_11_27 by Gilles
SUGGESTED 2019_09_30 by Gilles
Update the docker build
Mail::IMAPClient 3.38 to latest
Ok Mail::IMAPClient 3.42 in docker gilleslamiral/imapsync 1.967 a82b5d5ff2f0
DONE 2019_11_19 by Gilles revision 1.961
SUGGESTED 2019_11_13 by Flavio
Are --delete2foldersonly or --delete2folders or --delete2foldersbutnot
compatible with --subfolder2 ?
Yes but fix this
--delete2foldersonly SUB --subfolder SUB deletes SUB
while
--delete2foldersonly SUB. --subfolder SUB does not delete SUB
(seen the dot?)
DONE 2019_07_30 by Gilles release 1.955
SUGGESTED 2018_05_15 by Massimo Maggi
https://github.com/imapsync/imapsync/issues/141
Document --skipcrossduplicates and --debugcrossduplicates in imapsync --help
DONE 2019_04_28 by Gilles
SUGGESTED 2016_10_03 by Gilles
Do like tail -F on logfile when
"another imapsync may be curently running",
instead of quit.
Add --tail --notail and --tail on by default in cgi context.
DONE 2019_04_27 by Gilles
SUGGESTED 2019_04_25 by Gilles
Add length of $cache_dir to
Local cache directory: $cache_dir
DONE 2019/04/11 by Gilles revision 1.930
SUGGESTED 2016_08_07 by Gilles
Add a meaningful exit value to all die:
* fatal software dependency, Perl modules
* fatal parameter issue
* fatal connect issue
* fatal login issue
* fatal permission file issue (open)
* fatal IMAP issue
* fatal IMAP disconnection
Maybe replace all die by exit
Add a meaningful exit value to all exit
* exit at end but with errors
* exit at middle because of errormax
* exit by signal
DONE 2017_07_13
SUGGESTED 2016_03_11
Add a FAQ about Authentication failures and quoting.
FAQ.d/FAQ.Authentication_failure.txt
FAQ.d/FAQ.Passwords_on_Unix.txt
FAQ.d/FAQ.Passwords_on_Windows.txt
DONE 2017/05/24 by Gilles revision 1.813
SUGGESTED 2017_04_26 by Franco Fassio
Check that --maxbyteperseconds does not interfer with --timeout
by sleeping too long.
DONE by Gilles revision 1.795 date: 2017/04/22
SUGGESTED 2016_12_01 by Gilles
When working under with webmin and virtualmin imapsync
acts as a cgi, it should not. Find a way to avoid cgi mode
even under a web server. Solution: edit the sub under_cgi_context()
DONE 2019_04_08 by Gilles
SUGGESTED 2019_03_06 by Gilles
when exit by signal, exit by sending a signal to itself:
How to be a proper program
https://www.cons.org/cracauer/sigint.html
DONE 2018_11_20 by Gilles
Count the messages that are on host2 but not on host1.
(hum... see DONE 2018_09_01)
DONE 2018_09_01 by Gilles
Count messages on host2 that are not in host1
DONE 2018/04/18 revision 1.875 by Gilles
SUGGESTED 2018_06_29 by Michael Hesse
Remove the warning on Windows
No such signal: SIGUSR1 at script/imapsync line 11744.
DONE 2017_11_29 by Gilles
SUGGESTED 2017_04_11 by Stefano Lo Cascio
Add a --resyncflags on by default but allowing --noresyncflags
DONE 2017_10_27 by Gilles in revision 1.841
SUGGESTED 2017_09_02 by Gilles
Check that @ARGV is empty after readding it by GetOptions since
remaining arguments are not used and might signal something wrong
in the command line arguments.
DONE 2017_08_31 by Gilles
SUGGESTED 2016_12_14 by Gilles
Look at Pod::Usage and convert the actual doc to it
DONE 2017_04_24 by Gilles
SUGGESTED 2016_11_29 by Gilles
Fix bug passfile no exist => login
Should be fatal sooner.
DONE 2017/03/13 by gilles in revision 1.782
SUGGESTED 2016_06_29 by Gilles
Add --gmail1 --gmail2 --gmail12 to set automatically advices from
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
Add --exchange --office365 to set automatically advices from
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
Add --domino1 --domino2
DONE by Gilles revision 1.755 date: 2017/01/12
SUGGESTED by Gilles
Try to act as --tls if CAPABILITY allows it and if there is no --notls.
Fallback without --tls if --tls fails.
DONE 2016_09_29 by Gilles
SUGGESTED 2016_04_13 by Gilles
Split --noabletosearch in --noabletosearch1 --noabletosearch2
SUGGESTED 2016_09_29 by Gilles
DONE with revision 1.764 2017/01/19
Try to use TLS or SSL by default unless not asked to do so.
DONE 2016_07_27 by Gilles
SUGGESTED 2016_06_15 by Gilles
Fix sub check_lib_version and its call.
It was ok because of the Perl line "use Mail::IMAPClient 3.30".
SUGGESTED 2016_06_22 by Gilles
DONE 2016_07_29 by Gilles
Make reconnections launched by a signal Ctrl-c, aka INT signal.
2 consecutive Ctrl-c within 1 second then exit the program.
Inline help when Ctrl-c is hit
SUGGESTED 2016_07_15 by Alan Williams
DONE 2016_07_20 for IO::Socket::SSL
Add --inet4 --inet6 to force AF_INET or AF_INET6 ip connexion,
aka ipv4 or ipv6.
SUGGESTED 2016_06_24 by Jean-Dominique Delyon
DONE 2016_06_30 by Gilles
Add --delete1emptyfolders
Destroy host1 empty folders.
Be carefull of non-empty subfolders!
SUGGESTED 2016_07_01 by Gilles
DONE 2016_07_01 by Gilles
Add exit explanation after errorsdump
DONE 2016_01_06. Write a Unix tutorial, TUTORIAL_Unix.html.
DONE by Gilles revision 1.677 date: 2016/01/19
Move --help documentation into the man page so that description is easier to find.
DONE 2015_12_26. WANTED 2015_03_24
Add --sslargs with usage like:
imapsync ... --sslargs 'SSL_version=SSLv3' --sslargs 'SSL_use_cert=1' \
--sslargs 'SSL_verify_mode=SSL_VERIFY_PEER'
See perldoc IO::Socket::SSL for all possibilities.
DONE 2015_12_25 Gilles Lamiral
Add --ssldebug 0-4
DONE 2015_10_06. Find a way to avoid passwords in --debugimap unless needed.
Proposed in Mail-IMAPClient via $imap->Showcredentials()
in patch at
See https://rt.cpan.org/Public/Bug/Display.html?id=107592
DONE 2015_12_03 WANTED 2015_11_24 Jens Herrmann
Add --logdir to allow imapsync choosing the filename while allowing
user to choose the dirname part.
DONE. 2015_08_10.
Guess separators and prefixes when NAMESPACE is not available,
instead of forcing the user to guess and set them.
DONE 2015_08_03. WANTED 2015_07_21
Fix W/learn/imap_utf7_encode
--regextrans2 "s,El&AOk-ments,&AMk-l&AOk-ments,"
DONE 2014/10/06 by Gilles revision 1.597
SUGGESTED 2016_08_31 by Gilles & David Carter
(I don't know why the DONE date older than the SUGGESTED date)
Detect equivalent folders by upper/lower case on host1
and add a way to not merge them when host2 is case insensitive.
DONE. Add a FAQ entry about long path over than 260 character on Win32.
DONE. Build and distribute a standalone Darwin Mac OS X binary.
It's called imapsync_bin_Darwin
DONE. Add a NOP for host2 for each fake copy in --dry mode.
Goal is to avoid timeouts happening only because of --dry
DONE 2015_05_09 WANTED 2015_04_25
Add an option "--subfolder2 FOO" to move all folders to a subfolder FOO:
On Windows:
--regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," \
--regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"
DONE. Quota are read if available and warning is printed if 90% quota reached.
Can you setup an option to make it stop if the destination mailbox reports
that it is over quota?
DONE. Add stdin/stdout filter before transfer:
"Now i asked me, how to modify your perl program to work with
that - in example, to write each mail to stdout, pipe that to the
conversion program, and read the result from stdin - and this all before
the mail will transfer to the target imap-server"
http://www.courier-mta.org/maildrop/reformime.html
Look at IPC::Open2, assuming a good and safe pipe,
or write more code handling all the bad stranger behaviours possible.
Option name: --pipemess "command arg1 arg2 ..."
DONE. Transform messages with too long linelength.
Office365 and Exchange abort connexion
when they encounter a message with more than 10000 characters on a single line.
Could be done with the previous --pipemess with reformime.
Thanks to Larry Moore for suggesting reformime tool!
Usage: --maxlinelength 9900 maxlinelengthcmd 'reformime -r7'
DONE. Add DavMail in Similar Softwares section. http://davmail.sourceforge.net/
DONE. Convert folder names to utf-8 and print them next to the utf-7 ones.
Look at imapsync/W/learn/
./imap_utf7 data_utf7
DONE. Separate the online site in a folder site/ or www/ and make the upper directory simple.
DONE. --skipemptyfolders
Is it possible to run imapsync in a way that empty folders on host1
aren't created at all on host2? Yes with --skipemptyfolders
NO. Add an option to remove empty folders on host1, except INBOX.
Use --skipemptyfolders instead
DONE. Subject is output when APPEND fails. Done since a long time.
Suggestion: it's very difficult to track down messages which are behaving
funny during the sync. It would be great - and presumably easy to code -
to have an option to have imapsync display e.g. the subject of an
e-mail when it gets synced, rather than just the message ID and the date/time.
DONE. Add W/learn/imap_utf7 in imapsync, the folders lists output looks a good place,
if transcription is needed.
DONE. Add options --log and --logfile to log both on screen and to a file.
DONE. Document the tee command solution (Unix and Windows) in the FAQ.
NEVER. Add an option to implement the faq entry about copying a contact folder.
(write a dedicated software instead!)
NEVER. Use examine() instead of select() in --dry mode.
Add a method doing the switch automagicaly.
DONE. Write --disarmreadreceipts that does:
--regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms'
DONE. With --foldersizes make the listing on host2 independent of case
(avoid "not yet created" if the folder exists in another case string)
DONE. Write --maxmessagespersecond and --maxbytespersecond for staying under Microsoft's throttling policies.
DONE. Inform about --addheader when the problem occurs "(no wanted headers so we ignore this message)"