-
Notifications
You must be signed in to change notification settings - Fork 0
/
NOTES.txt
176 lines (117 loc) · 5.17 KB
/
NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
Homepage: http://imapsync.lamiral.info/
Requires modules.
Test for missing modules with:
perl -mMail::IMAPClient -mDigest::MD5 -mTerm::ReadKey -mIO::Socket::SSL -mFile::Spec -mDigest::HMAC_MD5 -mAuthen::NTLM -mTime::HiRes -mData::Uniqid -mURI::Escape -e ""
Look for error messages and install modules with:
$ perl -MCPAN -e shell
cpan> install Mail::IMAPClient
cpan> (etc)
I just did cycles of install one, test, read the error,
install another, test again, etc.
Once all the modules are installed, test imapsync with
imapsync --noreleasecheck --help
The script has a PILE of options.
Errors look like this:
======================
- msg INBOX.mime/120 {61551} couldn't append (Subject:[Mime test message/partial]) to folder Migrated/INBOX/mime: 493 BAD Command Argument Error. 11
- msg INBOX.mime/126 {51671} couldn't append (Subject:[Mime test message/partial]) to folder Migrated/INBOX/mime: 499 BAD Command Argument Error. 11
For these particular errors the UIDs are 120 and 126 and the
cache folder also skips them (note 119_81 and 121_82, skipping 120_?)
119_81 121_82 122_83 123_84 124_85 125_86 127_87
ls: cannot access 120*: No such file or directory
ls: cannot access 126*: No such file or directory
Caching is in
-------
/tmp/imapsync_cache
unless --tmpdir /var/tmp
to inspect messages by UID, you can use
--------------------------------
imtest -m plain -s -a snizromj imap.exchange.mit.edu
. select INBOX.mime
. uid fetch 120 (body[header])
. uid fetch 126 (body[header])
viewing logs
------------
My launcher makes logs with long unwieldy names.
View logs with:
less `ls -rt1 synclog-* | tail -1`
EXAMPLES
========
imapsync --noreleasecheck \
--justfolders --dry \
--host1 "${host1}" --ssl1 --authmech1 "${auth1}" \
--user1 "${user1}" --passfile1 "${pwdfile1}" \
--host2 "${host2}" --ssl2 --authmech2 "${auth2}" \
--user2 "${user2}" --passfile2 "${pwdfile2}" \
--tmpdir /var/tmp \
--nofoldersizes \
--prefix1 '' \
--folder "INBOX.Sent Mail" \
--regextrans2 's#(.*)#Migrated/$1#' \
--useuid --usecache
++++ Looping on each folder
[INBOX.Sent Mail] -> [Migrated/INBOX/Sent Mail]
imapsync --noreleasecheck \
--dry \
--host1 "${host1}" --ssl1 --authmech1 "${auth1}" \
--user1 "${user1}" --passfile1 "${pwdfile1}" \
--host2 "${host2}" --ssl2 --authmech2 "${auth2}" \
--user2 "${user2}" --passfile2 "${pwdfile2}" \
--tmpdir /var/tmp \
--nofoldersizes \
--prefix1 '' \
--folder "INBOX.Sent Mail" \
--regextrans2 's#(.*)#Migrated/$1#' \
--useuid --usecache \
++++ Looping on each folder
[INBOX.Sent Mail] -> [Migrated/INBOX/Sent Mail]
cache directory: /var/tmp/imapsync_cache/po12.mail.mit.edu/jmorzins/imap.exchang
e.mit.edu/snizromj/INBOX/Sent Mail/Migrated/INBOX/Sent Mail
CACHE h1 h2: 0 files
msg INBOX.Sent Mail/6 copying to Migrated/INBOX/Sent Mail (not really since --dry mode)
msg INBOX.Sent Mail/7 copying to Migrated/INBOX/Sent Mail (not really since --dry mode)
msg INBOX.Sent Mail/8 copying to Migrated/INBOX/Sent Mail (not really since --dry mode)
exec "${imapsync}" --noreleasecheck \
--host1 "${host1}" --ssl1 --authmech1 "${auth1}" \
--user1 "${user1}" --passfile1 "${pwdfile1}" \
--host2 "${host2}" --ssl2 --authmech2 "${auth2}" \
--user2 "${user2}" --passfile2 "${pwdfile2}" \
--tmpdir /var/tmp \
--nofoldersizes \
--prefix1 '' \
--folder "INBOX.Sent Mail" \
--regextrans2 's#(.*)#Migrated/$1#' \
--useuid --usecache \
++++ Looping on each folder
[INBOX.Sent Mail] -> [Migrated/INBOX/Sent Mail]
cache directory: /var/tmp/imapsync_cache/po12.mail.mit.edu/jmorzins/imap.exchange.mit.edu/snizromj/INBOX/Sent Mail/Migrated/INBOX/Sent Mail
CACHE h1 h2: 0 files
msg INBOX.Sent Mail/6 {7761} copied to Migrated/INBOX/Sent Mail/177 3.10 msgs/s 23.522 KiB/s
msg INBOX.Sent Mail/7 {453} copied to Migrated/INBOX/Sent Mail/178 3.79 msgs/s 15.203 KiB/s
msg INBOX.Sent Mail/8 {831} copied to Migrated/INBOX/Sent Mail/179 5.08 msgs/s 14.952 KiB/s
Use --include to get folders by regexp
exec "${imapsync}" --noreleasecheck \
--host1 "${host1}" --ssl1 --authmech1 "${auth1}" \
--user1 "${user1}" --passfile1 "${pwdfile1}" \
--host2 "${host2}" --ssl2 --authmech2 "${auth2}" \
--user2 "${user2}" --passfile2 "${pwdfile2}" \
--tmpdir /var/tmp \
--nofoldersizes \
--prefix1 '' \
--include "INBOX.utf.*" \
--regextrans2 's#(.*)#Migrated/$1#' \
--useuid --usecache \
++++ Looping on each folder
[INBOX.utf] -> [Migrated/INBOX/utf]
cache directory: /var/tmp/imapsync_cache/po12.mail.mit.edu/jmorzins/imap.exchange.mit.edu/snizromj/INBOX/utf/Migrated/INBOX/utf
CACHE h1 h2: 0 files
msg INBOX.utf/4562 {7083} copied to Migrated/INBOX/utf/3 3.50 msgs/s 24.189 KiB/s
And now I see the flaw in the system.
The uid cache always appends and never removes messages.
If the cache slips or if a UID slips, you get duplicates.
SO... NUCLEAR OPTION!
--skipsize --nouseuid --usecache \
--include "INBOX.utf.*" \
--delete2duplicates --expunge2 \
Oh hell. It is getting different message sizes from the
different servers arrrrrrgh.