Core functionality (custom headers) doesn't work at all

[Binarus]

At first, thank you very much for that great add-on! It looks promising,

However, in its current state (beta 4), its core functionality fails in TB 78.5.1 (64 bit):

I have about 20000 messages in a folder which all contain the following two headers: X-Bin-MAIL-FROM and X-Bin-RCPT-TO.

Thunderbird shows these headers in the header area for each message, provided I have View -> Headers -> All turned on, so I definitely know that they are there. I furthermore have verified their existence by looking into the source code for quite a lot of these messages (Ctrl + U), where I can find these headers as expected.

I wanted to see these headers' payload as column in my message pane, so I installed ColumnsWizard beta 4, went into its settings and switched to the Custom Columns tab. Then I created two custom columns like that:

Enabled    Searchable    ID              Mail Header        Column Name    Tooltip
x          x             xbinmailfrom    x-bin-mail-from    Env-From       Sort by Env-From
x          x             xbinrcptto      x-bin-rcpt-to      Env-To         Sort by Env-To

When I went back to the folder with the messages and chose Columns to display, I indeed had the two new columns available. When I chose them, they appeared as columns in the message window as expected, besides the other columns. So far, so good, but ...

The problem is that they were completely empty. Obviously, the add-on can't extract any bit of the header data, although both headers' payload is pretty simple: It is always something like <[email protected]>. That is, in the messages' source code, there always is something like

X-Bin-MAIL-FROM: <[email protected]>
X-Bin-RCPT-TO: <[email protected]>

I could very well imagine that the problem is due to the angle brackets. Whatever, I really hope that you can fix it - I was searching for an add-on like this for a very long time, and ColumnsWizard came to my attention just today.

If it would help, I could save one of these messages as .eml file and provide it here, or send it to you via PN.

[cleidigh]

@Binarus
b4 definitely have several big problems with custom headers.
this is a major effort for 78 so it's a bit of a slog.
I am close to releasing b6 which is a big improvement , but I am still
working on several things.
in the process I will also check the things you noted
one thing you should know is that custom columns do not get populated
for e-mails already received. you have to reindex the mail store (best to do a backup)
for the new headers to be populated. but do not forget b4 has its issues
thanks a lot!
cheers
Christopher

[Binarus]

Thank you very much for the reply!

I did not know that I have to re-index received messages for the new headers to be populated. Out of curiosity: Did you document this somewhere? I believe that it would be very important to make users aware of it, because otherwise they believe that ColumnsWizard does not work, while in fact it works correctly.

I will now try to re-index the messages in question, and report the result.

Thanks again,

Binarus

[Binarus]

I now can confirm that ColumnsWizard works as expected, including custom headers, after I have re-indexed the messages.

This is absolutely great. Finally an addon which enables us to actually work with the headers which are added to incoming messages by our custom milters ...

I will now close the issue and mark it as solved because ColumnsWizard works correctly after re-indexing messages.

Best regards, and thanks again,

Binarus

[cleidigh]

@Binarus
just posted b5 with a bunch of fixes
one fix is maintaining custom, column widths
among other things
you are one of the few early testers using custom columns
feedback for any problems would be great!
cheers
Christopher

[Binarus]

@cleidigh

Thank you very much for caring, and please excuse the delay.

I didn't manage to test b5, but have just installed b6 (https://github.com/micz/ColumnsWizard/blob/v8.0.0/xpi/columnswizard-8.0.0-b6-tb.xpi). Unfortunately, this made things worse:

In Thunderbird, when I choose Tools -> Add-on options -> ColumnsWizard, then go to tab Custom Columns, and hit the button New ..., nothing happens. I restarted Thunderbird a few times and repeated the process, but to no avail. The New ... button does not seem to do anything in b6.

I also tabbed the focus to that button using the TAB key and then hit enter, but this didn't effect anything either (I did this test just to make sure that the problem is not due to glitches in mouse handling). I'll remove the [Solved] mark from the heading again ...

Cheers,

Binarus

P.S. I wish you a happy new year! You have deserved it (helping people to solve tasks which can't be solved otherwise free of charge).

[Binarus]

@cleidigh

I have just verified that the New ... button works in b5 (I did not test anything else in b5, though). The misbehavior must have been introduced from b5 to b6.

Thanks again,

Binarus

[cleidigh]

@Binarus
happy holidays!! thanks for the comments.
also thanks for the testing. b5 is probably better than b6. just for your understanding:
I have been wrestling with the checkbox management. what should be very simple
as been complicated by new security mechanisms causing lots of headaches.
I have to work through several things to get them to behave correctly.
also interrupted by some other things
hopefully I can get b7 soon.
thanks for the patience 👍
Christopher

[Binarus]

@cleidigh Thanks again for caring!

In the meantime, I have tested b5 further. For my use cases, it works as expected and as intended.

I have created three custom columns, displaying the contents of three different headers which our custom milters add to incoming messages, and have activated these columns in message lists with 20000+ messages.

That worked like a charm, and I even couldn't notice hiccups due to performance.

There was one difference to b4:

With b5, I did not need to re-index messages before the custom columns worked. Instead, I just had to define the custom columns and switch them on in the message lists of interest. This of course is a great improvement and a very good thing, but I haven't understood why this is the case.

The tests with b5 were in another TB installation and another version (this time TB 78.6.0) than the tests with b4. I am unsure whether both installations had the same settings, notably indexing and synchronization settings. So please be aware that the changes in behavior described in the previous paragraph might be due to different settings in TB, or the TB version.

Thanks again,

Binarus