Add main encoding detection functions

[nalimilan]

I had wrapped some functions to convert from one encoding to another, and then I realized you had just done the same a few days ago. ;-) So I went on and wrote wrappers for encoding detection functions, which seem to work relatively well in the few tests I did.

Something I'm not sure I did right is what to do when ucsdet_detect returns NULL. I could return C_NULL or nothing, which would break type stability. So for now I return an object with a null pointer, and I handle this in ucsdet_getName, by returning an empty encoding name.

[nalimilan]

Actually, I'd now be inclined to throwing an exception when ucsdet_detect returns NULL. That would allow using try, which is more natural than checking for a NULL pointer or an empty string. What do you think?

[nolta]

Thanks! Yes, having uscdet_detect throw an error sounds better.

[nalimilan]

Here's a new version. What do you think of the name of the exception?

[nolta]

I'm having second thoughts about the exception. Is there a scenario where both ucsdet_detect returns NULL and U_SUCCESS(err[1]) is true?

[nolta]

This should return an empty array instead of throwing an exception.

[nalimilan]

Really? I first wanted to do that, but I though for consistency with ucsdet_detect it would be better to throw an exception. As you like.

[nalimilan]

Actually, when no encoding matches, U_SUCCESS(err[1]) is true, but the function returns NULL. U_SUCCESS(err[1]) will only fail under exceptional circumstances (i.e. a bug somewhere), and I think it's right to distinguish this total failure from "normal" failure to find a matching encoding.

[nalimilan]

Bump! I have rebased this old branch.

[ScottPJones]

Could this be merged? We have a need for it also. Thanks!

[nalimilan]

Re-bump!

[ScottPJones]

You might want to use https://github.com/JuliaString/ICU.jl instead, which supports the character detection API, fixes a bunch of errors with leaking memory (by adding finalizers and keeping a pointer to the other Julia object to prevent premature GC), and is tested (I am working on getting it to 100% coverage)

[nalimilan]

Ah, cool. I guess it would make sense to redirect to your fork and register it in METADATA if you plan to maintain it in the future?

[ScottPJones]

Yes, that's what I was hoping for. I never did hear back from @nolta about that. If you get a chance, I'd love your comments on it, constructive criticism is very very much appreciated!