[libvoikko] Possible encoding bug in latest voikkospell

[Sjur Moshagen]

26. sep. 2014 kl. 17:49 skrev Harri Pitkänen <hatapitk at iki.fi>:
> 
> On Friday 26 September 2014 11:02:55 Sjur Moshagen wrote:
>> 25. sep. 2014 kl. 22:59 skrev Harri Pitkänen <hatapitk at iki.fi>:
>>> Anyway, maybe there is a real difference between C and C++ locales on OS
>>> X.
>>> Please test if the patch below helps.
>> 
>> Sorry, no:
>> 
>> $ echo giellla | voikkospell -s -d se
>> Could not find default speller, using syll syll
>> W: giellla
>> S: giella
>> S: giellal
>> S: giellala
>> S: giellula
>> S: giell?
>> 
>> I did uninstall before I installed, just to make sure that the new version
>> of voikkospell is there, but nothing changed.
> 
> I don't know what is going on then. It might make sense to test different 
> compilers (g++ or clang++, different Xcode versions).

I haven’t had the chance to test different compilers and xcode versions, but that seems like a reasonable suggestion, especially since this bug has manifestated itself after upgrading both the os and xcode. The bug is reproducible on another mac with the same system and xcode versions, and we found another variant of this bug that is even more disturbing today:

$ echo arrát | voikkospell -s -d se
W: arr?t
S: ?rrat
S: arv?t
S: varr?t
S: darr?t
S: jarr?t
$ echo áššši | voikkospell -s -d se
W: ?$ 
$ 

That is, the I/O stream is completely broken when the input contains the letter š.

I’ll try to have a look at this using different compilers in the near future. Feedback is welcome.

Sjur