[libvoikko] zhfst support on Windows

[Sjur Moshagen]

15. des. 2013 kl. 16:40 skrev Harri Pitkänen <hatapitk at iki.fi>:

> I was able to build a version of libvoikko for Windows where support for zhfst 
> spellers was enabled. Unfortunately it seems that hfst-ospell does really not 
> work on Windows.
[...]
> If I move away all zhfst spellers then "voikkospell.exe -d fi-x-dialects" with 
> the Malaga backend works just fine.

We have an issue with hfst-ospell as well, but it is restricted to the combination of zip decompression to RAM and TinyXML2. My collegue Børre is looking into it right now.

> The next step would be to start debugging hfst-ospell.exe to figure out what 
> is wrong with it. Unfortunately I have no useful debugging tools on my Windows 
> laptop so I would need to start adding debugging output to various parts of 
> hfst-ospell source to nail this bug. But right now I really don't have the 
> time for that (I have already spent half of the weekend on this). So I'll get 
> back to this when I have more time.

Thanks a lot for the effort you have put into this, it is much appreciated. In the best of cases, the bug that Børre is looking into is the cause of what you experience on Windows as well. But the symptoms are different: on the Mac, hfst-ospell seems to work fine whereas voikkospell crashes with this error:

$ voikkospell -l -p tools/spellcheckers/fstbased/hfst/
libc++abi.dylib: terminating with uncaught exception of type hfst_ol::ZHfstMetaDataParsingError
Abort trap: 6

According to Børre’s findings, the unzipping into RAM corrupts the xml metadata file by appending garbage symbols to the end of the xml in certain cases. Libxml2++ doesn’t care (becasue the xml up until the outermost end tag is fine), and so everything works fine. TinyXML2 barks, and causes crashes in some cases.

Børre will have more details.

Sjur