[libvoikko] Small patch to HFST2 backend stuff

[Flammie Pirinen]

2010-04-06, Harri Pitkänen sanoi:

> I did not test this yet at all. It seems that the suggestion
> generator was not added to SuggestionGeneratorFactory.cpp so it would
> not work out of the box. I can fix this once I have time to test it,
> probably at the end of this week. Same problem with hyphenator, I'll
> fix that too.

Yes, in fact I used versions of factories where hfst backends were used
unconditionally #if HAVE_HFST, so I excluded them from previous patch
as that surely isn't wanted in the development version.

> > I've also compiled TeX hyphenation patterns from hyph-utf8
> > distribution as HFST transducers that may be used for testing, 
> 
> The original patterns appear to be from Kauko Saarinen. I believe
> they are in public domain although the history of these patterns
> (especially the hyph-utf8 version) is really confusing:
> 
>   http://www.openoffice.org/issues/show_bug.cgi?id=74298

Yeah, specifically since these patterns have been around since forever,
I'm sure there may very well be complex issues. In fact the hyph-fi
patterns aren't all that interesting, the schoolbook algorithm I
implemented in couple of minutes performs nearly as well for Finnish.
For testing other languages the tex patterns should be more relevant.

-- 
Flammie, computer scientist bachelor, linguist master, free software
Finnish localiser, and more! <http://www.iki.fi/flammie/>