[libvoikko] HFST speller lexicon spec - draft 0.2

Flammie Pirinen flammie at iki.fi
Mon Nov 29 18:48:18 EET 2010

2010-11-11, Flammie Pirinen sanoi:

> 2010-11-07, Harri Pitkänen sanoi:
> > It's interesting that loading takes so long in OOo if hfst-ospell
> > only needs 15 seconds. It might be that something gets loaded and
> > unloaded more often than necessary. This could very well have gone
> > unnoticed since loading happens more or less instantly when Malaga
> > is used (we only map the lexicon to memory at load time but don't
> > actually read it).
> I've certainly noticed that OOo unloads and loads the dictionary
> during the use occasionally, since these delays are really
> noticeable. During my testing with Finnish HFST stuff I didn't see it
> either, since it is mostly unnoticeable in current version I've used
> (I attached voikko-hfst-ospell.patch for reference, I'll commit it
> after hfst-ospell library is released?)
> On slightly related story, if you want to test this specific thing
> I've mentioned already, I suppose it's ok to demonstrate it already;
> it's the greenlandic in divvun's svn
> <https://victorio.uit.no/langtech/trunk/st/kal> with the other patch I
> attached. Requires HFST 2, the optimized stuff and foma.

After quite some testing I haven't seen anything in the OOo debug
information that would be out of ordinary, so I'll assume the
greenlandic slowdown's caused by swapping. I pushed the abovementioned
patch to voikko now, so --enable-hfst will now only require hfst-ospell
installed and no main hfst libraries are required.

The languages in svn repository
<https://victorio.uit.no/langtech/trunk/> should all, in principle, be
testable using the greenlandic patch I mentioned earlier, in addition
to omorfi of course. 

Next things I'll do is to implement the zhfst archive version of
hfst-ospell library and make it initialise the spellers using language
codes so the voikko side will work with minimal modifications.

Flammie, computer scientist bachelor, linguist master, free software
Finnish localiser, and more! <http://www.iki.fi/flammie/>

More information about the Libvoikko mailing list