[libvoikko] HFST speller lexicon spec - draft 0.2

[Flammie Pirinen]

2010-11-11, Flammie Pirinen sanoi:

> 2010-11-07, Harri Pitkänen sanoi:
> 
> > It's interesting that loading takes so long in OOo if hfst-ospell
> > only needs 15 seconds. It might be that something gets loaded and
> > unloaded more often than necessary. This could very well have gone
> > unnoticed since loading happens more or less instantly when Malaga
> > is used (we only map the lexicon to memory at load time but don't
> > actually read it).
> 
> I've certainly noticed that OOo unloads and loads the dictionary
> during the use occasionally, since these delays are really
> noticeable. During my testing with Finnish HFST stuff I didn't see it
> either, since it is mostly unnoticeable in current version I've used
> (I attached voikko-hfst-ospell.patch for reference, I'll commit it
> after hfst-ospell library is released?)
> 
> On slightly related story, if you want to test this specific thing
> I've mentioned already, I suppose it's ok to demonstrate it already;
> it's the greenlandic in divvun's svn
> <https://victorio.uit.no/langtech/trunk/st/kal> with the other patch I
> attached. Requires HFST 2, the optimized stuff and foma.

After quite some testing I haven't seen anything in the OOo debug
information that would be out of ordinary, so I'll assume the
greenlandic slowdown's caused by swapping. I pushed the abovementioned
patch to voikko now, so --enable-hfst will now only require hfst-ospell
installed and no main hfst libraries are required.

The languages in svn repository
<https://victorio.uit.no/langtech/trunk/> should all, in principle, be
testable using the greenlandic patch I mentioned earlier, in addition
to omorfi of course. 

Next things I'll do is to implement the zhfst archive version of
hfst-ospell library and make it initialise the spellers using language
codes so the voikko side will work with minimal modifications.

-- 
Flammie, computer scientist bachelor, linguist master, free software
Finnish localiser, and more! <http://www.iki.fi/flammie/>