[libvoikko] Libvoikko 4.0.1rc1

[Harri Pitkänen]

Libvoikko 4.0.1rc1 is available for testing at

  http://www.puimula.org/htp/testing/libvoikko-4.0.1rc1.tar.gz

 * Tokenization functions for C/C++/Java have been optimized to perform 
reasonably even with long input texts (longer than typical text paragraphs).
 * HFST backend has been fixed to handle VOIKKO_OPT_ACCEPT_ALL_UPPERCASE 
option properly.
 * The Python module now tries to load shared library dependencies of 
libvoikko (such as hfstospell) if setLibrarySearchPath is used.
 * On Windows dictionaries are searched from user's local application data 
folder (C:\Documents and Settings\$username\Local Settings\Application 
Data\voikko or similar).
   - Previous methods of specifying the search path are of course still 
available.
 * Fixed unweighted VFST lookup code for some compilers including GCC for 
Windows.
 * Lookup performance of Finnish VFST backend has been improved by about 15 %.

One potentially risky change has been made in this release: We now require a 
C++ compiler supporting the C++11 standard for building the library. For now 
it is still easy to revert this requirement, just reverting commit 
2c63ad7fd53b57e9d28ac89281b89b0ad04cdd3c is enough. Please check that you are 
still able to build this code. If you have problems, please let me know. In 
case it turns out that we actually should support older compilers I will 
revert this change but if it is OK for everyone we will be able to modernize 
the code a bit in the future.

Harri