Note

This documentation is still a work in progress. If you have any issues or questions, please ask on the unitex-devel mailing list or file a bug in our issue tracker.

Persistent data access layer

Unitex language-related resources such as graphes .fst2, dictionaries .bin and .inf files, may become too large or complex, thus slow to load. To minimize the load time, the Unitex shared library provides some useful functions to preload these resources explicitly and reference them when calling other functions.

For each type of resource (dictionary, graph or alphabet), there exists a preload function. These functions return, in the buffer pointed to by persistent_filename_buffer, the name of the persistent resource. The number of bytes to return in the buffer (i.e., the buffer size) need to be equal to buffer_size.

Warning

After to preload a dictionary file, do not modify or remove it. Under the propietary VFS implementation, a file memory-mapping strategy might being used. As result, your program may produces unpredictable results or stop to working.

Note

The conventions used to create named persistent resources may change and depend upon the type of the persistent library installed. You should therefore be consistent in using the same file prefix ($: or *:) across all function calls.

C

persistence_public_load_dictionary

int persistence_public_load_dictionary(const char* filename,
                                       char* persistent_filename_buffer,
                                       size_t buffer_size);

persistence_public_is_persisted_dictionary_filename

int persistence_public_is_persisted_dictionary_filename(const char*filename);

persistence_public_unload_dictionary

void persistence_public_unload_dictionary(const char* filename);

persistence_public_load_fst2

int persistence_public_load_fst2(const char* filename,
                                 char* persistent_filename_buffer,
                                 size_t buffer_size);

persistence_public_is_persisted_fst2_filename

int persistence_public_is_persisted_fst2_filename(const char*filename);

persistence_public_unload_fst2

void persistence_public_unload_fst2(const char* filename);

persistence_public_load_alphabet

int persistence_public_load_alphabet(const char* filename,
                                     char* persistent_filename_buffer,
                                     size_t buffer_size);

persistence_public_is_persisted_alphabet_filename

int persistence_public_is_persisted_alphabet_filename(const char*filename);

persistence_public_unload_alphabet

void persistence_public_unload_alphabet(const char* filename);

Java

loadPersistentDictionary

public native static String loadPersistentDictionary(String filename);

freePersistentDictionary

public native static void freePersistentDictionary(String filename);

loadPersistentFst2

public native static String loadPersistentFst2(String filename);

freePersistentFst2

public native static void freePersistentFst2(String filename);

loadPersistentAlphabet

public native static String loadPersistentAlphabet(String filename);

freePersistentAlphabet

public native static void freePersistentAlphabet(String filename);