Hi Fellas,
We are researching Basex for our upcoming solution to one of our customers. We are required to search and add XML documents to the data set. Just to brief you on the amount of data we receive around 8 million XML message per customer for a month. These set of data needs to be quickly searched and at the same time to expand it on receiving messages.
I have created a fulltext index on the database. But as soon as I do insert a data item the fulltext index is gone and my searches are slowed down.
How shall I proceed forward on this thread? Please help !!
Regards, Abhishek
Hi Abhishek,
currently, incremental updates only exist for the text and attribute value index [1].
There are several solutions for your problem. In BaseX, a single XQuery expression can be used to query or update an arbitrary databases at the same time. If you want to include recently add documents to your search results, and if you have some free minutes throughout the day/night to recreate your index structures, one option is to work with two databases:
– one or several databases contain the archived documents; it will be fully indexed – a second database contains new documents; it will be merged with the main database, or indexed and treated as archived database as soon as it gets too large for unoptimized real-time requests.
Incremental full-text indexing will be included on a medium term in BaseX; I can give you no deadline, though, as there are currently numerous other challenges waiting to be accomplished.
Christian
[1] http://docs.basex.org/wiki/XQuery_Update#Indexes ___________________________
On Thu, Sep 27, 2012 at 9:50 AM, Abhishek Jain abhishek.abhisvits@gmail.com wrote:
Hi Fellas,
We are researching Basex for our upcoming solution to one of our customers. We are required to search and add XML documents to the data set. Just to brief you on the amount of data we receive around 8 million XML message per customer for a month. These set of data needs to be quickly searched and at the same time to expand it on receiving messages.
I have created a fulltext index on the database. But as soon as I do insert a data item the fulltext index is gone and my searches are slowed down.
How shall I proceed forward on this thread? Please help !!
Regards, Abhishek
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
basex-talk@mailman.uni-konstanz.de