Hello team,
First of all, congratulations for the awesome tool! BaseX is amazing. We are starting using it for querying some XML data that we are receiving from a queue.
I've developed a Nifi processor based on https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/or... class.
The problem that I'm facing is that write throughput is very slow. Do you know any way to increase the write throughput? All the XML files are stored in the same DB. So basically I'm doing:
1) open db 2) replace (xml id path, inputstream)
Thanks, Juan
Hi Juan,
Welcome to the list and sorry for the late reply.
- open db
- replace (xml id path, inputstream)
Some suggestions:
1. If you can use ADD instead of REPLACE, you can usually safe a lot of time.
2. If you need to replace docs, it may depend on what the "xml id path" is:
a) If it’s an attribute in the document, you may need to enable the updatable index (e.g. via the GUI, DBA, or the UPDINDEX option).
b) If it’s the database path, you could try to disable AUTOFLUSH, or provide us with more information on what you do exactly.
Best, Christian
Thanks Christian,
We are receiving from a repayable queue (Kafka topic), so the idea is if we replay the messages, the XML data is updated in the DB. We are taking the message ID as the file path. Basically:
clientSession.replace(messageId, XML file).
Juan
On Fri, 22 Dec 2017 at 11:41 Christian Grün christian.gruen@gmail.com wrote:
Hi Juan,
Welcome to the list and sorry for the late reply.
- open db
- replace (xml id path, inputstream)
Some suggestions:
- If you can use ADD instead of REPLACE, you can usually safe a lot of
time.
- If you need to replace docs, it may depend on what the "xml id path" is:
a) If it’s an attribute in the document, you may need to enable the updatable index (e.g. via the GUI, DBA, or the UPDINDEX option).
b) If it’s the database path, you could try to disable AUTOFLUSH, or provide us with more information on what you do exactly.
Best, Christian
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Juan Pablo Gardella