Hello,
We try to insert/replace a record (size of ~500KB) in a file of 100MB. The query requires more than 10minutes in order to completed. Could you please tell if the previous is normal?
Regards Yannis
Hello Yannis,
that is hard to tell. Please describe in more detail what you do, which commands/queries you run and possibly input data.
For sure it is not in a general case - I just tested it on a 112MB XMark file and a update of one node took around 200ms. But depending on what you do it might be normal (e.g. if you search the node with an exponential runtime it might very well be).
Cheers, Dirk
On 16/05/14 11:26, Ioannis Kavvouras wrote:
Hello,
We try to insert/replace a record (size of ~500KB) in a file of 100MB. The query requires more than 10minutes in order to completed. Could you please tell if the previous is normal?
Regards Yannis
Hi Yannis,
we need a little bit more information - as Dirk already pointed out.
It would be especially interesting to know what you mean with ‘record’. Are you trying to insert an element or a document? The internal process differs greatly for both operations.
Which version of BaseX are you using? What about Namespaces in the database?
Cheers, Lukas
On 16 May 2014, at 11:35, Dirk Kirsten dk@basex.org wrote:
Hello Yannis,
that is hard to tell. Please describe in more detail what you do, which commands/queries you run and possibly input data.
For sure it is not in a general case - I just tested it on a 112MB XMark file and a update of one node took around 200ms. But depending on what you do it might be normal (e.g. if you search the node with an exponential runtime it might very well be).
Cheers, Dirk
On 16/05/14 11:26, Ioannis Kavvouras wrote:
Hello,
We try to insert/replace a record (size of ~500KB) in a file of 100MB. The query requires more than 10minutes in order to completed. Could you please tell if the previous is normal?
Regards Yannis
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
I try the following insert query: insert node/<coverage id=''>///....</coverage>/ /into /server[@endpoint='anEndpointId']/formal/lastimage/harvested/coverages
* we insert a node/element, not a document. * the coverage node is approximately 500kB * node /coverages/ has a lot of /coverage nodes/ that differ with each other in the /id/ attribute.
We run BaseX782. The database has a lot of namespaces in general but none of them is outside the scope of coverage element. As you can see we use the full path in the insertion.
In general, get queries are fast, the only problem we have is with insertions and replacements.
Regards Yannis
On 05/16/2014 12:40 PM, Lukas Kircher wrote:
Hi Yannis,
we need a little bit more information - as Dirk already pointed out.
It would be especially interesting to know what you mean with ‘record’. Are you trying to insert an element or a document? The internal process differs greatly for both operations.
Which version of BaseX are you using? What about Namespaces in the database?
Cheers, Lukas
On 16 May 2014, at 11:35, Dirk Kirsten dk@basex.org wrote:
Hello Yannis,
that is hard to tell. Please describe in more detail what you do, which commands/queries you run and possibly input data.
For sure it is not in a general case - I just tested it on a 112MB XMark file and a update of one node took around 200ms. But depending on what you do it might be normal (e.g. if you search the node with an exponential runtime it might very well be).
Cheers, Dirk
On 16/05/14 11:26, Ioannis Kavvouras wrote:
Hello,
We try to insert/replace a record (size of ~500KB) in a file of 100MB. The query requires more than 10minutes in order to completed. Could you please tell if the previous is normal?
Regards Yannis
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
Hi Yannis,
between version 7.8.2 and 8.0 we identified a weak-sport regarding updates and namespaces - which is already fixed in the latest stable snapshot. I’m almost 100% sure that this is responsible for the performance drawbacks that you experience. In general you can expect BaseX to be really fast with updates.
I would recommend to switch to the latest stable snapshot, as since BaseX 7.8.2 some more things have been fixed and improved. Would be nice if you could report back whether the problem persists - and sorry for the inconveniences.
Cheers, Lukas
On 16 May 2014, at 11:49, Ioannis Kavvouras i.kavvouras@di.uoa.gr wrote:
I try the following insert query: insert node <coverage id=''>....</coverage> into /server[@endpoint='anEndpointId']/formal/lastimage/harvested/coverages • we insert a node/element, not a document. • the coverage node is approximately 500kB • node coverages has a lot of coverage nodes that differ with each other in the id attribute. We run BaseX782. The database has a lot of namespaces in general but none of them is outside the scope of coverage element. As you can see we use the full path in the insertion.
In general, get queries are fast, the only problem we have is with insertions and replacements.
Regards Yannis
On 05/16/2014 12:40 PM, Lukas Kircher wrote:
Hi Yannis,
we need a little bit more information - as Dirk already pointed out.
It would be especially interesting to know what you mean with ‘record’. Are you trying to insert an element or a document? The internal process differs greatly for both operations.
Which version of BaseX are you using? What about Namespaces in the database?
Cheers, Lukas
On 16 May 2014, at 11:35, Dirk Kirsten dk@basex.org wrote:
Hello Yannis,
that is hard to tell. Please describe in more detail what you do, which commands/queries you run and possibly input data.
For sure it is not in a general case - I just tested it on a 112MB XMark file and a update of one node took around 200ms. But depending on what you do it might be normal (e.g. if you search the node with an exponential runtime it might very well be).
Cheers, Dirk
On 16/05/14 11:26, Ioannis Kavvouras wrote:
Hello,
We try to insert/replace a record (size of ~500KB) in a file of 100MB. The query requires more than 10minutes in order to completed. Could you please tell if the previous is normal?
Regards Yannis
-- Dirk Kirsten, BaseX GmbH, http://basex.org
|-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
basex-talk@mailman.uni-konstanz.de
-
Dirk Kirsten -
Ioannis Kavvouras -
Lukas Kircher