Hi Christian,

Perhaps you can give me a hint as to why inserts slow down.  I was imagining that most of the indexing work would be in the Optimize afterwards.  Sounds like it's also a lot slower relative to giving a single file that contains the same as many documents too, right?  Somehow this doesn't rhyme in my mind, so I must be missing something.

I will try find the time to try out the latest snapshot, but from what I read I guess you're not expecting greater Add speeds, just faster Replace.

Gerald


On Thu, Sep 18, 2014 at 1:43 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Gerald,

yes, we are aware that database inserts slow down over the time. I
would be interested in your experience with the latest snapshot of
BaseX [1], which has an improved document index [2]. In some cases,
the insertion of new files may get slower, but the replacement of
existing files will be sped up a lot with this index.

Thanks in advance,
Christian

[1] http://files.basex.org/releases/latest/
[2] https://github.com/BaseXdb/basex/issues/804


On Thu, Sep 18, 2014 at 11:58 AM, Gerald de Jong <gerald@delving.eu> wrote:
> I'm finding that adding documents to a database starts at about 1ms, but
> gradually gets slower (5ms after about 700,000).  I'm doing this with
> autoflush off, and I've tried periodic flush an optimize commands but they
> have no effect.
>
> Is this expected behavior?  Is there any way to keep it speedy?
>
> --
> Delving BV, Vasteland 8, Rotterdam
> http://www.delving.eu
> http://twitter.com/fluxe
> skype: beautifulcode
> +31629339805



--
Delving BV, Vasteland 8, Rotterdam
http://www.delving.eu
http://twitter.com/fluxe
skype: beautifulcode
+31629339805