Dear Christian
Thanks a lot for our need full suggestions
>I also have a question: how many documents (or how many gigabytes in terms of input documents) do you store in a single database?
Lakhs of documents are stored in one single database,data base size is more than 80 gb in some cases.
Regards Sekhar
________________________________________ From: Christian Grün christian.gruen@gmail.com Sent: Friday, December 12, 2014 10:37 PM To: Chandrasekhar Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: Reg : Chunk By Chunk To DB
Dear Sekhar,
Please do the needful.
Sorry again, can't resist: http://www.reddit.com/r/sysadmin/comments/2eaa0b/do_the_needful/
I hope it's not interpreted the wrong way… I guess that Germans use lots of phrases that sound more than quirky to British and American speakers. Feedback is always welcome!
Still, I did the needful, and I extended the documentation for the ADD command to make it more obvious what's going on [1]. When sending the data via an input stream, there is no way to detect in advance that your input may not fit into main memory, but you want to enforce caching of large documents, you can use the ADDCACHE option [2].
There is currently no way to both speed up processing and reduce memory consumption, because we need to be sure that the requested input will be valid before storing it in the database.
I also have a question: how many documents (or how many gigabytes in terms of input documents) do you store in a single database?
All the best, Christian
[1] http://docs.basex.org/wiki/Commands#ADD [2] http://docs.basex.org/wiki/Options#ADDCACHE The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.