BaseX-Talk

basex-talk@mailman.uni-konstanz.de

1 participants
5252 discussions

path index does not seem to be used
by Fabrice Etanchaud 14 Feb '13

14 Feb '13

Dear all at BaseX, In a indexed database, Query info shows that a request like Count(/a/b/c) does not use the statistics data available with index:facets(). Could you please tell me if this is really the case, and if there is a way to tell BaseX to use it ? Best regards, And thank you for your great XNDB ! Fabrice Here is the query : Query: declare namespace exch = 'http://www.epo.org/exchange'; count(/exch:exchange-documents/exch:exchange-document) Query plan: <QueryPlan> <FNAggr name="count(item)"> <IterPath> <DBNodeSeq size="38"> <DBNode name="DOCDB_RU" pre="0"/> <DBNode name="DOCDB_RU" pre="4198620"/> <DBNode name="DOCDB_RU" pre="7369614"/> <DBNode name="DOCDB_RU" pre="11083006"/> <DBNode name="DOCDB_RU" pre="15423603"/> </DBNodeSeq> <IterStep axis="child" test="exch:exchange-documents"/> <IterStep axis="child" test="exch:exchange-document"/> </IterPath> </FNAggr> </QueryPlan> Here is the db:info() : <database> <databaseproperties> <name>DOCDB_RU</name> <size>3391 MB</size> <nodes>135589551</nodes> <documents>38</documents> <binaries>0</binaries> <timestamp>2013-01-29-23-16-31</timestamp> </databaseproperties> <resourceproperties> <inputpath>C:/data/work/docdb/RU</inputpath> <timestamp>2012-12-28-06-53-45</timestamp> <encoding>UTF-8</encoding> <whitespacechopping>ON</whitespacechopping> </resourceproperties> <indexes> <uptodate>true</uptodate> <textindex>ON</textindex> <attributeindex>ON</attributeindex> <fulltextindex>OFF</fulltextindex> <updindex>ON</updindex> <maxcats>10000</maxcats> <maxlen>96</maxlen> </indexes> </database>

3 4

XMLPrague: Talks and Slides
by Christian Grün 13 Feb '13

13 Feb '13

Dear subscribers, it was great to see and talk to many of you at the XMLPrague conference! The slides of most talks are now available online: http://files.basex.org/xmlprague2013/ Video clips of the presentations are in process and will be added soon. Have fun, Christian

1 0

access to dom representation of result in server mode or with xqj
by Fabrice Etanchaud 13 Feb '13

13 Feb '13

Dear all at BaseX, Is there a way to gain access the dom representation of a query result in server mode, or with xqj ? (in simple words, could a client side parser work directly on the internal server representation, for example by returning a 'eventified' serialization of the result ?) Thank you for your wonderful NXDB ! Best regards, Fabrice

2 1

file:list - returns no directories
by Hans-Juergen Rennau 13 Feb '13

13 Feb '13

Dear BaseX team, the documentation states: "Lists all files and directories found in the specified $dir."; however the function does not always return file names. (1) If parameter $recursive=true, the contents are returned, but not the directories themselves. (2) The call "file:list($dir, false(), '*') does not return directories In case of (1), this behavior may often be desirable, but sometimes not, namely if I want to construct a directory tree. Perhaps a further control option would be worth considering? I am not sure. At any rate, if the present behavior is intended, the documentation should be adapted. (2) seems to me a bug, as the call should be equivalent to a one-parameter call, shouldn't it? With kind regards, Hans-Juergen

2 1

loading basex from a cron job
by rsr 13 Feb '13

13 Feb '13

I am just getting started with basex and have been attempting to load data into a basex db via a cron job. I am running the basexclient inside my job redirecting a bxs command file to the client. this works well when i run my script off the command line by fails when run via cron. i've checked the obvious stuff (relative paths, permissions etc). Does anyone have experience loading basex via cron that could point me in the right direction. thanks /home/xxx/basex/bin/basexclient -Uxxx -Pxxx < /home/xxx/load.bxs

5 10

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 12 Feb '13

12 Feb '13

Hi Fabrice and list I am dealing with data-centric XML rather than documents and so there is a fairly high node to content ratio. I have about 250 million nodes and I find that having about 15 million nodes per database seems to work well, but this is just a guesstimate and I am really looking for some performance profiles or some heuristics so that I can limit the numbers of nodes in each database before the performance degrades. Cheers Peter > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, fetanchaud(a)questel.com, >BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Tue, 12 Feb 2013 09:07:40 +0000 > >>Dear Peter, >> >>I'm just a BaseX user, and Christian's team will correct me, but >from my experience, document size does not matter, at least for >querying. >> >>Why do you talk about distributing data ? Did you reach the 2 >billion nodes limit ? >> >>As BaseX indexes all nodes, depending on the values distribution, >creating a new collection containing hand made indices can speed up >your queries. >> >>For example, for append only collections, I'm used to creating a >index collection like this : >><index> >> <item value='value to be indexed'> >> the 'pre' pointer to the indexed element >> </tem> >> <item>... >></index> >> >>And access that 'index' something like this : >> >>for $i in >> //item[@value='searched value'] >>return >> db:open-pre('mydb', $i) >> >> >>And a big number of documents may slow down the properties window >display in the GUI, because of the document tree view. >> >> >>Question to the BaseX 's team : would 'user defined' indices be a >interesting feature ? >> >> >>Regards >> >>-----Message d'origine----- >>De : pw(a)themail.co.uk [mailto:pw@themail.co.uk] >>Envoyé : lundi 11 février 2013 17:13 >>À : Fabrice Etanchaud; pw(a)themail.co.uk; >BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : RE: [basex-talk] handling large files: is there a >streamingsolution? >> >>Thanks Fabrice, I am making good progress following your advice. Do >you have any heuristics for the best way to distribute data for >performant searches and subsetting of data? Am I better having lots >of small files or a few large files in a collection? >> >>> >>> >>> >>>---- Original Message ---- >>>From: fetanchaud(a)questel.com >>>To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >>>Subject: RE: [basex-talk] handling large files: is there a >>>streamingsolution? >>>Date: Mon, 11 Feb 2013 14:38:54 +0000 >>> >>>>Dear Peter, >>>> >>>>Did you try to create a collection with the files (CREATE command) >? >>>>You should start that way, I don't see the point in using file: >>>module for import. >>>>I think that once in the database, file size does not matter >(until >>>you reach millions of file in the collection, and do a lot of >document >>>related operations (list, etc...)) >>>> >>>> >>>> >>>>-----Message d'origine----- >>>>De : basex-talk-bounces(a)mailman.uni-konstanz.de >>>[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >>>pw(a)themail.co.uk >>>>Envoyé : lundi 11 février 2013 15:33 >>>>À : BaseX-Talk(a)mailman.uni-konstanz.de >>>>Objet : [basex-talk] handling large files: is there a streaming >>>solution? >>>> >>>>Hello List >>>>I am wanting to do a join with some large (3-400Mb) XML files and >>>would appreciate guidance on the optimal strategy. >>>>At present these files are on the filesystem and not in a database >>>> >>>>Is there any equivalent to the Zorba streaming xml:parse()? >>>> >>>>Would loading the files into a database directly be the approach, >or >>>is it better to split them into smaller files? >>>> >>>>Is the file: module a suitable route through which to import the >>>files? >>>> >>>>Thanks for your help >>>> >>>>Peter >>>> >>>>_______________________________________________ >>>>BaseX-Talk mailing list >>>>BaseX-Talk(a)mailman.uni-konstanz.de >>>>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >> >> >>

1 0

XQDoc for BaseX
by John Snelson 12 Feb '13

12 Feb '13

Hi all, The xquerydoc project is a 100% XQuery 1.0 project which can parse XQuery modules and extract XQDoc style documentation. I've recently extended it to be able to parse XQuery Update and XQuery Full Text, which I believe solves the majority of problems if you want to use it with modules written for BaseX. The project is available here: https://github.com/xquery/xquerydoc I'm happy to add any other syntax that BaseX supports as well, but I'll need help understanding what that is if there is anything. Enjoy your XQuery documentation! John

3 3

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 12 Feb '13

12 Feb '13

Thanks Fabrice, I am making good progress following your advice. Do you have any heuristics for the best way to distribute data for performant searches and subsetting of data? Am I better having lots of small files or a few large files in a collection? > > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Mon, 11 Feb 2013 14:38:54 +0000 > >>Dear Peter, >> >>Did you try to create a collection with the files (CREATE command) ? >>You should start that way, I don't see the point in using file: >module for import. >>I think that once in the database, file size does not matter (until >you reach millions of file in the collection, and do a lot of >document related operations (list, etc...)) >> >> >> >>-----Message d'origine----- >>De : basex-talk-bounces(a)mailman.uni-konstanz.de >[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >pw(a)themail.co.uk >>Envoyé : lundi 11 février 2013 15:33 >>À : BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : [basex-talk] handling large files: is there a streaming >solution? >> >>Hello List >>I am wanting to do a join with some large (3-400Mb) XML files and >would appreciate guidance on the optimal strategy. >>At present these files are on the filesystem and not in a database >> >>Is there any equivalent to the Zorba streaming xml:parse()? >> >>Would loading the files into a database directly be the approach, or >is it better to split them into smaller files? >> >>Is the file: module a suitable route through which to import the >files? >> >>Thanks for your help >> >>Peter >> >>_______________________________________________ >>BaseX-Talk mailing list >>BaseX-Talk(a)mailman.uni-konstanz.de >>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

handling large files: is there a streaming solution?
by pw＠themail.co.uk 11 Feb '13

11 Feb '13

Hello List I am wanting to do a join with some large (3-400Mb) XML files and would appreciate guidance on the optimal strategy. At present these files are on the filesystem and not in a database Is there any equivalent to the Zorba streaming xml:parse()? Would loading the files into a database directly be the approach, or is it better to split them into smaller files? Is the file: module a suitable route through which to import the files? Thanks for your help Peter

2 1

Load xml files to BaseX in a loop
by Nicholas OR Rita Stevens 10 Feb '13

10 Feb '13

I am new at BaseX so please pardon the newbie question. I am trying to run an xquery script to load a set of xml files in a directory into to BaseX 7.5. If I run replace.xq, shown below, it fails with Out of Main Memory declare %updating function local:upd($path,$file) { db:replace("eams",$path,$file) }; let $files:=for $file in file:list("d:\xmlfiles") return "d:\xmlfiles\"||$file for $file in $files return local:upd("pd",$file) But if I run file replace.bxs, it updates with no problem <commands> <replace path="p1">d:\xmlfiles\drr1.xml</replace> <replace path="p1">d:\xmlfiles\drr2.xml</replace> <replace path="p1">d:\xmlfiles\drr3.xml</replace> <replace path="p1">d:\xmlfiles\dw1.xml</replace> <replace path="p1">d:\xmlfiles\dlp1.xml</replace> </commands> (The same set of files is being processed by both scripts; I checked) The only idea I have is to use xquery to generate the file replace.bxs and then run it. A better way? Thanks.

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk