- BaseX-Talk - mailman.uni-konstanz.de

Additional info on the invoke problem
by Arto Viitanen 22 Feb '13

22 Feb '13

I am not sure if this tells more of the problem (xquery:invoke does not open the collection), but: if the query does something with the collection plus predicates retrieved, the query is evaluated. So, in my example collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production'] if I add some extra content that forces the evaluation, like <a>{collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production']}</a>/* I.e. add the result to a -element and retrieve it, the xquery:invoke("file-name.xq") evaluates the query, even if the database (suuryritys_com) is closed! -- Arto Viitanen Microteam Oy Finland

2 1

BaseX 7.6, XQJ API 1.2.3 and XQuery 3.0 features - syntax error for simple map operator
by Markus Ackermann 22 Feb '13

22 Feb '13

Hello BaseX-community, first of all, I really have to say that I really enjoy using BaseX and that I am really impressed by its active development and its ongoing improvement. I am still a little unsure about the Query 3.0 support via the BaseX XQJ driver from net.xqj, though. According to the Changelog of the website ([1]), XQuery 3.0 via XQJ is now supported: BaseX 7.3, 18.06.2012 [...] - Charles Foster's latest XQJ Driver supports XQuery 3.0 and the Update and Full Text extensions: http://xqj.net/basex/ The README file of Charles Foster's driver also states that XQuery 3.0 is supported since driver version 1.1.0: Version 1.1.0: * Supports XQuery 3.0 + XQuery Update Facility + XQuery Full Text extensions. * Various bug fixes. However, take this minimal example using the simple map operator (query A): BaseXXQDataSource dataSource = new BaseXXQDataSource(); dataSource.setServerName("localhost"); dataSource.setPort(1984); XQConnection con = dataSource.getConnection("dict_wsa", "lidell"); con.createExpression().executeQuery( "(1 to 3) ! (. * 2))"); //query A con.createExpression().executeQuery( "let $m := map { 'one' := 1, 'two' := 2 }" + "return ($m('one')"); //query B Executing it yield a syntax exception: Exception in thread "main" javax.xml.xquery.XQException: XQJQS001 - Invalid XQuery syntax, syntax does not pass static validation. Root Cause: net.xqj.basex.bin.bn: Lexical error at line 1, column 11. Encountered: " " (32), after : "!" at XQ3Test.main(XQ3Test.java:21) at XQ3Test.main(XQ3Test.java:21) For query B I also receive a syntax exception: Exception in thread "main" javax.xml.xquery.XQException: XQJQS001 - Invalid XQuery syntax, syntax does not pass static validation. Root Cause: net.xqj.basex.bin.bj: Encountered " "{" "{ "" at line 1, column 15. Was expecting one of: "=" ... "order" ... "," ... Is there another option I have to set to circumvent the 'static validation' or did I overlook anything else? Regards and thanks in advance for any information or advice, Markus Ackermann PS: Information on the execution environment: JVM: 1.7.0_13 (Oracle Corporation 23.7-b01) OS: Linux 3.0.0-30-generic amd64 (Ubuntu 11.10) BaseX 7.6

2 1

Re: [basex-talk] BaseX-Talk Digest, Vol 38, Issue 36
by Arto Viitanen 22 Feb '13

22 Feb '13

22.2.2013 11:04, basex-talk-request(a)mailman.uni-konstanz.de wrote: > Message: 3 > Date: Fri, 22 Feb 2013 09:00:09 +0100 > From: Marco Lettere <marco.lettere(a)dedalus.eu> > To: basex-talk(a)mailman.uni-konstanz.de > Subject: Re: [basex-talk] Indirect evaluation with xquery eval or > invoke on closed collection > Message-ID: <51272589.3000600(a)dedalus.eu> > Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" > > This looks like the same stacktrace reported here: > > https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-February/004683.h… > > BTW, I got also a major corruption of the DB. Your DB remains coherent > when this happens? > M. > > No, but then the query was readonly. Another, but similar query by my colleagues also gave some random answer; where as my gave the stack trace. They used C# api, so it might be partially reason for the difference. -- Arto Viitanen

1 0

corruption when adding documents via the GUI in a almost full database
by Fabrice Etanchaud 22 Feb '13

22 Feb '13

Dear all at BaseX, Under the GUI (7.5.1 beta), When adding documents in a almost full database (Properties/Resources panel), An error is raised, and then the node count gets negative (Properties/General panel). Is there a way to safely add documents until the maximum node number is reached, Or do I have to add logic based on db:info()/databaseproperties/nodes of the input and destination data ? Best regards, Fabrice

2 4

Indirect evaluation with xquery eval or invoke on closed collection
by Arto Viitanen 22 Feb '13

22 Feb '13

Following error comes from BaseX version 7.5, but it seems to exist also on 7.6: We have a query that uses collection: ("suuryritys" stands for "big company"): collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production'] When evaluated on basexgui (or via C# api), the query evaluates, i.e. the database is opened, document (/suuryritys.com/data/ou) is opened ,data fetched and database closed. But, when we try to evaluate the query using either xquery:eval or xquery:invoke xquery:eval("collection('suuryritys_com/suuryritys.com/data/ou')//*[@cat='production']") instead of result, we get error message: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.5 Java: Oracle Corporation, 1.7.0_02 OS: Linux, i386 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 57 org.basex.util.Compress.pull(Compress.java:139) org.basex.util.Compress.unpack(Compress.java:113) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:389) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.io.serial.Serializer.serialize(Serializer.java:84) org.basex.query.iter.ValueBuilder.serialize(ValueBuilder.java:80) org.basex.query.iter.ValueBuilder.serialize(ValueBuilder.java:75) org.basex.core.cmd.AQuery.query(AQuery.java:91) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.gui.GUI.exec(GUI.java:389) org.basex.gui.GUI$6.run(GUI.java:343) Query: xquery:invoke("invoke-file.xq") Query plan: <QueryPlan> <FNXQuery name="invoke(string[,bindings])"> <Str value="invoke-file.xq" type="xs:string"/> </FNXQuery> </QueryPlan> Same error comes, when the query is on a text file, and it is evaluated with xquery:invoke. The funny part is, if the database (suuryritys_com) is opened by hand (on BaseXGui's command line with "open suuryritys_com"), the query returns the result. It also seems, that if I first open some other database, then close it, and then open the database suuryritys_com, I get the same error. -- Arto Viitanen Microteam Oy Finland

2 1

Re: [basex-talk] BaseX-Talk Digest, Vol 38, Issue 33
by elangovan MuthuSwamy 20 Feb '13

20 Feb '13

Hi All, There was a misspell in end of first para. I could get the results within 5 milliseconds using SAX parser misspelled as 5 seconds. Sorry for the inconvenience caused. Thanks, Elango. > Hi All, > > I am trying to use BaseX Client and server with large XML files(13 MB > size). I have created a database for the XML and evaluated an xpath in > that. When I executed an xpath it takes around 250 milliseconds to fetch > the results. In case of SAX parser I could get the response for the same > xpath within 5 seconds. > > I used following BaseXClient code for this. > > BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); > > // code to fetch the large XML file size. Read from the file. > String xmlString = ....... > > InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); > session.create("database", bais); > > String expression = "/configuration/protocol[starts-with(name, > 'http')]/name"; > > *session.query(expression).execute();* // this line takes 250 milliseconds > to give the result. Where as same xpath can be executed within 5 > milliseconds in SAX parser. > > Am I basically missing anything or something else. > > > Please advice me. > > Thanks in advance. > > -- > Regards, > Elango. >

1 0

Performance problem
by Nuria Garcia 20 Feb '13

20 Feb '13

We are testing basex to use with a huge database, and we have a performance problem. At the moment we have two small databases with thousands of documents to test: · Authors, 2.000 documents, 2MBs aprox. · Books (3 per author) with 6.000 documents, 5MBs aprox. We need join and group results, and this takes a long time: Query: <result>{for $Author in (db:open('Authors')//businessEvent), $Book in (db:open('Books')//businessEvent) for $income in distinct-values($Author//author/country) where $Author//author/idauthor=$Book//book/idauthor group by $income return <line><label>{ $income }</label><data>count($Book )</data></line>}</result> Compiling: - rewriting where clause to predicate(s) - pre-evaluating db:open("Authors") - simplifying descendant-or-self step(s) - pre-evaluating db:open("Books") - simplifying descendant-or-self step(s) - simplifying descendant-or-self step(s) - swapping operands: descendant::*:book/*:idauthor = $Author/descendant::author/idauthor - simplifying descendant-or-self step(s) - simplifying descendant-or-self step(s) Result: element result { for $Author in (document-node { "1" }, document-node { "2" }, ...)/descendant::*:businessEvent for $Book in (document-node { "1-1" }, document-node { "1-2" }, ...)/descendant::*:businessEvent[descendant::*:book/*:idauthor = $Author/descendant::author/idauthor] for $income in fn:distinct-values($Author/descendant::author/country) group by $income := $income return element line { element label { $income }, element data { "count($Book )" } } } Timing: - Parsing: 1.01 ms - Compiling: 3.99 ms - Evaluating: 100725.38 ms - Printing: 0.37 ms - Total Time: 100730.77 ms We have no idea why it’s taking so long time to show the results. If we don’t group results it takes a long time too. Regards, Nuria

2 3

basex docs - bulk export?
by Colin McEnearney 20 Feb '13

20 Feb '13

Hi, Is there a way to bulk export the basex docs wiki? Or, if it is backed by a db, get a dump of it? Travelling offline and would love to have access (the built-in pdf creator seems nice but not for getting *everything*) Many thanks, Colin

4 7

can't change password in basex web app
by marco fregonese 18 Feb '13

18 Feb '13

Hello everybody, I'm trying to change the default user name and password in basex web app. I changed the param-value in WEB-INF/web.xml but when deployed in Tomcat or using as standalone application by running BaseX HTTP Server, the web app is still requiring as user name and password "admin", "admin". What am I doing wrong? <context-param> <param-name>org.basex.user</param-name> <param-value>myusername</param-value> </context-param> <context-param> <param-name>org.basex.password</param-name> <param-value>mypassword</param-value> </context-param> Thanks.

2 3

problem keeping basexserver as an alive service
by Richard Alexander Castro Mamani 18 Feb '13

18 Feb '13

Hello, I am having problems keeping the basexserver service alive, you see every time I active the service it works for a couple of hours and then it just stops and I the application cannot connect to the database. It is very inconvenient to keep on typing the following command every time to active the service. richard@0113:~$ basexserver BaseX 7.0.2 [Server] Server was started. Could you tell me what I am doing wrong?. I installed it without any inconvenient but now I don't know how to solve this issue. Thanks in advance for any answer. Greetings Richard -- Br. Richard Castro http://unsaac.academia.edu/RichardAlexanderCastroMamani Grupo de Investigación Hinantin UNSAAC skype: richardtk_1 rcastro(a)hinantin.com http://hinantin.unsaac.edu.pe

3 3

WebDAV Locking of BaseX Source on Tomcat
by Sean Healy 17 Feb '13

17 Feb '13

BaseX 7.6 Tomcat 7 Oxygen Editor 14 Ubuntu 12.10 We would like writers to edit BaseX source via WebDAV and require level 2 WebDAV conformance (locking). We have uncommented: ... <servlet> <servlet-name>WebDAV</servlet-name> <servlet-class>org.basex.http.webdav.WebDAVServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>WebDAV</servlet-name> <url-pattern>/webdav/*</url-pattern> </servlet-mapping> … within .../tomcat7/webapps/bxApp/WEB-INF/web.xml. The class org.basex.http.webdav.WebDAVServlet does not seem to implement low-level, server-side methods such as LOCK. To verify that locking is possible with our toolset, we followed this generic Tomcat/WebDAV tutorial (http://www.mulesoft.com/tomcat-webdav). Tutorial's web.xml: <?xml version="1.0" encoding="UTF-8"?> <web-app ... <display-name>Tomcat webDAV TEST 1</display-name> <description>webDAV HTTP Services</description> <servlet> <servlet-name>webdav</servlet-name> <servlet-class>org.apache.catalina.servlets.WebdavServlet</servlet-class> <init-param> <param-name>debug</param-name> <param-value>0</param-value> </init-param> <init-param> <param-name>listings</param-name> <param-value>true</param-value> </init-param> <init-param> <param-name>readonly</param-name> <param-value>false</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>webdav</servlet-name> <url-pattern>/webdav/*</url-pattern> </servlet-mapping> <security-constraint> <web-resource-collection> <web-resource-name>myApp</web-resource-name> <url-pattern>/*</url-pattern> <http-method>PROPFIND</http-method> <http-method>PROPPATCH</http-method> <http-method>COPY</http-method> <http-method>MOVE</http-method> <http-method>LOCK</http-method> <http-method>UNLOCK</http-method> </web-resource-collection> </security-constraint> <security-role> <description>WebDAV User</description> <role-name>webdav</role-name> </security-role> </web-app> The tutorial demonstrated for us that locking is possible. What is the relationship/interaction between org.apache.catalina.servlets.WebdavServlet, with its full compliment of methods, and org.basex.http.webdav.WebDAVServlet? Or more to the point, how should web.xml within our BaseX servlet be configured within for webDAV on Tomcat? Does org.basex.http.webdav.WebDAVServlet even enter into the equation for our use (e.g., locking). An example web.xml would be much appreciated. Also, is authentication for WebDAV using credentials from .../tomcat/config/tomcat-users.xml (as in the tutorial), from .basexperm, or from some other source? Thank you in advance for your help. Sean

2 1

Call for papers, XML London 2013
by Charles Foster 16 Feb '13

16 Feb '13

Calling all BaseX and XML aficionados. The XML London 2013 conference takes place on 15/16th June 2013 at University College London. We are looking for speakers for this event. Speakers go for free. Call for papers is open and closes 15th March. Submissions do not HAVE to be final papers by that date, extended abstracts which succinctly describe the contents of a final paper will suffice. If successful, final / polished papers will need to be completed by 15th May. The conference has themed topics. However, papers of outstanding quality or novelty are ALWAYS welcome at XML London, regardless of whether they fit into any of the conference theme topics. The conference theme topics are as follows: Applied XML Real life use cases of where XML technologies have been beneficial. Document Databases The future of search across structured and unstructured data. Financial Markup The role of XML in finance and trading. Semantic Web XML working with Linked Data, Ontologies, Topologies, Graphs. Teaching XML XML tech is cool, why so? what can it do for people who are not using it? XML Elephant Coping with and processing huge volumes of XML and Big Data effectively. XML Librarian Publishing content to books, the web and a world of gadgets. For more information, please visit the XML London website: http://xmllondon.com Regards, Charles XML London

1 0

Lassen Sie uns Suche nach der Wahrheit für ein besseres Leben.
by Administrator 15 Feb '13

15 Feb '13

1 0

path index does not seem to be used
by Fabrice Etanchaud 14 Feb '13

14 Feb '13

Dear all at BaseX, In a indexed database, Query info shows that a request like Count(/a/b/c) does not use the statistics data available with index:facets(). Could you please tell me if this is really the case, and if there is a way to tell BaseX to use it ? Best regards, And thank you for your great XNDB ! Fabrice Here is the query : Query: declare namespace exch = 'http://www.epo.org/exchange'; count(/exch:exchange-documents/exch:exchange-document) Query plan: <QueryPlan> <FNAggr name="count(item)"> <IterPath> <DBNodeSeq size="38"> <DBNode name="DOCDB_RU" pre="0"/> <DBNode name="DOCDB_RU" pre="4198620"/> <DBNode name="DOCDB_RU" pre="7369614"/> <DBNode name="DOCDB_RU" pre="11083006"/> <DBNode name="DOCDB_RU" pre="15423603"/> </DBNodeSeq> <IterStep axis="child" test="exch:exchange-documents"/> <IterStep axis="child" test="exch:exchange-document"/> </IterPath> </FNAggr> </QueryPlan> Here is the db:info() : <database> <databaseproperties> <name>DOCDB_RU</name> <size>3391 MB</size> <nodes>135589551</nodes> <documents>38</documents> <binaries>0</binaries> <timestamp>2013-01-29-23-16-31</timestamp> </databaseproperties> <resourceproperties> <inputpath>C:/data/work/docdb/RU</inputpath> <timestamp>2012-12-28-06-53-45</timestamp> <encoding>UTF-8</encoding> <whitespacechopping>ON</whitespacechopping> </resourceproperties> <indexes> <uptodate>true</uptodate> <textindex>ON</textindex> <attributeindex>ON</attributeindex> <fulltextindex>OFF</fulltextindex> <updindex>ON</updindex> <maxcats>10000</maxcats> <maxlen>96</maxlen> </indexes> </database>

3 4

XMLPrague: Talks and Slides
by Christian Grün 13 Feb '13

13 Feb '13

Dear subscribers, it was great to see and talk to many of you at the XMLPrague conference! The slides of most talks are now available online: http://files.basex.org/xmlprague2013/ Video clips of the presentations are in process and will be added soon. Have fun, Christian

1 0

access to dom representation of result in server mode or with xqj
by Fabrice Etanchaud 13 Feb '13

13 Feb '13

Dear all at BaseX, Is there a way to gain access the dom representation of a query result in server mode, or with xqj ? (in simple words, could a client side parser work directly on the internal server representation, for example by returning a 'eventified' serialization of the result ?) Thank you for your wonderful NXDB ! Best regards, Fabrice

2 1

file:list - returns no directories
by Hans-Juergen Rennau 13 Feb '13

13 Feb '13

Dear BaseX team, the documentation states: "Lists all files and directories found in the specified $dir."; however the function does not always return file names. (1) If parameter $recursive=true, the contents are returned, but not the directories themselves. (2) The call "file:list($dir, false(), '*') does not return directories In case of (1), this behavior may often be desirable, but sometimes not, namely if I want to construct a directory tree. Perhaps a further control option would be worth considering? I am not sure. At any rate, if the present behavior is intended, the documentation should be adapted. (2) seems to me a bug, as the call should be equivalent to a one-parameter call, shouldn't it? With kind regards, Hans-Juergen

2 1

loading basex from a cron job
by rsr 13 Feb '13

13 Feb '13

I am just getting started with basex and have been attempting to load data into a basex db via a cron job. I am running the basexclient inside my job redirecting a bxs command file to the client. this works well when i run my script off the command line by fails when run via cron. i've checked the obvious stuff (relative paths, permissions etc). Does anyone have experience loading basex via cron that could point me in the right direction. thanks /home/xxx/basex/bin/basexclient -Uxxx -Pxxx < /home/xxx/load.bxs

5 10

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 12 Feb '13

12 Feb '13

Hi Fabrice and list I am dealing with data-centric XML rather than documents and so there is a fairly high node to content ratio. I have about 250 million nodes and I find that having about 15 million nodes per database seems to work well, but this is just a guesstimate and I am really looking for some performance profiles or some heuristics so that I can limit the numbers of nodes in each database before the performance degrades. Cheers Peter > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, fetanchaud(a)questel.com, >BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Tue, 12 Feb 2013 09:07:40 +0000 > >>Dear Peter, >> >>I'm just a BaseX user, and Christian's team will correct me, but >from my experience, document size does not matter, at least for >querying. >> >>Why do you talk about distributing data ? Did you reach the 2 >billion nodes limit ? >> >>As BaseX indexes all nodes, depending on the values distribution, >creating a new collection containing hand made indices can speed up >your queries. >> >>For example, for append only collections, I'm used to creating a >index collection like this : >><index> >> <item value='value to be indexed'> >> the 'pre' pointer to the indexed element >> </tem> >> <item>... >></index> >> >>And access that 'index' something like this : >> >>for $i in >> //item[@value='searched value'] >>return >> db:open-pre('mydb', $i) >> >> >>And a big number of documents may slow down the properties window >display in the GUI, because of the document tree view. >> >> >>Question to the BaseX 's team : would 'user defined' indices be a >interesting feature ? >> >> >>Regards >> >>-----Message d'origine----- >>De : pw(a)themail.co.uk [mailto:pw@themail.co.uk] >>Envoyé : lundi 11 février 2013 17:13 >>À : Fabrice Etanchaud; pw(a)themail.co.uk; >BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : RE: [basex-talk] handling large files: is there a >streamingsolution? >> >>Thanks Fabrice, I am making good progress following your advice. Do >you have any heuristics for the best way to distribute data for >performant searches and subsetting of data? Am I better having lots >of small files or a few large files in a collection? >> >>> >>> >>> >>>---- Original Message ---- >>>From: fetanchaud(a)questel.com >>>To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >>>Subject: RE: [basex-talk] handling large files: is there a >>>streamingsolution? >>>Date: Mon, 11 Feb 2013 14:38:54 +0000 >>> >>>>Dear Peter, >>>> >>>>Did you try to create a collection with the files (CREATE command) >? >>>>You should start that way, I don't see the point in using file: >>>module for import. >>>>I think that once in the database, file size does not matter >(until >>>you reach millions of file in the collection, and do a lot of >document >>>related operations (list, etc...)) >>>> >>>> >>>> >>>>-----Message d'origine----- >>>>De : basex-talk-bounces(a)mailman.uni-konstanz.de >>>[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >>>pw(a)themail.co.uk >>>>Envoyé : lundi 11 février 2013 15:33 >>>>À : BaseX-Talk(a)mailman.uni-konstanz.de >>>>Objet : [basex-talk] handling large files: is there a streaming >>>solution? >>>> >>>>Hello List >>>>I am wanting to do a join with some large (3-400Mb) XML files and >>>would appreciate guidance on the optimal strategy. >>>>At present these files are on the filesystem and not in a database >>>> >>>>Is there any equivalent to the Zorba streaming xml:parse()? >>>> >>>>Would loading the files into a database directly be the approach, >or >>>is it better to split them into smaller files? >>>> >>>>Is the file: module a suitable route through which to import the >>>files? >>>> >>>>Thanks for your help >>>> >>>>Peter >>>> >>>>_______________________________________________ >>>>BaseX-Talk mailing list >>>>BaseX-Talk(a)mailman.uni-konstanz.de >>>>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >> >> >>

1 0

XQDoc for BaseX
by John Snelson 12 Feb '13

12 Feb '13

Hi all, The xquerydoc project is a 100% XQuery 1.0 project which can parse XQuery modules and extract XQDoc style documentation. I've recently extended it to be able to parse XQuery Update and XQuery Full Text, which I believe solves the majority of problems if you want to use it with modules written for BaseX. The project is available here: https://github.com/xquery/xquerydoc I'm happy to add any other syntax that BaseX supports as well, but I'll need help understanding what that is if there is anything. Enjoy your XQuery documentation! John

3 3

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 12 Feb '13

12 Feb '13

Thanks Fabrice, I am making good progress following your advice. Do you have any heuristics for the best way to distribute data for performant searches and subsetting of data? Am I better having lots of small files or a few large files in a collection? > > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Mon, 11 Feb 2013 14:38:54 +0000 > >>Dear Peter, >> >>Did you try to create a collection with the files (CREATE command) ? >>You should start that way, I don't see the point in using file: >module for import. >>I think that once in the database, file size does not matter (until >you reach millions of file in the collection, and do a lot of >document related operations (list, etc...)) >> >> >> >>-----Message d'origine----- >>De : basex-talk-bounces(a)mailman.uni-konstanz.de >[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >pw(a)themail.co.uk >>Envoyé : lundi 11 février 2013 15:33 >>À : BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : [basex-talk] handling large files: is there a streaming >solution? >> >>Hello List >>I am wanting to do a join with some large (3-400Mb) XML files and >would appreciate guidance on the optimal strategy. >>At present these files are on the filesystem and not in a database >> >>Is there any equivalent to the Zorba streaming xml:parse()? >> >>Would loading the files into a database directly be the approach, or >is it better to split them into smaller files? >> >>Is the file: module a suitable route through which to import the >files? >> >>Thanks for your help >> >>Peter >> >>_______________________________________________ >>BaseX-Talk mailing list >>BaseX-Talk(a)mailman.uni-konstanz.de >>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

handling large files: is there a streaming solution?
by pw＠themail.co.uk 11 Feb '13

11 Feb '13

Hello List I am wanting to do a join with some large (3-400Mb) XML files and would appreciate guidance on the optimal strategy. At present these files are on the filesystem and not in a database Is there any equivalent to the Zorba streaming xml:parse()? Would loading the files into a database directly be the approach, or is it better to split them into smaller files? Is the file: module a suitable route through which to import the files? Thanks for your help Peter

2 1

Load xml files to BaseX in a loop
by Nicholas OR Rita Stevens 10 Feb '13

10 Feb '13

I am new at BaseX so please pardon the newbie question. I am trying to run an xquery script to load a set of xml files in a directory into to BaseX 7.5. If I run replace.xq, shown below, it fails with Out of Main Memory declare %updating function local:upd($path,$file) { db:replace("eams",$path,$file) }; let $files:=for $file in file:list("d:\xmlfiles") return "d:\xmlfiles\"||$file for $file in $files return local:upd("pd",$file) But if I run file replace.bxs, it updates with no problem <commands> <replace path="p1">d:\xmlfiles\drr1.xml</replace> <replace path="p1">d:\xmlfiles\drr2.xml</replace> <replace path="p1">d:\xmlfiles\drr3.xml</replace> <replace path="p1">d:\xmlfiles\dw1.xml</replace> <replace path="p1">d:\xmlfiles\dlp1.xml</replace> </commands> (The same set of files is being processed by both scripts; I checked) The only idea I have is to use xquery to generate the file replace.bxs and then run it. A better way? Thanks.

2 1

Binding external variable to sequence
by Lansing, Jeff J CIV SPAWARSYSCEN-PACIFIC, 56250 09 Feb '13

09 Feb '13

How can I bind an external variable to a sequence? I see the syntax trick for escaping commas, but I still don't get a bound sequence. I tried this: import org.basex.core.*; import org.basex.core.cmd.*; import org.junit.*; public class BindingTest { String var = "$ids="; String seq = "('a',,'b',,'c')"; String query = "" + "declare variable $ids external; " + "let $count := count($ids) " + "return if($count gt 1) then element ids { $ids } " + "else error(QName('urn:err', 'not_enough_ids'), xs:string($count)) " ; /** * test binding external variable to sequence */ @Test public void test() { try { Context context = new Context(); new Set(Prop.BINDINGS, var + seq).execute(context); String result = new XQuery(query).execute(context); context.close(); System.out.println(result); } catch(Exception e) { System.err.println(e); } } } I expect to get <ids>...</ids> but instead I get the error. I'm using v7.5, Java 7u13.

2 1

Document modification timestamp
by Bob Myers 09 Feb '13

09 Feb '13

Sorry for the newbie question, but how do I obtain a modification timestamp for a document in a database? Is there any clean way to integrate with a GNU make-based make system? -- Bob

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk