BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5269 discussions

Base-X basics :-}
by Niels Grundtvig Nielsen 06 May '15

06 May '15

I've used ColdFusion and (reluctantly) php with MySQL, so I have some idea about combining database query with html tags to get formatted output … but I'd welcome a pointer to a tutorial about doing the same thing with Base-X Also, what would you recommend as an introduction to xQuery? w3schools … I seem to remember this was a bit brief, and this afternoon I can't get it to load anyway; wikiBooks on xQuery already incudes exist-DB specific code in the second of its "Beginning examples". Thanks in advance!

2 1

Re: [basex-talk] Pulling files from multiple zips into one DB
by Fabrice Etanchaud 04 May '15

04 May '15

Dear Constantine, In my experience, commands are always faster than db:* calls. Maybe someone @basex could confirm that, and that commands do not use the Pending Update List ? Are you sure you disabled ADDRAW ? If there are many raw files along the xml files, you may have better results extracting and rearchiving only xml before. I have the same problem with patent archives, where each xml file may come with many pdf and gif. Best regards, Fabrice De : Hondros, Constantine (ELS-AMS) [mailto:C.Hondros@elsevier.com] Envoyé : lundi 4 mai 2015 14:01 À : Fabrice Etanchaud Objet : RE: Pulling files from multiple zips into one DB Is that going to be any faster do you think? I tried it and it took a looooong time to read through the zips, so I am hoping there might be a faster more direct way of doing it. From: Fabrice Etanchaud [mailto:fetanchaud@questel.com] Sent: 04 May 2015 13:56 To: Hondros, Constantine (ELS-AMS) Subject: RE: Pulling files from multiple zips into one DB Hello Constantine, Why don't you simply create a new collection with ADDARCHIVES=true ? Best regards, Fabrice De : basex-talk-bounces(a)mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Hondros, Constantine (ELS-AMS) Envoyé : lundi 4 mai 2015 13:50 À : basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Objet : [basex-talk] Pulling files from multiple zips into one DB Hello all, I need to merge any XML files located in 500 GB of zips into a single DB for further analysis. Is there any faster or more efficient way to do it in BaseX than this? TIA. for $zip in file:list($src, false(), '*.zip') let $arch := file:read-binary(concat($src, '\', $zip)) for $a in archive:entries($arch)[ends-with(., 'xml')] return db:add('my_db', archive:extract-text($arch, $a), $a) TIA, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 3

Pulling files from multiple zips into one DB
by Hondros, Constantine (ELS-AMS) 04 May '15

04 May '15

Hello all, I need to merge any XML files located in 500 GB of zips into a single DB for further analysis. Is there any faster or more efficient way to do it in BaseX than this? TIA. for $zip in file:list($src, false(), '*.zip') let $arch := file:read-binary(concat($src, '\', $zip)) for $a in archive:entries($arch)[ends-with(., 'xml')] return db:add('my_db', archive:extract-text($arch, $a), $a) TIA, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

1 0

Querying a subset
by Menashè Eliezer 04 May '15

04 May '15

Hello, I'm using Java org.xmldb.api package for accessing the Basex server (xmldb:basex://...) After getting the resultSet I need to make further queries about the requested subset (for reporting, etc.) I have seen that getId() cannot be used since the Resource will be anonymous if it is obtained as the result of a query. Source: http://xmldb-org.sourceforge.net/xapi/api/index.html The queries are not fixed and are based on end user selection. I couldn't find a way to have a direct access/reference to a document. In my case the ID is simply the filename, but it doesn't seem to be so efficient. I wonder if creating a temporal collection for the subset would be faster than making query/queries similar to the following example in which there are only two ids, but I can have thousands of them: let $ids := ('360836','300139') for $x in collection("collection_name") let $filename := substring-after(base-uri($x),'/') let $id := substring-before($filename,'.') where $id = $ids return ... -- With kind regards, Menashè

2 5

Poor runtime performance of node insertion in BaseX 8.1.1 (compared to BaseX 7.6)
by Werner Dworaczek 03 May '15

03 May '15

Hi Christian, I have found that BaseX 8.1.1 has a very poor performance in comparison to BaseX 7.6 regarding inserting XML elements into an XML document. In my application I very often add new data to the DB by using a construct like this insert node local:recordsWithObjectType(‚Type') into /rootDoc local:recordsWithObjectType() is an xQuery function returning typically several thousand XML nodes which has to be inserted info a given root document. Running this kind of stuff on BaseX 7.6 I typically have a runtime of 500 ms when adding round about 3.000 nodes. Running the exact same stuff on BaseX 8.1.1 (same data) the runtime is about 5.000 ms (factor 10 more). Is there a way to boost the insert performance in BaseX 8.1.1 to a value comparable to BaseX 7.6? Thanks in advance, Werner

2 1

programming a statefull xquery-module
by Rob Stapper 03 May '15

03 May '15

Hi Christian, To support debugging my code I'm writing a module with debugging-functions in xquery. To reduce the impact of the debug-functions on my code I want this debug-module to be statefull. So every time a debug-function is called it remembers it's previous debug state. This way I don't have to carry this debug state all throughout my already complex code. I've tried several options but without success. My latest attempt uses file-read and -write( see attached file), but I don't understand how these functions behave. Can you give me a hint how to approach this? Thanx in advance, Rob Stapper PS. This debug-state could look something like: <debugState functionName 'child-child-function-name' level='3'> <debugState functionName 'child-function-name' level='2'> <debugState functionName 'parent-function-name' level='1'/> </ debugState> </ debugState> --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. http://www.avast.com

2 1

Problem in STORE TO command
by Mh Djamila 03 May '15

03 May '15

Hi I am using BaseX and I want to execute a query on a database and then save the result using the STORE TO command.The query is well executed but when i tried to save the results i had this error : RUN C:/xquery.xq ....... ...... ...... Query "xquery.xq"executed in 4661.28 ms. > STORE TO C:/test.xml Stopped at ,1/21: Syntax: STORE (TO [path])[input] Store raw data Stores raw data to the specified [path] Any help you can provide will be appreciated. Thanks in advance

2 1

Connection refused
by Mh Djamila 03 May '15

03 May '15

Hi I have created in BaseX 8.1 a huge database named 'dblp'(~1,5 Go) and I am trying to communicate with it from Java (Eclipse IDE). I have used the Example.java and BaseXClient.java classes provided in BaseX documentation (basex-examples) in order to connect to the BaseX server try { // initialize timer final long time = System.nanoTime(); // create session final BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); // version 1: perform command and print returned string System.out.println(session.execute("info")); // version 2 (faster): perform command and pass on result to output stream final OutputStream out = System.out; session.execute("xquery 1 to 10", out); // close session session.close(); // print time needed final double ms = (System.nanoTime() - time) / 1000000d; System.out.println("\n\n" + ms + " ms"); } catch(final IOException ex) { // print exception ex.printStackTrace(); } But the connection is always refused. The console shows this error java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.net.PlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at dblp.BaseXClient.<init>(BaseXClient.java:47) at dblp.Example.main(Example.java:24) So could you please show me where is the problem? Thank you in advance

2 1

BaseX trace() output
by cmarchand＠oxiane.com 30 Apr '15

30 Apr '15

in basexGui, running trace((),'foo') displays this in Info panel : Evaluating: - fooempty-sequence() Query: trace((),'foo') Result: - Hit(s): 0 Items - Updated: 0 Items - Printed: 0 Bytes - Read Locking: none - Write Locking: none Timing: - Parsing: 0.75 ms - Compiling: 0.05 ms - Evaluating: 0.35 ms - Printing: 0.27 ms - Total Time: 1.43 ms Query plan: <QueryPlan> <FNInfo name="trace(value,label)"> <Empty size="0"/> <Str value="foo" type="xs:string"/> </FNInfo> </QueryPlan> There is no output, but in the query plan. Is this what is expected ? Le 2015-04-30 12:00, basex-talk-request(a)mailman.uni-konstanz.de a écrit : > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk [1] > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > Today's Topics: > > 1. Re: Database files across multiple directories > (julia.jacobson(a)arcor.de) > 2. Re: Database files across multiple directories (Christian Gr?n) > 3. Re: Where are the debugging infos emit by trace(...) > (Christian Gr?n) > 4. Re: Database files across multiple directories (Goetz Heller) > 5. Running commands on BaseX on Tomcat (Kristian Kankainen) > 6. Re: Running commands on BaseX on Tomcat (Christian Gr?n) > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 29 Apr 2015 17:59:53 +0200 (CEST) > From: julia.jacobson(a)arcor.de > To: basex-talk(a)mailman.uni-konstanz.de > Cc: heller(a)hellerim.de > Subject: Re: [basex-talk] Database files across multiple directories > Message-ID: > <1680192231.162639.1430323193071.JavaMail.ngmail(a)webmail22.arcor-online.net> > > Content-Type: text/plain; charset=ISO-8859-1 > > Thanks a lot for your answers so far. What I was thinking of as a use case are many small xml-Documents in many different (sub)directories that should be used together as one database, like for example one xml file for one address and address arranged into folders according to towns so one would have 9 xml files in a folder named "berlin" and 5 ones in another folder named "munich" and so on. > Does BaseX create an index of these files so that I could for example query for example for persons with the same name? If I want to change something in these files, will the changes be also made in the xml documents? > > Kind regards, > Julia > > ------------------------------ > > Message: 2 > Date: Wed, 29 Apr 2015 18:14:56 +0200 > From: Christian Gr?n <christian.gruen(a)gmail.com> > To: julia.jacobson(a)arcor.de > Cc: Goetz Heller <heller(a)hellerim.de>, BaseX > <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Database files across multiple directories > Message-ID: > <CAP94bnMhv9j1CNP0x-ovEh-jAH-6gT=Xydk48mtU+mynLQQZoQ(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi Julia, > >> Does BaseX create an index of these files so that I could for example query for example > for persons with the same name? > > BaseX will create a database from these files. This can e.g. be done > by starting the GUI, choosing "Database" ? "New?", selecting your XML > directory, and pressing OK. > > All documents will then be stored, and the textual contents will be > indexed. Next, you can use XQuery, which is not only a query language, > but a powerful programming language, to do all kinds of things with > the data. See e.g. [1 [2]] or [2 [3]] for some examples. You probably don't > want to do that by yourself, but XQuery can be used to write full web > applications. > >> If I want to change something in these files, will the changes be also made in the xml documents? > > Currently, there is no automatic synchronisation between databases and > local file directories, so you'll either have to export the database > after updates, or create a new database after files have been changed. > > Cheers, > Christian > > [1] http://docs.basex.org/wiki/XQuery_3.0 [4] > [2] http://www.w3schools.com/xquery/ [3] > > If I want to change something in these files, will the changes be > also made in the xml documents? > >> Kind regards, >> Julia > > ------------------------------ > > Message: 3 > Date: Wed, 29 Apr 2015 18:25:21 +0200 > From: Christian Gr?n <christian.gruen(a)gmail.com> > To: cmarchand <cmarchand(a)oxiane.com> > Cc: BaseX <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Where are the debugging infos emit by > trace(...) > Message-ID: > <CAP94bnMnA7Sb6LbRHpi0fLp-ODpqr-XmR0munVVN8Jqh8z2G4w(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi Christophe, > >> I'm looking for the trace() output in gui and server mode... I've checked >> System.out / System.err, but nothing. > > In the client/server mode, trace output will be redirected to the > client (try -V on command-line, or use SET QUERYINFO ON and > query.info() when using one of the bindings). In the GUI, the output > wil be visible in the Info panel. > > Hope this helps, > C. > > ------------------------------ > > Message: 4 > Date: Wed, 29 Apr 2015 19:37:47 +0200 > From: "Goetz Heller" <heller(a)hellerim.de> > To: <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Database files across multiple directories > Message-ID: <00ec01d082a3$35b09730$a111c590$(a)hellerim.de> > Content-Type: text/plain; charset="iso-8859-1" > > Hi Julia, > this looks to me as if you had hybrid data - some relational (tabular) data > (addresses etc.) and other semi-structured data. If so, then think about > storing the tabular data in a relational database management system (RDBMS) > like Maria SQL or MS SQL Server and accessing it from within your xQueries. > This is something which can be achieved very easily since the BaseX > implementation accommodates adding Java code to your queries simply by > dropping your jar files into the lib directory and adding a namespace > declaration to the prolog of your query - it's just a snap. You can do > anything you want with the relational data - insert, update, delete, and, of > course, select, you can run stored procedures, and still access your XML > data as easily, combine results from both sources, create great reports, and > so on. > > Think of xQuery as a domain-specific language (DSL) for querying XML data, > just as SQL is a DSL for dealing with relational data, which you can enhance > according to your needs. > > If you need to modify only the relational part of the data, you can do this > easily via the RDBMS since this is standard, and BaseX still remains the > perfect choice for handling the XML part of your data. Although many RDBMSs > offer also some XML and even xQuery functionality, compared to a system like > BaseX this is only rudimentary. E.g. Oracle and SQL Server implement only > xQuery 1.0, and even this only partially. > > I did something similar with BaseX and SQL Server, and - just as a proof of > concept - also with Oracle. E.g., you can create SQL code with xQuery on the > fly so it incorporates current context data, and pass it to some generic > database access function. With JDBC, you've got the glue to connect to a > wide range of database servers. > > I did not figure out whether you can make a database server and BaseX > participate in the same distributed transaction because I had no necessity > to do so; maybe Christian can tell you something more about it. > > Kind regards, > > Goetz > > -----Urspr?ngliche Nachricht----- > Von: julia.jacobson(a)arcor.de [mailto:julia.jacobson@arcor.de] > Gesendet: Mittwoch, 29. April 2015 18:00 > An: basex-talk(a)mailman.uni-konstanz.de > Cc: heller(a)hellerim.de > Betreff: Re: [basex-talk] Database files across multiple directories > > Thanks a lot for your answers so far. What I was thinking of as a use case > are many small xml-Documents in many different (sub)directories that should > be used together as one database, like for example one xml file for one > address and address arranged into folders according to towns so one would > have 9 xml files in a folder named "berlin" and 5 ones in another folder > named "munich" and so on. > Does BaseX create an index of these files so that I could for example query > for example for persons with the same name? If I want to change something in > these files, will the changes be also made in the xml documents? > > Kind regards, > Julia > > ------------------------------ > > Message: 5 > Date: Thu, 30 Apr 2015 10:12:23 +0300 > From: Kristian Kankainen <kristian(a)keeleleek.ee> > To: BaseX <basex-talk(a)mailman.uni-konstanz.de> > Subject: [basex-talk] Running commands on BaseX on Tomcat > Message-ID: <5541D5D7.5000301(a)keeleleek.ee> > Content-Type: text/plain; charset=utf-8; format=flowed > > Hello! > > I have BaseX 8.11 running on Tomcat and need to execute some commands or > XQuery for deploying xml files from elsewhere. I allready have a XQuery > function for this and can do it on my local setup using for example the > basexclient. > > My problem is that the Tomcat is available only through HTTP and I > wouldn't like to open up the function for the outside world using > RESTXQ. Is there a way to "talk" with the BaseX in the Tomcat? I have > full access to the machine via SSH. Another way I thought of would be to > just upload the data folder from my local setup, but it doesn't feel the > right way to do it. > > Best regards > Kristian K > > ------------------------------ > > Message: 6 > Date: Thu, 30 Apr 2015 10:47:23 +0200 > From: Christian Gr?n <christian.gruen(a)gmail.com> > To: Kristian Kankainen <kristian(a)keeleleek.ee> > Cc: BaseX <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Running commands on BaseX on Tomcat > Message-ID: > <CAP94bnNjxDFGo8ttNeE0c4v7FfYgCoDtHc=K-fDgZo=RNC65Lw(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi Kristian, > >> My problem is that the Tomcat is available only through HTTP and I wouldn't >> like to open up the function for the outside world using RESTXQ. Is there a >> way to "talk" with the BaseX in the Tomcat? > > Have you already thought about adding the DBA interface [1 [2]]? It might > be extended with new functionality to store and invoke server-side > code, and you can already use it to run arbitrary XQuery expressions. > > Another choice is our classical REST interface, which can also be used > to run server-side queries or command scripts. > > Just remember that these interfaces can potentially be accessed by > other people as well, even those who are not supposed to access it, so > you should e.g. change the default admin password, etc. > > Best, > Christian > > [1] http://docs.basex.org/wiki/DBA [2] > > I have full access to the > >> machine via SSH. Another way I thought of would be to just upload the data >> folder from my local setup, but it doesn't feel the right way to do it. >> >> Best regards >> Kristian K > > End of BaseX-Talk Digest, Vol 64, Issue 58 > ****************************************** Links: ------ [1] https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk [2] http://docs.basex.org/wiki/DBA [3] http://www.w3schools.com/xquery/ [4] http://docs.basex.org/wiki/XQuery_3.0

2 1

Re: [basex-talk] Database files across multiple directories
by Goetz Heller 29 Apr '15

29 Apr '15

Small correction; I should have said "reference each database with doc()" instead of "open each database". My apologies. Kind regards, Goetz

3 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk