- BaseX-Talk - mailman.uni-konstanz.de

insert an xml structure in database
by Oleg Duvanayev 12 Apr '16

12 Apr '16

2 2

Re: [basex-talk] XLSX to XML
by Dirk Kirsten 11 Apr '16

11 Apr '16

Hello Florian, please remember to always include the list when replying as it allows others to benefit from our exchange as well and also allows others to help you. I just want to point out, again, that you it doesn't make sense to say "convert the excel file to xml", because it already is XML. Yes, there might be multiple XML files and they reference each other, but this is just a very normal thing for XML and for every reasonably complex system to reference each other. So I guess what you really want is an XQuery module which allows you to easily manipulate xlsx files without the need to worry about internal ooxml format stuff like shared strings. This if course makes a lot of sense! However, as the format is ridiculously complicated it is a hard task to write a general-purpose library for all kinds of manipulations. As Christian indicated we wrote for ourself some small helpers functions which dies the stuff we need in our projects, but is very far from being complete on the xlsx standard. Cheers Dirk On 04/06/2016 12:35 PM, Florian Eckey wrote: > Hello Dirk, > > thanks. That was my idea as well. But the format xlsx is really complicated, because the content (sheet01.xml) in the cells is referenced to an other document (stringValues.xml) using an index. I guess anyone has implemented a simple xquery to convert the excel file to xml? > But if nobody has done that before, i have to spend time for the implementation on my own. :) > > Thanks, best, > Florian > > > > > Am 06.04.16, 12:26 schrieb "Dirk Kirsten" <dk(a)basex.org>: > >> Hello Florian, >> >> xlsx is just a zip file containing many xml files. you can simply unzip >> the xlsx (e.g. by using the BaseX zip module), modify the xml files >> inside using standard XQuery and rezip it again as xslx. >> >> Cheers >> Dirk >> >> On 04/06/2016 12:18 PM, Florian Eckey wrote: >>> Hi guys, >>> >>> are there any ideas how to convert excel's xlsx (not xls) files to xml >>> with XQuery or to use a Java library which can be imported? It looks >>> like BaseX has no internal functions as for instance MarkLogic. >>> >>> Any ideas or example implementations to do that in XQuery or Java? >>> >>> Best, >>> Florian >> -- >> Dirk Kirsten, BaseX GmbH, http://basexgmbh.de >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >> `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22 >> -- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

5 4

slow GUI-editor when using package-manager eXPath
by Rob Stapper 10 Apr '16

10 Apr '16

Hi Christian, New tools, new challenges. I've rearranged my source-code in packages/libraries conform the eXPath-specs in BaseX now. But since than the GUI-editor is becoming very slow, even unworkably slow. What is causing this and can you give me any tips how speed up the GUI-editor? I've got 7 libraries now. The one with the largest interface counts 11 entries in file:expath-pkg.xml, all public entries, the smallest library counts one entry. TIA, Rob

3 12

RESTXQ path preference
by Tim Thompson 10 Apr '16

10 Apr '16

Hello, Reading the draft RESTXQ spec[1], I have a question about path preference. Namely, can a bare trailing slash serve as a valid path segment? I am thinking about the pattern used by some linked data services to accommodate httpRange-14[2]. For example, VIAF (Virtual International Authority File), a popular service for library data, uses a trailing slash to differentiate between the URI for a real-world object and the URI for the Web resource that represents it. So, a GET request on the URI http://viaf.org/viaf/71387829 triggers a 303 redirect to http://viaf.org/viaf/71387829/ The RESTXQ implementation in BaseX does not seem to permit this pattern at present. If I create a resource function with a path annotation that ends in a trailing slash, the GET request returns 404. Any chance this feature could be supported? :) Best regards, Tim [1] http://exquery.github.io/exquery/exquery-restxq-specification/restxq-1.0-sp… [2] https://en.wikipedia.org/wiki/HTTPRange-14 -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

2 2

RESTXQ and Website-Building
by Günter Dunz-Wolff 10 Apr '16

10 Apr '16

Hi, I'm planning to relaunch my website (kleist-digital.de) with OpenShift as BaseX-Server. After some difficulties and a lot of help from Christian and Andy the server is running now. I took a look at RESTXQ and it seems quite interesting to build the whole frontend for my app with it. Some questions I have: 1. I'm working with a basic layout for all pages. This basic layout-file includes header, footer and changing content. What is the best approach with RESTXQ. How to build a modular concept with RESTXQ? 2. Is there any site-structure preferred? 3. Most of the site is dynamic. But I need also static files like images, css, javascript. Where to put them best (so that the Openshift-Server can handle them)? Thanks for any advice and help. Günter

4 3

Is Basex scanning static before each http-request?
by Lars Johnsen 08 Apr '16

08 Apr '16

Hi all It seems that BaseX is scanning the webapp directory including static for each request. Is that right? And is it possible to turn it off ? Best, Lars

3 3

node-basex
by kleist-digital 08 Apr '16

08 Apr '16

Hi Andy, me again. Just saw, that you’ve also written a node-based Basex-Client. My frontend is a node-app. So far I’m using AJAX-calls, to communicate via the REST-Interface with the Basex-Server, in generell I use the run command to execute different queries. Are there any advantages (disadvantages) to change to a classical Server-Client Scenario (performance, less resources, security)? You know, I want to use it in an OpenShift-Environment for the future. Thanks for any advice. Günter

2 3

XQUERY3 option
by George Sofianos 08 Apr '16

08 Apr '16

Hello, I wonder if there is any way at the moment to run scripts with xquery version 1 only. I can't find a way to set XQUERY3 option to false. I have some scripts that fail to run (process runs forever) with BaseX and I want to see if the version is causing this. I have set xquery version "1.0" encoding "UTF-8"; as the first line of my script, but it doesn't seem to do anything as well. Also is there any way to see where BaseX is consuming so much time/processing power? I can use a profiler but It's not easy to see what part of the script is causing this huge delay. Thanks.

2 3

Symlinked files with RESTXQ
by Lars Johnsen 08 Apr '16

08 Apr '16

Dear all What must be done to access symlinked files when using BaseX as http server? The scenario is that a fairly large collection of audio-files are accessed using BaseX which serves them up over http. However, it seems that symlinking (the folder with audio files exists as a symlinked copy in the static directory of the BaseX web folder) prevents the browser from accessing them. Is there a parameter in the settings I have overlooked? Best, Lars G. Johnsen National Library of Norway

3 2

Fwd: RESTXQ and Website-Building
by Günter Dunz-Wolff 08 Apr '16

08 Apr '16

Hi Christian, thanks for the tipp. I haven't noticed the files so far at all. I will have a closer look and I think a lot of questions :) Günter P.S.: Sorry for the second email, I didn't send it to the list. > > > >> Am 07.04.2016 um 17:55 schrieb Christian Grün <christian.gruen(a)gmail.com>: >> >> Hi Günter, >> >> This is just a quick reply, but you could have a look at the DBA >> XQuery code embedded with BaseX. Here, similar problems have been >> resolved: All pages have the same layout, static files are served as >> well, etc. >> >> Feel free to ask if you need some more help with the code. It’s >> definitely not perfect; for example, the function-based generation of >> buttons, tables, etc. is just one of many options, and I am convinced >> there should be better alternatives. >> >> Christian >> >> >> On Thu, Apr 7, 2016 at 5:08 PM, Günter Dunz-Wolff >> <kleist(a)mail.dunzwolff.de> wrote: >>> Hi, >>> >>> I'm planning to relaunch my website (kleist-digital.de) with OpenShift as BaseX-Server. After some difficulties and a lot of help from Christian and Andy the server is running now. >>> >>> I took a look at RESTXQ and it seems quite interesting to build the whole frontend for my app with it. Some questions I have: >>> >>> 1. I'm working with a basic layout for all pages. This basic layout-file includes header, footer and changing content. What is the best approach with RESTXQ. How to build a modular concept with RESTXQ? >>> >>> 2. Is there any site-structure preferred? >>> >>> 3. Most of the site is dynamic. But I need also static files like images, css, javascript. Where to put them best (so that the Openshift-Server can handle them)? >>> >>> Thanks for any advice and help. >>> >>> Günter >> >

1 0

Fwd: RESTXQ and Website-Building
by Günter Dunz-Wolff 08 Apr '16

08 Apr '16

Hi Marco, your approach sounds great. It would be a very, very great help, to see some snippets :-) so the picture becomes more clear for me. Ciao, Günter P.S.: Sorry for the second email, I didn't send it to the mailing-list > >> Am 08.04.2016 um 09:13 schrieb Marco Lettere <m.lettere(a)gmail.com>: >> >> Hi Günter, >> we have worked on several web applications that include a frontend and we are very happy with using basex for serving both static (scripts, css, images) and dynamic (markup) content. >> One pattern that we've used often and that we find very productive is to split markup content into different .xqm modules. >> The modules are called through the xquery:invoke function in order to "explode" their output into the main page. >> In our opinion this is a good approach because it is applicable recursively to sub-modules. Content of modules may be dynamic as well and parametric values are passed into the module through external variables. >> Finally, it is possible to set up an automatic way of mapping requests to module invocations by exploiting REST path segments for example. >> I hope I have been able to explain it well enough otherwise, just let me know and as soon as possible I'll provide an example code snippet that demonstrates the approach. >> >> Ciao, >> Marco. >> >> On 07/04/2016 17:08, Günter Dunz-Wolff wrote: >>> Hi, >>> >>> I'm planning to relaunch my website (kleist-digital.de) with OpenShift as BaseX-Server. After some difficulties and a lot of help from Christian and Andy the server is running now. >>> >>> I took a look at RESTXQ and it seems quite interesting to build the whole frontend for my app with it. Some questions I have: >>> >>> 1. I'm working with a basic layout for all pages. This basic layout-file includes header, footer and changing content. What is the best approach with RESTXQ. How to build a modular concept with RESTXQ? >>> >>> 2. Is there any site-structure preferred? >>> >>> 3. Most of the site is dynamic. But I need also static files like images, css, javascript. Where to put them best (so that the Openshift-Server can handle them)? >>> >>> Thanks for any advice and help. >>> >>> Günter >> >> >

1 0

Problem deleting jar packaged item from repo
by Andy Bunce 08 Apr '16

08 Apr '16

Hi, If I install HelloWorld.jar to the repository as described at [1] It all works. However attempts to remove the item generate: [bxerr:BXRE0001] Package 'org.basex.modules.Hello' does not exist. /Andy [1] http://docs.basex.org/wiki/Repository#Java

3 11

XLSX to XML
by Florian Eckey 07 Apr '16

07 Apr '16

Hi guys, are there any ideas how to convert excel's xlsx (not xls) files to xml with XQuery or to use a Java library which can be imported? It looks like BaseX has no internal functions as for instance MarkLogic. Any ideas or example implementations to do that in XQuery or Java? Best, Florian

4 3

XdmItem
by Kendall Shaw 06 Apr '16

06 Apr '16

Is XdmItem going to move out of tests to be part of the API at some point? Kendall

1 0

Issue with db:add via restxq in 8.4.2
by Vincenzo Cestone 06 Apr '16

06 Apr '16

Hi guys, in the new basex 8.4.2 version I cannot write on the db via restxq. For example if you create an empty 'testdb', set in the .basex conf file the option MIXUPDATES = true, and call the '/dbaddtest' restxq implemented as in the following: --- module namespace o = "test"; declare %rest:path("/dbaddtest") %rest:GET function o:dbaddtest() { let $o := db:add('testdb', <aaa/>, '111.xml') return <ok/> }; --- it will not work in the basex 8.4.2 i.e. the document is not added to the db, but it will return <ok/> without errors. On the contrary the db:add executed in the BaseX.jar client is working well. And also the above restxq code is working as expected in the basex 8.3.1 version. Thank you in advance. Regards, Vincenzo

2 2

Function as argument
by Paul Swennenhuis 06 Apr '16

06 Apr '16

Hi all, I am now using functions as argument in BaseX for the first time, borrowing the way to do that from some old list archive contribution. Apparently you have to use the name of the function followed by #1 like this: let $a := local:someFunc($nodeSet, local:someOtherFunc#1) But it is unclear to me if this is indeed the right way to do it. The #1 suggests that there may also be a #2, #3 etc. (and possibly a #0). I could not find the documentation on using functions as argument on the Wiki pages. (I am happy to see that it works as expected though) Paul

2 2

Re: [basex-talk] XQJ status
by Marc van Grootel 06 Apr '16

06 Apr '16

Hi all, Thanks both Charles and Jürgen for your comprehensive and enlightening answers. It is very good to hear that there is hope for XQJ even in an XQuery 3 world ;-) Beefing up the spec to XQuery 3.1 would be fantastic. For the shorter term I'm thinking along the lines of a XDM transport serialization that includes type info and aims at efficiency rather than being human readable. As my host language of choice would be Clojure I am currently looking at Transit[1] which has both a Java and Clojure library and seems like a nice candidate but there are many others such as Avro, Thrift, Protocol buffers, Hessian etc etc. I might read up a little on those too[2]. [1] https://github.com/cognitect/transit-format On Tue, Apr 5, 2016 at 9:04 PM, Charles Foster <charles(a)cfoster.net> wrote: > Hi Marc, > > I plan to open source all the XQJ drivers I’ve written, (in the long-run). > > The XQJ specification was designed to support XQuery 1.0 only, however I don’t see why it can’t be made to support XQuery 3.0 and 3.1 syntax to some extent. > > Regarding XDM Maps and Arrays being bound to external variables and returned in result sequences along with the conversion between XDM and Java types wouldn’t be too difficult to achieve in practice. > > Java’s List and Map classes spring to mind, along with XQItemAccessor getObject(...) [1] to retrieve and XQDynamicContext.bindObject(...) [2] to bind where you would of course have to do manual casting. The datatypes of Java's Map and Array entries and conversions between their XDM counterparts could be determined by following the rules already laid out in the XQJ spec itself [3]. > > One would of course be going above and beyond the original XQJ specification here, but I think technically it would still be within the rules and providing it was well documented I’m sure user’s could cope. I would love Oracle to update their existing XQJ specification to cater for XQuery 3.1 properly, I would even help them get there if they wished, but I don’t know if they would be up for something like this. > > I think once the drivers are open sourced, the world of XQJ will be in healthier place for the community to update going forward. > > Specifically regarding BaseX XQJ - for the moment, please assume that any of the driver’s provided by BaseX themselves are the latest and greatest. > > Kind Regards, > > Charles > > P.S. Please come to XML London in June - it will be great! > > [1] http://xqj.net/javadoc/javax/xml/xquery/XQItemAccessor.html > [2] http://xqj.net/javadoc/javax/xml/xquery/XQDynamicContext.html > [3] https://en.wikipedia.org/wiki/XQuery_API_for_Java#Default_data_type_mapping > > > On 5 Apr 2016, at 14:41, Marc van Grootel <marc.van.grootel(a)gmail.com> wrote: > >> Hi Christian, >> >> Thanks. Very curious to hear about XQJ's fate. >> >> I am currently using BaseX for data integration which got me thinking >> about integration of XQuery in other web applications >> (node/python/clojure etc.). I was triggered by the recent thread about >> node-basex[1] and Hans Jürgen and your Balisage paper [2]. I was >> considering what the most performant way would be to return XQuery >> results as native types. Eg. function returns an array with maps or >> XML nodes and it gets converted to the host application's native data >> types so the application can use the results as if they were native >> data types. >> >> Maybe the only viable way to tackle this *is* via HTTP and a smart >> serialization (EDN[3] for example is such a, non-JSON, serialization >> that includes type info) but I was thinking if there was an existing >> lower-level way (without the BaseX HTTP layer). XQJ would enable this >> (at least for XQuery < 3) but I have doubts if it would deserialize >> non-atomic XDM values (like nested maps/arrays) as this is new to >> XQuery 3, I didn't check but I figure that the BaseXClient on which it >> is based also doesn't provide the XDM metadata for each nested item. >> >> Just pondering how to bridge from XQuery to other languages. >> >> [1]: http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg07882.h… >> [2]: http://www.balisage.net/Proceedings/vol15/html/Rennau01/BalisageVol15-Renna… >> [3]: https://github.com/edn-format/edn >> >> >> On Tue, Apr 5, 2016 at 11:58 AM, Christian Grün >> <christian.gruen(a)gmail.com> wrote: >>> Hi Marc, >>> >>>> Is it right that the current XQJ driver for BaseX doesn't support >>>> XQuery 3 syntax? Everytime I try to use a query that includes some of >>>> the newer features it barks at me with Invalid XQuery syntax, syntax >>>> does not pass static validation. Is there way to get around that, say, >>>> when the result would not use these XQuery 3 data types? >>>> >>>> I'm wondering what the fate is of XQJ in general. Will it remain relevant? >>> >>> I decided to pass on your question directly to Charles Foster; he >>> might be able to give you some outlook! >>> >>>> Also I am confused why files.basex /maven repo has XQJ driver 1.6.0 >>>> and xqj.net has 1.4.0 as the latest or what the difference is. >>> >>> I have sent a pull request to Charles’ repo; it may be included in the >>> next version there. >>> >>>> The reason I was looking into XQJ was that it provides ways to get XDM >>>> type info with the result. I would like to be able to deal with the >>>> result from a query at a higher level than just strings. I've briefly >>>> looked into BaseXClient.java which seems to be able to return XDM >>>> metadata but then I saw that XQJ could do this out of the box. >>> >>> Right, that’s only possible with XQJ. >>> >>>> In general I would be interested to hear what the best way would be to >>>> get a client to deal with query responses as native (client-side) >>>> types (including hashmaps and arrays) instead of strings. Would it be >>>> better to focus on using BaseXClient and forget about XQJ? >>> >>> From my perspective, it’s good to use XQuery whenever possible and use >>> our Client Module to address BaseX server instances [1] ;) >>> >>> Cheers, >>> Christian >>> >>> [1] http://docs.basex.org/wiki/Client_Module >> >> >> >> -- >> --Marc >> > -- --Marc

1 0

XQJ status
by Marc van Grootel 05 Apr '16

05 Apr '16

Hi, Is it right that the current XQJ driver for BaseX doesn't support XQuery 3 syntax? Everytime I try to use a query that includes some of the newer features it barks at me with Invalid XQuery syntax, syntax does not pass static validation. Is there way to get around that, say, when the result would not use these XQuery 3 data types? Also I am confused why files.basex /maven repo has XQJ driver 1.6.0 and xqj.net has 1.4.0 as the latest or what the difference is. I'm wondering what the fate is of XQJ in general. Will it remain relevant? The reason I was looking into XQJ was that it provides ways to get XDM type info with the result. I would like to be able to deal with the result from a query at a higher level than just strings. I've briefly looked into BaseXClient.java which seems to be able to return XDM metadata but then I saw that XQJ could do this out of the box. In general I would be interested to hear what the best way would be to get a client to deal with query responses as native (client-side) types (including hashmaps and arrays) instead of strings. Would it be better to focus on using BaseXClient and forget about XQJ? Cheers, --Marc

3 4

XQuery Introduction in French
by Dirk Kirsten 05 Apr '16

05 Apr '16

Bonjour, just a little pointer: If you speak french and are new to XQuery (and especially if you are in the digital humanities) the following chapters from Farid Djaïdjas thesis might be of interest to you: "XQuery pour les Humanités Numériques" http://files.basex.org/publications/dummies/XQuery%20pour%20les%20Humanit%e… This introduction and some others introductions and further material to get you started can be found at http://docs.basex.org/wiki/Getting_Started#Tutorials_and_Slides Bien cordialement Dirk -- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

1 0

BaseX optimizer performance on REx-generated parser
by Gunther Rademacher 05 Apr '16

05 Apr '16

Hi, in a recent update of REx parser generator (v5.37), some type specifications were removed from generated XQuery and XSLT functions. This affects tail-recursive functions, and it was done because it turned out that MarkLogic fails to optimize tail calls, presumably due to an explicit type check caused by the type specification (see http://markmail.org/message/gxi26da4crk2v5ge ). Now it was reported that this change causes performance problems with BaseX (see https://twitter.com/apb1704/status/714219874441146368 ). I reproduced the problem as follows: - download the XQuery 3.1 grammar from http://bottlecaps.de/rex/CR-xquery-31-20151217.ebnf - generate a parser from it on http://bottlecaps.de/rex/ using command line options: -xquery -tree -main - run the generated parser, using this command line basex -binput={42} CR-xquery-31-20151217.xquery This works OK, but it takes about 80 seconds. Some analysis showed that the time can be influenced by putting back the type specification to function p:map2, i.e. declaring 'as xs:integer' as the result type: declare function p:map2($c as xs:integer, $lo as xs:integer, $hi as xs:integer) as xs:integer This variant completes in less than 3 seconds. But even when declaring a return type of as xs:integer? which might be the type that can be inferred statically, it completes fast. Is this possibly a problem with the optimizer? Which variant of generated code would be preferable for BaseX - typed or untyped? Thanks, Gunther

3 21

Problem with module-import via relative / absolute path (WEBPATH, REST-Interface)
by Günter Dunz-Wolff 04 Apr '16

04 Apr '16

Hi all, in my queries I have to import module namespace, for example in the file test.xq something like import module namespace kleist = "http://kleist-digital.de/ns/kleist" at "../modules/kleist-functions.xqm"; I'm calling the query via REST like http://localhost/rest?run=xqs/test.xq The webpath is configured like # HTTP Services WEBPATH = /Users/xxx/Documents/kleist/BaseXWeb RESTPATH = When I'm using the absolute path, everything works fine: import module namespace kleist = "http://kleist-digital.de/ns/kleist" at "file:///Users/xxx/Documents/kleist/BaseXWeb/modules/kleist-functions.xqm"; When I'm using the relative path for importing the module namespace like import module namespace kleist = "http://kleist-digital.de/ns/kleist" at "../modules/kleist-functions.xqm"; I'll get the following error: "Could not retrieve module: /Users/modules/kleist-functions.xqm" The correct address should be: Users/xxx/Documents/kleist/BaseXWeb/modules/kleist-functions.xqm What's wrong in my configuration? Btw, I need the relative path for portability to my remote server. Any help would be great, thanks a lot, Best regards, Günter

4 6

Async Module
by Christian Grün 03 Apr '16

03 Apr '16

Hi everyone, As some of you know, James Wright has presented his excellent work on the Promise Module in Prague [1,2]. The promise:fork-join function is now part of BaseX, along with some additional functions for asynchronous query processing [3,4]. Your feedback is welcome as usual, Christian [1] https://github.com/james-jw/xq-promise/ [2] http://archive.xmlprague.cz/2016/files/xmlprague-2016-proceedings.pdf [3] http://docs.basex.org/wiki/Async_Module [4] http://files.basex.org/releases/latest/

3 3

[ANN] Openshift BaseX quick start v0.6
by Andy Bunce 03 Apr '16

03 Apr '16

Hi, A new version is available at https://github.com/Quodatum/openshift-basex-quick-start. New features: - Enable REST and WebDAV - support for setting initial users - support for setting initial data - Use Java 8 Thanks to Michael Sperberg-McQueen and Günter Dunz-Wolff for helpful comments on the previous version. /Andy

3 6

Expected behaviour with full disk
by James Ball 03 Apr '16

03 Apr '16

Hello, I’m having some problems with BaseX in situations where there it runs out of disk space. I’m delivering XML files to BaseX 8.4 using RestXQ. A request is made and the XML is added to a database depending on some criteria. I receive many hundreds of thousands of requests and everything works well unless I run out of disk space. What I would expect to happen is that BaseX would try to add the document, find it couldn’t due to the disk being full and return a server error (e.g. 500) and the transaction would be aborted leaving the database as it was before the request. Any further requests would all get errors returned. What I’m seeing is that when a request is made that causes the disk to become full BaseX gets part way through adding the document, fails and returns an error but also leaves the database corrupted. Usually the server has to be restarted and then any database that has been affected has to be deleted - it’s rare to be able to save anything from it. The errors vary but it is usually a NullPointer exception and then java.nio.channels.OverlappingFileLockException. for further requests. I’ve tried turning off logging and setting parallel to 1 to see if that made any difference but it doesn’t. Is what I’m seeing unique to my setup or do others see this? Is there anything I can do to make BaseX just stop accepting requests when disk space runs out to protect my databases? Many thanks, James

4 5

Internal optimization
by Gabriel Tessarolli 30 Mar '16

30 Mar '16

Hi, First of all, thank you for the excellent software you produce and maintain! Keep up the good work. I've been using BaseX for some academic experiments on XQuery processing, and I got this situation that you guys can probably explain. Here is some context: - I am using version 8.2.3. - Database 'expdb' was created with default options of that version, using 'auction.xml' document generated from Xmark benchmark [1]. - BaseX is running with default options too. - What the query does is irrelevant. When I execute this query: for $pe in doc('expdb/auction.xml')/site/people/person for $cat in doc('expdb/auction.xml')/site/categories/category[position() >= 1 and position() < 101] where count($pe/profile/interest) > 3 and $pe/profile/interest/@category = $cat/@id return <match> <person>{$pe/name}</person> <category>{$cat/name}</category> </match> the resulting optimized query (in 'Query Info' window, on GUI) is this: for $pe_0 in *db:open-pre*("expdb",0)/*:site/*:people/*:person[3.0 < count(*:profile/*:interest)] for $cat_1 in *db:open-pre*("expdb",0)/*:site/*:categories/*:category[position() = 1 to 100][(@*:id = $pe_0/*:profile/*:interest/@*:category)] return element match { (element person { ($pe_0/*:name) }, element category { ($cat_1/*:name) }) } If I change the original query to (note that I am only switching the position of 'for' clauses): for $cat in doc('expdb/auction.xml')/site/categories/category[position() >= 1 and position() < 101] for $pe in doc('expdb/auction.xml')/site/people/person where count($pe/profile/interest) > 3 and $pe/profile/interest/@category = $cat/@id return <match> <person>{$pe/name}</person> <category>{$cat/name}</category> </match> the optimized query changes to: for $cat_0 in *db:open-pre*("expdb",0)/*:site/*:categories/*:category[position() = 1 to 100] for $pe_1 in *db:attribute*("expdb", $cat_0/@*:id)/self::*:category/parent::*:interest/parent::*:profile/parent::*:person[3.0 < count(*:profile/*:interest)] return element match { (element person { ($pe_1/*:name) }, element category { ($cat_0/*:name) }) } You can see that BaseX was able to use attribute index to optimize last query, reducing its execution time (by a lot!). My question is: what is the explanation for this behavior? Thank you in advance! Gabriel Tessarolli --- [1] http://www.xml-benchmark.org/

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk