March 2016 - BaseX-Talk - mailman.uni-konstanz.de

How to check an element's type against an XSD simple type
by Ulrike Schaper - SideStep Business Solutions GmbH 12 Mar '16

12 Mar '16

Hi everybody, I'm trying to compose XML documents for SEPA transactions via BaseX. I would like to check certain data against the corresponding schema before putting them into the document, e.g. BIC identifiers. In the xsd, there's a simpleType definition for BICs: <xs:simpleType name="BICIdentifier"> <xs:restriction base="xs:string"> <xs:pattern value="[A-Z]{6,6}[A-Z2-9][A-NP-Z0-9]([A-Z0-9]{3,3}){0,1}"/> </xs:restriction> </xs:simpleType> I hoped it would be as simple as declare default element namespace "urn:iso:std:iso:20022:tech:xsd:pain.001.003.03"; let $node := <BIC>GEN0DEM1MPW</BIC> return if(string($node) instance of BICIdentifier) then (true() (: process... :)) else (false() (: report error :)) But, as I learned now, BaseX doesn't support import of schemas. Thus, I receive the error message "Unknown type: Q{urn:iso:std:iso:20022:tech:xsd:pain.001.003.03 }BICIdentifier". Is there some workaround (other than rebuilding regular expressions from schema to xquery)? Best regards, Ulrike.

2 4

Using SaxonEE 9.7 XQuery Processor with BaseX
by Jörn Goeke 11 Mar '16

11 Mar '16

Hello together, I just recently startet using BaseX in one of my Projects and since I was developing on the GUI of BaseX I had no Problems with anything. Unfortunately, for reasons, I need to use the Saxon Processor to Query my BaseX DB. I tried different approaches, but, everything endet up with the following error message: XPST0017: Saxon does not allow external function to be declared The query I tried looks as followed: xquery version "3.0"; declare namespace vm = "http://www.viamedici.de"; import module namespace db = "http://basex.org/modules/db" at "test/db.xqm"; (: Retrieve documents from database :) db:open("hades")//template Since I could not get usable information from google I hope some of you guys can help me. Thanks a lot, Jörn Jörn Uli Goeke Projektleiter / Projektentwickler Viamedici Software GmbH Hertzstr. 14 D-76275 Ettlingen Germany Phone: +49(0)7243/9498-36 Fax: +49(0)7243/9498-99 e-mail: j.goeke(a)viamedici.de URL: www.viamedici.de<http://www.viamedici.de> HRB 362615, Amtsgericht Mannheim Geschaeftsfuehrer: Juergen Mueller

2 1

write permission and create permission
by C. M. Sperberg-McQueen 10 Mar '16

10 Mar '16

In a function intended to be called from the REST interface, I have code whose essential expression is db:add('my-db', doc('http://example.com/docs/foo.xml'), 'foo.xml') I was surprised at first that this requires CREATE permission rather than just WRITE permission, but eventually realized that the CREATE is needed not by the db:add() function but by the doc() function. It's not important for my current needs (I can run the query under a userid with CREATE permissions), but in the interests of knowledge I wonder: Is there a way to load a document from the Web that does not require CREATE permission? (And before anyone asks: yes, I could PUT the new document, if I could figure out how to do that from PHP, and if performing the necessary sanity checks were as easy in PHP as it is in XQuery. But neither of those is true at the moment; the first may change, but the second is not ever going to change: given the choice, I'd rather do any work possible in XQuery rather than PHP.) Thanks! Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

2 1

Query plans - feature request
by Hans-Juergen Rennau 09 Mar '16

09 Mar '16

Dear BaseX team, the query plans produced by xquery:parse-uri represent the state reached after resolving all imports. In particular, if module A imports module B and B contains a function F called by A, then function F is contained in the plan of module A, although the definition of function F is contained by module B, not by module A. And I think there is not way of finding out that fact. I understand that the behaviour is exactly what is required for compiling and executing the plans, but for purposes of analysis it would be great if it were possible to determine whether module A CONTAINS OR IMPORTS the definition of function F. A boolean flag (e.g. "<StaticFunc imported='true' ...") would suffice! Then, given a pool of modules, one could determine where to find what. As a hilarious fact, this feature (those boolean flags) would be very helpful in order to repair applications run with (or also with) the Saxon processor and "broken" by recent changes of Saxon: formerly, Saxon was tolerant about calling functions contained by *indirectly* imported modules (e.g. module A imports B, B imports C, A was allowed to call functions in C). This is now changed (A must import C explicitly), and repair of large-scale application would be greatly facilitated if the query plans could be evaluated as sketched. In fact, one could easily produce a report telling exactly which imports to add to which modules. So BaseX could lend a helping hand to Saxon users. Apart from that, of course, the feature would generally be great for supporting applications of code analysis. Cheers,Hans-Jürgen

2 2

Database corruption on Optimize All
by France Baril 09 Mar '16

09 Mar '16

Hi, I'm on 8.2.2, but I just tried it with BaseX 8.4 and got the same result. Here is a simple way to reproduce the error: 1. From BaseX GUI, I create a new database from a folder that contains mostly .xml, .xsd and .dtd files (attached). I use the default options from the GUI except: GENERAL - Input format XML: *.xml, *.xsd - Check 'Add other files as raw' - Uncheck 'Parse files in archives' PARSING - Check 'Use internal XML parser' - Uncheck 'Chop whitespaces' 2. Once the content is loaded, go to the 'Database properties' window for the DB and you'll see that all files are there and that you are able to browse the collections. 3. Click 'Optimize All' and try to browse the collections again. It doesn't work and a 'Stream Closed' error is generated in the terminal windows (details attached)! France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 4

$node//text()[1], why doesn't it return the 1st text node?
by France Baril 08 Mar '16

08 Mar '16

Hi, I was having issues manipulating text nodes. I found a way around my issue, but I'm still thinking that my 1st syntax to get the first text node of a sequence should have worked. My question is given all the example below, is #2 returning the expected value? I was expecting it would work like the syntax for elements in #3 and return the same result as #4 and #5 let $node := allo! Il pleut des clous. let $node2 := <ul><li>1</li><li>2</li></ul> let $text-node := $node//text() return <ul> <li>{$text-node}</li> <li>{$node//text()[1]}</li> <li>{$node2//li[1]}</li> <li>{($node//text())[1]}</li> <li>{$text-node[1]}</li> <li>{$node//text()[1][1]}</li> <li>{for $i in $node//text() return {$i}}</li> <li>{for $i in $text-node return {$i}}</li> </ul> <ul> <li>allo! Il pleut des clous.</li> <li>allo! Il pleut des clous.</li> <li> <li>1</li> </li> <li>allo!</li> <li>allo!</li> <li>allo! Il pleut des clous.</li> <li> allo! Il pleut des clous. </li> <li> allo! Il pleut des clous. </li> </ul> -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 5

Query plans without namespace bindings?
by Hans-Juergen Rennau 08 Mar '16

08 Mar '16

Dear BaseX team, the fabulous query plans delivered by xquery:parse and xquery-parse-uri reference functions with their lexical names, but do not contain the namespace bindings,e.g.: <StaticFuncCall name="i:writeNameFilter" tailCall="false"> <VarRef> <Var name="$deprecate" id="652"/> </VarRef> </StaticFuncCall> It seems to me that using this tool for the analysis of dependencies (e.g. "which modules require function f:foo?", "in which module do I find the definition of function f:bar?"), the namespace bindings would be required. And such analysis would certainly be very useful, especially when dealing with large applications and code bases. What do you think? Cheers,Hans-Jürgen

1 0

Run tests / standards compliance
by Maximilian Gärber 07 Mar '16

07 Mar '16

Hi all, do you have an overview about the tests you typically run for a release? I know there are unit tests but there is also a basex-tests repo which contains other types of tests. The readme says some tests are 'broken' so how to know which ones are good? Regards, Max

2 1

Re: [basex-talk] out of main memory
by michele.greco2＠email.it 04 Mar '16

04 Mar '16

Hi Christian, i solved and managed to increase the memory,changing basex.bat and not basexgui.bat. Thanks for your help. Regards, Michele --------- Original Message -------- Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> To: Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>, "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Oggetto: Re: [basex-talk] out of main memory Data: 03/03/16 14:38 > I Christian, i tried and this is my error: > Error:Could not create the Java Virtual Machine Did you exactly proceed as described in my last mail? How does the modified line in your script look like? > > > --------- Original Message -------- > Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> > To: > Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>, "BaseX" > <basex-talk(a)mailman.uni-konstanz.de> > Oggetto: Re: [basex-talk] out of main memory > Data: 03/03/16 11:57 > >> I tried to do what i suggested Costantine with the value 1024m, but my >> memory is alwais 495m. > > Please try the following: > > 1. Download the ZIP version of BaseX > (http://files.basex.org/releases/latest/) > > 2. Unzip it, edit bin/basexgui.bat (provided that you are using Windows): > > OLD: > set BASEX_JVM=-Xmx512m %BASEX_JVM% > > NEW: > set BASEX_JVM=-Xmx4g %BASEX_JVM% > > 3. Start bin/basexgui.bat > > 4. Click on the lower right memory bar and compare it with the > attached screenshot. > > Does this help? > Christian > > > > On Thu, Mar 3, 2016 at 12:26 PM, <michele.greco2(a)email.it> wrote: >> >> --------- Original Message -------- >> Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> >> To: >> Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>, "BaseX" >> <basex-talk(a)mailman.uni-konstanz.de> >> Oggetto: Re: [basex-talk] out of main memory >> Data: 03/03/16 11:03 >> >> >> >>> queries work. The Query on the extraction of text from files. I 495m >>> available and would like increase it to at least 1024m. >> >> So which value did you assign via -Xmx? Do you start BaseX via >> basexgui.bat? >> >> >>> Michele >>> >>> --------- Original Message -------- >>> Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> >>> To: >>> Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>, "BaseX" >>> <basex-talk(a)mailman.uni-konstanz.de> >>> Oggetto: Re: [basex-talk] out of main memory >>> Data: 03/03/16 10:42 >>> >>> >>> >>> Hi Michele, >>> >>> How much memory do you have available on your system, and how much did >>> you assign? What kind of operations are triggering the out-of-memory >>> errors? >>> >>> Christian >>> >>> >>> >>> On Thu, Mar 3, 2016 at 9:34 AM, <michele.greco2(a)email.it> wrote: >>>> Hi Constantine, thanks for your answer. >>>> I'm trying to do what you suggested but all my tests so far have had no >>>> effect. >>>> Do you have any particular which have to be careful as you suggest? >>>> >>>> Regards, >>>> Michele >>>> >>>> --------- Original Message -------- >>>> Da: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com> >>>> To: "basex-talk(a)mailman.uni-konstanz.de" >>>> <basex-talk(a)mailman.uni-konstanz.de> >>>> Oggetto: RE: [basex-talk] out of main memory >>>> Data: 01/03/16 10:07 >>>> >>>> Hi Michele, >>>> >>>> 1. Go to <BASEX_INSTALL>/bin >>>> 2. Open basexgui.bat (Windows) or basexgui (*nix) >>>> 3. Increase value of Xmx in BASEX_JVM variable, from 512 MB to >>>> something that your system can support >>>> 4. Restart the GUI >>>> >>>> For example, here is the line from my config I allocate almost 5 >>>> Gigabytes of memory >>>> >>>> set BASEX_JVM=-Xmx5000m %BASEX_JVM% >>>> >>>> Regards, >>>> Cosntantine >>>> >>>> >>>> From: basex-talk-bounces(a)mailman.uni-konstanz.de >>>> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of >>>> michele.greco2(a)email.it >>>> Sent: 01 March 2016 10:48 >>>> To: basex-talk(a)mailman.uni-konstanz.de >>>> Subject: [basex-talk] out of main memory >>>> >>>> Hi, i get the error "out of main memory", when i run my query on a >>>> collection in basex gui. >>>> How can i fix? >>>> How can i increase my memory? >>>> Regards >>>> Michele >>>> >>>> ---- >>>> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >>>> tutti i dettagli clicca qui >>>> >>>> Sponsor: >>>> Idee regalo classiche o alternative? Trova l'offerta migliore in un >>>> click >>>> Clicca qui >>>> >>>> ________________________________ >>>> Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The >>>> Netherlands, Registration No. 33156677, Registered in The Netherlands. >>>> >>>> >>>> ---- >>>> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >>>> tutti i dettagli clicca qui >>>> >>>> Sponsor: >>>> Idee regalo classiche o alternative? Trova l'offerta migliore in un >>>> click >>>> Clicca qui >>> >>> >>> ---- >>> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >>> tutti i dettagli clicca qui >>> >>> Sponsor: >>> Caselle con tuo dominio su piattaforma Zimbra, fino a 30 GB di spazio, >>> sincronizzazione dati e backup >>> Clicca qui >> >> >> ---- >> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >> tutti i dettagli clicca qui >> >> Sponsor: >> Soluzioni di email hosting per tutte le esigenze: dalle caselle gratuite a >> quelle professionali su piattaforma Zimbra, da quelle su proprio dominio a >> quelle certificate PEC. Confronta le soluzioni >> Clicca qui > > > ---- > ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per > tutti i dettagli clicca qui > > Sponsor: > Idee regalo classiche o alternative? Trova l'offerta migliore in un click > Clicca qui -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Soluzioni di email hosting per tutte le esigenze: dalle caselle gratuite a quelle professionali su piattaforma Zimbra, da quelle su proprio dominio a quelle certificate PEC. Confronta le soluzioni Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13326&d=4-3

1 0

Export database to SQL?
by Bhander, Gurbakhash S. 03 Mar '16

03 Mar '16

Hi, Is there any way to export data from the database to other databases? Thanks -Raj From: basex-talk-bounces(a)mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Klavs Prieditis Sent: Wednesday, March 02, 2016 11:21 AM To: Christian Grün <christian.gruen(a)gmail.com>; Dirk Kirsten <dk(a)basex.org> Cc: BaseX <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] Parallel execution Hi, Thanks Dirk and Christian! My apologies for using such a strong word to describe such a minor issue. It is not my intention to underestimate your work. My main concern is that `async:fork-join()` is too heavy. I mean, in essence, I am solving the same problem in these two examples. ```xquery for $i in 1 to 10 return fetch:text("http://example.com/magic-story/for-number/" || $i) ``` ```xquery async:fork-join( for $i in 1 to 10 return function() {fetch:text("http://example.com/magic-story/for-number/" || $i)} ) ``` If you solve the same problem in Java 8 using Streams API you get the following two solutions. ```java rangeFrom1To10AsList.stream().map(fetchText).collect(...); ``` ```java rangeFrom1To10AsList.parallelStream().map(fetchText).collect(...); ``` Considering these two Java 8 solutions, I guess what I actually want is the `async` keyword. So that the second Java example could be rewritten as follows. ```xquery for async $i in 1 to 10 return fetch:text("http://example.com/magic-story/for-number/" || $i) ``` Problem here is that you need to introduce a new keyword to the language. I am just trying to figure out if there could be a solution like this. ```xquery for $i in 1 to 10 return fetch:text-with-implicit-parallel-execution-characteristics("http://example.com/magic-story/for-number/" || $i) ``` Thanks for your input! Currently I lack in-depth knowledge on XQuery, I'll try to get a spare time to study the XQuery specification so that I am able reason about the various consequences. 2016-03-02 11:36 GMT+02:00 Christian Grün <christian.gruen(a)gmail.com<mailto:christian.gruen@gmail.com>>: Hi Klāvs, Dirk already gave some good reasons why implicit parallelization is not the default in BaseX, although it may seems to be the most natural and desirable choice. Indeed it’s fairly easy to create FLWOR expressions which are slower when executed in parallel. The XQuery specification, by the way, does not say anything about parallelization, so it’s completely up to the implementation to choose the most promising evaluation strategy. You asked if it was not possible to introduce sth. like a `can-be-optimized-for-parallel-execution` flag, and you may be interested to hear that we had some first thoughts on an 'async' keyword, which could be used in FLWOR expressions [1]. In the GitHub issue, only the most basic cases have been considered so far, but if we manage to push this further, we would have a syntax that will be more easily accessible to most users. Ideally, some time in future, we could then apply this pattern automatically if it will be absolutely clear that parallelization leads to faster results. Everyone: If you notice that the async:fork-join function makes your code faster, don’t hesitate and send us your queries. Hope this helps, Christian [1] https://github.com/james-jw/xq-promise/issues/15 On Wed, Mar 2, 2016 at 10:05 AM, Dirk Kirsten <dk(a)basex.org<mailto:dk@basex.org>> wrote: > Labrīt Klāvs, > > well, I would the word "broken" is a strong word for something, which > still works correctly and returns the correct result. It might not be > perform optimal, but broken is something different. Also, given that the > feature is still not even one day old, you might have to be a bit more > patient with us in applying optimization techniques. > > The problem here has several dimensions: > - XQuery returns results in order. So in order to not break BaseX we > would have to cache the results of the fetch and return it in the > correct order > - This caching might be much more complex, if it is a more complicated > expression > - It might not always be desirable to turn on parallelisation. In this > case you call an external HTTP server 10 times, in this case there might > be a speedup. But using fetch:text() could also point to a local URI and > to parallelize I/O is not always beneficial. > > So given the many different use cases here I don't think an implicit > parallelization will work here to the satisfaction of everyone. > Also, I invite you to read this closely related question about Haskell > on SO and the linked research: > https://stackoverflow.com/questions/15005670/why-is-there-no-implicit-paral… > > Cheers > Dirk > > On 03/02/2016 01:41 AM, Klāvs Priedītis wrote: >> Hello, BaseX team and community! >> >> Disclosure: I am not yet an expert on these issues and I have not >> fully studied the specs of XQuery. >> >> Now that Async Module is incorporated, I am wondering about some basic >> properties of xquery. >> >> Example: >> ```xquery >> for $i in 1 to 10 >> return fetch:text("http://example.com/magic-story/for-number/" || $i) >> ``` >> >> It feels to me that the xquery processor is broken if it cannot figure >> out on its own that the fetch can be done in parallel. >> Why does this not work out of the box? Can it work out of the box? >> Wouldn't it be able to do this in parallel if a flag >> `can-be-optimized-for-parallel-execution` was defined for the >> `fetch:text`? >> >> -- >> Veiksmi vēlot, >> Klāvs Priedītis > > -- > Dirk Kirsten, BaseX GmbH, http://basexgmbh.de > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22 > -- Veiksmi vēlot, Klāvs Priedītis

3 6