BaseX-Talk December 2017

basex-talk@mailman.uni-konstanz.de

17 participants
21 discussions

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

oXygen XML, content negotiation and #1220
by Omar Siam 20 Feb '18

20 Feb '18

Hi list! I like my oXygen editor to request raw XML data from my RESTxq endpoints to test XSL stylesheets. When I do this with RESTxq paths that have different %rest:produces it fails with the BASX0003 multiple functions error. Turns out oXygen sends application/xml, */* as Accept header without any quality. I understand that as documented in https://github.com/BaseXdb/basex/issues/1220 this is known not to work. My/our workaround is to branch in a single XQuery function using sth. like: if (some $a in tokenize(request:header("ACCEPT"), ',') satisfies $a = ('text/xml', 'application/xml')) then $xml else $xhtml I hope this is still on the agenda for 8.7. Best regards Omar Siam

3 7

Preparing BaseX 8.5.4 (please don't forget Issue #1341)
by Jan Dölle 20 Feb '18

20 Feb '18

Hi, we noticed you prepare release 8.5.4. Please don't forget to fix issue: SQL with BaseX 8.5.3 #1341 Thx Jan -- Jan Dölle E-Mail: basex-list(a)id1.de Telefon +49-69-244502-0 Home: www.id1.de Information Design One AG, Baseler Straße 10, 60329 Frankfurt am Main Registereintrag: Amtsgericht Frankfurt am Main, HRB 52596 Vorstand: Robert Peters, Aufsichtsrat: Christian Hecht (Vorsitz)

4 5

java.lang.NegativeArraySizeException
by Stefanie Nageldiek 15 Jan '18

15 Jan '18

Hi! I'm trying to move a XSLT Transformation into XQuery. First step is Creating a db and trying to run the XSLT with XQuery. I have a small input file (50 MB) and a large input file (2GB) for testing. So I set up the db with the small input file, ran a parametrizised query calling the XSLT Transformation and everything went well. To be sure everything will work on the bigger db too, I tried to do the same, which fails - I had to allow for more memory which I did. (To be sure the stylesheet works with the big input file for the db I checked it (it takes 20 Minutes to produce a ~1 GB File). This is the error I get: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.6.7 Java: Oracle Corporation, 1.8.0_151 OS: Windows 10, amd64 Stack Trace: java.lang.NegativeArraySizeException at java.util.Arrays.copyOf(Unknown Source) at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:25) at org.basex.io.out.PrintOutput.print(PrintOutput.java:76) at org.basex.io.out.NewlineOutput.print(NewlineOutput.java:33) at org.basex.io.serial.MarkupSerializer.printChar(MarkupSerializer.java:254) at org.basex.io.serial.MarkupSerializer.attribute(MarkupSerializer.java:131) at org.basex.io.serial.Serializer.node(Serializer.java:416) at org.basex.io.serial.Serializer.node(Serializer.java:158) at org.basex.io.serial.StandardSerializer.node(StandardSerializer.java:105) at org.basex.io.serial.AdaptiveSerializer.node(AdaptiveSerializer.java:75) at org.basex.io.serial.Serializer.serialize(Serializer.java:109) at org.basex.io.serial.AdaptiveSerializer.serialize(AdaptiveSerializer.java:66) at org.basex.query.value.Value.serialize(Value.java:222) at org.basex.query.value.Value.serialize(Value.java:205) at org.basex.query.func.xslt.XsltTransform.read(XsltTransform.java:75) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:47) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:33) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:65) at org.basex.query.QueryContext.value(QueryContext.java:406) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:185) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:93) at org.basex.query.scope.MainModule$1.next(MainModule.java:125) at org.basex.query.QueryContext.cache(QueryContext.java:618) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:112) at org.basex.core.cmd.AQuery.query(AQuery.java:86) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.exec(GUI.java:474) at org.basex.gui.GUI.access$4(GUI.java:428) at org.basex.gui.GUI$6.run(GUI.java:416) Compiling: - pre-evaluate root() to document-node() - pre-evaluate map { "param1":"default", "param2":"products" } to map - pre-evaluate map { "method":"xml", "indent":"no", "omit-xml-declaration":"no" } to map - inline $path_2 - pre-evaluate concat("c:/work/xslt_to_xquery/", "xslt_post_processing_bmwp-2526.xsl") to xs:string - inline $style_3 Optimized Query: let $document_4 := xslt:transform(db:open-pre("big-xml",0)/*, "c:/work/xslt_to_xquery/i-know-which.xsl", map { "param1": "default", "param2": "products", ... }) return file:write(concat("c:/work/xslt_to_xquery/", db:name(.), "_out_", "default", "products", ".xml"), $document_4, map { "omit-xml-declaration": "no", "method": "xml", ... }) Query: declare variable $param1 external := "default"; declare variable $param2 external := "products"; let $path := 'c:/work/xslt_to_xquery/' let $style := concat($path, 'i-know-which.xsl') let $document := xslt:transform(/*, $style, map { "param1": $param1, "param2": $param2 }) return file:write(concat($path, db:name(.), '_out_', $param1, $param2, '.xml'), $document, map { "method": "xml", "indent": "no", "omit-xml-declaration":"no" }) Query plan: <QueryPlan compiled="true"> <GFLWOR> <Let> <Var name="$document" id="4"/> <XsltTransform name="transform(input,stylesheet[,params[,options]])"> <IterPath> <DBNode name="big-xml" pre="0"/> <IterStep axis="child" test="*"/> </IterPath> <Str value="c:/work/xslt_to_xquery/i-know-which.xsl" type="xs:string"/> <Map size="2"> <Str value="param1" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="param2" type="xs:string"/> <Str value="products" type="xs:string"/> </Map> </XsltTransform> </Let> <FileWrite name="write(path,data[,params])"> <FnConcat name="concat(atom1,atom2[,...])"> <Str value="c:/work/xslt_to_xquery/" type="xs:string"/> <DbName name="name(node)"> <ContextValue/> </DbName> <Str value="_out_" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="products" type="xs:string"/> <Str value=".xml" type="xs:string"/> </FnConcat> <VarRef> <Var name="$document" id="4"/> </VarRef> <Map size="3"> <Str value="omit-xml-declaration" type="xs:string"/> <Str value="no" type="xs:string"/> <Str value="method" type="xs:string"/> <Str value="xml" type="xs:string"/> <Str value="indent" type="xs:string"/> <Str value="no" type="xs:string"/> </Map> </FileWrite> </GFLWOR> </QueryPlan> Please let me know which information you need. Many thanks Steffie

2 1

Isilex on GitHub
by Xavier-Laurent SALVADOR 02 Jan '18

02 Jan '18

Hi List, just a small message to draw your attention on our new Git Repo dedicated to Isilex. We put all our stuff into : https://github.com/Isilex/download and all the web documentation at: https://isilex.github.io/easy-xml/ <https://isilex.github.io/easy-xml/> and https://isilex.github.io/easy-xml-fr/ Cheers, -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*.

2 1

Basex API: repeated execution of same query over different input
by Hondros, Constantine (ELS-AMS) 27 Dec '17

27 Dec '17

I want to be able to execute the same XQuery hundreds of thousands (even millions) of times over different XML input. Is there a way to avoid re-compiling the query itself every time? It looks to me like using either XQuery.execute(ctx) or the QueryProcessor class will end up re-parsing the query itself. I know that Basex is fast in this, but I will end up with a performance hit over time. Just wondering whether things have evolved since this thread: https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg03064.ht… Thanks, Constantine Hondros Senior Manager Content Analytics & Content Engineering Digital Transformation Elsevier BV Radarweg 29 | 1043 NX Amsterdam | the Netherlands Tel: +31 20 4852772 | Mob: +31610735630 E-mail: c.hondros(a)elsevier.com<mailto:m.nauta@elsevier.com> [cid:image001.jpg@01CBEEC6.C17A8AC0] Amsterdam | Chennai | Exeter | Oxford | Shannon | St. Louis ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Properly sending/receiving JSON?
by Robert Crews 24 Dec '17

24 Dec '17

First off, thank you for the fast answer to my previous question. Now, I'm having trouble understanding how to send JSON to BaseX. Ultimately, I want to send JSON and store it (allowing BaseX to convert it to XML first, of course). For now, however, I am having trouble just receiving the JSON data. Here's my test XQM file: module namespace hwxdocs = "urn:com:hortonworks:docs"; declare %rest:POST("{$body}") %rest:path("hello/json") %rest:consumes("application/json") %output:media-type("application/json") function hwxdocs:hello2( $body ) { "Request body: " || $body }; I send data with this command: curl -iL -X POST \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --data-binary '{"foo":{"bar":"baz"}}' \ http://localhost:8984/hello/json I expect to get Request body: {"foo":{"bar":"baz"}} (I’m assuming I can pass '{"foo":{"bar":"baz”}}’ to json:parse to convert to a BaseX XML that can be stored in the database.) However, instead, I get Request body: baz Can someone tell me what my error is? Thank you, Robert

2 1

Real time DB population using Nifi
by Juan Pablo Gardella 22 Dec '17

22 Dec '17

Hello team, First of all, congratulations for the awesome tool! BaseX is amazing. We are starting using it for querying some XML data that we are receiving from a queue. I've developed a Nifi processor based on https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/o… class. The problem that I'm facing is that write throughput is very slow. Do you know any way to increase the write throughput? All the XML files are stored in the same DB. So basically I'm doing: 1) open db 2) replace (xml id path, inputstream) Thanks, Juan

2 2

Handling exceptions in the Python Client API
by Jonathan Robie 21 Dec '17

21 Dec '17

I am now happily using BaseX from Jupyter Notebooks for Greek data - see these links: http://jonathanrobie.biblicalhumanities.org/blog/2017/12/08/jupyter-tutoria… http://jonathanrobie.biblicalhumanities.org/assets/greeksyntax-tutorial-pro… But I need to know how to handle exceptions, e.g. if the notebook user writes a query with an error in it. And I would also like to return some result if the query returns an empty sequence. Here is the current code: https://github.com/biblicalhumanities/greek-new-testament/blob/master/labno… How can I improve what I am doing? Jonathan

2 4

Setting a default collation for a database
by Jonathan Robie 19 Dec '17

19 Dec '17

I would like to set a default collation for a database, a collation that will be used for all queries unless the user asks for a different collation. My use case: a database in Greek will never be sensibly sorted using an English collation, and I don't want users to need to specify collations over and over. Is this possible? Jonathan

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk December 2017