BaseX-Talk August 2019

basex-talk@mailman.uni-konstanz.de

28 participants
28 discussions

Reg : ArrayIndexOutOfBoundsException
by chandra Sekhar 09 Aug '19

09 Aug '19

Dear basex team, We are getting the below error while accessing the collection. Exception ::org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.7.0_151 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException Collection size : 37 GB And did INSPECT command on the collection,It is showing success. Regards, Chandrasekhar

2 1

Serialization mismatch using Saxon for xslt:transform
by Majewski, Steven Dennis (sdm7g) 08 Aug '19

08 Aug '19

Continuing with my test.xqm experiments… I have copied Saxon jar into WEB-INF/lib/ so xslt:transform is using Saxon. I have the following functions declared: declare %rest:path("test/xqdoc.xml") %output:method('xml') function test:doc() { inspect:xqdoc( $test:module_name ) }; declare %rest:path('test/xdoc') %output:method('html’) (: I’ve tried various values for method and html-version here ! :) %output:html-version('5.0') function test:htmldoc() { (: WTF? :) xslt:transform( test:doc(),'static/xsl/html-module.xsl', map { 'source' : 'test.xqm' }) }; Where html-module.xsl initially was: https://github.com/xquery/xquerydoc/blob/master/src/lib/html-module.xsl <https://github.com/xquery/xquerydoc/blob/master/src/lib/html-module.xsl> Accessing /basex/test/xdoc returns error message: Stopped at /usr/local/tomcat/webapps/basex/test.xqm, 41/20: [FODC0002] "" (Line 69): The entity "nbsp" was referenced, but not declared. Which was initially very puzzling, as I could not find nbsp in either my test.xqm, html-module.xsl, or inspect:xsdoc() output. Turns out, even though it was encoded as " ” in html-module.xsl, Saxon was encoding it as “ ” on output of the transform. Initially, I wasn’t sure if Saxon or BaseX was doing the serialization. I guess it seems to be both. Original output method for that stylesheet is: <xsl:output method="html" indent="yes" encoding="UTF-8"/> And changing output:method on the function to xhtml or html doesn’t have any effect on results. Changing output method on the stylesheet to xhtml causes Saxon to serialized it as “ ” instead of “ ” , and the serialization method on the basex RESTXQ function doesn’t seem to give an errors with any output methods. I take from these results that the output from Saxon xslt:transform is serialized according to the stylesheet, and then parsed again by BaseX as xml on the way to being serialized again on output from the function, and the error is coming from that implicit parse. I don’t suppose there is a way to short circuit the parsing of the xslt:transform function output and just output the results directly (?) I tried xslt:transform-text() but it escapes all of the element tags. Or a way to get it to parse the results of that transform function as html ? Clearly changing the stylesheet is the easy solution for this test file, but for some of my other real cases, I may be trying to repurpose stylesheets that will be used in different contexts: within BaseX and outside of that context, so keeping different versions of the stylesheets is going to be annoying. I suppose I could read in the stylesheets and modify the xml:output line in XQuery or XSLT before using it within BaseX. — Steve M.

3 2

Error from function-name(request:query#0): only from RESTXQ, not REST context.
by Majewski, Steven Dennis (sdm7g) 08 Aug '19

08 Aug '19

I’ve been learning how to use the BaseX webapp, and below is one of the first test scripts for the REST interface I wrote. No problems with this script. import module namespace request = "http://exquery.org/ns/request"; declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:omit-xml-declaration "no"; declare option output:method "xhtml"; (: http://docs.basex.org/wiki/XQuery_3.0#Serialization :) (: declare variable $contains := try { request:parameter( 'contains' ) } catch basex:http { '' }; Simpler to use implicit assignment: http://docs.basex.org/wiki/REST#Assigning_Variables "All query parameters that have not been processed before will be treated as variable assignments" :) declare variable $contains as xs:string external := ""; declare function local:f0line( $f ) as node() { (: build a list item with function name and function values :)I <li><b>{fn:function-name($f)}:</b> {$f()}</li> }; declare variable $CTX := try { . } catch err:XPDY0002 { () }; declare variable $DB := try { db:name( $CTX[1] ) } catch err:XPTY0004 { "" }; (: Could be, if (): [XPTY0004] node() expected, empty sequence found. but not sure if that's the only possible case. :) <html><head><title>test</title></head><body> <div id="context"> <h3>Database: { $DB }</h3> <h3>Context documents (.)</h3> <ol> { for $doc in $CTX return <li><b>{base-uri($doc)}</b></li> } </ol> </div> <div id="request"> <h3>HTTP Request</h3> <ul> { for $f in ( request:method#0, request:scheme#0, request:hostname#0, request:port#0, request:path#0, request:query#0, request:uri#0, request:context-path#0, request:address#0, request:remote-hostname#0, request:remote-address#0, request:remote-port#0 ) return local:f0line($f) } </ul> </div> <div id="params"> <h3>Request Parameters:</h3> <ul> { for $param in request:parameter-names() return <li><b>{$param}:</b> {request:parameter($param)}</li> } </ul> </div> <div id="headers"> <h3>Headers</h3> <ul> { for $header in request:header-names() return <li><b>{$header}:</b> {request:header($header)}</li> } </ul> </div> <div id="cookies" > <h3>Cookies</h3> <ul> { for $cookie in request:cookie-names() return <li><b>{$cookie}:</b> {request:cookie($cookie)}</li> } </ul> </div> <div id="xslt"> <h4>XSLT</h4> <ul> <li><b>xslt:processor: { xslt:processor()},   xslt:version: { xslt:version()}</b></li> </ul> </div> </body></html> I’m now trying to learn RESTXQ programming, so I was trying to do the same sort of thing from a RESTXQ module. I was getting an error from the line where (: request:query#0, :) is now commented out, and added the line: <li>{ request:query( ), function-name(request:query#0)}</li> And I see that what happens is that it’s the latter function-name(request:query#0) that is generating an error, but only when there is no query string in my request. An empty query string, i.e. “http://localhost/basex/test? <http://localhost/basex/test?>” , for example, works. But “http://localhost/basex/test <http://localhost/basex/test> returns: Stopped at /usr/local/tomcat/webapps/basex/test.xqm, 33/66: [XPTY0004] Cannot convert empty-sequence() to xs:string: (). I assume this has something to do with how RESTXQ processes form and request parameters, and I probably don’t actually need to do it this way, but I was wondering what exactly is the source of the problem with doing something like this or if there is an obvious work around. Maybe writing another local function to wrap around request:query and to use in that loop. ( One reason I chose that test to convert to RESTXQ was that it wasn’t obvious to be from the docs if I could mix those different access methods, and so what was surprising was that it mostly worked — only that one function failed (so far)). module namespace test = 'http://localhost/test'; import module namespace request = "http://exquery.org/ns/request"; declare function test:f0line( $f ) as node() { (: build a list item with function name and function values :) <li><b>{fn:function-name($f)}:</b> {$f()}</li> }; (:~ : Generates a welcome page. : @return HTML page :) declare %rest:path("test") %output:method("xhtml") %output:omit-xml-declaration("no") %output:doctype-public("-//W3C//DTD XHTML 1.0 Transitional//EN") %output:doctype-system("http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd") function test:test( ) as element(Q{http://www.w3.org/1999/xhtml}html) { <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>BaseX HTTP Services</title> <link rel="stylesheet" type="text/css" href="static/style.css"/> </head> <body> <div class="right"><a href='/'><img src="static/basex.svg"/></a></div> <h1>TEST</h1> <div id="debug" > <ul> <li>{ request:query( ), function-name(request:query#0)}</li> </ul> </div> <div id="request"> <h3>HTTP Request</h3> <ul> { for $f in ( request:method#0, request:scheme#0, request:hostname#0, request:port#0, request:path#0, (: request:query#0, :) request:uri#0, request:context-path#0, request:address#0, request:remote-hostname#0, request:remote-address#0, request:remote-port#0 ) return test:f0line($f) } </ul> </div> </body> </html> };

2 1

comment()
by Marc 08 Aug '19

08 Aug '19

Hi, I want to write a comment in the xml I produce with my XQuery. I try to use the comment() function, but each time I want to put something inside the () i have a syntax error. for example : <test> {comment("test"), let $r := random:integer(100) + 1 for $j in (/*)[$r] return ( $r,$j) } </test> or (who is more what I want to do) <test> { let $r := random:integer(100) + 1 for $j in (/*)[$r] return (comment("$r"),$j) } </test> I don't find any example on how to use the comment function. Anyone have example about that use? Thanks Marc

3 4

Coding help
by Greg Kawchuk 08 Aug '19

08 Aug '19

Hi everyone, I'm wondering if someone could provide what I think is a brief script for a scientific project to do the following. The goal is to identify XML documents from a very large collection that would be too big to load into a database all at once. Here is how I see the functions provided by the code. 1. In the script, the user could enter the path of the target folder (with millions of XML documents). 2. In the script, the user would enter the number of documents to load into a database at a given time (i =. 1,000) depending on memory limitations. 3. The code would then create a temporary database from the first (i) xml files in the target folder. 4. The code would then search the 1000 xml documents in the database for a pre-defined text string. 5. If hits exist for the text string, the code would write those documents to a unique XML file. 6. Clear the database. 7. Read in the next 1000 files (or remaining files in the folder). 8. Return to #4. There would be no need to append XML files in step 5. The resulting XML files could be concatenated afterwards. Thank you in advance. If you have any questions, please feel free to email me here. Greg *************************************************** Greg Kawchuk BSC, DC, MSc, PhD. Professor, Faculty of Rehabilitation Medicine University of Alberta greg.kawchuk(a)ualberta.ca 780-492-6891

5 4

BaseX in AWS Lambda - has anyone tried it?
by James Ball 07 Aug '19

07 Aug '19

Good afternoon, I’m currently working on a project where I get lots - maybe thousands - of XML files and I have to run a series of tests on them. I get a whole batch then maybe none for a week. I have all my code in XQuery in BaseX but with some locking and update issues it can take a long time to process. So I thought I might give AWS Lambda a try as all the tests can run independently. Before I jump in I wondered if anyone else has tried it? Any thoughts, suggestions, issues? I did a search of the list but didn’t see anything. Many thanks, James

1 0

Stopping BaseX HTTP server
by Markus Wittenberg 02 Aug '19

02 Aug '19

Hi all, we have a problem trying to stop a BaseX HTTP server with the Windows task scheduler. The server ist started with a maven profile, calling mvn exec:java -Pstart pom.xml extract see below, and can also be stopped directly with the corresponding stop profile by a batch file: cd /D "%~dp0" mvn exec:java -Pstop But calling this batch profile in task scheduler works fine on our local computers, but does not work on the server (Windows Server 2012). On this machine, executing the stop profile by the task scheduler results in a message HTTP server was started (port: 8985). in the console window of the process to be stopped, but it continues running anyway. Any ideas regarding this strange behavior? Can we stop the server with a simple HTTP request? Calling localhost:8985 in the browser also results only in the message above. Thanks in advance for any hints. Best regards, Markus Extract from pom.xml: <profile> <id>start</id> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.4.0</version> <configuration> <mainClass>org.basex.BaseXHTTP</mainClass> <arguments> <argument>-h9894</argument> <argument>-s8985</argument> <argument>-p1985</argument> <argument>-Urestxq</argument> </arguments> <additionalClasspathElements> <additionalClasspathElement>src/main/resources</additionalClasspathElement> </additionalClasspathElements> </configuration> </plugin> </plugins> </build> </profile> <profile> <id>stop</id> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.4.0</version> <configuration> <mainClass>org.basex.BaseXHTTP</mainClass> <arguments> <argument>-s8985</argument> <argument>stop</argument> </arguments> </configuration> </plugin> </plugins> </build> </profile> <profile> -- Markus Wittenberg Tel +49 (0)341 248 475 36 Mail wittenberg(a)axxepta.de ---- axxepta solutions GmbH Lehmgrubenweg 17, 88131 Lindau Amtsgericht Berlin HRB 97544B Geschäftsführer: Karsten Becke, Maximilian Gärber

3 12

Forcing download of JSON doc
by Tim Thompson 01 Aug '19

01 Aug '19

Hello, I am posting a JSON doc to a RESTXQ function from XForms and trying to set a Content-Disposition header to trigger a file download. Is this possible to do without first saving the JSON doc as a file? Right now, the RESTXQ function returns the correct headers, but does not force a file download dialog. Thanks in advance! Tim -- Query: xquery version "3.1"; module namespace wrap = "http://example.org/wrap"; declare %rest:path("/wrap") %rest:POST("{$data}") %output:method("json") function wrap:process( $data as node() ) as item()* { let $name := data($data/data/upload/file/@name) let $json := json-to-xml($data/data/upload/file) let $header := ``[attachment; filename="`{$name}`";]`` return ( <rest:response> <http:response status="200" message="OK"> <http:header name="Content-Type" value="application/json"/> <http:header name="Content-Disposition" value="{$header}"/> </http:response> </rest:response>, <map xmlns="http://www.w3.org/2005/xpath-functions"> <map key="Profile"> <array key="resourceTemplates">{$json}</array> <string key="title">Wrapper_Profile</string> <string key="id">Wrapper:Profile:BF2</string> <string key="description">Profile wrapper for resource templates</string> <string key="author">Wrapper</string> <string key="schema"> https://ld4p.github.io/sinopia/schemas/0.0.9/profile.json</string> <string key="date">{current-dateTime()}</string> </map> </map> => xml-to-json() => parse-json() => serialize(map {'method': 'json', 'use-character-maps' : map {'/' : '/'}}) ) }; -- Tim A. Thompson Discovery Metadata Librarian Yale University Library

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk August 2019