- BaseX-Talk - mailman.uni-konstanz.de

Automatically generating test cases. Hidden feature?
by Marc van Grootel 13 Jan '16

13 Jan '16

Hi, Been a long time since posting. Work is very busy recently. Luckily for me a lot of it involves XQuery ;-) I am elaborating some examples [1] that show off the use of my Origami [2] library. The most promising ones I will turn in to blog posts or a wiki at some point. Recently I've been reading a bit about generative testing (or property-based testing) and libraries like Haskell's QuickCheck. I hope to be able to show that we can use such techniques in combination with RelaxNG and the concepts that Eric van der Vlist describes in Examplotron [3] to mutate XML instances guided by a schema that I can then turn into automatically generated tests. It's still early days and time is limited but I did stumble onto something interesting that I wanted to share here. If you already use similar techniques I would be very interested to hear about it. I was contemplating if it was possible to create anonymous unit test functions that I can invoke with generated test input. So, a bit to my amazement, the following works: let $equals := %unit:test function($a,$b) { unit:assert-equals($a,$b, 'Not equal') } return $equals(4,3) This function raises the unit:UNIT0001 exception with the message "Item 1: 3 expected, 4 returned". Cool! So if this is possible can we also catch a failure and turn it into data? let $equals := %unit:test function($a,$b) { unit:assert-equals($a,$b, 'Not equal') } return try { $equals(4,3) } catch * { <error> <code>{ $err:code }</code> <description>{ $err:description }</description> <value>{ $err:value }</value> <module>{ $err:module }</module> <line-number>{ $err:line-number }</line-number> <column-number>{ $err:column-number }</column-number> </error> } This too works and produces an XML fragment recording the failure. Very cool! <error> <code>unit:UNIT0001</code> <description>Item 1: 3 expected, 4 returned.</description> <value>Not equal</value> <module>.../origami-examples/check/demo.xq</module> <line-number>23</line-number> <column-number>24</column-number> </error> I am not sure if this a hidden or an intentional feature of the BaseX unit module. I sure hope that it doesn't go away in future releases as I intend to leverage this in an alternative testing module. BaseX, the gift that keeps on giving, thanks to all that make it possible. --Marc [1] https://github.com/xokomola/origami-examples [2] https://github.com/xokomola/origami [3] http://examplotron.org/

3 3

minor issue in the project-view of BaseX-GUI with error marking
by Rob Stapper 13 Jan '16

13 Jan '16

Hi, I found that when a directory contains both a program- and a module-file with the same name but with different extensions ( .xq and .xqm) both file are marked as erroneous in the project-view if only one file, on my case the module-file, is indeed erroneous and the other file is ok. Regards, Rob Stapper PS. It is my unittest-directory that contains both program- and module-files.

2 4

XML embedded in JSON
by E. Wray Johnson 12 Jan '16

12 Jan '16

We have JSON that has string/text values which are XML. Is there a way o have the XML as sub-elements value and not be encoded/escaped string? Wray Johnson

2 1

XQUF 3.0, MIXUPDATES
by Hans-Juergen Rennau 12 Jan '16

12 Jan '16

Dear BaseX team, version 3.0 of the XQuery Update Facility (XQUF) allows expressions to perform an updating operation *and* return a value (e.g. [1]). Presently, BaseX supports probably only version 1.0 of the XQUF, so the mixing of update and value creates an error - and option MIXUPDATES equal "true" does not change this.: Query:=============================== declare option db:mixupdates 'true'; 100, insert node <elem/> into * ================================>Stopped at C:/projects/act/updateQuery.xq, 5/1: [XUST0001] List expression: all expressions must be updating or return an empty sequence. In my opinion, the possibility to mix updating and value construction is very important, as otherwise it is impossible to develop versatile tools which may, dependent on call parameters, either return a result or perform an update. So my Q U E S T I O N is - are there already plans to support XQUF 3.0 - or as a first step the mixing of update and value - soon? Would be phantastic! Cheers,Hans-Jürgen [1] http://www.w3.org/TR/xquery-update-30/#id-comma-expr"The operand expressions of the comma expression are evaluated,and the XDMinstances and pending update lists returned areconcatenated in operand order. Pending update lists are concatenatedby the upd:mergeUpdates operation."

2 2

XML Commands
by E. Wray Johnson 12 Jan '16

12 Jan '16

How do you execute XML commands from the BaseX GUI? Why can't multiple XML commands be executed from the BaseXClient.java? How can database options (e.g. autooptimize) be changed from the GUI without opening the database and locking it?

2 1

Adding basex-talk to markmail
by Joe Wicentowski 12 Jan '16

12 Jan '16

Hi all, I wasn't able to find basex-talk in markmail.org - a fantastic site for searching mailing list archives. Would the list owners/administrators consider submitting this list to markmail? Information about this can be found at http://markmail.org/docs/feedback.xqy. Best, Joe

2 1

Fulltext Stemming
by kleist 12 Jan '16

12 Jan '16

Hi all, with query options: contains text "grün" all using diacritics sensitive using stemming using language 'German' I'll get: "grüne", "grüner", "Grüns" etc. But I also get "Gruß", "Grüße", "grüßen" or something like "Gruner" (Eigenname) Does the stemming have problems with german umlaute and "ß"? with query options: contains text "grün" all using stemming using language 'German' same results, but less. Why is that? The indexes of the database are the following: Indexes Up-to-date: true TEXTINDEX: true ATTRINDEX: true FTINDEX: true TEXTINCLUDE: ATTRINCLUDE: FTINCLUDE: LANGUAGE: German STEMMING: true CASESENS: true DIACRITICS: true STOPWORDS: UPDINDEX: false AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 INDEXSPLITSIZE: 0 FTINDEXSPLITSIZE: 0 Optimized Query is the following: Optimized Query: let $hits_137 := ft:mark((db:open-pre("kleist-searchindex",147544), ...)/(((descendant::tei:s union descendant::tei:l)))[descendant::text() contains text "grün" all using diacritics sensitive using stemming using language 'German']) return element div { (element p { ("grün") }, element p { (count($hits_137)) }, element ul { (for $hit_139 in $hits_137 return element li { ($hit_139/descendant-or-self::node()/ancestor::tei:TEI/descendant-or-self::node()/tei:classCode[position() = 1]/string(), $hit_139) }) }) } Best, Günter

2 1

group-by behaviour for clustering XML fragments
by Hondros, Constantine (ELS-AMS) 11 Jan '16

11 Jan '16

Hello all, I'm using BaseX to cluster a set of millions of small XML fragments which look something like this: <affiliation> <organization>Institut für Organische Chemie der Universität Heidelberg</organization> <country iso-code="DEU"/> </affiliation> I need to cluster based on fragment similarity - so taking into account elements, attributes and text nodes. If I use the entire XML fragment as a grouping key, something like this: for $a at $c in db:open('DB')/item/*/affiliation group by $val := $a ... then will the grouping be equivalent to the functionality of the deep-equal function? First results seem to suggest this, but I want to make sure that grouping is not done on text node value alone or anything like that. Incidentally, BaseX is simply unbelievably fast at executing this - a million fragments clustered and written out to another DB in 16 seconds on a laptop. My congratulations on an amazing product. Regards, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

3 3

Strange behaviour on db:add inside restxq
by Marco Lettere 11 Jan '16

11 Jan '16

Hi all, We've run into a strange issue that we never experienced beforehand even if it's something we should have done thousands of time. Consider storing a resource sent through a RestXQ POST directly to a database. The restxq module module namespace o = "test"; (:sendMessage: Send a new message:) declare %rest:path('test/savemsg') %rest:POST('{$body}') %rest:consumes("application/xml") function o:save-message($body as document-node()) { db:add('hereiam', $body, 'saved.xml') }; The client request: http:send-request( <http:request method="POST" href="http://localhost:8984/test/savemsg"> <http:body media-type="application/xml"> <a> </a> </http:body> </http:request> ) The resource gets stored into the DB with all the extra text nodes containing white spaces (codepoints 10 and 32) something like: <a> </a> That's pretty strange because if I PUT the resource through Basex REST interface it doesn't create the spaces and the same if I directly db:add the resource. We noticed this oddity on BaseX 8.3.1 and could verify it on a Basex 8.2 and 8.1.1 as well. It works neatly on an older 7.9 version we still have around The applies same if the resource is sent through curl or Chrome Postman. What are we doing wrong? Thanks. Marco.

2 6

Lookups and arrows
by Tim Thompson 10 Jan '16

10 Jan '16

Hello, I'm testing some XQuery 3.1 features against a JSON-LD[1] document and had a few questions. In the JSON-LD format, the "@" symbol has special semantics in key names, but seems to cause problems with the 3.1 lookup operator. For example: json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld")("@context") works as expected, but json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld")?@context throws an error: [XPST0003] No specifier after lookup operator: '@'. Also, when using the "=>" operator, should it be possible to perform a lookup on the last expression in a chain, if that expression returns a map? For example: json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld")("@context") evaluates to "http://iiif.io/api/presentation/2/context.json" and json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld")("@context") => json-doc() returns another map object. So, how would one achieve this: json-doc(json-doc("http://lae.princeton.edu/catalog/0bp35.jsonld ")("@context"))("@context") using the arrow operator? Thanks, Tim [1] http://www.w3.org/TR/json-ld/ -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

3 5

Mimicking RDBMS uniqueness constraints
by E. Wray Johnson 09 Jan '16

09 Jan '16

What is the best practice in BaseX to implement a uniqueness constraint similar to a RDBMS? Given a database, a path to the documents, an optional RegEx expression on files in the path, and one or more XPaths to the constraint elements/attributes. Can I create an index of this nature? Can the index not allow duplicates?

2 3

Database folders with numbers after
by James Ball 09 Jan '16

09 Jan '16

Hello, I’m seeing regular occurrences where I end up with extra folders in my database location with numbers after, for example: ODF-PUSH-T-JU ODF-PUSH-T-JU_1079560404 These have some but not all of the files from the original database with the same name, crucially don’t have the inf.basex file and so prevent the /dba from listing databases. What creates these folders? * I have a job that runs each night to optimise all the databases that have changed that day, I think these are coming about due to a failure to optimise but I’m not sure, I’m trying to understand what might creates them to be created so I can look deeper into the cause. Running 8.4 beta 5687c30 on Linux. Many thanks for your help, James

2 5

problems with BaseX under OpenShift (connection failed on port 15007)
by C. M. Sperberg-McQueen 08 Jan '16

08 Jan '16

I'm running into a problem that perhaps others here can help with. It appears to be almost identical to the problem reported by "P.C." on 5 March 2014 under the subject line 'BaseX server.close() "Connection refused" in Openshift' [1]; the mail record in the basex-talk archive doesn't show a resolution to P.C.'s problem. The main difference I see is that P.C. was deploying BaseX under Tomcat 7 on OpenShift, and I'm using Andy Bunce's quickstart method. [1] https://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03861.… I'm experimenting with Andy Bunce's BaseX quick-start for OpenShift, which I like a lot, and I am experiencing the following sequence of events: 1 Following the instructions at [2], I use rhc to make a new app, add the Quodatum repository as a remote, merge it into the application source, and push the result to OpenShift. (Note: no changes made between the commands "git pull -s recursive -X theirs upstream master" and "git push origin master". The first time I tried it, I did change the admin password, but I thought that that might have something to do with the problem, so I omitted that step for this test.) [2] https://github.com/Quodatum/openshift-basex-quick-start/ The OpenShift app is now available as promised at the URI indicated in [1], namely http://TestAPP-CMSMcQ.rhcloud.com/ (except that TestAPP is not really the name of the app, and CMSMcQ is not the name of the domain). It's a very nice looking console app. 2 I make some change -- any change -- to a file in the repository. To keep things simple, I'll add a comment line reading (: Kilroy was here :) to the file $app-name/basex/repo/quodatum/basex/env.xqm. I'm pretty sure I didn't break anything in the env.xqm file in making the change, and I didn't change anything in any other file. 3 I check the change in to the local git repository: # confirm that we have not touched anything else git status # commit the change git add basex/repo/quodatum/basex/env.xqm git commit -m "Vacuous change to env.xqm, just to have something to check in" 4 I attempt to push the change to the deployed app. The response is shown below (in the extract below, 'Salerno' is the local name of the machine, 'TestAPP' stands in for the name of the test application, and "CMSMcQ" for the OpenShift domain -- I've changed them here, because as can be seen from the log below, the app is currently open to anyone who knows the default admin userid and password). Salerno:TestAPP cmsmcq$ git push origin master Counting objects: 7, done. Delta compression using up to 8 threads. Compressing objects: 100% (5/5), done. Writing objects: 100% (7/7), 649 bytes | 0 bytes/s, done. Total 7 (delta 2), reused 0 (delta 0) remote: Stopping DIY cartridge remote: HOST: 127.12.13.129 remote: PORT: 15005 remote: DEBUG: true remote: USER: admin remote: SERVERHOST: 127.12.13.129 remote: STOPPORT: 15007 remote: SERVERPORT: 15005 remote: PASSWORD: admin remote: java.io.IOException: Connection failed (port: 15007). remote: at org.basex.BaseXHTTP.stop(BaseXHTTP.java:367) remote: at org.basex.BaseXHTTP.stop(BaseXHTTP.java:180) remote: at org.basex.BaseXHTTP.<init>(BaseXHTTP.java:94) remote: at org.basex.BaseXHTTP.main(BaseXHTTP.java:50) remote: Connection failed (port: 15007). remote: An error occurred executing 'gear prereceive' (exit code: 1) remote: Error message: CLIENT_ERROR: Failed to execute: 'control stop' for /var/lib/openshift/568d73d67628e13ca2000004/diy remote: remote: For more details about the problem, try running the command again with the '--trace' option. To ssh://568d73d67628e13ca2000004@TestAPP-CMSMcQ.rhcloud.com/~/git/TestAPP.git/ ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://568d73d67628e13ca2000004@TestAPP-CMSMcQ.rhcloud.com/~/git/TestAPP.git/' Salerno:TestAPP cmsmcq$ I've logged in to the application using ssh and run the basexhttp stop command manually, and what one gets then is essentially the same. I think the root of the matter here is the line reading "remote: java.io.IOException: Connection failed (port: 15007)", but I'm having trouble thinking of a fix. One could imagine that OpenShift doesn't like application code using a port like 15007. But since it seems to have no trouble with BaseX using 15005, for the SERVERPORT, that doesn't seem to make sense. One could suppose that BaseX isn't actually listening on port 15007. The debugging output produced by -d, however does seems to show that BaseX knows that port 15007 is the stop port. Or does it? Perhaps it means only that BaseX has successfully parsed the -s option on the 'stop' command, which reads: ${OPENSHIFT_DATA_DIR}/basex/binbasexhttp -d \ -n ${OPENSHIFT_DIY_IP} -h ${OPENSHIFT_DIY_PORT} \ -p ${PORT_DATABASE} -s ${PORT_STOP} \ -U ${BASEX_USER} -P ${BASEX_PASSWORD} \ stop which in turn expands to .../basex/binbasexhttp -d \ -n 127.12.13.129 -h 8080 \ -p 15005 -s 15007 \ -U admin -P admin \ stop The startup command uses the same values for the -p and -s options, so it seems plausible that the start command is successfully setting the port. Several questions arise in connection with this problem, apart from the obvious and most important one, namely Q1 Does anyone reading this list understand where the problem might lie? The others that occur to me are: Q2 Is there a server command or an admin query that will show what ports a running instance of BaseX is listening to? Or would that be too broad a security violation? (It would be nice to confirm that BaseX is actually listening for a stop signal on port 15007. Q3 A search through the file system of the application shows that logs are being written to $APPLICATION_HOME/app-root/data/basex/data/.logs/ -- is there any way to find out from a running server what directory it's writing the logs to, to save the trouble of sshing to the application server and running find . -print0 | xargs -0 grep "User was logged in" or the equivalent? The admin:logs() function was helpful, but it would have been even more helpful if it had helped me find the logs directory. Of course, I see now, looking at the documentation, that logs are always (? or only by default?) written to $DATABASE_DIRECTORY/.logs -- if I had been confident of knowing what directory was the database directory in this (still unfamiliar) setup, I would have known where to look. Q4 Is there a command or admin-module query that indicates what directories BaseX regards as the base directory and the database directory? It could be very helpful, if it's not regarded as too great a security risk, for users trying to set things up in a new configuration. Q5 OpenShift's documentation claims that it really wants application logs to go into $APPLICATION-HOME/app-root/logs/ -- is there any way to induce BaseX to write its logs there? The main advantage I can currently see is that it then becomes possible to examine them with the OpenShift rhc tools. Any light anyone can shed on these issues would be very welcome. Thanks! Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

3 13

Re: [basex-talk] XQuery Update: remove-elements-not-contents
by kleist-digital 08 Jan '16

08 Jan '16

Christian, I tried the proposed way, but it doesn’t work as expected. I’ll give a shortened example: Input is: <s> <w>u.<reg>und</reg> </w> <w>für</w> <w>die</w> <w>Folge</w> <w>da</w> <w>werden</w> <w>Sie</w>, <persName> <w>Gnädigste<reg>gnädigste</reg></w> <lb/> <w>Tante<c>,</c></w> </persName> <w>schon</w> <w>sorgen</w>. </s> Output should be: <s>u. für die Folge da werden Sie, Gnädigste Tante, schon sorgen.</s> With db:create('kleist_index', for $item in db:open('kleist-data') return $item update delete node .//(tei:note|tei:rdg|tei:lb|tei:del|tei:reg|tei:sic), db:open('kleist-data')/db:path(.)) I’ll get <s> <w>u.</w> <w>für</w> <w>die</w> <w>Folge</w> <w>da</w> <w>werden</w> <w>Sie</w>, <persName> <w>Gnädigste</w> <w>Tante<c>,</c></w> </persName> <w>schon</w> <w>sorgen</w>. </s> That’s what I expected. With db:create('kleist_index', for $item in db:open('kleist-data') return $item update (delete node .//(tei:note|tei:rdg|tei:lb|tei:del|tei:reg|tei:sic), for $n in .//tei:w return replace value of node $n with string($n)), db:open('kleist-data')/db:path(.)) I’ll get <s> <w>u.und</w> <w>für</w> <w>die</w> <w>Folge</w> <w>da</w> <w>werden</w> <w>Sie</w>, <persName> <w>Gnädigstegnädigste </w> <w>Tante,</w> </persName> <w>schon</w> <w>sorgen</w>. </s> The w-element isn’t removed and the reg-element isn’t deleted, but the value of w-element is replaced with string of w and reg. Any idea, how to reach <s>u. für die Folge da werden Sie, Gnädigste Tante, schon sorgen.</s>>? Best regards, Günter

2 5

Notes on running BaseX under OpenShift (using Andy Bunce's BaseX Quick Start for OpenShift)
by C. M. Sperberg-McQueen 07 Jan '16

07 Jan '16

For the benefit of any readers of this list who have been wondering whether to try Andy Bunce's Openshift quick start for BaseX, I have written up some notes on the subject [1] which are a little less terse than Andy's README file and perhaps slightly less forbidding for people (like me) who haven't used OpenShift before. The short version is: yes, if you have any interest in XQuery-driven Web applications, this is definitely worth trying: very quick to install, very straightforward. I've also updated my earlier notes on installing BaseX to run under Tomcat in a commercial Java hosting environment [2] to cover Tomcat 7 and BaseX 8.3.1. As always, my thanks to the BaseX team for the great tool, and to Andy Bunce for the quick-start, without which I would certainly not have been able to understand OpenShift well enough to get BaseX running. -C. M. Sperberg-McQueen [1] http://cmsmcq.com/mib/?p=1395 [2] http://cmsmcq.com/mib/?p=1368 -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

1 0

best approach to add dtd file into baseX
by Wang, Yvonne 07 Jan '16

07 Jan '16

Hi, I have sample dtd file such as below, I want to add the dtd file as is in baseX. Since the dtd file is not in xml format, there are currently two option I can thought about 1. Use TEXT as input format When I add it using baseX client UI, I got following output. Since we just need store the dtd file and later get it back as is. So I can not use doc function to retrieve it back as is. <text> <line><dtd></line> <line></line> </text> 2. Format it to xml using <dtd> <!-original data --> </dtd> when add, and remove those dtd and comment tag when download using doc() function. And there is extra handling required for nested xml comment. Is there any other better approach for adding dtd file which can be retrieved as its original content? Thanks Yvonne =========================================================================================================================== Sample.dtd <?xml encoding="UTF-8"?>  <!ELEMENT first (#PCDATA)> <!ELEMENT last (#PCDATA)> <!ELEMENT user (first,last)> <!ATTLIST user xmlns:xsi CDATA #FIXED 'http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation CDATA #FIXED 'http://ariba.com.com/Core_after_xsd.xsd'>

3 3

Remove me from mailing list
by Toni-Marie Hamilton 07 Jan '16

07 Jan '16

Please remove me from your mailing list Sent from Yahoo Mail on Android

2 1

Question about namespace prefix bindings
by Tim Thompson 07 Jan '16

07 Jan '16

Hello, I have a small database of library catalog records that I am trying to run some simple queries on, but I'm running into a namespace problem. Some docs in the database have the following structure, with a wrapper collection element and two record elements: <marc:collection xmlns:marc="http://www.loc.gov/MARC21/slim" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"> <marc:record type="Bibliographic"> ... </marc:record> <marc:record type="Holdings"> ... </marc:record> </marc:collection> But others lack a wrapper collection and have only a single record element, which also lacks a namespace declaration, like so: <marc:record type="Bibliographic"> ... </marc:record> So, I run the following query (using the DBA query interface): declare namespace marc = "http://www.loc.gov/MARC21/slim"; let $m := collection("marc-data")/marc:record[@type = "Bibliographic"] return count($m) And the count returns 0. However, if I change the predicate in the let clause to: let $m := collection("marc-data")/*[local-name() = "record"][@type = "Bibliographic"] the count returns 45. If I am declaring the namespace in my query prolog and associating it with the same prefix as the one used in my XML documents, why don't I get the desired prefix/namespace mapping? I'm probably missing something basic, but this didn't immediately make sense to me. Thank you, Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

2 2

Remote vs local performance
by E. Wray Johnson 07 Jan '16

07 Jan '16

Using a Java client, we are seeing operations take much longer (e.g. 2x+) when BaseX is remote vs when local (on the same machine). All machines are virtual Windows. Any ideas why?

2 1

db:add speed proportional to DB size?
by Hondros, Constantine (ELS-AMS) 06 Jan '16

06 Jan '16

Hello all, Processing hundreds of thousands of zips, using db:add to to append small XML fragments from each into a single DB, I notice that the process becomes successively slower. Without having done any proper profiling, and aware that I might be looking in the wrong direction here, would it be reasonable to expect db:add to become steadily slower proportional to the size of the database being added to? Cheers, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 2

Subject: Re: BaseX request from Saxon XSL transformation
by ski n 06 Jan '16

06 Jan '16

Hi Christophe, It's possible to query Basex directly from XSLT. However this can be cumbersome because you need a lot of escapes to correctly execute the query. If you only need to query a relative small XML you can avoid this by getting by getting the complete XML and use a normal xpath in the xslt (in stead of xquery): 1) First create a function in a separate XSLT: <?xml version="1.0"?> <xsl:stylesheet version="2.0" exclude-result-prefixes="xs basex saxon" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:functx="http://www.functx.com" xmlns:org="http://www.example.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:saxon="http://saxon.sf.net/" xmlns:basex="java:org.basex.server.ClientSession"> <xsl:function name="org:getBasexDoc"> <xsl:param name="database"/> <xsl:variable name="session" select="basex:new('localhost', 1984, 'admin','admin')"/> <xsl:variable name="query"> <xsl:text>XQUERY for $doc in db:open('</xsl:text><xsl:value-of select="$database"/><xsl:text>') return $doc</xsl:text> </xsl:variable> <xsl:variable name="result"> <xsl:value-of select="basex:execute($session,string($query))"/> </xsl:variable> <xsl:copy-of select="saxon:parse($result)"/> </xsl:function> </xsl:stylesheet> 2) Include the stylesheet in you main stylesheet: xsl:include href="basex.xsl"/> 3) Set the XML document in a variable <xsl:variable name="dbname"> <xsl:copy-of select="drs:getBasexDoc('dbname')"/> </xsl:variable> 4) Query the document with xpath <xsl:value-of select="$dbname//xpath"/> See also: https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-March/004776.html Raymond

1 0

Error Java ArrayIndexOutOfBoundsException
by kleist 06 Jan '16

06 Jan '16

Hello again, I got the following error: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.3.1 Java: Oracle Corporation, 1.8.0_45 OS: Mac OS X, x86_64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException after running: xquery version "3.0"; declare namespace tei = "http://www.tei-c.org/ns/1.0"; db:create('kleist_index', for $item in db:open('kleist-data') return $item update delete node .//(tei:note|tei:rdg|tei:lb|tei:del|tei:reg|tei:sic), db:open('kleist-data')/db:path(.)) This error only appears on my second computer (both MacBooks). The one, which works, runs java version "1.8.0_31" Java(TM) SE Runtime Environment (build 1.8.0_31-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode) the other, where my xq is not working java version "1.8.0_45" (Java 8-Update 66). Any ideas, what's the problem? Best, Günter

2 1

XQuery Update: remove-elements-not-contents
by kleist 06 Jan '16

06 Jan '16

Hi Christian, hello all, to write a new index to get better results out of mostly mixed-content elements, I have to delete some elements: db:create('index', for $item in db:open('data') return $item update delete node .//(note|sic), db:open('data')/db:path(.)) After that I have to remove some elements, but not the content, like the FunctX-function functx:remove-elements-not-contents does (http://www.xqueryfunctions.com/xq/functx_remove-elements-not-contents.html), to get This is only an illustrative example, not a real text. from This is only an illustrative <hi>example</hi>, not a <q>real</q> text. Is there any way, to do it with the help of BaseX XQuery Update? Best regards, Günter

2 1

XQery Performance-Problem when using variables
by kleist 05 Jan '16

05 Jan '16

Dear members, dear Christian, when using a for-loop inside of a for-loop I run into serious performance-problems, when I'm using variables to return results. For example this code-line takes 36780ms (!!!) for evaluation: return <li><a>{$title}</a>{$quote}</li> When I substitute the variable $time with the defining code, it takes only 711ms for evaluation (!!): <li><a> { if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]/string() else ()} </a>{$quote}</li> The only difference in the code is using a variable and it differs in performance by a factor of more than 50. In my real application (using more variables) it differs by factor more than 150. I tried to simulate the problem with a simple factbook-query. Here is the code: xquery version "3.0"; let $query := "Pa.*" for $city in doc('factbook')//city/name for $hits in ft:mark($city[.//text() contains text {$query} using wildcards]) let $country_of_city := $city/ancestor::country/name return (: fast version: Evaluating 12.54ms :) <hit><city>{$hits}</city><country>{$city/ancestor::country/name}</country></hit> (: slow version: Evaluating 35.78 :) (: <hit><city>{$hits}</city><country>{$country_of_city}</country></hit> :) Also here is a performance bottleneck (not as dramatically: factor 3). Is there any solution to the problem? Now I'm working in my code without variables in the result part of the xquery, but it makes realy ugly non maintainable code. Any help would be appreciated. - Günter

2 8

Guidance on Indexing
by Mansi Sheth 05 Jan '16

05 Jan '16

Hello, A very happy new year to all of you !!! I have some very basic questions with indexing. 1. Most of my xqueries are of below nature '/Archives/descendant::apiCalls[contains(@name,"com.sun")]/@name', where apiCalls could be 3-4 level under 'Archives'. Xqueries are accessed via REST Based on this, I used attribute indexing, after each update to DB. Am I correct ? Should I have been using fulltext indexing instead ? Why ? 2. I have 1000s of documents, spanning over 100 XML DB, with total space around 400 GB currently. Each query is taking roughly 30 mins, to run. Though expectable performance, but I know I can do better with indexing. Currently, when I looked at one of the DBs, > open bi_output_3 Database 'bi_output_3' was opened in 38.22 ms. > info db Database Properties Name: bi_output_3 Size: 3938 MB Nodes: 16193129 Documents: 35 Binaries: 0 Timestamp: 2016-01-03T13:40:40.000Z Resource Properties Timestamp: 2016-01-03T13:40:40.776Z Encoding: UTF-8 CHOP: true Indexes Up-to-date: false TEXTINDEX: false ATTRINDEX: false FTINDEX: false LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: false AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 When looked at its HDD footprint: ubuntu@<abc>/BaseXDB/bi_output_3$ ls -l total 4032992 -rw-rw-r-- 1 ubuntu ubuntu 2209449064 Jan 1 17:00 atv.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 atvl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 atvr.basex -rw-rw-r-- 1 ubuntu ubuntu 6414 Jan 3 13:40 doc.basex -rw-rw-r-- 1 ubuntu ubuntu 6 Jan 1 17:00 ftxx.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxy.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxz.basex -rw-rw-r-- 1 ubuntu ubuntu 829 Jan 3 13:40 inf.basex -rw-rw-r-- 1 ubuntu ubuntu 28 Jan 1 17:00 swl.basex -rw-rw-r-- 1 ubuntu ubuntu 1916444672 Jan 3 13:40 tbl.basex -rw-rw-r-- 1 ubuntu ubuntu 3796037 Jan 3 13:40 tbli.basex -rw-rw-r-- 1 ubuntu ubuntu 45462 Jan 1 17:00 txt.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 txtl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 txtr.basex ubuntu@<abc>/BaseXDB/bi_output_3$ pwd /veracode/msheth/BaseXDB/bi_output_3 ubuntu@<abc>/BaseXDB/bi_output_3$ My concern is, at each DB update, I am using attribute indexing, but info command on basex prompt tells me otherwise. Am I misreading something ? Is there a way to fix this once DB is created ? Its takes me 48 hours, to create DBs from scratch... :) Reading thru UPDINDEX and AUTOOPTIMIZE ALL commands, tells me to open each DB and run these commands. Is that my option ? Do we have a xquery script somewhere which I can use to do this ? Thanks, - Mansi

4 14

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk