BaseX-Talk January 2016

basex-talk@mailman.uni-konstanz.de

39 participants
68 discussions

Start a nNew thread

Remove me from mailing list
by Toni-Marie Hamilton 07 Jan '16

07 Jan '16

Please remove me from your mailing list Sent from Yahoo Mail on Android

2 1

Question about namespace prefix bindings
by Tim Thompson 07 Jan '16

07 Jan '16

Hello, I have a small database of library catalog records that I am trying to run some simple queries on, but I'm running into a namespace problem. Some docs in the database have the following structure, with a wrapper collection element and two record elements: <marc:collection xmlns:marc="http://www.loc.gov/MARC21/slim" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd"> <marc:record type="Bibliographic"> ... </marc:record> <marc:record type="Holdings"> ... </marc:record> </marc:collection> But others lack a wrapper collection and have only a single record element, which also lacks a namespace declaration, like so: <marc:record type="Bibliographic"> ... </marc:record> So, I run the following query (using the DBA query interface): declare namespace marc = "http://www.loc.gov/MARC21/slim"; let $m := collection("marc-data")/marc:record[@type = "Bibliographic"] return count($m) And the count returns 0. However, if I change the predicate in the let clause to: let $m := collection("marc-data")/*[local-name() = "record"][@type = "Bibliographic"] the count returns 45. If I am declaring the namespace in my query prolog and associating it with the same prefix as the one used in my XML documents, why don't I get the desired prefix/namespace mapping? I'm probably missing something basic, but this didn't immediately make sense to me. Thank you, Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

2 2

Remote vs local performance
by E. Wray Johnson 07 Jan '16

07 Jan '16

Using a Java client, we are seeing operations take much longer (e.g. 2x+) when BaseX is remote vs when local (on the same machine). All machines are virtual Windows. Any ideas why?

2 1

db:add speed proportional to DB size?
by Hondros, Constantine (ELS-AMS) 06 Jan '16

06 Jan '16

Hello all, Processing hundreds of thousands of zips, using db:add to to append small XML fragments from each into a single DB, I notice that the process becomes successively slower. Without having done any proper profiling, and aware that I might be looking in the wrong direction here, would it be reasonable to expect db:add to become steadily slower proportional to the size of the database being added to? Cheers, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 2

Subject: Re: BaseX request from Saxon XSL transformation
by ski n 06 Jan '16

06 Jan '16

Hi Christophe, It's possible to query Basex directly from XSLT. However this can be cumbersome because you need a lot of escapes to correctly execute the query. If you only need to query a relative small XML you can avoid this by getting by getting the complete XML and use a normal xpath in the xslt (in stead of xquery): 1) First create a function in a separate XSLT: <?xml version="1.0"?> <xsl:stylesheet version="2.0" exclude-result-prefixes="xs basex saxon" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:functx="http://www.functx.com" xmlns:org="http://www.example.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:saxon="http://saxon.sf.net/" xmlns:basex="java:org.basex.server.ClientSession"> <xsl:function name="org:getBasexDoc"> <xsl:param name="database"/> <xsl:variable name="session" select="basex:new('localhost', 1984, 'admin','admin')"/> <xsl:variable name="query"> <xsl:text>XQUERY for $doc in db:open('</xsl:text><xsl:value-of select="$database"/><xsl:text>') return $doc</xsl:text> </xsl:variable> <xsl:variable name="result"> <xsl:value-of select="basex:execute($session,string($query))"/> </xsl:variable> <xsl:copy-of select="saxon:parse($result)"/> </xsl:function> </xsl:stylesheet> 2) Include the stylesheet in you main stylesheet: xsl:include href="basex.xsl"/> 3) Set the XML document in a variable <xsl:variable name="dbname"> <xsl:copy-of select="drs:getBasexDoc('dbname')"/> </xsl:variable> 4) Query the document with xpath <xsl:value-of select="$dbname//xpath"/> See also: https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-March/004776.html Raymond

1 0

Error Java ArrayIndexOutOfBoundsException
by kleist 06 Jan '16

06 Jan '16

Hello again, I got the following error: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.3.1 Java: Oracle Corporation, 1.8.0_45 OS: Mac OS X, x86_64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException after running: xquery version "3.0"; declare namespace tei = "http://www.tei-c.org/ns/1.0"; db:create('kleist_index', for $item in db:open('kleist-data') return $item update delete node .//(tei:note|tei:rdg|tei:lb|tei:del|tei:reg|tei:sic), db:open('kleist-data')/db:path(.)) This error only appears on my second computer (both MacBooks). The one, which works, runs java version "1.8.0_31" Java(TM) SE Runtime Environment (build 1.8.0_31-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode) the other, where my xq is not working java version "1.8.0_45" (Java 8-Update 66). Any ideas, what's the problem? Best, Günter

2 1

XQuery Update: remove-elements-not-contents
by kleist 06 Jan '16

06 Jan '16

Hi Christian, hello all, to write a new index to get better results out of mostly mixed-content elements, I have to delete some elements: db:create('index', for $item in db:open('data') return $item update delete node .//(note|sic), db:open('data')/db:path(.)) After that I have to remove some elements, but not the content, like the FunctX-function functx:remove-elements-not-contents does (http://www.xqueryfunctions.com/xq/functx_remove-elements-not-contents.html), to get <p>This is only an illustrative example, not a real text.</p> from <p>This is only an illustrative <hi>example</hi>, not a <q>real</q> text.</p> Is there any way, to do it with the help of BaseX XQuery Update? Best regards, Günter

2 1

XQery Performance-Problem when using variables
by kleist 05 Jan '16

05 Jan '16

Dear members, dear Christian, when using a for-loop inside of a for-loop I run into serious performance-problems, when I'm using variables to return results. For example this code-line takes 36780ms (!!!) for evaluation: return <li><a>{$title}</a>{$quote}</li> When I substitute the variable $time with the defining code, it takes only 711ms for evaluation (!!): <li><a> { if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="short"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="article"]/string() else if ($ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]) then $ele/ancestor::tei:TEI//tei:titleStmt/tei:title[@type="main"]/string() else ()} </a>{$quote}</li> The only difference in the code is using a variable and it differs in performance by a factor of more than 50. In my real application (using more variables) it differs by factor more than 150. I tried to simulate the problem with a simple factbook-query. Here is the code: xquery version "3.0"; let $query := "Pa.*" for $city in doc('factbook')//city/name for $hits in ft:mark($city[.//text() contains text {$query} using wildcards]) let $country_of_city := $city/ancestor::country/name return (: fast version: Evaluating 12.54ms :) <hit><city>{$hits}</city><country>{$city/ancestor::country/name}</country></hit> (: slow version: Evaluating 35.78 :) (: <hit><city>{$hits}</city><country>{$country_of_city}</country></hit> :) Also here is a performance bottleneck (not as dramatically: factor 3). Is there any solution to the problem? Now I'm working in my code without variables in the result part of the xquery, but it makes realy ugly non maintainable code. Any help would be appreciated. - Günter

2 8

Guidance on Indexing
by Mansi Sheth 05 Jan '16

05 Jan '16

Hello, A very happy new year to all of you !!! I have some very basic questions with indexing. 1. Most of my xqueries are of below nature '/Archives/descendant::apiCalls[contains(@name,"com.sun")]/@name', where apiCalls could be 3-4 level under 'Archives'. Xqueries are accessed via REST Based on this, I used attribute indexing, after each update to DB. Am I correct ? Should I have been using fulltext indexing instead ? Why ? 2. I have 1000s of documents, spanning over 100 XML DB, with total space around 400 GB currently. Each query is taking roughly 30 mins, to run. Though expectable performance, but I know I can do better with indexing. Currently, when I looked at one of the DBs, > open bi_output_3 Database 'bi_output_3' was opened in 38.22 ms. > info db Database Properties Name: bi_output_3 Size: 3938 MB Nodes: 16193129 Documents: 35 Binaries: 0 Timestamp: 2016-01-03T13:40:40.000Z Resource Properties Timestamp: 2016-01-03T13:40:40.776Z Encoding: UTF-8 CHOP: true Indexes Up-to-date: false TEXTINDEX: false ATTRINDEX: false FTINDEX: false LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: false AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 When looked at its HDD footprint: ubuntu@<abc>/BaseXDB/bi_output_3$ ls -l total 4032992 -rw-rw-r-- 1 ubuntu ubuntu 2209449064 Jan 1 17:00 atv.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 atvl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 atvr.basex -rw-rw-r-- 1 ubuntu ubuntu 6414 Jan 3 13:40 doc.basex -rw-rw-r-- 1 ubuntu ubuntu 6 Jan 1 17:00 ftxx.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxy.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 17:00 ftxz.basex -rw-rw-r-- 1 ubuntu ubuntu 829 Jan 3 13:40 inf.basex -rw-rw-r-- 1 ubuntu ubuntu 28 Jan 1 17:00 swl.basex -rw-rw-r-- 1 ubuntu ubuntu 1916444672 Jan 3 13:40 tbl.basex -rw-rw-r-- 1 ubuntu ubuntu 3796037 Jan 3 13:40 tbli.basex -rw-rw-r-- 1 ubuntu ubuntu 45462 Jan 1 17:00 txt.basex -rw-rw-r-- 1 ubuntu ubuntu 4 Jan 1 16:35 txtl.basex -rw-rw-r-- 1 ubuntu ubuntu 0 Jan 1 16:35 txtr.basex ubuntu@<abc>/BaseXDB/bi_output_3$ pwd /veracode/msheth/BaseXDB/bi_output_3 ubuntu@<abc>/BaseXDB/bi_output_3$ My concern is, at each DB update, I am using attribute indexing, but info command on basex prompt tells me otherwise. Am I misreading something ? Is there a way to fix this once DB is created ? Its takes me 48 hours, to create DBs from scratch... :) Reading thru UPDINDEX and AUTOOPTIMIZE ALL commands, tells me to open each DB and run these commands. Is that my option ? Do we have a xquery script somewhere which I can use to do this ? Thanks, - Mansi

4 14

ft:search and map-function
by kleist 05 Jan '16

05 Jan '16

Hi all, in the documentation for the ft:search-function is the following example: ft:search("db", ("A", "B"), map { "mode": "all words", "distance": { "max": "5", "unit": "words" } }) It's throwing this error for line 3: "Invalid value, simple expression expected." I'm using BaseX Version 8.3.1. Did the notation change or what's going wrong? Best regards, Günter

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk January 2016