BaseX-Talk July 2016

basex-talk@mailman.uni-konstanz.de

33 participants
52 discussions

retrieve a sequence of all values within an attribute index
by Alex Muir 10 Oct '16

10 Oct '16

Greetings, I need to generate a list of all the distinct values within an index. Is there a way to retrieve a sequence of all values within an attribute index? Regards Alex tech.jahtoe.com bafila.jahtoe.com

3 9

fork-join and recursive function
by Carl Leitner 11 Sep '16

11 Sep '16

Hi All, I have hierarchical information encoded an XML document that looks something like: <organization entityID=“1”> <organization entityID=“2"> <parent entityID=“1”> </organization> <organization entityID=“3"> <parent entityID=“2”> </organization> <organization entityID=“4"> <parent entityID=“1”> </organization> There are around 80,000 entries like this and I need to regular do extractions of sub-hiearchies (see the commented out version of the below function). The commented-out version runs in about a minute using a modest amount of memory. Hoping to take advantage of this <https://github.com/BaseXdb/basex/commit/ac86bbbc3cd1f71461ce94d803cab46f21e…>, I modified the function (the uncommented version) which tries to use xquery:fork-join in 8.5.2beta (the July 12th snapshot). The parallelized one chews up the 3g of available memory, unceremoniously throws exceptions (Exception in thread "qtp198198276-19” ), with the occasional: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1857) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2073) Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp198198276-19" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp198198276-14" java.lang.OutOfMemoryError: GC overhead limit exceeded and runs for tens of minutes (perhaps more - I always kill the process). Any ideas on what I can do to improve the situation? Thanks in advance. Cheers, -carl declare function csd_bl:get_child_orgs($orgs,$org) { let $org_id := $org/@entityID return if (functx:all-whitespace($org_id)) then () else let $c_orgs := $orgs[./parent[@entityID = $org_id]] let $t0 := trace($org_id, "creating func for ") let $t1 := trace(count($c_orgs), " func checks children: ") let $c_org_funcs:= for $c_org in $c_orgs return function() { ( trace($org_id, "executing child func for ") , $c_org, csd_bl:get_child_orgs($orgs,$c_org))} return xquery:fork-join($c_org_funcs) (: let $c_orgs := if (functx:all-whitespace($org_id)) then () else $orgs[./parent[@entityID = $org_id]] return for $c_org in $c_orgs let $t0 := trace($org_id, "processing children for ") return ($c_org,csd_bl:get_child_orgs($orgs,$c_org)) :) };

2 18

multiple inserts/update/deletes in single transaction
by Wang, Genneva 11 Aug '16

11 Aug '16

Hi I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section: RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction. I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback. I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI. OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added. Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ? Thank you very much for your help.

4 17

BaseX performance 7.x vs 8.x
by Bondeson, Carl 04 Aug '16

04 Aug '16

I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson(a)ct.gov<mailto:carl.bondeson@ct.gov> [logosmall]

4 10

Inconsistency of regular expression support in BaseX and GUI
by Kristian Kankainen 03 Aug '16

03 Aug '16

Hello! Today I noticed that I can successfully use the following regular expression inside XQuery but the same regexp fails when using it in the GUI's searchboxes (the ones appearing when pressing Ctrl+F) when enabling regular expression search. This is the regular expression and probably it is the block escape that is causing the error: (¹|²|³|\p{IsSuperscriptsandSubscripts}) The error message says: "Regular Expression: java.util.regex.PatternSyntaxException: Unknown character property name {issuperscriptsandsubscripts} near index 37" Probably the lowercaseing of the block escape name is causing the error and hopefully it can be fixed without too complex logics. Best regards Kristian K

2 1

users.xml change outside basex
by Maximilian Gärber 02 Aug '16

02 Aug '16

Hi, it seems all of users.xml will be rewritten on changes. I just tried to add a <groups> element next to <users>. Any way to preserve changes in the users.xml ? Maybe allow for a <custom> section or similar? By the way: what was the reason users.xml is not stored as a basex database but on the file system? Regards, Max

2 3

db:optimize questions
by Baysdon, Joseph 02 Aug '16

02 Aug '16

I a new baseX user and I have a few questions regarding the db:optimize command. Does 'optimize all' perform all available optimizations or are there other parameters (indexing or full-text as mentioned at http://docs.basex.org/wiki/Database_Module#db:optimize) to supply? Is read/write possible to a database undergoing optimization? I’m guessing no because I get an error (“Database ‘foo’ is currently opened by another process”) whenever another connection to the database is open. On what size or content of databases does optimization have a significant impact? Currently on a 400Mb database, I don't see any performance benefits after optimization. (Note: all of my performance testing is using a simple db:list command.) Thanks in advance for any assistance you can provide. Joe

2 7

Re: [basex-talk] Validating with XML schema
by Christian Grün 31 Jul '16

31 Jul '16

Hi Chahd (cc to the list), > Can I do it with the Rest service? Yes you can [1]: http://localhost:8984/rest/MyFirstBase/text.xml?query=validate:xsd(., db:open('MyFirstBase/text.xml')) I agree this doesn’t look very straightforward. If you are interested in spending some more time with XQuery, you could use RESTXQ [2] to provide more intuitive URLs. An example: Your URI: http://localhost:8984/restxq/validate/text.xml?schema=validate:xsd Your server-side RESTXQ code (see the documentation and the example in the zip/exe distributions for more details): declare %rest:path('/validate/{$doc}') %rest:query-param('schema', '{$schema}') function local:validate($doc, $schema) { let $db := 'MyFirstBase' return validate:xsd(db:open($db, $doc), db:open($db, $schema)) }; Christian [1] http://docs.basex.org/wiki/REST#GET_Method [2] http://docs.basex.org/wiki/RESTXQ On Sat, Jul 30, 2016 at 11:41 PM, Chahd Chahd <chahd.chahddd(a)gmail.com> wrote: > > 2016-07-30 22:24 GMT+02:00 Christian Grün <christian.gruen(a)gmail.com>: >> >> Hi Chahd, >> >> > I am using the web application of BaseX. How can I validate a base (that >> > contains a XML file) with a XML schema file? >> >> I invite you to have a look into our documentation [1]. >> >> Hope this helps >> Christian >> >> [1] http://docs.basex.org/wiki/Validation_Module#XML_Schema_Validation > >

1 0

Path of a resource
by Chahd Chahd 31 Jul '16

31 Jul '16

Hi, I am using the Database Administration interface. I created a base named "MyFirstBase" and I add 2 resources (test.xml and test.xsd). How can I write the paths of the resources properly in a query? I tried: validate:xsd('test.xml', 'test.xsd') but I have a *[FODC0002] Resource 'test.xml' does not exist.* *Thank you*

2 1

Validating with XML schema
by Chahd Chahd 30 Jul '16

30 Jul '16

Hi, I am using the web application of BaseX. How can I validate a base (that contains a XML file) with a XML schema file? thank you

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk July 2016