BaseX-Talk August 2018

basex-talk@mailman.uni-konstanz.de

34 participants
39 discussions

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

BaseX 9.0: The Spring Edition!
by Christian Grün 01 Nov '18

01 Nov '18

Dear BaseX aficionados, We are very happy to announce the release of BaseX 9.0! The new version of our XML database system and XQuery 3.1 processor includes some great new features and a vast number of minor improvements and optimizations. It’s both the usage of BaseX in productive environments as well as the valuable feedback of our open source users that make BaseX better and better, and that allow and motivate us to keep going. Thanks to all of you! Along with the new release, we invite you to visit our relaunched homepage: http://basex.org/. Java 8 is now required to run BaseX. The most prominent features of Version 9.0 are: XQUERY - Comprehensive extensions in the internal XQuery optimizer framework - Dynamic hash joins rewritings of general comparisons - Register query jobs as persistent services - Process very large CSV files via the new CSV 'xquery' format - Update Module: higher-order functions for performing updates - Unified static typing, including maps, arrays and function items - COPYNODE: Lightweight copying of XML fragments - ENFORCEINDEX: Enforce rewriting for index access STORAGE - Improved database compression (short strings, whitespaces) WEB APPLICATIONS - Permissions layer: Unified definition of RESTXQ access patterns - RESTXQ: Full support for client- and server-side quality factors - REST: Run BaseX command scripts - Web server upgrade to Jetty 9 REPOSITORIES - Combined packaging mechanism (XQuery and Java) - Java Packaging: Bundling of additional libraries DISTRIBUTIONS - Revised detection and configuration of BaseX home directory - Windows executable: better detection of Java libraries DBA: BASEX DATABASE ADMINISTRATION - Improved editing and evaluation of XQuery modules - Revised file, session and job management More details can be found on our homepage. We are looking forward to your feedback. Enjoy and have fun, Christian, BaseX Team

7 8

Database Updates
by Dave Kopecek 25 Oct '18

25 Oct '18

Hi All, Given a database created from catalog.xml below. What's the best way to update & add records received daily as in file daily-update.xml ? Coming from relational DBs & new to this. I ultimately need to script/automate this. Hoping there's some magic command I'm missing & looking for best way to approach the problem. Thanks, -Dave catalog.xml <?xml version="1.0" encoding="UTF-8"?> <CATALOG> <ENTRIES> <ENTRY> <SKU>1</SKU> <TITLE>Empire Burlesque</TITLE> <ARTIST>Dylan</ARTIST> <PRICE>10.90</PRICE> </ENTRY> <ENTRY> <SKU>2</SKU> <TITLE>Hide your heart</TITLE> <ARTIST>Bonnie Tyler</ARTIST> <PRICE>9.90</PRICE> </ENTRY> </ENTRIES> </CATALOG> daily-update.xml <?xml version="1.0" encoding="UTF-8"?> <CATALOG> <ENTRIES> <ENTRY> <SKU>1</SKU> <PRICE>19.90</PRICE> <NEWTHING>I wasn't here before the update</NEWTHING> </ENTRY> <ENTRY> <SKU>1971</SKU> <TITLE>Tupelo Honey</TITLE> <ARTIST>Van Morrison</ARTIST> <PRICE>8.20</PRICE> </ENTRY> </ENTRIES> </CATALOG>

3 5

diacritics sensitive not working
by Ron Katriel 03 Sep '18

03 Sep '18

Hi, The following from your website (docs.basex.org/wiki/Full-Text) appears to be syntactically incorrect "'Äpfel' will not be found..." contains text "Apfel" diacritics sensitive In the BaseX GUI the keyword diacritics is underlined in red and the following error is reported Unexpected end of query: 'diacritic sens...'. This happens in version 8.6.4 and also the latest (9.0.2). Thanks, Ron *Ron Katriel, Ph.D. *| Principal Data Scientist | Medidata Solutions <http://www.mdsol.com/> 350 Hudson Street, 7th Floor, New York, NY 10014 rkatriel(a)mdsol.com <tbrophy(a)mdsol.com> | direct: +1 201 337 3622 <//201%20337%203622> | mobile: +1 201 675 5598 <//mobile:%20+1%20201%20675%205598> | main: +1 212 918 1800 <//+1%20212%20918%201800>

4 15

BaseX insert/delete node performance
by BIRKNER Michael 01 Sep '18

01 Sep '18

Hello, I asked this question in StackOverflow concerning some performance problems I experienced when inserting nodes into a BaseX database: https://stackoverflow.com/questions/51595210/basex-inserting-nodes-performa… I already made some progress, especially when it comes to querying all data I need for the updates. I work a lot with the indexes now. But I still have problems with inserting - and also deleting - nodes. It doesn't matter if I insert/delete nodes via a Java program or in the editor of the BaseX GUI: Both is quite slow. Inserting just one node in the GUI with an XQuery like this one takes up to 3 seconds: insert node <related_record><title>Test title</title><author>Joe Lastname</author></related_record> into db:open-id('Database_Name', 7947561) Deleting a node with the following command takes up to 7 seconds: delete node db:open-id('Database_Name', 88085737) The problem is that in my case, I have to do about 150000 inserts and deletes, so it would take too much time. Maybe my database is just too big to be performant? Or some settings are wrong? I'm very new to BaseX (and XML databases in general) so maybe there are just some errors I don't see. I also give you some information on my database that I copied from the info screen of the BaseX GUI: Database Properties NAME: Database_Name SIZE: 2568 MB NODES: 135607105 DOCUMENTS: 1 BINARIES: 0 TIMESTAMP: 2018-08-07T07:05:56.000Z UPTODATE: true Resource Properties INPUTPATH: /path/to/file.xml INPUTSIZE: 1774 MB INPUTDATE: 2018-07-24T14:32:58.000Z Indexes TEXTINDEX: true ATTRINDEX: true TOKENINDEX: false FTINDEX: false TEXTINCLUDE: ATTRINCLUDE: TOKENINCLUDE: FTINCLUDE: LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: true AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 SPLITSIZE: 0 Best regards, Michael Beachten Sie, dass Sie uns ab sofort unter einer geänderten Rufnummer erreichen. Bitte speichern Sie gleich Ihren Kontakt zur AK Wien ein unter 501 65 1, gefolgt von der gewohnten Durchwahl. Dieses Mail ist ausschließlich für die Verwendung durch die/den darin genannten AdressatInnen bestimmt und kann vertrauliche bzw rechtlich geschützte Informationen enthalten, deren Verwendung ohne Genehmigung durch den/ die AbsenderIn rechtswidrig sein kann. Falls Sie dieses Mail irrtümlich erhalten haben, informieren Sie uns bitte und löschen Sie die Nachricht. UID: ATU 16209706 I https://wien.arbeiterkammer.at/Datenschutz_(DSGVO).html

2 9

Bug (?) in fn:generate-id
by Hans-Juergen Rennau 30 Aug '18

30 Aug '18

Dear BaseX team, I face an issue with fn:generate-id. Given: a document $doc parsed from the file system.Nodes obtained by xquery_eval(EXPR, map{'':$doc}) and nodes obtained by $doc/EXPR have the same identity (per $node1 is $node2), but the values generated by fn:generate-id differ. Interestingly, if the document is constructed within the query, rather than read from a file, the values generated by fn:generate-id do not differ. For a test see below. At first sight, the issue may appear too obscure to justify efforts to remove it. HOWEVER, there is an important use case involved: (a) expressions are read from a configuration and evaluated via xquery:eval; a map is constructed which associates for each node in the expression value its node id with other information (e.g. which of the configured expressions yielded this node) ;(b) then, when later nodes are visited via XPath, that other information should be retrieved (lookup in the map, using generate-id($node) as key) - AND THIS FAILS, as the XPath-provided nodes have ids which are different from the xquery:eval-provided nodes. Could you please have a look? With kind regards,Hans-Jürgen Test query:======== let $doc1 := <root><foo/><bar/></root> let $doc2 := doc('foobar.xml')/* let $evalNodes1 := xquery:eval('*', map{'': $doc1}) let $evalNodes2 := xquery:eval('*', map{'': $doc2}) let $xpathNodes1 := $doc1/* let $xpathNodes2 := $doc2/* let $evalIds1 := $evalNodes1/concat(local-name(.), ' id=', generate-id(.)) let $xpathIds1 := $xpathNodes1/concat(local-name(.), ' id=', generate-id(.)) let $evalIds2 := $evalNodes2/concat(local-name(.), ' id=', generate-id(.)) let $xpathIds2 := $xpathNodes2/concat(local-name(.), ' id=', generate-id(.)) return ( 'local doc - evalNodes1 = xpathNodes1 ?', for $i in 1 to 2 return $evalNodes1[$i] is $xpathNodes1[$i], '', 'local doc - evalNodes, ids:', $evalIds1, 'local doc - xpathNodes, ids:', $xpathIds1, '', 'file doc - evalNodes1 = xpathNodes1 ?', for $i in 1 to 2 return $evalNodes2[$i] is $xpathNodes2[$i], '', 'file doc - evalNodes, ids:', $evalIds2, 'file doc - xpathNodes, ids:', $xpathIds2) Document 'foobar.xml':================= <root><foo/><bar/></root> OUTPUT:======= local doc - evalNodes1 = xpathNodes1 ? true # OK! true # OK! local doc - evalNodes, ids: foo id=id10 bar id=id12 local doc - xpathNodes, ids: foo id=id10 # OK! bar id=id12 # OK! file doc - evalNodes1 = xpathNodes1 ? true # OK! true # OK! file doc - evalNodes, ids: foo id=id17 bar id=id18 file doc - xpathNodes, ids: foo id=id31 # SHOULD BE id17bar id=id32 # SHOULD BE id18

1 0

DB connection between classes
by Karina C. Malavazi 30 Aug '18

30 Aug '18

Hello, I'm new on Java programming and also on nonSQL database... hope you can give me some support. I have a group of testcases where every time that a test is executed the database has to be connected again, which lets my tests very slow. Example: public class AllTests { . public static Test suite() throws IOException { TestSuite suite = new TestSuite(); suite.addTestSuite(NumberTest.class); suite.addTestSuite(LetterTest.class); suite.addTestSuite(ImageTest.class); ... } } In the example above, NumberTest.class, LetterTest.class and ImageTest.class are tests made for some interfaces that use the same document from the database. However, each one of them finish (NumberTest and LetterTest), I have to connect to the database again. I'm not "closing" it in any place, only opening. So, why is this being closed every time that a new test is executed? Is there any way to keep the db connection opened during all the tests and close it only in the end? Any idea or tip will be helpful. Thanks, Karina.

2 1

rest parallel
by Ветошкин Владимир 29 Aug '18

29 Aug '18

2 9

ThreadLocal error in Tomcat
by Simon Chatelain 29 Aug '18

29 Aug '18

Hello all, I am currently using BaseX in embedded mode as a XQuery processor in a tomcat webapplication. I am receiving some XML, create a BaseX in memory database, execute some XQuery, then close the context. But when I stop tomcat or reload the web application, I have a lot of errors saying: SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [webap] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal] (value [java.lang.ThreadLocal$SuppliedThreadLocal@60506a99]) and a value of type [org.basex.query.expr.path.PathCache] (value [org.basex.query.expr.path.PathCache@665cfafa]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. Is there a way to avoid that error, by forcibly clearing these cache or something ? Thanks Simon

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk August 2018