BaseX-Talk

basex-talk@mailman.uni-konstanz.de

5251 discussions

OutOfMemoryError at Query#more()
by Simon Chatelain 22 Sep '17

22 Sep '17

Hello, I am facing an issue while retrieving some big amount of XML documents from a BaseX collection. Each document (as an XML file) is around 10 KB, and in the problematic case I must retrieve around 70000 of them. I am using Session#query(String query) then Query#more() and Query#next() to iterate through the result of my query. try (final Query query = l_Session.query(“query”)) { while (query.more()) { String xml = query.next(); } } If there is more than a certain amount of XML document in the result of my query I get a OutOfMemoryError (full stack trace in attached file) when executing query.more(). I did the test with BaseX 8.6.6 and 8.6.7, Java 8, VM arguments –Xmx1024m Increasing the Xmx value is not a solution as I don’t know what the maximum amount of data I will have to retrieve in the future. So what I need is a reliable way of executing such queries and iterate through the result without exploding the heap size. I also try to use QueryProcessor and QueryProcessor#iter() instead of Session#query(String query). But is it safe to use it knowing that my application is multithreaded and that each thread has its own session to query or add elements from/to multiple collections? Moreover, for now all access to BaseX are done through a session, so my application can run with an embedded BaseX or with a BaseX server. If I start using QueryProcessor, then it will be embedded BaseX only, right? I also attached a simple example showing the problem. Any advice would be much appreciated Thanks Simon

3 6

Server Variables, cached vars, etc
by Erik Peterson 21 Sep '17

21 Sep '17

How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?

6 15

BaseX 8.6.7
by Christian Grün 21 Sep '17

21 Sep '17

Hi all, As we’ll tackle BaseX 8.7 in the next weeks, we’ve released yet another patch release today [1]. As usual, it contains minor performance tweaks (better typing in FLWOR expressions) and bug fixes (pre-evaluation of fn:tail, GUI XQuery highlighting). Have fun, Christian [1] http://basex.org/download/

1 0

DBA deployed on JBoss (Wildfly)
by Marco Lettere 21 Sep '17

21 Sep '17

Hi all, We created an application web app based on the BaseX .WAR distribution. If we deploy it under tomcat we can access all DBA related functionalities after the login prompt. If we try to do the same under WildFly 10, the application starts without errors but trying to access the DBA pages the browser is just bounced back to the login page. Please check the snippets at [1] and [2] to observe the behaviour reflected in the log files. After arriving to the final page one more request to login?path=databases is received. Is there anyone having experience with running BaseX under Jboss / Wildfly? Thanks a lot, Marco. [1] Tomcat DBA login 18:49:13.754 0:0:0:0:0:0:0:1:55352 admin 302 76.47 ms 18:49:13.754 0:0:0:0:0:0:0:1:55352 admin 200 0.1 ms 18:49:13.756 0:0:0:0:0:0:0:1:55352 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/databases 18:49:13.778 0:0:0:0:0:0:0:1:55352 admin 302 21.5 ms 18:49:13.778 0:0:0:0:0:0:0:1:55352 admin 200 0.08 ms 18:49:13.780 0:0:0:0:0:0:0:1:55352 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/login?path=databases 18:49:13.792 0:0:0:0:0:0:0:1:55352 admin 200 11.91 ms 18:49:13.806 0:0:0:0:0:0:0:1:55352 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/style.css 18:49:13.808 0:0:0:0:0:0:0:1:55353 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/js.js 18:49:13.821 0:0:0:0:0:0:0:1:55352 admin 200 15.94 ms 18:49:13.824 0:0:0:0:0:0:0:1:55353 admin 200 16.12 ms 18:49:13.827 0:0:0:0:0:0:0:1:55352 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/basex.svg 18:49:13.841 0:0:0:0:0:0:0:1:55352 admin 200 14.16 ms 18:49:50.466 0:0:0:0:0:0:0:1:55510 admin REQUEST [POST] http://localhost:8080/mywebapp/dba/login-check 18:49:50.537 0:0:0:0:0:0:0:1:55510 admin INFO DBA user was logged in: admin 18:49:50.537 0:0:0:0:0:0:0:1:55510 admin 302 71.72 ms 18:49:50.537 0:0:0:0:0:0:0:1:55510 admin 200 0.09 ms 18:49:50.540 0:0:0:0:0:0:0:1:55510 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/databases 18:49:50.577 0:0:0:0:0:0:0:1:55510 admin 200 37.53 ms 18:49:50.591 0:0:0:0:0:0:0:1:55510 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/style.css 18:49:50.594 0:0:0:0:0:0:0:1:55511 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/js.js 18:49:50.605 0:0:0:0:0:0:0:1:55510 admin 200 14.41 ms 18:49:50.607 0:0:0:0:0:0:0:1:55511 admin 200 12.49 ms 18:49:50.613 0:0:0:0:0:0:0:1:55511 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/basex.svg 18:49:50.622 0:0:0:0:0:0:0:1:55511 admin 200 9.8 ms [2] Wildfly DBA login 18:57:29.915 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/ 18:57:30.594 127.0.0.1:57601 admin 302 681.49 ms 18:57:30.594 127.0.0.1:57601 admin 200 0.18 ms 18:57:30.599 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/databases 18:57:30.660 127.0.0.1:57601 admin 302 61.28 ms 18:57:30.660 127.0.0.1:57601 admin 200 0.1 ms 18:57:30.663 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/login?path=databases 18:57:30.684 127.0.0.1:57601 admin 200 20.65 ms 18:57:30.699 127.0.0.1:57602 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/js.js 18:57:30.699 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/style.css 18:57:30.718 127.0.0.1:57601 admin 200 19.23 ms 18:57:30.721 127.0.0.1:57602 admin 200 21.71 ms 18:57:30.723 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/basex.svg 18:57:30.745 127.0.0.1:57601 admin 200 22.75 ms 18:57:50.379 127.0.0.1:57601 admin REQUEST [POST] http://localhost:8080/mywebapp/dba/login-check 18:57:50.442 127.0.0.1:57601 admin INFO DBA user was logged in: admin 18:57:50.442 127.0.0.1:57601 admin 302 63.28 ms 18:57:50.442 127.0.0.1:57601 admin 200 0.1 ms 18:57:50.446 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/databases 18:57:50.463 127.0.0.1:57601 admin 302 17.4 ms 18:57:50.463 127.0.0.1:57601 admin 200 0.09 ms 18:57:50.466 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/login?path=databases 18:57:50.476 127.0.0.1:57601 admin 200 10.89 ms 18:57:50.490 127.0.0.1:57602 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/js.js 18:57:50.493 127.0.0.1:57601 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/style.css 18:57:50.502 127.0.0.1:57602 admin 200 11.72 ms 18:57:50.505 127.0.0.1:57601 admin 200 12.17 ms 18:57:50.509 127.0.0.1:57602 admin REQUEST [GET] http://localhost:8080/mywebapp/dba/static/basex.svg 18:57:50.523 127.0.0.1:57602 admin 200 13.51 ms

2 2

Library for BaseX
by Eliúd Santiago Meza y Rivera 20 Sep '17

20 Sep '17

Hi! I've been using BaseX for a while mainly for personal purpose and I have written this library to read and make simple updates for Open Office Open Workbook files (Excel files), I hope someone could find it useful, and all comments are welcome. I leave the link below: https://github.com/eliudmeza/OOXML-Library-XQuery-BaseXdb Regards! -- LSC Eliud Santiago Meza y Rivera

2 1

db:text() vs XPath
by Giuseppe Celano 19 Sep '17

19 Sep '17

I am using BaseX 8.6.4 and I am trying to do a group-by/order-by operation, and I see that two logically equivalent queries perform very differently: one cannot see the end, while the other can (and fast). I can provide further details if necessary, but these are the queries (look at the last line of both, where the difference is): (This does not work:) declare variable $p := db:open("db2")/text/line/text(); (: returns a list of values like ΠΕΡΙ#n-s---mv- :) for $u in db:open("db1")/text/s/t let $a := $u/f/text() (: returns one value like ΠΕΡΙ :) let $b := $u/(a)o/data(.) (: returns one value like n-s---mv- :) group by $k := $a || "#" || $b (: builds a value like ΠΕΡΙ#n-s---mv- :) let $n := count($u) order by $n descending return if ($p = $k) then $k || " " || $n else () (This works:) for $u in db:open("db1")/text/s/t let $a := $u/f/text() (: returns one value like ΠΕΡΙ :) let $b := $u/(a)o/data(.) (: returns one value like n-s---mv- :) group by $k := $a || "#" || $b (: builds a value like ΠΕΡΙ#n-s---mv- :) let $n := count($u) order by $n descending return if (db:text("db2", $k)) then $k || " " || $n else () The problem is with directly accessing the database with db:text() or indirectly using XPATH. I would tend to use XPATH and expect to get an underlying translation into db:text(), but this seems not to happen: why? Best, Giuseppe

4 6

java.lang.NoClassDefFoundError: Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder
by Liam R. E. Quin 19 Sep '17

19 Sep '17

It seems with the latest Java 1.8 - java-1.8.0-openjdk-headless-1.8.0.144-0.b01.el7_4.x86_64 on Centos 7, I can no longer drop a database, any ideas? This is with both 8.5.3 and 8.6.6, and also with the latest snapshot, BaseX867-20170824.195627.zip [[ $ bin/basexclient -p 1994 Username: admin Password: BaseX 8.6.6 [Client] Try 'help' to get more information. > open rdf Database 'rdf' was opened in 1.61 ms. > close Database 'rdf' was closed. > drop db rdf Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.5.3 Java: Oracle Corporation, 1.8.0_131 OS: Linux, amd64 Stack Trace: java.lang.NoClassDefFoundError: Could not initialize class java.nio.file.FileSystems$DefaultFileSystemHolder at java.nio.file.FileSystems.getDefault(FileSystems.java:176) at java.nio.file.Paths.get(Paths.java:84) at org.basex.io.IOFile.toPath(IOFile.java:335) at org.basex.io.IOFile.delete(IOFile.java:243) at org.basex.io.IOFile.delete(IOFile.java:240) at org.basex.core.cmd.DropDB.drop(DropDB.java:77) at org.basex.core.cmd.DropDB.run(DropDB.java:46) at org.basex.core.Command.run(Command.java:253) at org.basex.core.Command.execute(Command.java:99) at org.basex.server.ClientListener.run(ClientListener.java:136) > open rdf Database 'rdf' was opened in 1.56 ms. > xquery count(//image) 7370 Query executed in 68.2 ms. ]] -- Liam Quin, W3C, http://www.w3.org/People/Quin/ Staff contact for Verifiable Claims WG, XQuery WG Web slave for http://www.fromoldbooks.org/

2 3

SQL2XML
by iCloud 18 Sep '17

18 Sep '17

Hello, I wrote a small app for my own needs to convert any kind of MS-SQL tables to a XML file. Feel free to download it, use it, distribute it. FREE OF CHARGE ! Still be a BETA for the moment !! Thanks for your feedback This is the download area : https://rink.hockeyapp.net/apps/71fcf4452594429d8cf81e5f2569e8e8 Have fun Michel

1 0

A few general questions about BaseX
by Anastasiou A. 18 Sep '17

18 Sep '17

Hello everyone I am trying to load BaseX with a large number of XML files (~500), each one a few hundreds of MBs big. BaseX fails with a message along the lines "This is too big for one database". Can I please ask: 1) Are there any logs, beyond the DB logs? If yes, where can I find them? a. The reason I am asking is because once basexgui gives the message, there is no indication about the error. Ideally, I would like to know if this is a limitation on memory amount or number of items (?). 2) The parser options include reading XML files from archives, which is very convenient, but once the file has been parsed, does BaseX require the "originals" for queries / returning results? 3) Is it possible to do federation with BaseX? In other words, let's say I split a database in two large parts (as per #1), is it possible to launch two baseX servers and then have them talk to each other so that ultimately I just query one of them and get back unified results? All the best

5 5

retrieving the name of the archive?
by Graydon Saunders 18 Sep '17

18 Sep '17

Hello -- BaseX will happily consume zip archives; this is just splendid for loading up a bunch of docx files. Now I find myself wanting the name of the docx file -- the original name of the archive -- and I don't know how to retrieve that. (or if it can be!) But I think it must be there somewhere because db:path repeats the standard OOXML file paths: [Content_Types].xml word/document.xml word/footnotes.xml word/footer1.xml word/endnotes.xml word/theme/theme1.xml word/settings.xml docProps/custom.xml customXml/itemProps2.xml docProps/app.xml customXml/item2.xml customXml/itemProps1.xml word/fontTable.xml customXml/item1.xml customXml/item3.xml customXml/itemProps3.xml customXml/item4.xml customXml/itemProps4.xml word/numbering.xml word/styles.xml word/webSettings.xml docProps/core.xml word/people.xml over and over; if they were all going to exactly that there'd be one copy, and all several hundred docx files are there by content. (db:list-details tells me about > 4000 individual xml files.) If I can get the name of the original archive, how do I do that? Thanks! Graydon

2 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk