BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

locking problem
by Lizzi, Vincent 22 Oct '14

22 Oct '14

This is an update, and further problem, on a problem I posted here a few months ago. I've written a program to import XML documents into BaseX in one database, and populate a second database with a small amount of metadata about each document. This works but crashes every so often, and I have to restore the databases from backup and restart the import. To get through a large set of documents I now have the program create a backup (using the CREATE BACKUP command) after every 1000 documents imported. Since my earlier post, I found that the connection between the client and server was occasionally being dropped. I've switched to opening and closing a connection for each document instead of using one long-running connection. The client and server are running on the same machine. I'm using the latest version of BaseX (BaseX80-20141013.213520.zip). Here is the error message reported by my program when it crashes. The error occurs on the Query.execute() method of org.basex.examples.api.BaseXClient.java. Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.0 beta a91f5c2 Java: Oracle Corporation, 1.7.0_67 OS: Windows 7, amd64 Stack Trace: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source) at sun.nio.ch.SharedFileLockTable.add(Unknown Source) at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source) at org.basex.io.random.TableDiskAccess.lock(TableDiskAccess.java:139) at org.basex.io.random.TableDiskAccess.<init>(TableDiskAccess.java:87) at org.basex.data.DiskData.init(DiskData.java:119) at org.basex.data.DiskData.<init>(DiskData.java:80) at org.basex.core.cmd.Open.open(Open.java:70) at org.basex.core.cmd.Open.run(Open.java:36) at org.basex.core.Command.run(Command.java:360) at org.basex.core.Command.execute(Command.java:94) at org.basex.server.ClientListener.run(ClientListener.java:145) The server instance reports the error with this stack trace: C:\Programs\basex\basex\bin>basexhttp BaseX 8.0 beta a91f5c2 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) java.io.FileNotFoundException: C:\Programs\basex\basex\data\article_xml_meta\doc.basex (The requested operation cannot be performed on a file with a user-mapped section open ) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.<init>(Unknown Source) at java.io.FileOutputStream.<init>(Unknown Source) at org.basex.io.out.DataOutput.<init>(DataOutput.java:47) at org.basex.io.out.DataOutput.<init>(DataOutput.java:36) at org.basex.index.resource.Docs.write(Docs.java:73) at org.basex.index.resource.Resources.write(Resources.java:51) at org.basex.data.DiskData.write(DiskData.java:141) at org.basex.data.DiskData.flush(DiskData.java:249) at org.basex.data.DiskData.finishUpdate(DiskData.java:239) at org.basex.query.up.ContextModifier.apply(ContextModifier.java:124) at org.basex.query.up.Updates.apply(Updates.java:129) at org.basex.query.QueryContext.iter(QueryContext.java:351) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:80) at org.basex.server.ServerQuery.execute(ServerQuery.java:128) at org.basex.server.ClientListener.query(ClientListener.java:492) at org.basex.server.ClientListener.run(ClientListener.java:109) Here is the update statement where the error occurs, along with a bit of the surrounding Java. BaseXClient.Query queryIndex = client.query("declare variable $indexDb external; declare variable $hash external; declare variable $database external; declare variable $path external; let $p := <properties><database>{$database}</database><path>{$path}</path><sha1>{$hash}</sha1><loadedDate>{current-dateTime()}</loadedDate></properties> return db:replace('" + indexDatabaseName + "', $path, $p)"); try { queryIndex.bind("$indexDb", indexDatabaseName); queryIndex.bind("$database", managedDatabaseName); queryIndex.bind("$path", path); queryIndex.bind("$hash", hash); queryIndex.execute(); } finally { queryIndex.close(); } Previously this code used a variable for the first parameter of db:replace() but I changed it to "hard code" the database name into the query after reading about transactions and locking in the wiki. This change doesn't appear to have helped. To put this in context, the above update is run after the XML document is imported using BaseXClient.replace(). The next thing that happens in the program is an XQuery db:replace to add more properties for the document to the metadata database. I've added a Thread.sleep(300), to create a small delay between loading each document and this has helped the program to import more documents before crashing. Adjusting the delay doesn't seem to completely stop the crashes though. I'm not sure if this is a bug or improper use. Any help would be appreciated. Thanks, Vincent

2 2

Architecture Question
by Mansi Sheth 22 Oct '14

22 Oct '14

Hello, I have been going thru and comparing different Native XML Databases and so far I am liking BaseX. However, there are still a few questions unanswered, before I make a final choice: 1. I have 1000s of XML files (each between 50MB-400MB) and this is going to grow exponentially (~200 / per day). So, my question is how scalable is BaseX ? Can I configure it to use data from my external HDD, in my initial prototype ? 2. I plan to heavily use XPATH, for data retrieval. Does BaseX, use any multi-processing, multi-threading to speed up search ? Any concurrent processing ? 3. Can I do some post-processing on searched and retrieved data ? Like sorting, unique elements etc ? - Mansi

2 11

Using TcpCatcher as a proxy with Basex Client
by Arto Viitanen 20 Oct '14

20 Oct '14

I am debugging a Basex client built in C#. The client is based on the Git Hub example https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/c%23. The problem is, every now and then the client drops the session. It might be, that the client drops the connection if the server takes too much time to answer (or something). So I need some way to emulate slow connection or connection dropping packages. I downloaded TcpCatcher (http://www.tcpcatcher.org/) and set it as a proxy (client talks to 21984; TcpCatcher catches packages and send them to BaseX listening to 11984). But It seems that BaseX does not respond. I took BasexClient.cs and Example.cs from the github and tried with them. I get three packages; one coming from the server and two going to client; but after that nothing. The Example prints "Access Denied". I guess this comes from: socket = new TcpClient(host, port); stream = socket.GetStream(); ehost = host; string ts = Receive(); Send(username); Send(MD5(MD5(pw) + ts)); if (stream.ReadByte() != 0) { throw new IOException("Access denied."); } that is: client gets ts (timestamp?), sends username and password with timestamp; but receives nothing. So how can I use TcpCatcher (or other proxy program) with Basex client? (Basex 7.7; windows 7) Arto Viitanen Microteam Oy

3 3

Templating
by Emmanuel Chateau 18 Oct '14

18 Oct '14

Hi, I’ve been trying to use the templating proposed by Andy Bunce here http://cubeb.blogspot.fr/2012/11/xquery-templating-engines-and-txq.html https://mailman.uni-konstanz.de/pipermail/basex-talk/2012-November/004209.h… I’ve tried with txq module in different ways. But, I can’t figure out why I couldn’t reproduce it. Here is the simpler exemple I’ve tried : In the webapp folder an xml file, named `view1.xml` wich contains : ```xml <div>{map:get($map, 'greeting')}</div> ``` In webapp a restxq file with : ```xquery module namespace test = 'test.test'; import module namespace xquery = "http://basex.org/modules/xquery"; declare function test:render($template as xs:string, $map as map(*)){ xquery:invoke($template,$map) }; declare %restxq:path('/test') %restxq:GET function test:test(){ let $map := map{ "greeting" : "hello", "who": "world"} return test:render('view1.xml', $map ) }; ``` Arrêté à /Users/emmanuelchateau/Documents/basex/webapp/view1.xq, 1/15: [XPST0008] Undefined variable $map. I’m running BaseX version 7.9 Is there still anybody with this kind of templating running ? Thanks a lot for your help. Emmanuel

2 4

Web hosting
by spaelti 16 Oct '14

16 Oct '14

I have now tried BaseX web server on my own machine, and it works great. But what I would really like to do is to use it on my website. This website is on a commercial web provider. Is it possible to use BaseX in such a situation? How does one go about this? Would I need to ask the commercial provider to install BaseX? Or is there some other web site hosting method, that one can use? Thanks in advance for any help or suggestions. Philip Spaelti spaelti(a)mac.com

2 1

Fw: Suggestion for a requirement
by Sateesh 16 Oct '14

16 Oct '14

? Hi Basex Team, We are starting a new project in that we have selected to use basex,i would tell you my requirement,request you to please suggest the best way to use basex. We have started a web application in that we have a requirement to create xml db's per user based means each user who logs in we need to create DB's on his xml files in the backend,for this we dont want to use the client-server architecture of the basex as we dont want that extra time to hit a http server request to basex server and get the response,what we would want is to create the DB's execute queries and drop the DB's with in the same web application and also please note that we dont have a requirement to edit the DB's data we would only read the data from those DB's by executing the queries and also note that volumes of the xml files are very huge they might go up to gb's also and query output will also be in huge volume and also we must make sure that this huge volumes of query output will not eatup all the heap memory as the number of parallel users using this site are very high and we cannot afford to go out of memory.Also we would need to configure the DB paths as this are huge volumes we cannot store them in the default locations to. Considering all the above points request you to please suggest the best way to use basex. Our web application is developed in java.Though we have used basex in our earlier projects some where i feel that we did not used it properly and gone through problems and the kind of support we got from your team in solving them earlier was immense and this time we want to go in a more systematic way so request you to please help us by suggesting the best way. Thanks & Regards SAteesh.A The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.

4 8

Crashed BaseX import
by Arjan Loeffen 16 Oct '14

16 Oct '14

This is the report, bold is my logging steps in the process: *DELETE core-cbnl-20140901.xml* Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.7.2 Java: Oracle Corporation, 1.7.0_15 OS: Windows Server 2012, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 372 at org.basex.io.random.TableDiskAccess.flush(TableDiskAccess.java:124) at org.basex.data.DiskData.finishUpdate(DiskData.java:212) at org.basex.core.cmd.Add.run(Add.java:109) at org.basex.core.Command.run(Command.java:345) at org.basex.core.Command.exec(Command.java:321) at org.basex.core.Command.execute(Command.java:78) at org.basex.server.LocalSession.execute(LocalSession.java:122) at org.basex.server.LocalSession.execute(LocalSession.java:114) at org.basex.server.Session.execute(Session.java:50) at org.basex.http.rest.RESTCommand.run(RESTCommand.java:40) at org.basex.http.rest.RESTPost.run(RESTPost.java:108) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:14) at org.basex.http.BaseXServlet.service(BaseXServlet.java:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2430) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2419) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) * ADD "C:\Servers\xslweb\home\webapps\CBNL-browser-v2\work\contexts\core-cbnl-20140901.xml" CREATE INDEX ATTRIBUTE (2d) Database 'CBNL' is currently opened by another process. CREATE INDEX TEXT (2e) Database 'CBNL' is currently opened by another process. CREATE INDEX FULLTEXT (2f) * Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.7.2 Java: Oracle Corporation, 1.7.0_15 OS: Windows Server 2012, amd64 Stack Trace: java.lang.RuntimeException: Data Access out of bounds: - pre value: 0 - #used blocks: 0 - #total locks: 0 - access: -1 (0 > -1] at org.basex.util.Util.notexpected(Util.java:53) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:488) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:189) at org.basex.data.Data.kind(Data.java:282) at org.basex.query.value.node.DBNode.<init>(DBNode.java:53) at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:201) at org.basex.build.Builder.parse(Builder.java:73) at org.basex.build.DiskBuilder.build(DiskBuilder.java:90) at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:126) at org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) at org.basex.core.Command.run(Command.java:345) at org.basex.core.Command.exec(Command.java:321) at org.basex.core.Command.execute(Command.java:78) at org.basex.server.LocalSession.execute(LocalSession.java:122) at org.basex.server.LocalSession.execute(LocalSession.java:114) at org.basex.server.Session.execute(Session.java:50) at org.basex.http.rest.RESTCommand.run(RESTCommand.java:40) at org.basex.http.rest.RESTPost.run(RESTPost.java:108) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:14) at org.basex.http.BaseXServlet.service(BaseXServlet.java:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2430) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2419) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) *OPTIMIZE ALL * Any idea what's going on here? Hope this is enough info. -- Arjan Loeffen, Armatiek BV 06-12918997

2 2

Xpath gives multiple results, finding the one originating from the latest imported document?
by Sam Eklund 15 Oct '14

15 Oct '14

A quick question, I have this simple xquery to retrieve a node that sometimes has multiple hits, and for this reason I am using the function: head. This gives me the first node that matches the xpath, but is there a way to retrieve the node that originates from the latest imported document? My current query: xquery head (//event[eventID="12345ABC"]) // Sam

2 3

Question re starting basexhttp
by spaelti 12 Oct '14

12 Oct '14

I have been trying to run BaseX as a web application on my mac. After some difficulties I have gotten it to work. However there is one thing that I am still unsure about. Whenever I start the web server using ‘basexhttp’ I get the following message: /usr/local/bin/basexhttp: line 11: cd: /usr/local/Cellar/basex/7.9/libexec/../basex: No such file or directory However everything thereafter seems to work fine. So why do I get this message, or what does it mean? In my installation (which I got through the brew script) there is no ‘basex’ file/folder at the specified location (which would be in the ‘7.9’ folder). Should there be? Any help or advice would be much appreciated. best Philip This is what /usr/local/Cellar/basex looks like: ---- basex 7.9 bin changelog.txt INSTALL_RECEIPT.json libexec BaseX.jar bin lib webapp license.txt readme.txt ---- --------------------------------------------------- Philip Spaelti Kobe Shoin Women's University, Graduate school ---------------------------------------------------

4 3

zsh plugin
by Dirk Kirsten 12 Oct '14

12 Oct '14

Hello, if any of you uses the excellent zshell, you might be interested in this small plugin for BaseX for the zsh: https://github.com/dirkk/zsh-basex. Aliases might also be useful for every other shell user, because they basically shorten all basex commands (e.g. bx is defined as an alias for basex). Cheers, Dirk -- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk