BaseX-Talk

basex-talk@mailman.uni-konstanz.de

5251 discussions

Basex server ports
by Bram Vanroy 31 Aug '16

31 Aug '16

Hi all I am trying to initialise multiple server on a single machine. This will probably be possibly by listening to different ports. The question that I have is that I am not sure which values to change in the .basex file. Candidates are: PORT SERVERPORT EVENTPORT STOPPORT Which of these cannot be the same when running a BaseX server on the same machine? Thank you in advance Bram Vanroy

3 3

REST API performance, massive disk IO
by Braunschweig, Björn 29 Aug '16

29 Aug '16

Hello everybody, Im using the basexhttp docker image: root@koala:/tmp# docker images | grep basexhttp basex/basexhttp 8.4.1 54d1e0001449 3 months ago 628.1 MB On ubuntu 14.04 [1] and the python client library: <https://github.com/lucalianas/pyBaseX> https://github.com/lucalianas/pyBaseX I have a performance problem with the setup. The database gets filled up by up to 64 parallel clients using the REST api. Currently the database has a size of 2GB, 74899811 Nodes, 305756 Documents. For every document I add, I can see a lot of disk I/O on my host with up to 80MB/s [2] by the org.basex.BaseXHTTP class. The size of the xml documents cannot be the cause, they are < 10KB. The corresponding response time of the api is in the range of 10-30 seconds [3]. Furthermore the basex docker container log shows a HTTPException and a strange Creating Database... for every request [4]. strace of the basex process shows massive open/write calls [5]. What am I doing wrong? :-) Thx, Björn Braunschweig [1] root@koala:~# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=14.04 DISTRIB_CODENAME=trusty DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS" root@koala:~# uname -a Linux koala 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [2] accumulated disk io after a couple of seconds root@koala:/tmp# iotop -a -P Total DISK READ : 3.88 K/s | Total DISK WRITE : 53.39 M/s Actual DISK READ: 3.88 K/s | Actual DISK WRITE: 52.84 M/s PID PRIO USER DISK READ DISK WRITE> SWAPIN IO COMMAND 62061 be/4 1984 8.00 K 2.20 G 0.00 % 0.01 % java -cp /usr/src/basex/basex-api/target~.2.jar: -Xmx2048m org.basex.BaseXHTTP d [3] my application log: 18 seconds till response 2016-08-15 14:52:00,337 | dmp043 | 46912496392256 | INFO | mdqi | add_document | 241 | saving document: urn:diasid:ast:kop01:0201502181219395 340000/mets.xml to db: mets 2016-08-15 14:52:18,164 | dmp043 | 46912496392256 | INFO | mdqi | add_document | 243 | RESPONSE (status code 201): 0 resource(s) replaced in 62.67 ms. [5] docker logs basex Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: _ REQUEST _________________________________ Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: (PUT /rest/mets/urn:diasid:ast:kop01:0201502150401114120000/mets.xml)@843076472 <mailto:org.eclipse.jetty.server.Request@32405378> org.eclipse.jetty.server.Request@32405378 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Host: koala.gwdg.de:8984 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Accept: */* Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Authorization: Basic xxx Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Content-Length: 7558 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - User-Agent: python-requests/2.10.0 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Connection: keep-alive Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: - Accept-Encoding: gzip, deflate Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: _ RESPONSE ________________________________ Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: HTTP/1.1 201 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: Content-Type: text/plain;charset=UTF-8#015 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: #015 Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: Creating Database... Aug 12 13:44:21 koala docker/basex/ead004c4ce42[1167]: . 1.98 ms (458 MB) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: org.basex.http.HTTPException: 0 resource(s) replaced in 52.02 ms. Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.basex.http.HTTPCode.get(HTTPCode.java:57) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.basex.http.rest.RESTServlet.run(RESTServlet.java:31) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137 ) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java :231) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java :1086) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java: 193) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java: 1020) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135 ) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:1 16) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.Server.handle(Server.java:370) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpCo nnection.java:494) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnecti on.java:982) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(Abstr actHttpConnection.java:1043) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java :82) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint. java:696) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.j ava:53) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java: 608) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:5 43) Aug 12 13:44:22 koala docker/basex/ead004c4ce42[1167]: #011at java.lang.Thread.run(Thread.java:745) [4] root@koala:/tmp# strace -t -f -p 9675 -e trace=open -o /tmp/trace.log root@koala:/tmp# root@koala:/tmp# head trace.log 38813 14:23:12 open("/srv/BaseXData/mets/pth.basex", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 129 38797 14:23:12 open("/srv/BaseXData/mets/inf.basex", O_RDONLY) = 76 38797 14:23:12 open("/srv/BaseXData/mets/tbli.basex", O_RDONLY) = 76 38797 14:23:12 open("/srv/BaseXData/mets/tbl.basex", O_RDWR|O_CREAT, 0666) = 76 38797 14:23:12 open("/srv/BaseXData/mets/txt.basex", O_RDWR|O_CREAT, 0666) = 121 38797 14:23:12 open("/srv/BaseXData/mets/atv.basex", O_RDWR|O_CREAT, 0666) = 127 38797 14:23:12 open("/srv/BaseXData/mets/upd.basex", O_WRONLY|O_CREAT|O_EXCL, 0666) = 128 38797 14:23:12 open("/srv/BaseXData/mets/pth.basex", O_RDONLY) = 128 38797 14:23:12 open("/srv/BaseXData/mets/tbli.basex", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 128 38797 14:23:12 open("/srv/BaseXData/mets/inf.basex", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 128

4 4

Docker run basexhttp and permission issues
by Michael Seiferle 28 Aug '16

28 Aug '16

Hi BaseX-Talk, in case any of you is using our docker images, in recent builds there seems to be a bug related to the ENTRYPOINT and the PARENT mvn Image we are using. If you encounter the following error message (in fact you should if you docker pulled recently ;-)): > ⇒ docker run basex/basexhttp > touch: cannot touch ‘/root/.m2/copy_reference_file.log’: Permission denied > Can not write to /root/.m2/copy_reference_file.log. Wrong volume permissions? You might want to override the ENTRYPOINT yourself: > docker run --entrypoint=basexhttp basex/basexhttp I already proposed a fix but I’d like someone with docker-expertise have a look, maybe s/o might want to add her or his 5cts ;-) https://github.com/BaseXdb/basex/pull/1346 <https://github.com/BaseXdb/basex/pull/1346> Best Michael -- Michael Seiferle, BaseX GmbH, http://www.basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Fon: +49 7531 916 82 77

2 1

ignoring macos turds via webdav
by Kendall Shaw 19 Aug '16

19 Aug '16

Finder and /bin/cp leave turds wherever they go in some situations. If I use finder or cp in macos to write a file to a basex db mounted via webdav, the log shows something like: [propfind ] …/a.xml [put] …/a.xml [propfind] …/._a.xml [lock] .../a.xml [propfind] .../a.xml [propfind] …/._. [propfind] …/ Then finder complains “some data could not be read or written”, or cp complains “interrupted system call”. I imagine some sort of intermediate between Milton and basex could respond in a way that ignores the turds and responds as expected to macos. Assuming that it is the turds that are causing the problem, do you know of an existing server-side solution? Kendall

1 0

Strange Exception
by Christophe Marchand 19 Aug '16

19 Aug '16

Hello, I have a strange exception : Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.2.3 Java: Oracle Corporation, 1.8.0_91 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Data Access out of bounds: - pre value: 120241192 - #used blocks: 469694 - #total locks: 469770 - access: 469693 (469694 > 469693] at org.basex.util.Util.notExpected(Util.java:60) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:458) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:148) at org.basex.data.Data.kind(Data.java:306) at org.basex.query.value.node.DBNode.<init>(DBNode.java:51) at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:68) at org.basex.query.value.seq.DBNodeSeq.itemAt(DBNodeSeq.java:22) at org.basex.query.value.seq.Seq$1.next(Seq.java:77) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:58) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:36) at org.basex.query.expr.gflwor.For$1.next(For.java:79) at org.basex.query.expr.gflwor.GroupBy$1.init(GroupBy.java:131) at org.basex.query.expr.gflwor.GroupBy$1.next(GroupBy.java:98) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:174) at org.basex.query.expr.gflwor.For$1.next(For.java:99) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:95) at org.basex.query.expr.List$1.next(List.java:135) at org.basex.query.MainModule.cache(MainModule.java:91) at org.basex.query.QueryContext.iter(QueryContext.java:352) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:80) at org.basex.core.cmd.AQuery.query(AQuery.java:89) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.server.ClientListener.run(ClientListener.java:136) The XQuery is : for $fs in collection('FichiersBase')/forms group by $s := $fs/@supp return let $mv := max($fs/@omversion) for $f in $fs return if (empty($mv) or $f/@omversion >= $mv) then () else db:delete('FichiersBase', db:path($f)) ), db:optimize('FichiersBase') After this, trying a OPTIMIZE ALL on DATABASE FichiersBase, I have this exception : > OPEN FichiersBase Database 'FichiersBase' was opened in 744.68 ms. > OPTIMIZE ALL Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.2.3 Java: Oracle Corporation, 1.8.0_91 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 1 at org.basex.data.Namespaces.values(Namespaces.java:201) at org.basex.data.Data.namespaces(Data.java:450) at org.basex.io.serial.Serializer.node(Serializer.java:449) at org.basex.io.serial.Serializer.node(Serializer.java:176) at org.basex.io.serial.Serializer.serialize(Serializer.java:126) at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:202) at org.basex.build.Builder.parse(Builder.java:73) at org.basex.build.DiskBuilder.build(DiskBuilder.java:85) at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:131) at org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:47) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.server.ClientListener.run(ClientListener.java:136) Any idea ? Does this mean my database is corrupted ? Best, Christophe

2 3

Python API error
by Sayth Renshaw 19 Aug '16

19 Aug '16

I created an issue on github #1342 it has been closed until mailing list confirms its a bug On Windows and Fedora with Python 2.7 and 3.5 I cannot import Session from the BaseXClient >> import BaseXClient >>> from BaseXClient import Session Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name Session Not sure if there is an obvious thing I am missing I was following the example on github https://github.com/BaseXdb/basex/blob/master/basex-api/src/main/python/Quer… Is there something super simple missing? Sayth

1 0

Example concatenates iterable to database name
by Sayth Renshaw 19 Aug '16

19 Aug '16

Hi Running 8.5.3 on windows. Currently open database is RaceXML1. I am running the example from here http://docs.basex.org/wiki/Databases for $i in 1 to 100 return db:open('books' || $i)//book/title as for $i in 1 to 100 return db:open('RaceXML1' || $i)//meeting/race What occurs though is that the iterable is concatenated to the database name and subsequently not found. Error: Stopped at C:/Users/Sayth/Documents/firstquery.xq, 2/15: [bxerr:BXDB0002] Database 'RaceXML11' was not found. To new to xquery to know why this is wrong. Full error is Error: Stopped at C:/Users/Sayth/Documents/firstquery.xq, 2/15: [bxerr:BXDB0002] Database 'RaceXML11' was not found. Compiling: - pre-evaluating (1 to 100) - rewriting descendant-or-self step(s) Optimized Query: for $i_0 in (1 to 100) return db:open(concat("RaceXML1", $i_0))/descendant::meeting/race Query: for $i in 1 to 100 return db:open("RaceXML1" || $i)//meeting/race Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$i" id="0"/> <RangeSeq from="1" to="100"/> </For> <CachedPath> <DbOpen name="open(database[,path])"> <FnConcat name="concat(atom1,atom2[,...])"> <Str value="RaceXML1" type="xs:string"/> <VarRef> <Var name="$i" id="0"/> </VarRef> </FnConcat> </DbOpen> <IterStep axis="descendant" test="meeting"/> <IterStep axis="child" test="race"/> </CachedPath> </GFLWOR> </QueryPlan> Can you help? Sayth

2 7

XPath query with special char & fails
by Martin Manthey 19 Aug '16

19 Aug '16

Dear all, we use BaseX version 7.9 BaseX nodes and the XPath query of the getNodes()-method contains the char "&" and an exception is thrown. Example: //File[@originalLocation='E:\data\bugs\prom-9366\file & codemodel']/CopiedLocations/CopiedLocation If we use JDOM Nodes the query does not fail in this case. Why JDom can execute this query and BaseX cannot? Is there a general possibility to avoid this special char in a BaseX query, except masked it, or could it be solved in a higher version of BaseX? best regards

2 1

Using BaseXServer and BaseXHTTP at the same time?
by Kendall Shaw 18 Aug '16

18 Aug '16

Hi, Does the fact that basexhttp starts servers on 1984 and 8984 mean that I can have one program using client/server and another using rest using the same databases at the same time? Kendall

2 1

question on XML with prolog
by Wang, Genneva 18 Aug '16

18 Aug '16

Hi Christian, As I was testing some xml files, I realized that xml file with prolog, the prolog will always get removed when persisted in basex. Is that the expected behavior? Is there some option that the prologs can be saved along with the xml file ? Any tips would be greatly appreciated. Thanks! Regards, -Genneva

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk