BaseX-Talk

basex-talk@mailman.uni-konstanz.de

9 participants
5241 discussions

Question regards "add path"; does it break existing indexes?
by Sandra Maria Silcot 25 Aug '10

25 Aug '10

Hi all, Using Basex v6.2.3 jar under linux, sun jdk1.6.0_18. I have a database named 'linkdb' with a few documents and a full set of indexes built. I would like to be able to add/delete docs and simply optimise the indexes without needing to rebuild them from scratch. But I am having problems doing so. Before adding a document, the "info" command reports: Database Properties Name: linkdb Size: 29338 KB Nodes: 1085033 Height: 6 Database Creation ... Documents: 3 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON All is well. I then add another document and I can access the new document via xquery, but now, even after an "optimize", "info db" reports: Database Properties Name: linkdb Size: 29414 KB Nodes: 1088588 Height: 6 Database Creation ... Documents: 4 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: OFF Attribute Index: OFF Full-Text Index: OFF So what has just happened to the text, attribute and fulltext indexes? The only process which seems to work is to completely rebuild them, like so: LOGIN admin OK SET pathindex on OK 0.19 ms SET textindex on OK 0.13 ms SET attrindex on OK 0.14 ms SET ftindex on OK 0.13 ms SET wildcards on OK 0.14 ms SET casesens off OK 0.13 ms SET scoring 2 OK 0.13 ms OPEN linkdb OK 0.69 ms CREATE INDEX TEXT OK 424.54 ms CREATE INDEX PATH OK 90.74 ms CREATE INDEX ATTRIBUTE OK 344.46 ms CREATE INDEX FULLTEXT OK 996.65 ms OPTIMIZE OK 1784.94 ms CLOSE OK 0.18 ms And now: Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: ON (wildcards) In the .basex file in the home dir of the user the basex server is run as, I attempt to set global options like so: TEXTINDEX = ON ATTRINDEX = ON FTINDEX = ON WILDCARDS = ON CASESENS = OFF SCORING = 2 but this seems to be always ignored. What am I doing wrong, or is this the intended behaviour? Where does the ".basex" file need to be to take effect as a set of defaults. The behaviour is repeatable irrespective of whether I use the java client or run the commands via a perl script. Many thanks for any advice, Cheers, Sandra.

2 1

defining external variables when using BaseX in server mode
by Godmar Back 25 Aug '10

25 Aug '10

Hi, is it possible to create bindings for external variables when using BaseX's server mode? For instance, we have a query that declares: declare variable $id as xs:string external; When using XQJ, 'id' would be bound as bindNode(). In our server-based implementation, we manually substitute all occurrences of $id with its value before running the query. Obviously, this is not robust: for instance, occurrences of $id in string literals would be substituted as well, even when they should not (e.g., "$id") Is there a way to leave $id declared as external, but assign a value for it when using BaseX's server mode? - Godmar

1 0

note on BaseXClient.java
by Godmar Back 23 Aug '10

23 Aug '10

Hi, The example file BaseXClient uses an unbuffered stream to write to the socket: out = socket.getOutputStream(); void send(final String s) throws IOException { for(final byte t : s.getBytes()) out.write(t); out.write(0); } This yields extremely bad performance, especially for large queries (every single call to write will trigger a separate system call to the OS, which in turn will trigger a separate TCP segment on the wire). On a typical Ethernet, this means that 100 minimum sized packets are transmitted rather than a single packet to transmit a 100-byte query. On a different note, why is BaseXClient not part of the "official" BaseX API, or at least provided as part of the distributed .jar files in an org.basex.* package? - Godmar

3 3

exception when interrupting BaseX server
by Godmar Back 23 Aug '10

23 Aug '10

When I interrupt BaseXServer with SIGINT, I occasionally see backtraces like this one. - Godmar Starting BaseXServer on port 1985 BaseX 6.2.3 [Server] Server was started. java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178) java.io.IOException: Stream closed at sun.nio.cs.StreamEncoder.ensureOpen(StreamEncoder.java:26) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:99) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203) at java.io.Writer.write(Writer.java:140) at org.basex.server.Log.write(Log.java:76) at org.basex.server.ServerProcess.run(ServerProcess.java:178)

2 2

Bulk Loading XML documents with Add(docPath).execute(CONTEXT)
by pw＠themail.co.uk 22 Aug '10

22 Aug '10

When adding documents from a directory I get things working using a few small documents and Add(docPath).execute(CONTEXT) but when I want to add hundreds of documents (average size 1Mb) I get a memory error. I am using Jython and have -J-Xms1200M and -J-Xmx1200m should I use an iterator and only add one document at a time? [this doesn't seem to me to be the way to use Add() ] TIA Peter

2 1

how do I embed character data in XQuerys?
by Godmar Back 22 Aug '10

22 Aug '10

Hi, following Christian's insinuation we're now abandoning XQJ and are porting our application to use the ClientSession API. I'm performing a textual replacement of the external variables in my XQuery expressions before sending them to BaseX's server. I'm facing difficulties with embedding character data in the XQuery expressions that are sent to the server. Notably: xquery <c-code>if (cond) { // a C comment }</c-code> produces: Stopped at line 1, column 33: [XPST0003] Expecting "}", found "C". whereas: xquery <c-code><![CDATA[ if (cond) { // a C comment } ]]></c-code> produces: <c-code> if (cond) { // a C comment } </c-code> Is this a bug in BaseX? I thought that the use of CDATA is optional and required only if one wishes to avoid escaping <, >, & in PCDATA. If CDATA is required, I would have to convert my XML document in this way. The problem did not occur in the XQJ binding. I haven't consulted the grammar; I'm following my intuition here that XQuery should allow embedded XML fragments. Is this not so? - Godmar

3 7

Student project based on BaseX
by Antonis Thomas 22 Aug '10

22 Aug '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, -- Antonis Thomas Information & Computing Sciences Utrecht University Utrecht, The Netherlands

2 13

how to separate multiple results when using BaseX's client/server interface
by Godmar Back 19 Aug '10

19 Aug '10

Hi, when I use the client/server interface and run a query that returns multiple nodes, BaseX appears to return the results concatenated (and, somewhat oddly, with interspersed newlines). For instance, xquery declare namespace atom='http://www.w3.org/2005/Atom'; doc('myfeed')//atom:title returns: <title xmlns="http://www.w3.org/2005/Atom">First Title</title> <title xmlns="http://www.w3.org/2005/Atom">Second Title</title> <title xmlns="http://www.w3.org/2005/Atom">Third Title</title> .... Does anybody happen to know how to parse those into separate XML documents in Java? (Short of preprocessing the XML to find the matching end tag for each element?) For background, classes such as javax.xml.parsers.DocumentBuilder can only parse one document at a time, and throw an error if the input source contains multiple documents/fragments. Alternatively, would it make sense for BaseX to support returning results one by one (perhaps using a cursor approach) to ease the client's task of parsing the returned XML? - Godmar

3 3

Re: [basex-talk] Error while creating an attribute index with the latest from SVN
by Huib Verweij 19 Aug '10

19 Aug '10

Hi Christian, Op 19 aug 2010, om 13:18 heeft Christian Grün het volgende geschreven: > The stack trace indicates that the merge of several temporary index > instances causes problems. Regarding the SVN history, the current > index builder shouldn't have been changed that much since… 6.1.8? As a > shortcut, it might help to give the JVM different -Xmx values; 1gb > should be completely more than enough for 780 mb of input data. In the > GUI, you should be able to observe how much RAM is actually used > during index construction. That was it. 1Gb didn't work, but 2Gb was enough to get it done in about 50 seconds. > >> I don't get anything else, if there's something I can do to debug it let me know. > > It might help if you could provide us with the original data (which > will of course be handled confidentially). It might take some time, > however, as I'm currently busy writing my thesis… I can't supply you with the original data I'm afraid. That would breach my contract with the MPI. Anyway, thanks for your help again and good luck with the thesis. Huib.

1 0

How is transaction supported in BaseX?
by 汤力丞 18 Aug '10

18 Aug '10

Recently I choosed BaseX as our cybermarketing data store. The most appealing feature for us is ACID transaction support as BaseX claims while eXist supports only invisible transactions limited to crash recovery. But I failed to find any example or command related such as ROLLBACK or COMMIT. So how can I fully utilize this important feature? Thanks! Best regards, Michael Tang

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk