- BaseX-Talk - mailman.uni-konstanz.de

Debugging jar loading (Java interface)
by Charles Duffy 21 Mar '12

21 Mar '12

Howdy -- I'm trying to include (and use) a jar in a package, and not having any luck of it just yet. Could I get any guidance on what I might be missing here? === BEGIN expath-pkg.xml <package xmlns="http://expath.org/ns/pkg" spec="1.0" name="http://dyfis.net/svnkit-wrapper" abbrev="svnkit" version="0.1"> <title>Subversion Wrapper</title> <xquery> <namespace>http://dyfis.net/svnkit-wrapper</namespace> <file>svn.xq</file> </xquery> </package> === END expath-pkg.xml === BEGIN basex.xml <package xmlns="http://expath.org/ns/pkg"> <jar>svnkit-wrapper.jar</jar> <class>net.dyfis.svnkit_wrapper.SvnWrapper</class> </package> === END basex.xml === BEGIN svnkit/svn.xq module namespace svnkit = "http://dyfis.net/svnkit-wrapper"; declare namespace j="java:net.dyfis.svnkit_wrapper.SvnWrapper"; declare function svnkit:cat($path as xs:string) as xs:string { j:cat($path) }; declare function svnkit:cat($path as xs:string, $rev as xs:integer) as xs:string { j:cat($path, $rev) }; === END svnkit/svn.xq ...where "svnkit-wrapper.jar" contains a thin wrapper around TMate's SVNKit library, allowing direct access to Subversion resources through a simpler API (providing static functions only, abstracting away session management and the like). However, when I try to call svnkit:cat($subversion_url), the following is returned: [XPST0017] Unknown Java function 'net.dyfis.svnkit_wrapper.SvnWrapper.cat(...)'. I can call net.dyfis.svnkit.wrapper.SvnWrapper("") through Jython and other JVM-based interpreters, so I'm confident that the jar correctly exposes it. Any pointers on what I might be doing wrong in the packaging?

1 2

Re: [basex-talk] Use of unsalted password hashes
by Dimitar Popov 18 Mar '12

18 Mar '12

Am Sonntag, 18. März 2012, 12:16:48 schrieb Charles Duffy: > On Sun, Mar 18, 2012 at 10:16 AM, Dimitar Popov < > Dimitar.Popov(a)uni-konstanz.de> wrote: > > > Am Sonntag, 18. März 2012, 09:08:36 schrieb Charles Duffy: > > > Caching accepted username/password pairs in RAM, and maintaining a > > > separate pool using the credentials for each such pair, would _work_, > > > but it would mean that you'd have to be careful not to let the memory > > > that information is stored in get swapped to disk -- writing users' > > > passwords to disk, even by way of the OS's swap operations, being a > > > serious no-no. > > > > The pairs username/password can be, of course, hashed and additionally, > > swap > > partitions can be encrypted, if such a level of security is required. > > > > Hashing could work, but adds a requirement that only basic auth be > supported. Given the number of other credential-proving mechanisms in the > world (Kerberos, NTLM, etc), this seems a very unfortunate limitation to > place on future development. > > As for encrypted swap -- in a typical commercial environment, that's > outside of the dev team's control unless they bother to do a great deal of > paperwork and politics; in short, not an acceptable solution. > > AFAIK, JCA provides a keystore mechanism, which I guess, provides enough > > means > > of protection against steal. I'm, however, too lazy to thoroughly read the > > JCA > > guide [1], but the first which comes to my mind is for example, external > > storage of keys. > > > > Again, to make that keystore secure, you're creating requirements that the > typical user won't implement. Better to design things to be secure > out-of-the-box, rather than "secure if only you do A, B, C and D". Well, then it seems after all, that the token alternative provides the highest degree of flexibility and security :) I guess, if there are no other suggestions, it should be discussed how to implement it. Regards, Dimitar

1 0

Use of unsalted password hashes
by Charles Duffy 18 Mar '12

18 Mar '12

Howdy -- I'm a bit concerned about storing password hashes unsalted -- compared to a salted hash, this makes a stolen database easier to retrieve user passwords from, dangerous if users use their passwords for other purposes as well. Would salting (and perhaps stretching) the hashes be considered a reasonable feature to support in the future?

4 16

Setting the context item via the native protocol
by Charles Duffy 17 Mar '12

17 Mar '12

Howdy -- Right now, I'm aware of two ways to set the context item via the native protocol: (1) Insert a "declare context item" header into the document - Disadvantage: Should only work when XQUERY3 is true - Disadvantage: Modifies line/column references for errors received - Disadvantage: Hard to ensure that user-provided XML document will evaluate to itself in XQuery (security risk) (2) Use the "CS" ["Context Set"] command - Disadvantage: Requires a database to be open (why?) - Disadvantage: Hard to ensure that user-provided XML document will evaluate to itself in XQuery (security risk) Is there a better approach?

2 8

REPOPATH prefixed twice on current snapshots?
by Charles Duffy 17 Mar '12

17 Mar '12

In my .basex file: REPOPATH = /home/cduffy/local/opt/basex-data/repo ...and running repo:list() shows my local package, with namespace http://dyfis.net/ as installed, but trying to use anything from it leads to an error similar to the following: releng.db.basex.rest.RemoteException: [XQST0059] Could not retrieve module '/home/cduffy/local/opt/basex-data/home/cduffy/local/opt/basex-data/repo/http-dyfis.net-0.1.140101/dyfis/reporting.xq'. This is happening with a local build of 230e16551cf397def9e6c59044ff605e48908e1c (current master on github).

2 8

after upgrading one big disaster
by jidanni＠jidanni.org 17 Mar '12

17 Mar '12

All I know back in December the Makefile and programs in http://radioscanningtw.jidanni.org/images/radioscanningtw/maps/ worked fine. Now with BaseX 7.1.1 it is one big disaster and will take days to rewrite.

3 6

Any way to persist OutputStream in DiskData?
by Dave Glick 16 Mar '12

16 Mar '12

Hi, I've been doing some profiling recently and noticed one area in particular that appears to be using a lot of cycles. In calls to DiskData.write() the final DataOutput.close() call results in 20% - 40% of overall execution time in a variety of cases. The following screen capture shows the aggregated call stack (though it might look a little funny since this is the cross-compiled version used for Nxdb). Admittedly, this is probably an issue that just impacts me. Based on the trace, I suspect the issue is related to the releasing of handles rather than the actual disk writes (though I could be wrong - I'm not entirely sure what the cross-compiler does behind the scenes to support Java file descriptors). In any case, my question boils down to wondering if there is a way to force the DiskData class to use a persistent DataOutput instance instead of creating and closing a new one for every write? Instead, could a DataOutput be kept open until the database is "unpinned" and simply flushed on each write? Would this cause any problems with the current GUI, server, etc. uses? Thanks, Dave

3 4

fulltext index: case sensitive vs. case insensitive
by Cerstin Mahlow 16 Mar '12

16 Mar '12

Hi, If I create the fulltext index with "stemming" switched on and "case sensitive" switched off, there is no way to enable case sensitive search, is this correct? So if users would like to search case sensitive as well as case insensitive, I would have to create two indices? Which would mean creating actually four indices to allow for fuzzy search or exact matches as well: w/ stemming and w/ case sensitive w/ stemming and w/o case sensitive w/o stemming and w/ case sensitive w/o stemming and w/o case sensitve. That's quite a lot of data! Would it be possible to handle case sensitivity rather as an option for the query than as an option for the index, like described here: <http://www.xmlmind.com/tutorials/XQueryFullText/index.html#d0e333> Best regards Cerstin -- Dr. phil. Cerstin Mahlow Universität Basel Deutsches Seminar Nadelberg 4 4051 Basel Schweiz Tel: +41 61 267 07 65 Fax: +41 61 267 34 40 Mail: cerstin.mahlow(a)unibas.ch Web: http://www.oldphras.net ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

Delete attributes
by William P. Heise 16 Mar '12

16 Mar '12

How do I delete attributes? (and just the attribute, not the node) Motivation: I have a db where 20% of the attributes is the same, 'title', which is never used. Deleting it could decrease the size of the db. /William ________________________________ Please consider the environment before printing. </pre> <div style="FONT-FAMILY: arial;color:#525759;font-size:9pt; margin-top:15px">This e-mail and any attachments are intended only for the named recipient(s) and may contain information that is legally privileged, confidential, or exempt from disclosure under applicable law. This message may be logged for archival purposes, may be reviewed by parties at Deltek other than those named in the message header, and may not necessarily constitute an official representation of Deltek. If you have received this message in error, or are not the named recipient(s), you may not retain copy or use this e-mail or any attachment for any purpose or disclose all or any part of the contents to any other person. Any such dissemination, distribution or copying of this e-mail or its attachments is strictly prohibited. Please immediately notify the sender and permanently delete this e-mail and any attachment from your computer.</div>

2 2

REPLACE fails when root has attributes
by Charles Duffy 15 Mar '12

15 Mar '12

> REPLACE inventory <root foo="bar">test</root> Stopped at line 1, column 24: Syntax: REPLACE [path] [input] Replace documents in database. Replaces documents in the current database.

1 1

ADD:REPLACE :: STORE:?
by Charles Duffy 15 Mar '12

15 Mar '12

Restated: "ADD is to REPLACE as STORE is to... what?" REPLACE, like ADD, expects the content given to be parsed rather than raw. ADDing the same content twice can result in two documents with the same name in the database; I'm presuming that duplicate STORE commands have similar peril attached. What is the appropriate mechanism to use when replacing one version of raw content with another?

2 1

nesting queries
by Cerstin Mahlow 15 Mar '12

15 Mar '12

Hi, I would like to have a nested xquery, but I am not sure if this is possible at all: When a user searches the corpus, he can save the hits he likes most. The result for the query is a form with checkboxes. The node IDs together with the query and a time stamp are saved in a second collection. This collections looks like this: <collection> <entry time="2012-03-04T17:24:53"> <node>1434455</node> <query>[text() contains text ('Daumen' ftand 'Auge' ftand 'halten') using stemming using language "de" distance at most 10 words ordered]</query> <person>xyz</person> <phraseme>Ad0024</phraseme> <selected>yes</selected> </entry> <entry time="2012-03-04T17:24:53"> <node>1475764</node> <query>[text() contains text ('Daumen' ftand 'Auge' ftand 'halten') using stemming using language "de" distance at most 10 words ordered]</query> <person>abc</person> <phraseme>Ad0024</phraseme> <selected>no</selected> </entry> </collection> When the user searches again with a different query, he will see the results as for the first search and is able to save only those that are not in the hit-collection. This is controlled by this part in the "return" part of the xquery in my perl client: { if (not (db:open("collect")//node[text() = \$node and ../phraseme[text() = "$phraseme"] and ../selected[text() = "yes"]])) then <input type="checkbox" name="NODE" value="{\node}"/> else "X" } Already saved hits get an "X" and new hits get the checkbox. For documentation, I would like to collect all queries a node could be found with. Thus, instead of printing "X", I would like to add an additional query-element to the entry-element, i.e., something like insert node <query>$query</query> after /collection/entry/node[text() = "\$node" and ../phraseme[text() = "$phraseme" and ../selected[text() = "yes"]]] to get: <collection> <entry time="2012-03-04T17:24:53"> <node>1434455</node> <query>[text() contains text ('Daumen' ftand 'Auge' ftand 'halten') using stemming using language "de" distance at most 10 words ordered]</query> <query>[text() contains text ('Daumen' ftand 'Auge') using stemming using language "de" distance at most 10 words ordered]</query> <query>[text() contains text ('Daumen' ftand 'Auge') distance at most 10 words ordered]</query> <person>xyz</person> <phraseme>Ad0024</phraseme> <selected>yes</selected> </entry> <entry time="2012-03-04T17:24:53"> <node>1475764</node> <query>[text() contains text ('Daumen' ftand 'Auge' ftand 'halten') using stemming using language "de" distance at most 10 words ordered]</query> <person>abc</person> <phraseme>Ad0024</phraseme> <selected>no</selected> </entry> </collection> However, I am not sure if it is possible at all to mix displaying (in the "then" part) and updating (in the "else" part) -- although I would like to update a different collection than I am searching. If it is possible, I guess I have to include the name of the collection for insertion by using db:open? Is it necessary to specify the exact place for insertion (as in my example) as this should be already determined by the condition? Best regards Cerstin -- Dr. phil. Cerstin Mahlow Universität Basel Deutsches Seminar Nadelberg 4 4051 Basel Schweiz Tel: +41 61 267 07 65 Fax: +41 61 267 34 40 Mail: cerstin.mahlow(a)unibas.ch Web: http://www.oldphras.net

2 2

Position information on errors for queries submitted via REST
by Charles Duffy 14 Mar '12

14 Mar '12

Howdy -- The GUI's feedback on errors, including line number and position, is an extremely useful feature. Is this information also available via the REST interface? If so, how is it enabled? Thanks!

2 5

BaseX: New XQuery 3.0 Features
by Christian Grün 14 Mar '12

14 Mar '12

Dear all, we have now added documentation to function items [1] and higher order functions [2] in our Wiki. These features are being introduced with XQuery 3.0, and they provide much more powerful ways to write real functional code with XQuery. We dare say that BaseX currently offers the most complete and up-to-date support for these new features! Thanks to our team member Leo, we will soon be able to offer the last pending 3.0 features, such as general flwor expressions, which allow an arbitrary order of FLWOR clauses (fot, let, order, group by, count, etc.) [3] By the way: BaseX 7.2 will be released in about 10 days. Stay tuned.. Christian [1] http://docs.basex.org/wiki/XQuery_3.0#Function_Items [2] http://docs.basex.org/wiki/Higher-Order_Functions [3] http://www.w3.org/TR/xquery-30/ ___________________________ BaseX Team Christian Grün Uni KN, Box 188 D-78457 Konstanz http://www.basex.org

3 5

Bind optional variables and/or bind node
by Ronny Möbius 14 Mar '12

14 Mar '12

Hello, I want to query a BaseX XML Database via Python (BaseXClient.py). Therefore I write a XQuery script, which executes a FLOWR expression combined with a few self-written functions. A bunch of options, which are generated within my Python script, have to externally bind to this query. For convenience it would be appropriate to have optional external variables which I do not have necessarily to bind, but which will have default values specified in the XQuery script. Is there a possibility to do this? I thought about an alternative, that is to declare the options with the help of a XML snippet. But then I could not figure out how to bind this snippet as a node instead of a string, because XQuery itself does not have a function to convert a String containing a XML document to a node. Of course I could create before each request a new options document in the database, but this seems to be a bit unhandy. All in all I have the impression, that those are quite easy problems, but for some reason I could not find any solution on the web, so I were glad about reading a solution to one of those problems. All the best, Ronny

3 3

full-text search with wildcard
by Shakila Shayan 13 Mar '12

13 Mar '12

Hello all, We are using Basex 7.0.2 and using wildcard for full-text search we ran into some problems when it comes to tokenization related issues. Our database contains these entries: bb (aa)bb bb(cc) (aa)bb(cc) We ran a test as following with the given results shown in each case: 1- .//value[text() contains text {'.*(bb)'} using wildcards] returned (aa)bb and (aa)bb(cc) 2- .//value[text() contains text {'.(bb).*'} using wildcards] returned bb(cc) and (aa)bb(cc) 3- .//value[text() contains text {'(bb)'} using wildcards] returned (aa)bb and (aa)bb(cc) and bb(cc) and bb so far so good, but the following case is the weird case: 4- .//value[text() contains text {'.*(bb).*'} using wildcards] returning only (aa)bb(cc) Can anyone explain why is the behavior of the last case different? Whereas it should be the most general case , it turns out to be the most exclusive one ? Are we missing something or is it a bug?

2 1

BaseX 7.1.1 WAR problems: Permissions and other issues
by pw＠themail.co.uk 12 Mar '12

12 Mar '12

Hello I am having the same problems with the WAR version of 7.1.1 as described for 7.0 earlier { https://mailman.uni-konstanz.de/pipermail/basex-talk/2011-October/0020 28.html } There is duplication of the servlet.jar that prevents it from starting. Then after deletion of the servlet.jar, there are problems with permissions (I don't know the reason for this) I would appreciate help in getting the WAR version working both with Tomcat and with Oxygen. Can anyone provide details of how to remedy the situation with BaseX711.war and also an 'idiots guide' on how to sort out permissions, connect Oxygen to the database, and access via webDAV and rest interfaces.? Many thanks Peter

3 2

say how to get "newline" working
by jidanni＠jidanni.org 11 Mar '12

11 Mar '12

On http://docs.basex.org/wiki/Serialization we see newline. But we have no idea how to make it work. Please add an example to that page! declare option output:method "text"; declare option output:newline "\n"; for $m in (1,2,3) return $m Nope no idea. basex: Installed: 7.1.1-2

7 20

Performance and the capital of Albania
by Andy Bunce 09 Mar '12

09 Mar '12

Using the factbook.xml database and a recent 7.1.2 beta, I am hitting some performance problems. To debug I have been trying some expressions in the GUI window. The goal is given a country id, get the capital. The examples use Albania ="f0_136" let $id:="f0_136" let $cap:=//country[@id=$id]/@capital return //city[@id=$cap] ------result------------ <city id="f0_1461" country="f0_136" longitude="10.7" latitude="46.2"> <name>Tirane</name> <population year="87">192000</population> </city> *Time: 0.7ms * Very nice. Sadly my code is not doing it like this! ( the xpath is being generated from some data description xml I have created) If I try in the GUI some Xpath variations they produce wildly different execution times let $id:="f0_136" return //city[@id=(//country[@id=$id]/@capital)] *26144ms* let $id:="f0_136" return //city[@id=//country[@id=$id]/@capital] *25157ms* let $id:="f0_136" return (//city)[@id=(//country[@id=$id]/@capital)] *550ms* let $id:="f0_136" return (//city)[@id=((//country)[@id=$id]/@capital)] *6087ms* I was expecting all these to perform about the same. They all produce the same result. /Andy

2 3

Gets name of collection
by Anh Thuấn 09 Mar '12

09 Mar '12

Hi BaseX team, I 've met a problem of getting the name of the collection when retrievingXQuery. I am looking for a node satisfying a certain condition on multiple collections (each collection is a database of BaseX), when it returns a result, I want to know the name of that collection. For example, here is my query: for $node in (Collection("Database1"),Collection(Database2"))/a[@attribute = "name"] return $node // I want to get of the name of the collection of the returned node here. I don't know if BaseX supports a certain function to get the name of the collection from a node or something similar. Or does it has a way to know the node belongs to which collection?

2 1

Hint for documentation of command: legal database names
by Andreas Rulle 06 Mar '12

06 Mar '12

Dear BaseX team, in this mail you receive a small suggestion for the command documentation http://docs.basex.org/wiki/Commands#CREATE_DATABASE The comment from the source code, see the class org.basex.core.Command and its method validName, "allows only letters, digits, the underscore, and dash." could be added as information about legal characters for database names to the page http://docs.basex.org/wiki/Commands#CREATE_DATABASE . Okay it is a wiki and therefore could be editited directly, but I want to inform the team about this suggetion first. Perhaps the page could have a paragraph "legal identifiers". With best regards Andreas

3 2

Query
by chausel＠arcor.de 06 Mar '12

06 Mar '12

Hallo! gibt es eine Möglichkeit in der GUI eine Query so zu formulieren, dass als Ergebnis *nicht* so was rauskommt: <xml>Das was ich will</xml> sondern so was: Das was ich will Viele Grüße Carsten

2 1

Importing two modules from the same package -> "Module is declared twice"
by Charles Duffy 05 Mar '12

05 Mar '12

Howdy -- please see the following example: === expath-pkg.xml <?xml version="1.0"?> <package xmlns="http://expath.org/ns/pkg" spec="1.0" name=" http://dyfis.net/example/ns-test" abbrev="ns-test" version="0.1.136892+build.20120224.105428"> <title>Namespace Trouble Test</title> <xquery> <namespace>http://dyfis.net/example/ns-test/foo</namespace> <file>foo.xq</file> </xquery> <xquery> <namespace>http://dyfis.net/example/ns-test/bar</namespace> <file>bar.xq</file> </xquery> </package> === ns-test/foo.xq module namespace foo = "http://dyfis.net/example/ns-test/foo"; declare function foo:test() { 1 + 1 }; === ns-test/bar.xq module namespace bar = "http://dyfis.net/example/ns-test/bar"; declare function bar:test() { 1 + 1 }; === ...in the basex-gui buffer... import module namespace foo="http://dyfis.net/example/ns-test/foo"; import module namespace bar="http://dyfis.net/example/ns-test/bar"; 1+1 === ...results in... Error: [XQST0047] Module is declared twice: ' http://dyfis.net/example/ns-test/bar'.

2 2

db:list-details() @modified-date not valid xs:dateTime?
by Charles Duffy 05 Mar '12

05 Mar '12

Howdy -- I note that the modified-date attribute given in db:list-details() is not a valid xs:dateTime value (being "DD.MM.YYYY HH:MM:SS" rather than "YYYY- MM-DDTHH:MM:SS"). Would a patch correcting this be accepted, or is its current form considered part of the public API? Thanks!

2 2

CHOP=false in a POST request ?
by Maud Ingarao 05 Mar '12

05 Mar '12

Hello, I need BaseX not to chop whitespaces while creating DBs out of my XML texts. I found this post : https://mailman.uni-konstanz.de/pipermail/basex-talk/2010-April/000213.html and this documentation : http://docs.basex.org/wiki/Options#CHOP But could someone please show me how to write it as a REST POST request ? I looked at examples here http://docs.basex.org/wiki/REST#Command_Line but always get a "Nodes must belong to "http://www.basex.org/rest"" error... Thanks a lot !! Maud -- Maud Ingarao IHPC (UMR 5037) Institut d’histoire de la pensée classique, de l’Humanisme aux Lumières Ecole Normale Supérieure de Lyon - Site Descartes - Bureau R322 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07 +33 4 37 37 65 79 - maud.ingarao(a)ens-lyon.fr cerphi.ens-lyon.fr - pensee-classique.ens-lyon.fr institutdesanti.ens-lyon.fr - hyperdonat.ens-lyon.fr mutec-shs.fr - projet-plume.org - ahn.ens-lyon.fr http://www.cahier.paris-sorbonne.fr/ - je suis absente le vendredi - out of office on Fridays -

3 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk