BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5269 discussions

my first question: how to deal with the duplicated files in basex?
by easy 11 Nov '13

11 Nov '13

Hi,all, I found basex db can add same file in same db and collection many times, when I use doc('') func, there will be a error: database paths ..... pointed more than one document. how to avoid this?? -- 此致 easy 莫愁前路无知己，天下谁人不识君。

2 1

Feature request: output params for db:replace
by France Baril 09 Nov '13

09 Nov '13

Hi, I have a feature request. I have a nice file with just the correct indent, that I was able to create by adding the xml:space="preserve" attribute to the root before making my inserts. [From previous thread] Ex. <block xml:space="preserve"> <block>This is a test sentence.</block> </block> Note: I can't use the output indent = no declaration at the top of my query because I'm in a library module, and I need this function to stay there for sharing. Now, I can't save the content like this (with @xml:space). My DTD doesn't allow for xml:space="preserve", and I don't want it to. Authors can't get the correct indent (based on our DTD) in the editor as long as the xml:space="preserve" remains. If I apply: *let* *$clean-file* := *copy* *$copy* := *$new* *modify* (*delete* *node* *$copy*/*/* @xml:space*) *return **$copy* file:write('file.xml', $clean-file, $param), where param contains indent="no". It saves correctly, indicating that removing the @xml:space did not cause a the insertion of an indent space between and . However, the db:replace('db', 'path', $clean-file) function forces an indent on the content between the and the . Since this is a BaseX function, and not an XQuery fonction, I'm hoping we could get db:replace('db', 'path', $clean-file, $param), where param can include output params such as indent=no. Thank you! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 2

Re: [basex-talk] Writeback option
by Christian Grün 09 Nov '13

09 Nov '13

…I’ve made it an GitHub issue: https://github.com/BaseXdb/basex/issues/786 On Fri, Nov 8, 2013 at 5:31 PM, Christian Grün <christian.gruen(a)gmail.com> wrote: >> - Concerning your comment on using the WRITEBACK option only for single >> file data bases: debugging the basex server yields that when using more >> than one xml document (via the ADD command), the "original" field in the >> MetaData class is not set, thus the WRITEBACK operation does not write >> anything in these situations. > > …exactly. The original URI of added resources will not be stored in > the database, and the optional "original" string you mentioned refers > to the path of a single resource, which was added in the database > creation process. > >> - When raw files reside in the database (added via STORE commands) in >> addition to the single initial xml data file, then an error is thrown >> upon every xquery write statement stating: >> Stopped at .../backend/restxq/ajax/addmodel.xqm, 10/10: [FOUP0002] >> ".../backend/data/gallery.xml" could not be written. > > I agree that it would make sense to reduce the number of unwanted side > effects caused by the WRITEBACK option. I even think the best solution > is to only write back files when updating documents that have been > created within a query, e.g... > > delete node doc("file-on-disk.xml")//abc > > and disallow updates on database instances. > > Thanks, > Christian > > >> At the same time the file gallery.xml is in fact updated to reflect the >> write operations changes. >> >> >> Than you >> Stefan >> >> Am 06.11.13 19:58, schrieb Christian Grün: >>> Dear Stefan, >>> >>>> - My impression is that the WRITEBACK option only works for databases >>>> directly created from an initial input xml file (CREATE DB <name> >>>> <input.xml>). Is that true? My other tests involving ADD <file.xml> >>>> commands failed to write back inserted or deleted nodes. >>> >>> that’s correct: the WRITEBACK option is only recommendable if you are >>> working with single files. Usually, the EXPORT command is the better >>> choice for writing database contents back to files. >>> >>>> - I checked out the latest git version and noticed that the WRITEBACK >>>> option is not read from the .basex file anymore. In version 7.7.2 that I >>>> used for the other tests it was read. >>> >>> Thanks for the observation; I have committed a fix and uploaded a new >>> snapshot, which should fix the problem [1]. >>> >>> Christian >>> >>> [1] https://github.com/BaseXdb/basex/commit/2d618a18f4b2ad14cc17b15d155320a088f… >>> >>> >>>> Am 25.10.13 18:04, schrieb Dirk Kirsten: >>>>> Dear Stefan, >>>>> >>>>> please be aware that this is a english-speaking mailing list. As we want >>>>> most of our users to benefit from all questions and answers I'll stick >>>>> to english in my response (and aa your web page is in English as well I >>>>> guess it is no obstacle for you). >>>>> >>>>> Regarding your question about saving plain xml files: You can use the >>>>> Export Command (see https://docs.basex.org/wiki/Commands#EXPORT) to >>>>> export all your documents. You can e.g. use a cron job to export them >>>>> regularly. There also is a writeback option >>>>> (https://docs.basex.org/wiki/Options#WRITEBACK), which will update your >>>>> original input nodes. >>>>> >>>>> Secondly, regarding your question about empty WebDAV files: Please take >>>>> a look at a recent discussion on the list (see >>>>> https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-September/005453.…), >>>>> which is about the same topic. Short answer: With another viewer it >>>>> should work. Computing the size is quite difficult as we store the data >>>>> in a special storage format to speed up querying. Seems like some >>>>> viewers don't like it if the file size isn't correct. >>>>> >>>>> Cheers, >>>>> Dirk >>>>> >>>>> On 25/10/13 17:27, Stefan Sechelmann wrote: >>>>>> Hallo, >>>>>> ich bin dabei BaseX für unser aktuelles Projekt zu evaluieren. Bis jetzt >>>>>> sieht alles danach aus als ob BaseX genau das ist wonach wir gesucht >>>>>> haben. Hier meine Fragen dazu: >>>>>> >>>>>> - Der Datenbankserver speichert seine XML-Daten optimiert in .basex >>>>>> Files. Gibt es die Möglichkeit den Server auch XML Files schreiben zu >>>>>> lassen (So wie er das mit raw Files tut), so dass ich alles auch ohne >>>>>> BaseX ansehen kann? Gibt es vielleicht einen automatischen Export? >>>>>> >>>>>> - Der Webdav-Support kommt mir unstabil vor. Wenn ich XML-Dateien >>>>>> kopiere sind diese danach teilweise leer oder werden vom Server als RAW >>>>>> erkannt. Ich benutze einen Mac und den Finder als Client. Sind das >>>>>> eventuell bekannte Probleme oder muss ich bei mir suchen? >>>>>> >>>>>> Vielen Dank >>>>>> Stefan Sechelmann >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Stefan Sechelmann >>>> DFG-Forschungszentrum Matheon >>>> Mathematik für Schlüsseltechnologien >>>> Technische Universität Berlin >>>> Sekretariat MA 8-3 Tel. 030/314 29 486 >>>> Straße des 17. Juni 136 Fax 030/314 79 282 >>>> 10623 Berlin sechel(a)math.tu-berlin.de >>>> http://www.math.tu-berlin.de/~sechel >>>> >>>> >>>> _______________________________________________ >>>> BaseX-Talk mailing list >>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >> >> >> -- >> Stefan Sechelmann >> DFG-Forschungszentrum Matheon >> Mathematik für Schlüsseltechnologien >> Technische Universität Berlin >> Sekretariat MA 8-3 Tel. 030/314 29 486 >> Straße des 17. Juni 136 Fax 030/314 79 282 >> 10623 Berlin sechel(a)math.tu-berlin.de >> http://www.math.tu-berlin.de/~sechel

1 1

~webdav locks + server stop = conflicted locks
by France Baril 08 Nov '13

08 Nov '13

This Issue occurs when the server stops while files are open/locked: When basex starts it doesn't show lock icons on files the user had locked before the server stopped. When users tries to open a file oxygen says the file is locked, even though the current user owns the lock. In other words, the lock seems to be tracking something that doesn't survive sessions. To recover, we delete the webdav db + restart Oxygen. Granted, doing all check-ins prior to stopping the server for maintenance would be a best practice. But in case of an unexpected failure, since the lock are saved in ~webdav, is there anyway we could get back to the locks as set in ~webdav? -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

3 3

Re: [basex-talk] xquery and hash:md5
by James Wright 07 Nov '13

07 Nov '13

Would this work $someDoc//@*/text() --- Original Message --- From: "buzz chopra" <buzz.chopra(a)gmail.com> Sent: November 2, 2013 11:38 PM To: "Dirk Kirsten" <dk(a)basex.org> Cc: "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] xquery and hash:md5 Hello Dirk, Yes in that case, the issue exists in the nature of our data - and my poor choice of the return value of fn:string as the content to hash. Our data looks like: <test guid="blah"> <e1 attr0="a" attr1="b"/> <e2 attr0="c" attr1="d" attr2="e"/> </test> As you state, fn:string behaves the way http://www.w3.org/TR/xpath-functions-30/#func-string says it should. i.e. the empty string is returned for the document above. I'll have to try and find something that returns all attribute values in each document. Thanks again for your help. Buzz On Fri, Nov 1, 2013 at 10:37 AM, Dirk Kirsten <dk(a)basex.org> wrote: > Hello Buzz, > > The result of string() depends on the kind of element as an input. For > an attribute value it will simply be the value of the attribute. For an > element node it will be a concatination of the node content and all > descendant nodes. So if string($r) for $r being an element node is > always empty I would guess it is an empty node (something like <reps > />), which will always yield an empty result. > > If you want to create different hashs for empty elements with different > attribute values you might want to do something like the following > > let $reps := fn:collection($coll)//reps > for $r at $lcv in $reps > let $l := string-join($r/@*, '') > let $hash := xs:hexBinary(hash:md5(fn:string($l))) > return <hash i="{$lcv}">{$hash}</hash> > > If this is not what you actually want to achieve it would be very > helpful to actually get a small example of your data (especiall the reps > nodes) to get an idea how it looks like. > > Cheers, > Dirk > > On 01/11/13 16:19, buzz chopra wrote: > > Dirk, > > > > Thank you for your quick response. > > > > Upon your suggestion (and prior to that) I checked that the values for $r > > are different. Indeed they are. But, what I didn't check until your > > suggestion is that the values of fn:string($r) are different. They in > fact > > are the same - thank you for your suggestion. > > > > Is it possible that there is some sort of type casting related issue > going > > on where I call fn:string($r)? > > > > I get different values for the hash of each attribute value when I select > > attributes in the collection like: > > > > let $reps := fn:collection($coll)//reps/@att0 > > for $r at $lcv in $reps > > let $hash := xs:hexBinary(hash:md5(fn:string($r))) > > return <hash i="{$lcv}">{$hash}</hash> > > > > > > when I select elements in the collection like: > > > > let $reps := fn:collection($coll)//reps > > for $r at $lcv in $reps > > let $hash := xs:hexBinary(hash:md5(fn:string($r))) > > return <hash i="{$lcv}">{$hash}</hash> > > > > I found that the value of fn:string($r) is blank for all values of $r > > (which are different) which is why the hash is the same. > > > > I appreciate your help. > > > > Buzz > > > > > > On Fri, Nov 1, 2013 at 2:52 AM, Dirk Kirsten <dk(a)basex.org> wrote: > > > >> Hello buzz, > >> > >> it is ok to also ask more general XQuery questions on this list. > >> > >> For testing I executed the following query: > >> > >> for $r in ("123", "234", "345") > >> let $hash := xs:hexBinary(hash:md5(fn:string($r))) > >> return $hash > >> > >> which yields different results for each value of $r. Are you absolutely > >> sure you have different values for $r (you can check using trace())? > >> Also, given they query I would be very suprised if indeed it shows the > >> same result. > >> > >> Cheers, > >> Dirk > >> > >> On 01/11/13 08:24, buzz chopra wrote: > >>> This may be a question better suited for an xquery forum but I'll start > >>> here since I'm using basex: > >>> > >>> Can someone suggest why I'd be getting the same result for the hash > call > >>> even though > >>> the content of $r should be changing? > >>> > >>> let $reps := fn:collection($coll)/Rep > >>> for $r at $lcv in $reps > >>> let $hash := xs:hexBinary(hash:md5(fn:string($r))) > >>> return <hash i="{$lcv}">{$hash}</hash> > >>> }; > >>> > >>> Please let me know if I should post this elsewhere (and please include > >>> where). > >>> > >>> Thanks in advance for any suggestions. > >>> > >>> Buzz > >>> > >>> > >>> > >>> _______________________________________________ > >>> BaseX-Talk mailing list > >>> BaseX-Talk(a)mailman.uni-konstanz.de > >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >>> > >> > >> -- > >> Dirk Kirsten, BaseX GmbH, http://basex.org > >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk(a)mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > -- > Dirk Kirsten, BaseX GmbH, http://basex.org > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 > _______________________________________________ BaseX-Talk mailing list BaseX-Talk(a)mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

1 0

Receiving user feedback (too much traffic for BaseX?)
by France Baril 07 Nov '13

07 Nov '13

Hi, Our online help offers a form to collect comments from end-users. We have a lot of users and when a new product hits the market, we get a lot of feedback. We have found that BaseX is unable to handle so many write operations. Is there a suggested setup/configuration to handle multiple write operations coming from http requests. Is it possible to queue them somehow? Or is it better to use a more traditional approach to feedback collection? I.e. use a different toolset? Right now, BaseX just goes blank (freeze) with all the feedback coming in. Expectations: Average 5 comments/minute, but bursts of 2 per second should not break anything. Dropping comments during bursts is acceptable to us for our current business case because we hope to identify the most important issues after a new product release/software upgrade. Major issues should come up often enough. Any suggestion would be appreciated. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 1

BaseX
by Stefan Sechelmann 06 Nov '13

06 Nov '13

Hallo, ich bin dabei BaseX für unser aktuelles Projekt zu evaluieren. Bis jetzt sieht alles danach aus als ob BaseX genau das ist wonach wir gesucht haben. Hier meine Fragen dazu: - Der Datenbankserver speichert seine XML-Daten optimiert in .basex Files. Gibt es die Möglichkeit den Server auch XML Files schreiben zu lassen (So wie er das mit raw Files tut), so dass ich alles auch ohne BaseX ansehen kann? Gibt es vielleicht einen automatischen Export? - Der Webdav-Support kommt mir unstabil vor. Wenn ich XML-Dateien kopiere sind diese danach teilweise leer oder werden vom Server als RAW erkannt. Ich benutze einen Mac und den Finder als Client. Sind das eventuell bekannte Probleme oder muss ich bei mir suchen? Vielen Dank Stefan Sechelmann -- Stefan Sechelmann DFG-Forschungszentrum Matheon Mathematik für Schlüsseltechnologien Technische Universität Berlin Sekretariat MA 8-3 Tel. 030/314 29 486 Straße des 17. Juni 136 Fax 030/314 79 282 10623 Berlin sechel(a)math.tu-berlin.de http://www.math.tu-berlin.de/~sechel

3 3

Adding WKT/WKB To Geo Module
by Zachary DeLuca 03 Nov '13

03 Nov '13

Hi All, apologies in advance for the long message, I have many documents that store WKB (base64-encoded) geometry data (going for speed not readability). I want to be able to do queries where I can use the geo predicate functions with the stored WKB geometry and a provided WKT geometry. Example: Sample document structure: <root><geoWKBdata type=xs:base64Binary/></root> Sample query: collection()/root[geo:intersects(*geo:fromWKB*(geoWKBData), * geo:fromWKT*("LINESTRING((0,0),(1,1))")) I have looked at the module code and the only problem is that all methods assume an input of GML. I don't want to just make functions to convert from WKB/WKT to GML on input because I think that would be very inefficient (slow). I could just hack something together for my own needs, but I would like some input so that ideally I can produce something good enough to contribute back to the project. Options: 1) Add copies of all methods to accept each possible encoding type (GML, WKB, WKT). I don't like this approach because it doesn't scale well, and I don't even know if it's possible? (Pretty sure you can't overload functions in XPath/XQuery) 2) Leave the Geo module the way it is and add "sister" modules for the additional encoding types. Yuck. 3) The only thing that makes sense, really. Define 3 new functions, *fromGML *, *fromBinary*, and *fromText* and change the signatures of all module functions to just accept Geometry objects. That way you can load your geometry however you want and then pass it to the desired function. I will also modify the functions *asText *and *asBinary *to accept a Geometry object, and add an additional *asGML* so that the many existing functions that already return GML (envelope, boundary, etc.) can instead return Geometry, and can then be converted as needed to the desired format. This should scale well too because if any future encoding formats are desired all you need to do is add *fromXYZ* and *asXYZ *functions. With these functions, you could also string together 2 calls to convert between any of the supported encoding types. Does this make sense? It seems easy enough, am I missing anything? Also, assume for a second that I implement #3. Refer to my sample document/query above. As I understand it, for each document in the collection, the back end will have to parse "LINESTRING((0,0),(1,1))" into a Geometry first in order to compare. Is there any way to make that happen only once for the duration of the query? Is it as easy as this?: let $data = geo:fromText("LINESTRING((0,0),(1,1))"); return collection()/root[geo:intersects(*geo:fromWKB*(geoWKBData), $data) This would work, right? (I'm new to XQuery, sorry) So, am I on the right track with this? Have I overlooked some absurdly simple solution to my problem? I would appreciate any and all input. As I said, I would like to go beyond meeting my own needs and actually get this into the code base. I'll leave you with this snippet taken from the EXPath Geo Module Spec: "The data model envisioned by EXPath Geo module is abstract in the sense that it may be possible for module implementers to support more than one encoding on input. Output data types are specified in GML and XML Schema, where appropriate. All that is necessary is that the input geometry be transformable to the model specified by OGC SF and supported by the EXPath implementation. -SNIP- *Implementations are not limited to GML, KML, nor even XML, on input. It is possible that geometries encoded as “Well Known Text”, GeoJSON, or even “Well Known Binary”, among other possibilities, could be supported by an implementation.*" Thanks, Zach

3 9

xquery and hash:md5
by buzz chopra 03 Nov '13

03 Nov '13

This may be a question better suited for an xquery forum but I'll start here since I'm using basex: Can someone suggest why I'd be getting the same result for the hash call even though the content of $r should be changing? let $reps := fn:collection($coll)/Rep for $r at $lcv in $reps let $hash := xs:hexBinary(hash:md5(fn:string($r))) return <hash i="{$lcv}">{$hash}</hash> }; Please let me know if I should post this elsewhere (and please include where). Thanks in advance for any suggestions. Buzz

2 4

Error message
by Lars Johnsen 31 Oct '13

31 Oct '13

When trying to create a database from a folder of approximately 5000 collections (each containing between 200 and 500 files) the processed stopped after two hours with an error something like this: Line too long for a single database {filename} and {line number}. The file itself looks ok, with a single tag in the referenced position. What is the recommended procedure from here? All the best Lars G Johnsen National Library of Norway

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk