- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] Adding documents to database while space on diskdrive runs out
by pw＠themail.co.uk 22 Nov '12

22 Nov '12

I think that eXist-db takes the latter approach - the database becomes read-only when the disk is limited in space > > >---- Original Message ---- >From: christian.gruen(a)gmail.com >To: thomas.kaltofen(a)risc.uni-linz.ac.at >Subject: Re: [basex-talk] Adding documents to database while space on >diskdrive runs out >Date: Thu, 22 Nov 2012 19:08:01 +0100 > >>Dear Thomas, >> >>Im sorry that your data got corrupted. I agree that such a test >would >>be a desirable addition, Im afraid its often difficult to guess >how >>much space will be occupied by a specific update operation. We could >>generally disable update operations when the remaining space falls >>below a given limit, but once again such a limit would be strongly >>dependent on the used system and use cases. >> >>Rollbacks are on our long-term todo list (along with, or even >without >>MVCC), similar to incremental backups, but if memory gets low, they >>may as well lead to unexpected behavior. >> >>All the best, >>Christian >>_________________________________________ >> >>> I just had the problem that the disk space of the drive where a >BaseX database instance was stored ran out while adding a new >document to the database via the server interface. The result was a >half-finished document in the database resulting in >ArrayIndexOutOfBounds-exceptions when querying the database and >strange behavior of the GUI (views are not shown and even disabled). >Would it be possible to check for sufficient disk space before >writing to the database files? Moreover, it would be nice if the >failed transaction of adding the document while the space ran out >could be rolled back somehow to get the database back into a >consistent state. >>> >>> Thank you for your support! >>> >>> Best regards, >>> Thomas >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>_______________________________________________ >>BaseX-Talk mailing list >>BaseX-Talk(a)mailman.uni-konstanz.de >>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

Unable to integrate with oxygen to run xquery
by Alex Muir 22 Nov '12

22 Nov '12

Hi, I've got basex set up using jetty and can query within the firefox browser to return data for example using the url http://localhost:8080/rest/DDEX?query=//MessageHeader however my equivalent oxygen xquery does not return data and rather continues to run not stopping I've recently downloaded the jars and configured the Data Source Drivers as per the http://docs.basex.org/wiki/Integrating_oXygen file:/mnt/xslt_volume/i4EnrichV7/analysis/resources/libs/basex-xqj-1.2.3.jar file:/mnt/xslt_volume/i4EnrichV7/analysis/resources/libs/xqj-api-1.0.jar file:/mnt/xslt_volume/i4EnrichV7/analysis/resources/libs/xqj2-0.1.0.jar In the basex connection which is enabled I've got the following settings port 8080 databaseName DDEX logLevel description loginTimeout 6000 serverName localhost user admin password admin I set up a trasformation scenario with BaseX transformer with the following xquery let $content := //MessageHeader return $content The scenario does not return any data Thoughts? Regards -- - Alex G. Muir Software Engineering Consultant Linkedin Profile : http://ca.linkedin.com/pub/alex-muir/36/ab7/125 Love African Kora Music? Take a moment to listen to Gambia's - Amadu Diabarte & Jali Bakary Konteh www.bafila.bandcamp.com Your support keeps Africa's griot tradition alive... Cheers!

3 4

Adding documents to database while space on disk drive runs out
by Thomas Kaltofen 22 Nov '12

22 Nov '12

Hi, I just had the problem that the disk space of the drive where a BaseX database instance was stored ran out while adding a new document to the database via the server interface. The result was a half-finished document in the database resulting in ArrayIndexOutOfBounds-exceptions when querying the database and strange behavior of the GUI (views are not shown and even disabled). Would it be possible to check for sufficient disk space before writing to the database files? Moreover, it would be nice if the failed transaction of adding the document while the space ran out could be rolled back somehow to get the database back into a consistent state. Thank you for your support! Best regards, Thomas

2 2

Re: [basex-talk] 3 steps
by Simon Knuijver 21 Nov '12

21 Nov '12

Hi Christian, I see but there you put the entire xslt in one place, Simon -----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: woensdag 21 november 2012 14:38 To: Simon Knuijver Cc: BaseX Subject: Re: [basex-talk] 3 steps >> The current way to do this is transform your request via XQuery and the XSLT module. We may add a parameter for XSLT transformations to our REST API [4]; details would first have to be specified, though. Yes, it can also be done in Java. The most compact solution I get in mind is: 1. Create a database X with your XML and XSLT file (called "example" in the following) 2. Go to your browser and visit http://localhost:8984/rest/X/example.xml?query=xslt:transform(.,db:open(.,'example.xslt'))" If we added an additional xslt parameter, the URL could look as follows: http://localhost:8984/rest/X/example.xml?xslt=example.xslt C.

2 1

Re: [basex-talk] 3 steps
by Simon Knuijver 21 Nov '12

21 Nov '12

Hi Christian, thanks for your swift reply. I'll get on it right away and I would be interested in that extra parameter :) To dumb this one down a little: >> 3. Connect to that database and query for the templates within another >> XSL file? You said: > The current way to do this is transform your request via XQuery and the XSLT module. We may add a parameter for XSLT transformations to our REST API [4]; details would first have to be specified, though. What would that look like, something like this (in Java)?: package org.basex.examples.rest; import java.io.*; import java.net.*; import org.basex.*; /** * This class is a simple example to demonstrate applying stored templates from the REST API. * * @author BaseX Mailing list, BSD License */ public final class RESTGet { /** * Runs the example. * @throws IOException I/O exception */ static void run() throws IOException { System.out.println("=== GET: execute a query ==="); // The java URL connection to the resource String base = "http://localhost:8984/rest/"; URL url = new URL(base + "templates?query=(//tag/styleVersion) "); System.out.println("\n* URL: " + url); // Establish the connection to the URL HttpURLConnection conn = (HttpURLConnection) url.openConnection(); // Print the HTTP response code int code = conn.getResponseCode(); System.out.println("\n* HTTP response: " + code + " (" + conn.getResponseMessage() + ')'); // Check if request was successful if(code == HttpURLConnection.HTTP_OK) { // Print the received result to standard output System.out.println("\n* Result:"); // Get and cache input as UTF-8 encoded stream BufferedReader br = new BufferedReader(new InputStreamReader( conn.getInputStream(), "UTF-8")); // Print all lines of the result for(String line; (line = br.readLine()) != null;) { System.out.println(line); } br.close(); } // Close connection conn.disconnect(); } /** * Main method. * @param args (ignored) command-line arguments * @throws Exception exception */ import javax.xml.transform.*; import javax.xml.transform.stream.StreamResult; import javax.xml.transform.stream.StreamSource; import java.io.File; import java.io.IOException; import java.net.URISyntaxException; public static void main(final String... args) throws Exception{//, IOException, URISyntaxException, TransformerException { // Start servers final BaseXHTTP http = new BaseXHTTP("-Uadmin", "-Padmin"); TransformerFactory factory = TransformerFactory.newInstance(); // Run example //run();//How does this call the RESTGET class? //for now (You may not want to do make RESTGET instance for every template, instead, it may be easier to ask for a set of templates at once and apply them as a stylesheet here) //For(tags) //Get template and add to stylesheet Source xslt = theBuiltStyleSheet; Transformer transformer = factory.newTransformer(xslt); Source text = new StreamSource(new File("input.xml")); transformer.transform(text, new StreamResult(new File("output.xml"))); // Stop servers http.stop(); } } Best, Simon

2 1

Re: [basex-talk] db:list via basexclients doesn't show all databases
by Andreas Weiler 21 Nov '12

21 Nov '12

The server creates an index of all existing databases on startup. Afterwards only by client created databases are added to that index. So all databases created by other processes (GUI, Console) are not in the index. Restart the server and you will see all databases. I guess the db:list() function uses that index and the db:list("dbname") function does not. -- Andreas Am 21.11.2012 um 13:01 schrieb Stefan Koch: > Strange thing is: Our application has access to the db. > ... found out that db:list('dbname') has access to the database, while db:list() can't see it. > > > > Am 21.11.2012 12:51, schrieb Andreas Weiler: >> uh same problem for me... >> >> steps to reproduce: >> >> 1. start server >> 2. start GUI and create db "test", close GUI >> 3. run list...DB "test" is not there, but can be accessed with open test >> >> I guess just client connections can add new databases to a running server instance. >> >> -- Andreas >> >> Am 21.11.2012 um 12:48 schrieb Stefan Koch: >> >>> ... the user has all privileges. I tried to added R/W Access via Grant (though basex standalone). >>> But the client still can't see it. >>> >>> Stefan >>> >>> >>> Am 21.11.2012 12:15, schrieb Andreas Weiler: >>>> Hi Stefan, >>>> >>>> does the user have permission to see all databases? >>>> >>>> Regards, >>>> Andreas >>>> >>>> Am 21.11.2012 um 12:21 schrieb Stefan Koch: >>>> >>>>> Hi, >>>>> >>>>> I was wondering why our backupskript isn't exporting all databases, one is missing. >>>>> The I found out why - our backuptool connects via Java and uses db:list to get the names of all databases. >>>>> >>>>> If I connect via basexclient - there is one missing, if I check via basex (from the shell) the database is shown. >>>>> I added the database also via basex and not like the others via the client. >>>>> >>>>> Is this intended behaviour? How can I tell the client, that there is one more database? >>>>> >>>>> Regards, >>>>> >>>>> Stefan >>>>> _______________________________________________ >>>>> BaseX-Talk mailing list >>>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> _______________________________________________ >>>> BaseX-Talk mailing list >>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

2 1

Re: [basex-talk] db:list via basexclients doesn't show all databases
by Andreas Weiler 21 Nov '12

21 Nov '12

uh same problem for me... steps to reproduce: 1. start server 2. start GUI and create db "test", close GUI 3. run list...DB "test" is not there, but can be accessed with open test I guess just client connections can add new databases to a running server instance. -- Andreas Am 21.11.2012 um 12:48 schrieb Stefan Koch: > ... the user has all privileges. I tried to added R/W Access via Grant (though basex standalone). > But the client still can't see it. > > Stefan > > > Am 21.11.2012 12:15, schrieb Andreas Weiler: >> Hi Stefan, >> >> does the user have permission to see all databases? >> >> Regards, >> Andreas >> >> Am 21.11.2012 um 12:21 schrieb Stefan Koch: >> >>> Hi, >>> >>> I was wondering why our backupskript isn't exporting all databases, one is missing. >>> The I found out why - our backuptool connects via Java and uses db:list to get the names of all databases. >>> >>> If I connect via basexclient - there is one missing, if I check via basex (from the shell) the database is shown. >>> I added the database also via basex and not like the others via the client. >>> >>> Is this intended behaviour? How can I tell the client, that there is one more database? >>> >>> Regards, >>> >>> Stefan >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

1 0

db:list via basexclients doesn't show all databases
by Stefan Koch 21 Nov '12

21 Nov '12

Hi, I was wondering why our backupskript isn't exporting all databases, one is missing. The I found out why - our backuptool connects via Java and uses db:list to get the names of alle databases. If I connect via basexclient - there is one missing, if I check via basex (from the shell) the database is shown. I added the database also via basex and not like the others via the client. Is this intended behaviour? How can I tell the client, that there is one more database? Regards, Stefan

2 1

3 steps
by Simon Knuijver 20 Nov '12

20 Nov '12

Hi, I have an xsl or xml file containing xsl templates: <xsl:template match="compcd" name="tata"> <compcd> <xsl:value-of select="string(ns0:LOCATIONSPEC[ns0:ASSETATTRID='VS_TYPE']/ns0:ALNVALUE) "></xsl:value-of> </compcd> </xsl:template> <xsl:template match="compcd" name="dada"> <equipnum> <xsl:value-of select="string(ns0:FXSTAMDATASet/ns0:LOCATIONS/ns0:FXCOMPANYNAME)"></xsl :value-of> </equipnum> </xsl:template> : Etc I have a noob question about this, in three quick steps, how do I: 1. Add these templates to a database 2. Put that db in a webservice/server 3. Connect to that database and query for the templates within another XSL file? Pointers to the proper manual would be fine, thanks! Best, Simon

2 1

slow query
by Mahlow Cerstin 20 Nov '12

20 Nov '12

Hi, I have a problem with slow queries. I'm not sure if it is due to the construction of the query or if there is something else going on. I'm querying two databases: "collect" which is opened at the beginning and contains several thousands entries like these two: <entry time="2012-05-22T11:27:09"> <node>40177618</node> <query>[text() contains text ('den' ftand 'Hals' ftand 'brachen') distance at most 10 words]</query> <person>britta</person> <phraseme>Ad0194</phraseme> <selected>yes</selected> </entry> <entry time="2012-05-22T11:27:09"> <node>40672561</node> <query>[text() contains text ('den' ftand 'Hals' ftand 'brachen') distance at most 10 words]</query> <person>britta</person> <phraseme>Ad0194</phraseme> <selected>no</selected> </entry> The node-ids refer to a second database: "TG-DTA-GerManC-stemming-ws". The queries contain the xquery the node was found with. For a particular phraseme, I display the original node using ft:mark to highlight the query terms. Additionally, I display some of the metadata for this node. The user will be able to "delete" some of the entries -- i.e., setting the value of "selected" from "yes" to "no". Therefore I render everything as a form element with a checkbox. The query is this one: for $i at $p in //entry[phraseme[text() = "Ad0194"] and selected[text() = "yes"]] let $query := $i/query let $node := $i/node let $prefix := fn:in-scope-prefixes($i) let $title := db:open-id('TG-DTA-GerManC-stemming-ws', $node) /ancestor::*:TEI[1]//*:fileDesc[1]//*:titleStmt[1]//*:title[1] let $author := db:open-id('TG-DTA-GerManC-stemming-ws', $node) /ancestor::*:TEI[1]//*:sourceDesc[1]//*:bibl[1]//*:author[1] let $note := db:open-id('TG-DTA-GerManC-stemming-ws', $node) /ancestor::*:TEI[1]//*:notesStmt//*:note let $expr := concat("ft:mark(db:open-id('TG-DTA-GerManC-stemming-ws', ", $node, ") ", $query, ")") let $time := data($i/@time) return <div> <hit count="{ $p}"> <input type="checkbox" name="NODE" value="{$node}"/> {$p} ({ if($prefix = "dta") then "DTA" else "TG"})Knoten: {$i/node} {xquery:eval($expr)} </hit> <bib> {$time} Bibliographie { data($author)}: { data($title)} Anmerkung: { data ($note) } Korpus: { if($prefix = "dta") then "Deutsches Textarchiv" else "TextGrid Digitale Bibliothek"} </bib> </div> However, for this particular phraseme ("Ad0194"), there will be 676 hits to be displayed (out of around 1000 stored) which takes around 15 seconds in the GUI. In the actual web application, the full site is shown after more than 20 seconds (there is some HTML to be displayed and another short xquery for displaying distinct queries for all entries in question), which is too slow, I think. Is there anything I can do to speed up the query? Can I somehow rewrite it to make execution faster? I think opening the second DB four times costs a lot of time. Both DBs have an optimized index already. Thanks in advance Cerstin -- Dr. phil. Cerstin Mahlow Universität Basel Deutsches Seminar Nadelberg 4 4051 Basel Schweiz Tel: +41 61 267 07 65 Fax: +41 61 267 34 40 Mail: cerstin.mahlow(a)unibas.ch Web: http://www.oldphras.net

3 6

RESTXQ: What to use for page layout / view templating?
by Daniel Kvasnička 19 Nov '12

19 Nov '12

Hi folks, another thing I'm trying to solve while working on a XQuery web app. What kind of templating system do you use to separate HTML templates from the rest of your code? - XSLT - the only problem I have with this is performance. I tested the XSLT Module with Saxon 9 and a primitive page and it was more than 3x slower compared to pure XQuery templates (XQuery fn taking $model). Other than that I think it's the best built-in option. I actually don't have any users-per-second expectations and those templates will be quite small... do I overemphasize the perf. question? - pure XQuery - not aimed at templating at all and it shows I'm afraid... just can't come up with a solution that would be clean enough (plus HTML, JS & CSS highlighting withing xq files is not good at all, at least in Vim) - I looked at Mustache.xq and quite liked it - depends on MarkLogic though :( - Any hidden gem I haven't googled yet? Leaning towards XSLT right now... Daniel -- danielkvasnicka.net

5 8

Saving xquery output to a file
by Alex Muir 19 Nov '12

19 Nov '12

Hi, I've create a standalone script which creates a database and run's an xquery although I'm not clear on how to save the result of the xquery to file. generally I would like to create a database and run a series of xqueries each going to file rather than to the console output. <?xml version="1.0" encoding="UTF-8"?> <commands> <show-databases/> <drop-db name="1997-i4enrich-10k"/> <create-db name="1997-i4enrich-10k"/> <add path="1997-i4enrich-10k">/mnt/xslt_volume/i4ContentOutput/DebugSet/10k-GHU-2009/MDAFalse/Completed/</add> <info-db/> <run file='../xquery/section-10k-metrics.xq'/> <close/> </commands> I saw this on the documentation however I couldn't get it to work Q{java:java.io.FileOutputStream}new("output.txt") btw.. It might be nice to be able to say <run file='../xquery/section-10k-metrics.xq' outputFile="filename.xml" /> Thanks -- - Alex G. Muir Software Engineering Consultant Linkedin Profile : http://ca.linkedin.com/pub/alex-muir/36/ab7/125 Love African Kora Music? Take a moment to listen to Gambia's - Amadu Diabarte & Jali Bakary Konteh www.bafila.bandcamp.com Your support keeps Africa's griot tradition alive... Cheers!

2 1

Re: [basex-talk] BUG# Multiple Thread try to execute some Query
by Samuel Lacmene 19 Nov '12

19 Nov '12

1 0

concurrent issues with 7.5 - OverlappingFileLockException
by Srinivasan Muthu 18 Nov '12

18 Nov '12

Hi, We started having some thread hanging issues when concurrent calls were being made to our basex 7.3 database (client-server mode) from java 6 web app on Mac (less prominent in Windows). We extracted the code into a simple java app to test this in two threads executing a simple doc('blah') command several times and were able to reproduce the thread hanging issue easily in basex 7.3. After Christian's suggestion in another forum conversation to use 7.5, we tried with 7.5, now the code does not hang but it throws java.nio.channels.OverlappingFileLockException frequently for some of the db requests. I am attaching the sample java program here as a jar file (source code inside as well). Here is a sample output of this program: Srini-MacBook-Pro:Downloads srini$ java -jar BasexTest.jar the time the xxxxxxxx 1 time the time the xxxxxxxx 2 time . . (some output lines hidden here for brevity) . the time the xxxxxxxx 30 time the time the xxxxxxxx 31 time java.io.IOException: java.nio.channels.OverlappingFileLockException at com.vmware.gts.vha.basex.BaseXClient$Query.exec(BaseXClient.java:446) at com.vmware.gts.vha.basex.BaseXClient$Query.execute(BaseXClient.java:405) at com.vmware.gts.vha.test.ParallelExecution.runXQuery(ParallelExecution.java:33) at com.vmware.gts.vha.test.ParallelExecution.access$0(ParallelExecution.java:30) at com.vmware.gts.vha.test.ParallelExecution$SampleRunnable.run(ParallelExecution.java:13) at java.lang.Thread.run(Thread.java:680) Is this a potential bug in BaseX 7.5? This issue is more prominent (happens frequently) in Mac than Windows. We tried with both DBLocking=true and false and the issue remains. Thanks, Srini

2 4

Re: [basex-talk] BUG# Multiple Thread try to execute some Query
by Samuel Lacmene 18 Nov '12

18 Nov '12

2 1

udpindex
by Mahlow Cerstin 17 Nov '12

17 Nov '12

Hi, as far as I can see, UPDINDEX cannot be set in the GUI when creating a database. Is this correct? So how would I proceed to change my existing database created without this option and updated and queried by users constantly into an database with this option to make retrieval faster? Is it correct that with UPDINDEX true there is no need anymore to "optimize" the index from time to time? Initially I thought of exporting the existing DB as XML and then using this as source for a new DB with UPDINDEX set true. How do I do this without the GUI? Oh, and could you please make it possible to choose UPDINDEX as an option in the GUI? Best regards Cerstin -- Dr. phil. Cerstin Mahlow Universität Basel Deutsches Seminar Nadelberg 4 4051 Basel Schweiz Tel: +41 61 267 07 65 Fax: +41 61 267 34 40 Mail: cerstin.mahlow(a)unibas.ch Web: http://www.oldphras.net

3 2

Updating single database locks entire server
by Srinivasan Muthu 17 Nov '12

17 Nov '12

First of all, great work you guys, thanks much. We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function: declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () }; This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery. Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server? Thanks, Srini

2 5

performance and collections
by pw＠themail.co.uk 15 Nov '12

15 Nov '12

dear list I have a web-based database (basex 7.5 running on tomcat) in which I have a collection with about 3000 files in various sizes from about 1k to 40k. Would I get better performance by splitting these files across several collections? what are the rules of thumb here? many thanks Pete

2 1

XQuery web app authentication - best practices?
by Daniel Kvasnička 15 Nov '12

15 Nov '12

Hi folks, I'd like to write an app using RESTXQ and I'd like to auth users using a regular form-based authentication and then on some XQuery functions check for an existing user session (and possibly user roles). I'd also like to add some social media login using OAuth (later). My question is - is it somehow possible to do this in a declarative way? For example custom annotations on XQuery handlers? Something like %auth:roles-allowed("admin") I definetly don't want to "if" at the beginning of every function that should be protected. No problem with implementing this in Java or XQuery. Just tell me how to approach this orthogonal concern in a reasonable way... Or should I equal app users to BaseX users and leverage Basex auth? Any tips appreciated (yes, you can even tell me BaseX RESTXQ is not a good tool for that). Daniel -- danielkvasnicka.net

4 5

Options in Basex 7.5
by Andy Bunce 15 Nov '12

15 Nov '12

There are many ways to set options [1] but it is not clear to me which one wins in the case where different values are specified for the same option by the different ways. I guess it is the 1st one to be set. So SystemProperty, then command line, then .basex? I am often getting caught out by the fact that in recent 7.5 builds the http properties are ignored unless they are set by editing jetty.xml. I wonder if the supplied jetty.xml could use <SystemProperty>. Something like: <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">  <Set name="host"><SystemProperty name="org.basex.serverhost" default=""/></Set> <Set name="port"><SystemProperty name="org.basex.serverport" default="8984"/></Set> <Set name="confidentialPort"><SystemProperty name="org.basex.??" default="??"/></Set> <Set name="maxIdleTime">60000</Set> <Set name="reuseAddress">false</Set> <Set name="Acceptors">2</Set> </New> /Andy [1] http://docs.basex.org/wiki/Options

2 3

for ... group by - minor bug
by Hans-Juergen Rennau 15 Nov '12

15 Nov '12

Hello, the support for XQuery 3.0 is really wonderful! Thank you. Just a trifle: I think there is a little bug concerning the "group by" clause and the case that a non-grouping variable is reused as grouping variable, but the shorthand notation for this is not used. According to the spec, these expressions should be equivalent: group by $x group by $x := $x I observed that in the shorthand notation $x references the grouping variable, that is, one value per group, as it should. But in the longhand notation, $x references the variable in the pre-grouping variable, and I suppose this should not be so. For a full example, see below. Kind regards, Hans-Juergen PS: Full example: Code: for $x in (1, 1, 2) group by $x := $x return <r x="{$x}"/> Output: <r x="1 1"/> <r x="2"/>

1 0

sql query result - formated xml from mysql database
by Jarosław Kowalewski 15 Nov '12

15 Nov '12

Hi, I've in my mysql database field with xml (formated xml). By using sql module i get something like that: <sql:column xmlns:sql="http://basex.org/modules/sql" name="xml"> <?xml version="1.0" encoding="UTF-8"?> <Document xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="wniosek.xsd"> </Document> and I can't query it like regular xml by xquery. It's possible to get only content of xml column? Or it's other way to query this content? Jarek

2 2

Re: [basex-talk] xquery sql module init driver
by Jarosław Kowalewski 15 Nov '12

15 Nov '12

Thanks a lot. That works!:) Mayby it's a good idea to write about that in manual or FAQ? Jarek 2012/11/15 <basex-talk-request(a)mailman.uni-konstanz.de>: > Message: 4 > Date: Wed, 14 Nov 2012 15:55:34 +0100 (CET) > From: Antoni Mylka <amy(a)basis06.ch> > To: basex-talk <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] xquery sql module init driver > Message-ID: <701444465.74583.1352904934339.JavaMail.root(a)basis06.ch> > Content-Type: text/plain; charset=utf-8 > > Hi Jarek, > > And what about > > java -cp pathto/mysql-connector-java-5.1.21-bin.jar:BaseX73.jar org.basex.BaseXGUI > > ? > > quoting from: > > http://stackoverflow.com/questions/945962/java-problem-running-a-jar-file-i… > > "If you're running a jar file with java -jar, the -classpath argument is ignored." > > -- > Antoni My?ka > Software Engineer > > basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22 > http://www.basis06.ch - source of smart business > > ----- Urspr?ngliche Mail ----- > Von: "Jaros?aw Kowalewski" <jaroslaw.kowalewski(a)students.waw.pl> > An: basex-talk(a)mailman.uni-konstanz.de > Gesendet: Mittwoch, 14. November 2012 13:16:37 > Betreff: [basex-talk] xquery sql module init driver > > Hi, > I try to connect with my database directly by sql module. > > sql:init("com.mysql.jdbc.Driver"), > > let $conn := sql:connect("jdbc:mysql://dbhost:port/dbname") > return sql:execute($conn, "SELECT * FROM coffees WHERE price < 10") > > but I've problem with and I get error "[BXSQ0007] Could not initialize > specified driver: 'com.mysql.jdbc.Driver'" > > java -cp pathto/mysql-connector-java-5.1.21-bin.jar -jar BaseX73.jar > > Thanks for a help > > Jarek >

1 0

Re: [basex-talk] Database corruption
by Antoni Mylka 14 Nov '12

14 Nov '12

Hi, Two further questions: 1. What's the cleanest way to get a log containing the actual query strings? AFAIU BaseX doesn't use any logging framework and the BaseXServlet.service has a finally block that prints the request headers (if Prop.debug is enabled) but not the request body. I thought about writing my own Filter for that purpose. 2. I've been trying to update BaseX to the latest snapshot, but the queries generated by my application, that used to work with 7.2.1 and 7.3 don't seem to work now. The body of a POST looks like this (\n means a newline character): <?xml version="1.0" encoding="UTF-8"?>\n <command xmlns="http://basex.org/rest"><text><![CDATA[\n ADD TO 5d03ebad-dc6a-492c-9527-bd2ce3830080\n <tx id="5d03ebad-dc6a-492c-9527-bd2ce3830080"\n dbId=""\n xmlns="http://our.namespace.com/xmlns/"\n xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"\n xmlns:xs="http://www.w3.org/2001/XMLSchema"\n <commit />\n <skip />\n </tx>\n \n ]]></text></command> Now the question is what's wrong with this query. I get a 400 Bad Request response with the following error message. Stopped at line 1, column 43:\n Syntax: ADD (TO [path]) [input]\n Add resource to database.\n \n Adds [input] as [path] to the database.\n As I said this used to work with 7.3. I tried to diff the StringParser class between 7.3 and the current trunk. It seems to use the InputParser on a single line and not on the entire query. I can't spot anything that would explain this behaviour though. All kinds of comments welcome. -- Antoni Myłka Software Engineer basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22 http://www.basis06.ch - source of smart business -- Antoni Myłka Software Engineer basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22 http://www.basis06.ch - source of smart business ----- Ursprüngliche Mail ----- Von: "Antoni Mylka" <amy(a)basis06.ch> An: "Christian Grün" <christian.gruen(a)gmail.com> Gesendet: Montag, 12. November 2012 15:42:42 Betreff: Re: [basex-talk] Database corruption Dear Christian, Thanks for the tip. Will fix the logging settings and retest with the latest snapshot. Best regards, -- Antoni Myłka Software Engineer basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22 http://www.basis06.ch - source of smart business ----- Ursprüngliche Mail ----- Von: "Christian Grün" <christian.gruen(a)gmail.com> An: "Antoni Mylka" <amy(a)basis06.ch> CC: "basex-talk" <basex-talk(a)mailman.uni-konstanz.de> Gesendet: Montag, 12. November 2012 15:34:23 Betreff: Re: [basex-talk] Database corruption Dear Antoni, without having read all the elaborate details in your mail (thanks), I would strongly advise to try the latest stable 7.5 snapshot [1], which contains some important improvements reg. concurrent updates. I'm well aware that it feels inconvenient to use beta releases in productive environments, but the current version of 7.5 is very stable, and will be released soon without any major enhancements or legacy issues. Hope this helps, Christian [1] http://files.basex.org/releases/latest/ ___________________________ On Mon, Nov 12, 2012 at 3:30 PM, Antoni Mylka <amy(a)basis06.ch> wrote: > Hi, > > We're developing an application that uses BaseX. In our tests, the application works properly for a couple of hours, makes queries to BaseX adds and removes data. Then suddenly some BaseX request will return the HTTP 500 response. From that point on all queries that make modifications in the database end in an error and the entire system grinds to a screeching halt. One of the experiments ran for four days, then we examined the access log to see that only the first 6 hours were OK, after that all BaseX queries would end in error. > > I'm going to investigate and will try to come up with as detailed a description of the issue as possible. For now the things I know are: > > 1. In our experiment, the application would make about 80000 successful queries over a period of 6 hours. Then it would get the first 500 error. > 2. Afterwards it "seems" that the queries which don't modify anything end with 200, while all the modification queries end with a 500 error. That's to be confirmed. > 3. BaseX 7.2.1 runs in a war deployed on Tomcat 7.0.32 on a linux system. > 4. The access logs don't show anything interesting. > 5. The catalina.out that shows the Tomcat standard output contains hundreds of exceptions of the following kind: > > Exception in thread "Thread-78197" java.lang.RuntimeException: Data Access out of bounds [pre:1422612, indexSize:5540, access:5540 > 5539] > at org.basex.util.Util.notexpected(Util.java:55) > at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:478) > at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:211) > at org.basex.data.Data.textOff(Data.java:432) > at org.basex.data.DiskData.text(DiskData.java:228) > at org.basex.index.value.DiskValues.get(DiskValues.java:295) > at org.basex.index.value.UpdatableDiskValues.index(UpdatableDiskValues.java:133) > at org.basex.data.DiskData$1.run(DiskData.java:461) > > The thread name is different every time and the numbers in square brackets are also different. I don't see any patterns (yet). > 6. BaseX is accessed from a single thread in a single application. AFAIK there are no concurrent requests, though that is to be confirmed. > 7. The GC log shows that basex stays below 100M of used heap, which is well below the allotted Xmx setting of 2G. > 8. We did three experiments and the same effect has been observed in all three, albeit after different periods of time: (59K queries and 4:25 hours, 37K queries and 2:56 hours, 76.8K queries and 6 hours). > > Things I intend to do: > 1. Set up proper logging for BaseX, so that these errors don't end up in catalina.out, but in the proper log file. > 2. Catch the first offending query, see if there is anything unusual about it. It certainly doesn't differ much from the successful queries. It simply updates the BaseX state to keep it in sync with an external data source. All the update queries are generated by the same template and there have been hundreds of successful ones before the first error occurs. > 3. Manipulate the log settings, to see if it's possible to get something more meaningful out of them. > 4. Try newer versions: 7.3 and the current trunk. > > I've seen emails similar about "Data Access out of bounds" but they don't seem relevant. > > As I said, I'll try to come up with more information, but if the error or the stack trace looks familiar - I'd be grateful for pointers. > > Thanks in advance, > > -- > Antoni Myłka > Software Engineer > > basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22 > http://www.basis06.ch - source of smart business > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 7

xquery sql module init driver
by Jarosław Kowalewski 14 Nov '12

14 Nov '12

Hi, I try to connect with my database directly by sql module. sql:init("com.mysql.jdbc.Driver"), let $conn := sql:connect("jdbc:mysql://dbhost:port/dbname") return sql:execute($conn, "SELECT * FROM coffees WHERE price < 10") but I've problem with and I get error "[BXSQ0007] Could not initialize specified driver: 'com.mysql.jdbc.Driver'" java -cp pathto/mysql-connector-java-5.1.21-bin.jar -jar BaseX73.jar Thanks for a help Jarek

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk