BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5269 discussions

Basex query with huge volume collection
by Chandrasekhar 28 Feb '15

28 Feb '15

Dear Basex, We have written a query to read nodes from collection which are in volumes of crores.Sample xml is as attached to this mail.In the xml there are crores of nodes with the tag "R",now i need to fetch all the tags with "R", i have used the below query to fetch it but the amount of time it is taking to do is going above 5 minutes and taking a memory of >1 GB. for $PSTNCallData in (collection("52_DATA_AUGUST-14_Billed_HugeDataTesting")/SUBCUSTBRK/ITEMIZED/TRANSACTION[data(@Type)=('PSTN')]/CALL_TYPES/DATA/R) return <R><bill_no>{$PSTNCallData/C1}</bill_no></R> Could you please help us in optimizing this query.Thank you in advance. Thanks & Regards ChandraShekar The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.

2 1

LocalSession delete document - No database opened
by Ketill Fenne 27 Feb '15

27 Feb '15

Hi! I am trying to delete a specific document from a database called Users, but get the error "org.basex.core.BaseXException; No database opened" The code is (input name is the document name): Context context = new Context(); LocalSession session = new LocalSession(context); try { new Open("Users").execute(context); session.execute("delete " + name); } catch (Exception e) { e.printStackTrace(); // let the caller handle exception throw e; } Do I need to open the database through the session? Haven't found out how.... BR Ketill Fenne

2 1

Re: [basex-talk] Executing Xquery update from java - hangs
by Christian Grün 26 Feb '15

26 Feb '15

Hi Ketill, I'm cc'ing this back to the mailing list, as it may also be of interest for other users. > Would you advice against mixing different "connection methods" to the > database(s) in the same application? (Queries through XQJ, updates via > QueryProcessor) Using a single API will usually cause you less headache in the middle or long term, but it may be fine to simply try out things. If you create QueryProcessor instances, no problems are to be expected as long as you dont't need to run concurrent queries. You could also work with LocalSession instances, which will be more stable in the long run. The (Local|Client)Session API could be further extended to meet more requirements, depending on the feedback we get. Hope this helps, Christian > On Wed, Feb 25, 2015 at 5:23 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Ketill, >> >> Welcome to our list. >> >> XQJ was written to be compliant with XQuery 1.0, so it may well be >> that the db:output extension cannot be used with our (well, Charles >> Foster's) implementation of XQJ (yet). Have you tried to set the >> MIXUPDATES flag to true instead? >> >> All the best, >> Christian >> >> >> On Wed, Feb 25, 2015 at 5:02 PM, Ketill Fenne <ketill.fenne(a)gmail.com> >> wrote: >> > Hi! >> > >> > I am executing all Xqueries through the same runQuery method in Java, >> > and I >> > am running the latest jar-files (8.0.1): >> > >> > XQDataSource ds = new BaseXXQDataSource(); >> > xqc = ds.getConnection(); >> > XQExpression xqe = xqc.createExpression(); >> > rs = xqe.executeQuery(query); >> > >> > The query is (in this case): >> > for $doc in (collection('Users/customer_1'))/* >> > return (replace value of node $doc/@name with ("My new customer name")) >> > >> > The update works fine in BaseX Gui, and I suspect the problem is that it >> > does not return anything, and my code expects XQResultSequence. I have >> > tried >> > adding db:output to return part of the query with no luck. >> > Do you have any suggestions? >> > >> > Kind regards >> > Ketill Fenne > >

1 0

Trying to get started with IntelliJ and BaseX
by Bridger Dyson-Smith 25 Feb '15

25 Feb '15

Hi all, I'm unsure of how many IntelliJ users there are here, but I'm hitting some embarrassingly simple problems and I hope someone can provide some guidance. I'm trying to use BaseX 8.0 as a data source for a simple query [1]. However I'm unable to apply the query against a database. I've added BaseX as an XQuery data source: *File* > *Settings* > *Other Settings* > *XQuery Data Sources* > "*+*" (*add a data source*) > Name: BaseX80-tei Host: localhost Port: 1984 Database name: tei-dump Username: admin Password: admin I've added a *Run* *Configuration*: Run > Edit Configurations > "+" (add a configuration) > XQuery Main Module > Name: name-query Configuration: Main file: /usr/home/bridger/src/basex/80/basex/repo/name.xq Data source: BaseX80-tei (from above) Java Configuration: VM options: blank Program arguments: blank Working directory: /usr/home/bridger/src/basex/80/basex Use classpath of module: basex When I try to run the configuration, I get a Java exception [2]. I'm fairly sure that this is user/operator error, but I'm not quite sure what I've done wrong. Thank you for reading. Cheers, Bridger [1] names.xq <results> { for $n in //*:name return $n } </results> [2] Java exception: /usr/local/openjdk7/bin/java -Didea.launcher.port=7532 -Didea.launcher.bin.path=/usr/home/bridger/bin/idea/bin -Dfile.encoding=UTF-8 -classpath /usr/home/bridger/.IntelliJIdea14/config/plugins/intellij-xquery/lib/intellij-xquery-rt.jar:/usr/home/bridger/.IntelliJIdea14/config/plugins/intellij-xquery/lib/xqj-api-1.0.jar:/usr/home/bridger/.IntelliJIdea14/config/plugins/intellij-xquery/lib/xqj2-0.2.0.jar:/usr/home/bridger/.IntelliJIdea14/config/plugins/intellij-xquery/lib/basex-xqj-1.3.0.jar:/usr/local/openjdk7/jre/lib/compilefontconfig.jar:/usr/local/openjdk7/jre/lib/management-agent.jar:/usr/local/openjdk7/jre/lib/javazic.jar:/usr/local/openjdk7/jre/lib/resources.jar:/usr/local/openjdk7/jre/lib/jce.jar:/usr/local/openjdk7/jre/lib/charsets.jar:/usr/local/openjdk7/jre/lib/jsse.jar:/usr/local/openjdk7/jre/lib/rt.jar:/usr/local/openjdk7/jre/lib/ext/zipfs.jar:/usr/local/openjdk7/jre/lib/ext/localedata.jar:/usr/local/openjdk7/jre/lib/ext/dnsns.jar:/usr/local/openjdk7/jre/lib/ext/sunpkcs11.jar:/usr/local/openjdk7/jre/lib/ext/sunjce_provider.jar:/usr/local/openjdk7/jre/lib/ext/sunec.jar:/usr/home/bridger/bin/idea/lib/idea_rt.jar com.intellij.rt.execution.application.AppMain org.intellij.xquery.runner.rt.XQueryRunnerApp /usr/home/bridger/.IntelliJIdea14/system/tmp/xquery-run8080433949304952694.xml Exception in thread "main" javax.xml.xquery.XQException: XQJNC001 - Connection refused Root Cause: java.net.ConnectException: Connection refused at net.xqj.basex.BaseXXQDataSource.getConnection(Unknown Source) at org.intellij.xquery.runner.rt.xqj.ConnectionFactory.getConnection(ConnectionFactory.java:36) at org.intellij.xquery.runner.rt.xqj.XQJRunnerApp.run(XQJRunnerApp.java:57) at org.intellij.xquery.runner.rt.XQueryRunnerApp.runConfigForOutputStream(XQueryRunnerApp.java:35) at org.intellij.xquery.runner.rt.XQueryRunnerApp.main(XQueryRunnerApp.java:30) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at net.xqj.basex.BaseXXQDataSource.getConnection(Unknown Source) at org.intellij.xquery.runner.rt.xqj.ConnectionFactory.getConnection(ConnectionFactory.java:36) at org.intellij.xquery.runner.rt.xqj.XQJRunnerApp.run(XQJRunnerApp.java:57) at org.intellij.xquery.runner.rt.XQueryRunnerApp.runConfigForOutputStream(XQueryRunnerApp.java:35) at org.intellij.xquery.runner.rt.XQueryRunnerApp.main(XQueryRunnerApp.java:30) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

4 5

Executing Xquery update from java - hangs
by Ketill Fenne 25 Feb '15

25 Feb '15

Hi! I am executing all Xqueries through the same runQuery method in Java, and I am running the latest jar-files (8.0.1): XQDataSource ds = new BaseXXQDataSource(); xqc = ds.getConnection(); XQExpression xqe = xqc.createExpression(); rs = xqe.executeQuery(query); The query is (in this case): for $doc in (collection('Users/customer_1'))/* return (replace value of node $doc/@name with ("My new customer name")) The update works fine in BaseX Gui, and I suspect the problem is that it does not return anything, and my code expects XQResultSequence. I have tried adding db:output to return part of the query with no luck. Do you have any suggestions? Kind regards Ketill Fenne

2 1

function:module
by Emmanuel Chateau 25 Feb '15

25 Feb '15

Hi, In the result of a function:inspect(), it seems that the @name of the function element is not a prefixed name when using the "declare default namespace" in a function module whereas it is prefixed name for the others. Is it me who miss something, or it should always be the same ? Thanks, Emmanuel

2 4

Performance of PUL commit
by Marco Lettere 25 Feb '15

25 Feb '15

It's me again .... sorry. ;-) One question that puzzles me in these last hours .. This is the scenario on 2 DBs (same server) with the following sizes: 136746 docs ~1 Gb 194608 docs ~0.5 Gb declare %rest:path('{$project}/some/url/') %rest:POST('{$body}') %rest:consumes("application/xml") function _:insert($project, $body) { prof:time( __:insert_impl($project, $body), false(), "POST ACCOUNT") }; The function insert() is an RestXQ entry point for document insertion. It delegates to the actual implementing function. Insertion occurs continuosly. The implementing function performs queries for checking uniqueness of what is inserted and then posts the actual update which is a db:add or db:replace (BTW does this make a difference from a performance point of view?????). Insertion might occur either on one or on both the DBs. The time T output by the profile:time is a steady amount of time. This means that queries and update schedule is working in constant time. The time reported by Jetty (and perceived at the client) is a T + d. The issue is that this d may vary from something around few hundreds of milliseconds to something which is 4 or 5 times T itself. And this happens in waves. When it's slow, it's slow for few minutes and then it gets better for a few minutes and then back again ... When staring at the tcpdump it's evident that all the time is spent between the ack of the request and the sending of the response. Thus network overhead is negligible. This makes me think that the overhead is in the commit of the PUL. Is this possible? I'm using basex 8.0 official. Openjdk 7 and monitoring through JMX doesn't show anything particularly critical. Any clues on what I have to better look into? Thanks and sorry for bothering ... M.

2 11

BaseX 8.0.1: Minor Patches
by Christian Grün 25 Feb '15

25 Feb '15

Hi there, We have just released a first post-Prague version of BaseX, which includes some minor optimizations and fixes (nothing critical): XQUERY - Faster execution of single, index-based results - Iterative evaluation of steps with multiple predicates FIXES - WebDAV locking - Archive Module - Adaptive serialization of arrays and maps - Digest Authentication - Save command-line history In future, we will try our best to release bug fixes more regularly. This way, you can stay up-to-date without the need to check for new versions five times a day. Have fun, Christian [1] http://basex.org/downloads

2 1

Re: [basex-talk] Modelling Checkin-Checkout using locks
by Christian Grün 24 Feb '15

24 Feb '15

I have added a GitHub issue on your locking request [1]. In the meanwhile, it's probably best to use RESTXQ (XQuery) and check if a resource is locked before returning it to a user. [1] https://github.com/BaseXdb/basex/issues/1083 On Mon, Feb 23, 2015 at 6:28 PM, <zopyxfilter(a)gmail.com> wrote: > Andreas Jung wrote: >> Christian Grün wrote: >>>> oops, this was intended for the exist-db guys...but the question >>>> are similar...anyone used BaseX with a checkout-checkin scenario? >>> First of all: I'm also interested to hear more about this from our >>> users.. >> >>> Talking about locking functions, we plan to generalize our WebDAV >>> locking feature to a general database locking mechanism. The WebDAV >>> locks already contain information on the user who created the lock, >>> a timeout, the scope (shared vs. exclusive), the lock id, etc., and >>> this info could be extended by the lock time and other (maybe even >>> application specific) info. Do you have some examples for data that >>> could be specific to an application? >> >> In our SGML CMS you were able to checkout parts of a bigger document >> for example a chapter. In this case we needed additional information >> inside the checked-out document - perhaps not on the lock itself - >> we used processing instructions for that as far as I recall. >> >> Being able to store arbitrary key-value pairs on a lock would be >> sufficient. Otherweise I would store such application specific >> data as an additional XML file containing these informations. > > At least needs to be some support for lock token in order detect > lock/unlock conflicts. > > Andreas

2 1

Xquery issue
by Rob Stapper 24 Feb '15

24 Feb '15

Hi guys, While debugging a xquery-program I saw something strange behavior( to me anyway). I reproduced it in the attached xquery-snipped. Looking at the evaluation data in the query-info screen of the basexGUI after running the program, one can see that function $f($a) is evaluated more than ones. The result is ok but I wondered if this would reduce the performance. I'm running version 8.01 beta d13ddc7 in local GUI-mode. Regards, Rob Stapper --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. http://www.avast.com

3 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk