BaseX-Talk June 2013

basex-talk@mailman.uni-konstanz.de

35 participants
39 discussions

BaseX Script memory leak . . ?
by Christopher.R.Ball 12 Jun '13

12 Jun '13

I have a BaseX script (.bxs) I am running that does queries in batches (sets of 5k documents), but as it progresses it bogs down in speed, does not release memory between sets even if I force it to close and reopen the db between queries, and eventually runs out of memory. But, if I break the same BaseX script into separate files still doing the same exact batches it is extremely fast and memory efficient. Very suggestive of a memory leak . . . I am running on BaseX 7.6.1 Beta. Any thoughts? Is there a way to force the script to do garbage collection?

5 7

Java DOM Interface and updating XQueries
by Ingo Dengler 12 Jun '13

12 Jun '13

2 1

Loading Large xml file in DB taking long time
by Pushpendra Singh Sengar 12 Jun '13

12 Jun '13

Hi, I am loading 5GB xml in DB it takes around 10-12 minutes. I need to load about 20-30 document of same size (5Gb) , So i created thread to loading these document in DB but its taking too much time ,I wait till 4 hrs after that I stop process to load document . Pls tell me the best solution to loading large document. my program is following : package org.basex.examples.query; import java.io.*; import javax.xml.namespace.*; import org.basex.build.*; import org.basex.core.*; import org.basex.core.cmd.*; import org.xml.sax.Parser; /** * This class demonstrates collection relevant queries. * It shows how to find and query specific documents. * * @author BaseX Team 2005-12, BSD License */ public final class QueryCollection implements Runnable{ Context context = new Context(); String dbName = null; public QueryCollection(String string) { dbName = string; } /** * Runs the example code. * @param args (ignored) command-line arguments * @throws BaseXException if a database command fails * @throws XQException */ public static void main(final String[] args) throws BaseXException { /** Database context. */ Context context = new Context(); System.out.println("=== QueryCollection ==="); // ------------------------------------------------------------------------ // Create a collection from all XML documents in the specified directory System.out.println("\n* Create a collection."); System.out.println(System.currentTimeMillis()/1000); Thread thread = new Thread(new QueryCollection("Collection1")); Thread thread1 = new Thread(new QueryCollection("Collection2")); Thread thread2 = new Thread(new QueryCollection("Collection3")); Thread thread3 = new Thread(new QueryCollection("Collection4")); Thread thread4 = new Thread(new QueryCollection("Collection5")); thread.start(); thread1.start(); thread2.start(); thread3.start(); thread4.start(); OutputStream outputStream =null; try { outputStream = new FileOutputStream(new File("result.xml")); } catch(FileNotFoundException e) { // TODO Auto-generated catch block e.printStackTrace(); } // ------------------------------------------------------------------------ // Evaluate a query on a single document System.out.println(System.currentTimeMillis()/1000); // new DropDB("Collection").execute(context); // ------------------------------------------------------------------------ // Close the database context context.close(); } public void createDB(String dbName){ // TODO Auto-generated constructor stub new CreateDB(dbName, "E:/downloads/temp/test.xml").run(context); } @Override public void run() { // TODO Auto-generated method stub System.out.println(dbName+" start time : "+System.currentTimeMillis()/1000); createDB(dbName); System.out.println(dbName+" end time : "+System.currentTimeMillis()/1000); } } -- Thanks & Regards , Pushpendra Singh

2 1

Binding variables in scripts (bxs)
by Wendell Piez 12 Jun '13

12 Jun '13

Hi, I'm sure you will forgive me if this is as dumb as I'm afraid it might be ... trying to cover all possibilities here. We recently upgraded (now on 7.7beta) and external scripts stopped working as expected -- variable bindings of the form SET BINDINGS $db=my-db,$start=20001,$count=10000 no longer had any effect. Instead, the logic fell back to use values declared for the variables in the XQuery, which had: declare variable $db := 'your-db'; declare variable $start := 1; declare variable $count := 1000; etc. When we removed the declarations from the XQuery, things started working again. (Note that my assumption of a causal relation here could be wrong.) Should I never have put those declarations in to begin with (even as defaults, a la XSLT parameter declarations)? Is this a BaseX question or an XQuery question? I would like to become enlightened. Thanks as ever, Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

3 2

Re: [basex-talk] Java DOM Interface and updating XQueries
by Ingo Dengler 12 Jun '13

12 Jun '13

Hi, Sorry for posting the first message in HTML format (damn web mailer...). Here is it again in Text only: I have a problem using BaseX Java DOM Node Interface and updating XQuerys in combination. The problem is that on updating the XML structure the node "pointer" moves to another location therefore the BXNode Object points to the wrong node. I have investigated a bit and the problem seems to be the 'pre' attribute in Class DBNode that points to the wrong/old location. There is already an uninque ID in ANode that could be used to identify the right node. But a ID->pre mapping table is missing. Any solution/workaround for this issue? Best regards, Ingo See following snippet that demonstrates the issue: import java.io.ByteArrayInputStream; import org.basex.api.dom.BXNode; import org.basex.core.Context; import org.basex.core.Prop; import org.basex.core.cmd.CreateDB; import org.basex.core.cmd.Set; import org.basex.query.QueryProcessor; import org.basex.query.value.item.Item; import org.basex.query.value.node.ANode; public class UpdateAndDOM { public static void main(String[] args) throws Exception { // create test DB in memory Context context = new Context(); new Set(Prop.MAINMEM, true).execute(context); ByteArrayInputStream bais = new ByteArrayInputStream("<root><node1/></root>".getBytes()); CreateDB cmd = new CreateDB("testDB"); cmd.setInput(bais); cmd.execute(context); //get DOM node "node1" QueryProcessor query = new QueryProcessor("//node1", context); query.execute(); Item item = query.iter().next(); Object elem = BXNode.get((ANode)item); BXNode node1 = (BXNode)elem; System.out.println("Expected node1 is: " + node1.getNodeName()); //updating query that inserts data before node1 String xpath = "insert node <newNode/> before /root/node1"; query = new QueryProcessor(xpath, context); query.execute(); System.out.println("Expected node1 is: " + node1.getNodeName()); } }

1 0

Bind Externals via RUN Command
by James Hughes 12 Jun '13

12 Jun '13

Hello, I'm wondering if it is at all possible to an external variable to a query when it is being executed via the RUN command? e.g. RUN /queries/country.xq Which has (condensed for brevity) declare variable $country_code external; //iati-activity/recipient-country[@code='{ $country_code }']/ Is this possible when calling from a client? James

2 1

CloudHub 101 Webinar [Register]
by Cameron Coles 11 Jun '13

11 Jun '13

Live Webinar <http://www.mulesoft.com> >From zero to integration in 15 minutes with CloudHub <http://www.mulesoft.com/webinars/cloudhub-integration-demo> The myth is that integration takes weeks to build and deploy, not minutes. Join this live demo and follow along as we show how to integrate SaaS and on-premise applications at cloud speed. Don’t forget to bring your stopwatch to test whether this is myth is confirmed or busted. This MuleSoft event will cover: - Creating a CloudHub account - Configuring a message queue to Salesforce integration flow - Deploying integration to CloudHub - Monitoring performance from CloudHub Insight The demo will be followed by an interactive Q&A session. Register Now » Thursday, June 13, 2013 8AM Pacific, 11AM Eastern, 16:00 BST Forward to a colleague » <http://resources.mulesoft.com/index.php/email/emailWebview?ftf=true&mkt_tok…> Presenter: Tom Motley, Sr. Solutions Consultant, MuleSoft Date: Thursday, June 13, 2013 Time: 8AM Pacific, 11AM Eastern, 16:00 BST Length: 45 Minutes <http://www.mulesoft.com/webinars/cloudhub-integration-demo> Presenter: Tom Motley Sr. Solutions Consultant, MuleSoft <http://www.mulesoft.com/webinars/cloudhub-integration-demo> <http://www.mulesoft.com> <mailto:info@mulesoft.com> <http://blogs.mulesoft.org/> <http://twitter.com/#!/mulesoft> <http://www.facebook.com/MuleSoft> ©2013 MuleSoft, Inc. All rights reserved. 77 Geary Street, Suite 400, San Francisco, CA 94108 If you no longer wish to receive these emails, go to the following link to unsubscribe: http://resources.mulesoft.com/UnsubscribePage.html?mkt_unsubscribe=1&mkt_to….

1 0

Catching BXFE0001 when fetching binary
by Alexander von Bernuth 11 Jun '13

11 Jun '13

Hi folks, I'm a student working on my bachelor thesis. I use BaseX to read large XML files and forward them to a PostgreSQL database. Additionally I want to fetch images via URI and add them to the database-records. While fetching these images, I have to test three different file extensions for each image, as my source does not use one single image format (but jpg, png and gif). First I tried checking with fn:doc-available($uri) weather that certain URI is available, but this obviously only works for XML-documents. Then I wanted to catch the err:BXFE0001 that is thrown by fetch:binary($uri), but this does not work, either; the error-message is displayed and the script stops. > declare variable $baseuri := 'http://www.bricklink.com/ML/'; > declare variable $itemid := 'sw008'; > > try { > fetch:binary($baseuri || $itemid || '.png') > } > catch err:BXFE0001 { > try { > fetch:binary($baseuri || $itemid || '.gif') > } > catch err:BXFE0001 { > try { > fetch:binary($baseuri || $itemid || '.jpg') > } > catch err:BXFE0001 { > 0 > } > } > } In this example, only the .jpg-image exists. But all I get is > [BXFE0001] Resource "http://www.bricklink.com/ML/sw008.png" not found. > Information: > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> > <HTML><HEAD><TITLE>The page cannot be found</TITLE> … and so on I would be very glad if you could help me with this, either by pointing out my coding-error or by showing me another, better solution for doing this. Thank you and have a nice day, Alex PS: My sources on Try/Catch and fetching -- | Alexander von Bernuth | alexander.von-bernuth(a)student.uni-tuebingen.de

2 2

baseX-core vs baseX-api
by David Dobbs 10 Jun '13

10 Jun '13

So, I have a couple of (hopefully simple) questions about the Java APIs provided by BaseX. Based on the comments in the examples, it looks like the function calls in the RunQueries.java example (https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/bas…) are strictly local calls that operate directly on local XML documents - there's no BaseX server involved, no client communication, no files created in org.basex.path, etc. Is that correct? What's the difference between the queries in RunQueries.java and QueryCollection.java? My guess from skimming the documentation is that, per above, RunQueries just uses BaseX's XQuery API, without taking advantage of BaseX's indexing, etc capabilities, whereas QueryCollection creates a (local) BaseX server, indexes the document, and then its queries would be accelerated using those indexes. Is this also correct? Thanks, -David

2 1

(x)query, db lookup, and update in one step
by buzz chopra 08 Jun '13

08 Jun '13

Hello All, I'm relatively new to basex and xquery - but am using basex as a large report repository. I'd like to try and perform the following xquery - but with some intermediate steps: xquery for $rep in collection('coll')/Report let $attr0 := $rep/@attr0 return (replace value of node $rep/@attr1 with 'zzz') The actual value represented by 'zzz' above must be based on a separate (physical) database lookup for each attribute value $attr0 in the collection ( i.e. the values of attr1 is a (non-trivial lookup) function of attr0. Is this a scenario in which multiple queries would have to be made based on the number of documents in the collection ? Perhaps there is some slick way of including a call out to some method to such that the 'zzz' above is replaced with something like: JavaModule.doLookup($attr0) ? I'm executing xquery from a jruby application so either the ruby or the java client can be used (perhaps that is somewhat irrelevant). Thanks in advance for any suggestions, Buzz

2 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk June 2013