- BaseX-Talk - mailman.uni-konstanz.de

concurrency within xquery
by Seenivasan Gunabalan 29 Jan '13

29 Jan '13

Hi all, Is there a way to control concurrency from inside xqueries? Though its possible to achieve concurrency through REST (and RESTXQ?) interface, I am looking for ways to declaratively ask an xquery to run subqueries in a concurrent way. Thanks in advance Seenivasan

5 4

Fwd: BaseX Performance Test - hitting a performance wall
by Meredith Gregory 29 Jan '13

29 Jan '13

Dear Christian, We've been using your experimental locking to improve performance of BaseX in our application. It definitely helps, but right behind this bottleneck is file-io blocking. i'd like to get a deeper understanding of how your file-io is organized and what your caching scheme looks like. What sort of file-io caching mechanism is in place now? What are your plans? Is there anything we can do to help that along? Best wishes, --greg ---------- Forwarded message ---------- From: Jason Klassen <Jason.Klassen(a)protegra.com> Date: Tue, Jan 22, 2013 at 9:35 AM Subject: BaseX Performance Test - hitting a performance wall ... I’ve run the performance tests with 7.5.1 servers using DB Locking and with varying levels of parallel reads. *Unfortunately there is no performance gain*. While the socket.io blocking is gone the disk io blocking right after impacts performance just as much. Results are:**** ** ** previous (8 reads)**** 32 reads**** 64 reads**** 192 reads**** first**** last**** first**** last**** first**** Last**** first**** last**** UAT 10 by 1000**** 8.76**** 14.16**** 8.79**** 12.99**** 9.12**** 12.73**** 8.4**** 12.9**** ** ** Running with 192 reads and causing 10 by 10 by 1000, 10 users by 10 concurrent searches on 300ms delay each, by an affiliate with 1000. (above is 10 users by 1 concurrent search by an affiliate with 1000)**** ** ** I’ve attached some screenshots showing what the basex jar is doing when the tests run. It seems only ~1-2 threads can read from disk at any given time. The red sections are all 3 common issues, TableDiskAccess.read1, TableDiskAccess.read2, DataAccess.readToken, the dumps are attached as well. ** ** ** ** Looking at the source for basex TableDiskAccess<https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/io/ran…> and DataAccess<https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/io/ran…>there is no file system caching. Every test is doing a buffer read on the exact same database even though it does not change from a write. So instead of basex realizing this, it just blindly goes to the disk each time. **** ** ** We’ll run some tests on a vm backed by ssd to see if that can resolve the disk access*. If it can’t, and without any kind of caching logic, this looks to be as far as we can go with basex performance and I’m not sure it will be enough.* Even if caching logic existed for read, I’m still concerned that many writes with basex will block it the same way as it doesn’t appear to do any log or page caching as most relational dbs do now to limit disk io impacts. Sharding the dbs across 16 servers would have helped for socket.io but it will not solve physical disk io. **** ** ** Jason**** ** ** *From:* Sasa Ivetic *Sent:* January-18-13 3:05 PM *...* ** ** Hi everyone,**** ** ** I’ve spent a few hours today writing a simple BaseX test (mimics the BaseXPersistTest in KVDB) in order to analyze performance of different machines we have available to us. Results are interesting, and demonstrate that the test is likely heavily I/O bound on our Production server hosted by Rackforce. I’ve attached two quick graphs I made from the results. “BaseX Test Results Chart.png” is a graph of total runtime based on specified parameters, with Y Axis being measured in ms (lower is better). “BaseX Test Results Chart – Queries Per Second” is a graph of total queries per second (higher is better).**** ** ** PARALLEL parameter is a BaseX Server setting indicating how many read/write processes can run concurrently**** T parameter is the number of client threads used to create read/write requests**** QPT parameter is the number of queries each thread runs**** ** ** Amazon EC2 is the High I/O instance, with 16 cores, and 1TB worth of SSD-backed storage**** My laptop is a Core i7 (4 cores w/ HT) with a 7200rpm drive**** Beta is our own internal VM**** Prod is the Rackforce VM, in their high performance cloud and with the high performance SAN (which we’re pretty sure is SSD backed as well) **** ** ** For purposes of the above test, 5% of total queries were write queries. On average, each test executed just under 19k read queries and just over 1k write queries (for a total of 20k queries). The test is only a micro-benchmark, but I feel it reflects daily Protunity usage reasonably well.**** ** ** I will also try to spend some time testing raw I/O throughput (and more importantly) latency with third-party software to get a better grasp of the differences between the various servers.**** ** ** ** ** _________________**** Sasa Ivetic**** 204-488-5697**** ** ** -- L.G. Meredith Managing Partner Biosimilarity LLC 7329 39th Ave SW Seattle, WA 98136 +1 206.650.3740 http://biosimilarity.blogspot.com

2 1

text() vs string()
by Wendell Piez 29 Jan '13

29 Jan '13

Hi, This may be a question about XQuery Full Text, or only about common usage (or misusage?) of XPath; in either case I hope it's on topic. Please tell me if not. In BaseX [A]: let $test := <test> The apple never falls far from the tree. Apples and trees. Trees and  apples. <fruit>Apple</fruit> trees. </test> return $test/*[text() contains text ('apple' ftand 'tree') using stemming using language 'en'] This returns  Apples and trees. As an experienced XPath user, this is what I expect, assuming "contains text" allows a sequence of nodes as its first argument (and returns true if any of them satisfies the test). Only the second 'p' element has a child text node whose value contains both "apple" and "tree". Of course the problem in the others is the mixed content: in the first, an element node 'em' intervenes, while in the third, a comment intervenes, so both these cases contain text nodes with either "apple" or "tree", but not both. In the case of the fourth 'p', there is no text node child containing "apple" at all, only a grandchild. Assuming I want all four back, I can write either: [B] return $test/*[string() contains text ('apple' ftand 'tree') using stemming using language 'en'] or [C] return $test/*[. contains text ('apple' ftand 'tree') using stemming using language 'en'] In the case of [B], the string() function casts the element to a string, flattening its structure. [C] passes the element itself to the "contains text" operation, which happily has the same effect. I have several related questions about this: 1. Unless I learn better, I'm going to prefer [B] or [C], because in my world, mixed content is common; is there any reason (performance or otherwise) to prefer [A] in cases where I know it will be robust? Is there any reason to prefer [B] or prefer [C]? 2. I see examples like [A] offered frequently in the XQuery literature, of "text()" being used apparently to refer to an element's string (text) value not to its text node children. And I see this usage in running code. I can only imagine that those who write it are simply not aware that mixed content will complicate their queries like this; maybe they have just never thought about it, or they don't know what text() actually does. In any case, the error is pernicious, since nothing tells you the query you gave isn't the one you intended -- it even works, until the day it doesn't, and the cases where gives correct but unwanted results may be rare. But maybe I'm wrong and they just know something about XQuery, XQuery FT, or their tools, that I don't. What do the experts say? Cheers, Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

5 4

Scientific Numbers in Scatter Plot
by IVES Greg 25 Jan '13

25 Jan '13

Hi I find BaseX a very powerful tool. One thing I noticed however is that it misunderstands numbers in scientific notation in the scatter plot display. Please see the attached plot. The samples should form a rough diagonal line. However, look at the scale on the left and you can see the exponent field is 'dis-continuous'. It looks like the chart formatter treats it as a non-scientific number or possibly even lexicographic order. Best Regards Greg

2 3

Collection related functions [SEC=UNCLASSIFIED]
by Thamm, Russell 24 Jan '13

24 Jan '13

UNCLASSIFIED Hi, Does baseX provide any collection related functions? In particular, I am after equivalents to the eXist functions: xmldb:create-collection xmldb:get-child-collections xmldb:get-child-resources I have searched the module library documentation and can't seem to find anything. I can duplicate the action of the latter two using db:list but that appears to be rather inefficient. BaseX doesn't seem to like empty collections, so the first might be superfluous. Cheers IMPORTANT: This email remains the property of the Department of Defence and is subject to the jurisdiction of section 70 of the Crimes Act 1914. If you have received this email in error, you are requested to contact the sender and delete the email.

2 2

Basex performance
by Mark Dwyer 24 Jan '13

24 Jan '13

Hi, I was considering the use of BASEX for an XPATH based solution I have which uses XPATH to select data from moderately large messages. Say between 20 and 50 Mb. Are there any relative performance stats in using BASEX over SUN/ORACLE Java implementations of XPATH ? Mark A copy of the LCH.Clearnet e-mail disclaimer can be found at: www.lchclearnet.com/disclaimer/email LCH.Clearnet Limited, Registered Office: Aldgate House, 33 Aldgate High Street, London EC3N 1EA. Recognised as a Clearing House under the Financial Services & Markets Act 2000. Reg. in England No. 25932. LCH.Clearnet SA, Siège Social: 18 rue du Quatre Septembre, 75002 Paris, Chambre de Compensation conformément au Code Monétaire et Financier. LCH.Clearnet LLC, Principal Office: 17 State Street, New York, NY 10004. Registered with the U.S. Commodity Futures Trading Commission as a derivatives clearing organization under the U.S. Commodity Exchange Act.

2 1

Performance with fulltext searches
by Charles Duffy 24 Jan '13

24 Jan '13

Howdy, all -- If I run the following query against a large, full-text-indexed store (output from "svn log --xml" on a repository with several hundred thousand entries), I get results back in well under a second: count(/log/logentry[./paths/path/text() contains text "trunk"]) However, the following variant takes multiple minutes to execute -- actually, has never yet completed before I've lost patience and cancelled it: count(/log/logentry[ ./paths/path/text() contains text "trunk" and not( (./paths/path/text() contains text "tags") or (./paths/path/text() contains text "branches") ) ]) Is there a way to get reasonable(ish) performance for the more complex query?

3 2

HTTPS
by pw＠themail.co.uk 24 Jan '13

24 Jan '13

Dear List Since Win7, Win2008 etc don't by default allow establishing WebDAV connections with basic authentication on http connections, what is the best way to configure the BaseX httpserver for https? TIA Peter

2 1

Re: [basex-talk] BaseX: Support for JSON
by Abishek N 22 Jan '13

22 Jan '13

Hello I am using twitter4j streaming api to capture tweets and feeding them into a basex database, converting them into xml using json api. While using the json:parse api, i get BXJS0001 error often. I assume it is because of encoding. Is there an option available in the json: api that would allow me to specify the enoding technique? Also if i store the documents as a json, instead would it facilitate json query? Thanks, Abishek

3 5

Fwd: dynamic web application, XQJ, BaseX
by marco fregonese 22 Jan '13

22 Jan '13

---------- Forwarded message ---------- From: marco fregonese <marcofregonese(a)gmail.com> Date: 2013/1/21 Subject: Re: [basex-talk] dynamic web application, XQJ, BaseX To: Christian Grün <christian.gruen(a)gmail.com> There is a class in my Java web application that has a method that, after connecting to my BaseX database, execute a query on it. This query (associated to the "xquerystring" variable), after importing a library module and pointing the processor to the file containing the library module("library_module.xq"), uses the function "lib:getparts()". Now, after executing this method, I noticed that: with BaseX 7.3 the Xquery processor (o whatever it is, I don't know) looks for "library_module.xq" in the filesystem $HOME/basex/bin, while with BaseX 7.5 the processor looks for "library_module.xq" in $HOME. My question basically is: what is that makes the processor (or whatever it is) look for the file "library_module.xq" in $HOME/basex/bin (with BaseX 7.3) and in $HOME (with BaseX 7.5)? Hope I've been clear. Thanks PS: in BaseX GUI everything works fine: the XQuery processor looks for the file containing the library module in the same folder where the main module is. Example of main module: import module namespace lib="http://www.example.com/my_library_module" at "library_module.xq"; lib:getparts() The following is the code of the class I've talked about: package xqu; import java.io.File; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.PrintStream; import javax.xml.namespace.QName; import javax.xml.xquery.XQConnection; import javax.xml.xquery.XQException; import javax.xml.xquery.XQExpression; import javax.xml.xquery.XQResultSequence; import con.Connection; public class Query { public static void getAllParts(String a, String sc) throws XQException, FileNotFoundException { File file = new File(sc); FileOutputStream fos = new FileOutputStream(file); PrintStream ps = new PrintStream(fos); System.setOut(ps); XQConnection con = Connection.getConnection(); XQExpression xqe = con.createExpression(); xqe.bindString(new QName("b"), a, null); String xqueryString = // using backslash for escaping " "import module namespace lib=\" http://www.example.com/library_module\"" + "at \"library_module.xq\";" + "declare variable $b external; " + "lib:getparts($b)"; XQResultSequence rs = xqe.executeQuery(xqueryString); String b = rs.getSequenceAsString(null); con.close(); System.out.println(b); } } 2013/1/20 Christian Grün <christian.gruen(a)gmail.com> > Hi Marco, > > sorry for the late feedback. Can you reproduce the issues in the BaseX > GUI or on command line? > > Best, > Christian > ___________________________________ > >

2 2

Hi all, and URL Rewriting/Request handling question
by Jay Straw 22 Jan '13

22 Jan '13

Hi all, I've been developing in eXist since 2008, and I'm now trying out your BaseX. I'm impressed so far, but how can I perform URL rewriting? I've tried using Jetty's Rewrite Handler via the example code at http://wiki.eclipse.org/Jetty/Feature/Rewrite_Handler but then the server fails to start, it simply dumps with: $ sh bin/basexhttp org.eclipse.jetty.rewrite.handler.RewriteHandler So I figure I need to compile it in (I grabbed from github and built with maven, but didn't know how to build the http server and downloaded the 5meg zip). Not sure how to do that. I'm not a Java guy at the moment, so any pointers would be very helpful. Thanks everybody! Jay

2 1

Use of BaseX
by Ankur Kasliwal 21 Jan '13

21 Jan '13

Hi, I am new to BaseX and documentation is also not very helpful for me. I am trying to deal with a collection of XML files present on the server. I am able to access those files using REST . What I want to do is read those XML files from a collection present in a certain folder and using Xpath/Xquery get certain nodes from these files and populate a RDBMS corresponding to them. For example: &gt bookstore &lt &gt book &lt abc &gt/ book &lt &gt price &lt 123 &gt /title &lt &gt title &lt xyz &gt /title &lt &gt /book &lt I have 10 of these files in my collection. So my question is how can I query all these Xml files at ones and get specific results like "Books with price value greater than 100". Once I get these results I want to store the result in a RDBMS so that at later point I can query that RDBMS to generate certain reports. NOTE: I am a graduate student and just doing this for learning and academic purpose. Hope to hear from someone soon. -- Ankur ankur.kasliwal.26(a)gmail.com 001-(916)673-7559

3 3

dynamic web application, XQJ, BaseX
by marco fregonese 20 Jan '13

20 Jan '13

Hi there, I'm developing a Java dynamic web application in Eclipse Juno using XQJ and BaseX (OS Ubuntu 12.04). I'm using an XQuery main module importing a library module in my Java-XQJ code to query BaseX, something like this: .... String xqueryString = // import library module and point the processor to the file containing the library module "import module namespace lib=\" http://www.example.com/mylibrary_module\" at \"mylibrary_module.xq\";" + "lib:myfunction()"; XQResultSequence rs = xqe.executeQuery(xqueryString); ... Two questions: 1. Why does the processor looks for "mylibrary_module.xq" in ~/basex/bin? 2. Is there any way to change this path? Thanks.

2 2

Fwd: basex_7.5-1_amd64.changes ACCEPTED into unstable
by Alexander Holupirek 17 Jan '13

17 Jan '13

To all the Debian users. BaseX 7.5 has been accepted in unstable. Sorry for letting you wait so long. Begin forwarded message: > From: Debian FTP Masters <ftpmaster(a)ftp-master.debian.org> > Subject: basex_7.5-1_amd64.changes ACCEPTED into unstable > Date: 18. Januar 2013 00:17:33 MEZ > To: Alexander Holupirek <alex(a)holupirek.de>, <tmancill(a)debian.org> > > Accepted: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Format: 1.8 > Date: Tue, 15 Jan 2013 10:00:12 +0100 > Source: basex > Binary: basex > Architecture: source all > Version: 7.5-1 > Distribution: unstable > Urgency: low > Maintainer: Alexander Holupirek <alex(a)holupirek.de> > Changed-By: Alexander Holupirek <alex(a)holupirek.de> > Description: > basex - XML database and XPath/XQuery processor > Closes: 696796 696804 697037 > Changes: > basex (7.5-1) unstable; urgency=low > . > * New upstream release (December 21, 2012) (closes: #697037) > XQUERY UPDATE > - bulk updates are now much faster than before > - insert and replace operations take much less memory > - databases can now be created via XQuery and db:create() > . > GUI TEXT EDITOR > - a fast and flexible search/replace panel has been added > - error highlighting has been extended to XML files > - the editor was improved for editing arbitrary text files > . > XQUERY 3.0 > - the code has been aligned with latest changes in the specs > - HTML5 support has been updated > . > WEB APPLICATIONS > - new modules have been added: Request, Session, Sessions > - logging has been revised and extended to HTTP requests > - SSL support added, switched to Jetty 8 > - all operation modes have been unified and simplified > - RESTXQ elements added to simplify forwarding and redirects > . > CORE FEATURES > - improved stability of concurrent read/write operations > - the INSPECT command performs database sanity checks > - EXECUTE triggers the execution of command scripts > . > * Packaging: > - Added example provided by jidanni(a)jidanni.org to basex.1 > Thanks! (closes: #696804) > - Added note for missing -q option to basex.1 (closes: #696796) > Reported issue upstream: > https://github.com/BaseXdb/basex/issues/620 > Checksums-Sha1: > f2557eb167c7833402ce771a1fb8b1bd7499a0a8 1856 basex_7.5-1.dsc > 9dd929c365333bc6bcc2681b782c4fcaaad3a94a 1315007 basex_7.5.orig.tar.gz > e8b7ff70d349bf4b69b588b74aa69f2b82dbb016 98461 basex_7.5-1.debian.tar.gz > 2e9f5a817befb3a680822cb58ad808f42e98c461 2489490 basex_7.5-1_all.deb > Checksums-Sha256: > 017ecab180dda4499b557f537f7f30813a7b615af47dc8d8bae827d02f47fe3d 1856 basex_7.5-1.dsc > 3fbc7b2fe2aa7427ad312cbe0aded3fcc86d029e6f6cd38476c5fc5d59ef7c30 1315007 basex_7.5.orig.tar.gz > 851bd420ecc0e9ac706a80251b7dd7ffbc9adec7cd83ecc99173a00705f26494 98461 basex_7.5-1.debian.tar.gz > 211f97595d0004e71ae2cd9d3822e13e54eb111b37a5ec25362d8ec9d0dfc315 2489490 basex_7.5-1_all.deb > Files: > 0a815272fac2cf303c9fce8350faace9 1856 database optional basex_7.5-1.dsc > 7028a0e81cecf02b73ba6d30cdc261fd 1315007 database optional basex_7.5.orig.tar.gz > a6b8dc8d172e8cfa93af05c18644b390 98461 database optional basex_7.5-1.debian.tar.gz > 3b3571e5b6d078867e40329bfc06ffb3 2489490 database optional basex_7.5-1_all.deb > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (GNU/Linux) > > iQIcBAEBAgAGBQJQ+IDGAAoJECHSBYmXSz6WT4IP/1tKOwPKJlzDQGHrDZM015vD > OFo300qJK32pSD9HDHfKBpp8m6+DqpgEzzE7xeCbo2/o/1r6y8JZHGbgNxIw+AhB > Vio2fcq7EYelj2/NEGaohAhMsCVqpwxGdAcO+WURjJZJUBphO7dZxv8uFDY2d/yD > 9NRLm4Ta63c9Whg5ozMaWh3KNJR/fc6AksPH1rMgKqWETJvnm5VgLuEYV2pd/+x4 > fH6QeDxeko1GGQZUaXQjf+bqWff+QRaCevy6rQOkF6Io/PYOfniuhBGIMEm4fs/d > +SGc8dGuXOgaJ2ALeb6yG0ZnMdeEVQ9vRO5LcJJv+YkMmIG75LewojXMpwwFGA/J > Af4o6WS4zFjHH+dLAbsKntrDhiLd1h3T8Urt9CA3T3cEP4xTUWDjNRAEwV5ebqRG > veD7qPzLXZErVKcrLKXaYXEg5sYS0DIjB0Q6sUYYPDfUmhYTIFTyRf3vaDuoc0Er > xW24/we8aVAnSAOcg63dPKajUz+slnxZPn6iNgqJUe+r1R1Qn/aMntPZfgeARmmx > YZKN5GXcABzCPx7N+LGyXTOPg4fkda+vFK8x70vm2Sf0Tet0Z4PBPvxKDMQzu9BF > H/7tHLAqM8aCWZNfHz8e12xbQctyaZiz6i24i9F1a+Dh94Vjjpvyd7HBZ7ZcHWnb > jo6iobCXljlk20aa/JlO > =/S7U > -----END PGP SIGNATURE----- > > > Thank you for your contribution to Debian.

1 0

Changing content through Data Source/Oxygen
by France Baril 17 Jan '13

17 Jan '13

Hi, Our .xqm uses a configuration file that is stored in a DB called AppResources. When we use the GUI to create/upload the content for AppResources, we can access our restxq pages and the result is based on our config. If we access the config file in Oxygen and change anything, even a space, it's like the .xqm cannot access the information anymore. We tried: - Updating/optimize the index/paths. - Stopping/starting the server. The file is saved properly. If we save it on disk from Oxygen and reload AppResources with the config file, we are able to access the information in the file again. So the issue is really the fact that the change occurs through webdav. What are we missing here? Regards, France -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 11

fastest way to add/replace/delete xml
by Fabrice Etanchaud 17 Jan '13

17 Jan '13

Dear all, I have to store and replace xml pieces, and am wondering what can be the fastest way to achieve that task. - Add provided files containing several xml pieces, and then use xquery update to replace nodes ? - Or split files to add a document per xml piece, and then use db:replace ? These databases have size near the physical limit. Regards, Fabrice Etanchaud Questel-Orbit

2 1

[XQuery] Sharing a value between expressions?
by Johannes Krampf 16 Jan '13

16 Jan '13

Hello, I would like to assign a sequence to a variable and then share this variable between two expressions. Particularly I want to create a list of documents to delete from the db and then return the number of deleted documents. Code sketch of what I am trying: let $nodes := [...] let $base-uris := for $node in $nodes return base-uri($node) let $distinct-base-uris := distinct-values($base-uris) for $base-uri in $distinct-base-uris return db:delete("db", $base-uri), return count($distinct-base-uris) --- The problem with this approach is that $distinct-base-uris is local to the first expressions and not in scope for the count call. I tried to just duplicate all code, but then I get an error message that "no updating expression [is] allowed". Is there any way to achieve what I want and put my value in a scope accessible by both expressions? Regards, Johannes

2 2

DB freezes on updating restxq - a simpler example
by France Baril 15 Jan '13

15 Jan '13

Hi, I have sent a request for this issue last month, but my example was too complex. Here is the simplest example I could get that reproduces the issue. This seems to be a problem with updating restqx functions, so I'm unsure how to reproduce it with a .bsx. If you use the attached db with the attached query on localhost:8984. The browser should freeze on BaseXMas when accessing debug/debug-freeze. - The xslt works. If you make the function none updating and return $result. You'll see the video.png in the browser. - The updating function works. If you replace the node $result with <hey/> and comment out the xslt:transform line. The document will be replace in the db. - The two don't work together. Specific characteristic if this xsl: - It uses a rest access to extract the path for the video. Without the doc('http://...rest…') line, this would work. Note that this used to work with prior versions of BaseX. Thanks, France -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 1

Database locking pre-release published
by Jens Erat 15 Jan '13

15 Jan '13

Dear BaseX community, we're glad to announce some pre-release-version of a much-requested and long-awaited feature: database locking. Database locking will enable higher parallelism, it will be possibly to perform updates to some database without blocking all other databases (actually even parallel updates to different databases). Please notice this is a pre-release developer version which is not yet fully tested and optimized. We will announce as soon as we regard it ready for production. It is included in most current [development release]. Have a look at our [wiki documentation] for more information on limitations and [how to enable]. During the next weeks, we will continue to include more optimizations and reduce the number of queries requiring global locks and add fixes if necessary. We will be glad receiving your feedback! As I'll be writing my thesis on these locking enhancements, I'd also be glad on some user stories and requirements (which could be considered in prioritizing efforts on optimizations, too). You can also mail me directly if you do not want to post to the list. Regards from Lake Constance, Germany, Jens Erat [development release]: http://files.basex.org/releases/latest/ [wiki documentation]: http://docs.basex.org/wiki/Transaction_Management [how to enable]: http://docs.basex.org/wiki/Transaction_Management#How_to_enable -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B

1 0

Determining whether a function exists
by Charles Duffy 15 Jan '13

15 Jan '13

Howdy -- I'm trying to determine whether sql:execute-prepared exists, to have a module that functions in both BaseX 7.3 and 7.5. However -- exists(function-lookup(xs:QName("sql:execute-prepared"), 2)) returns false even in BaseX 7.5, even though function-name(function-lookup(xs:QName("sql:execute-prepared"), 2)) properly returns "sql:execute-prepared". Am I going to have to use a try block to detect this case?

2 3

256 namespaces limit
by Imsieke, Gerrit, le-tex 15 Jan '13

15 Jan '13

Dear Team, Do you plan on increasing the 256 namespaces limit any time soon? I know that there is the STRIPNS option, but this does not fit my use case (which is to index all my XSLT and XProc files) too well. Gerrit

3 2

unzip
by Jason Harrop 14 Jan '13

14 Jan '13

Hello eXist and MarkLogic both have mechanisms for unzipping a resource when it is loaded into the database. In eXist, you can use a trigger and its unzip module. For MarkLogic, you can use its Office OpenXML Extract pipeline. See http://robotbeerbash.wordpress.com/2009/08/31/getting-started-with-open-xml… What's the easiest way to do this sort of thing in basex? thanks .. Jason

4 7

Re: [basex-talk] best way to partition large data sets among collections
by pw＠themail.co.uk 14 Jan '13

14 Jan '13

Hi Christian Thank you for directing me to the profiling module, I think that is just what I need Cheers Peter > > > >---- Original Message ---- >From: christian.gruen(a)gmail.com >To: pw(a)themail.co.uk >Subject: Re: [basex-talk] best way to partition large data sets among >collections >Date: Mon, 14 Jan 2013 19:13:12 +0100 > >>Hi Peter, >> >>> Do you have any information to guide me here; what sorts of XQuery >>> expressions should I match with large numbers of collections, and >>> which with small numbers of collections? >> >>hmm, there is no answer I get in mind that could give you general >>guidance here, as XQuery provides just too many possibilities for >>writing slow and fast queries. Its similar to the question how to >>write efficient Java, Perl of whatever.. To do get more information >on >>why your queries are not as fast as they ideally should be, you >could >>e.g... >> >> check in the InfoView/query info if the relevant index structures >are used >> use functions of the Profiling Module [1] to track down >bottlenecks >> use -Xmx get do low level profiling >> >>Next, you may pass us on snippets of your code that may, in your >>opinion, be subject to being optimized. >>Christian >> >>[1] http://docs.basex.org/wiki/Profiling_Module >> >> >>> If I am using data-rich XML there is a high ration of nodes to >>> content. What are the rules of thumb for this type of content? >>> >>> Are there any recommendations specifically for GML? >>> >>> Many thanks >>> >>> Peter >>> >>>> >>>> >>>> >>>>---- Original Message ---- >>>>From: christian.gruen(a)gmail.com >>>>To: pw(a)themail.co.uk >>>>Subject: Re: [basex-talk] best way to partition large data sets >among >>>>collections >>>>Date: Mon, 14 Jan 2013 13:05:00 +0100 >>>> >>>>>Hi Peter, >>>>> >>>>>thanks for the link. Theres no general answer for your question, >as >>>>>an application may both run flawlessly with a single or hundreds >of >>>>>databases, depending on how your XQuery expressions look like. If >>>>you >>>>>do regular updates, I suggest to split your data into fixed >>>>instances >>>>>that will never change, and use all indexes, and updating >instances >>>>>that may eventually be merged with the fixed instances if no more >>>>>changes are expected. >>>>> >>>>>Christian >>>>>___________________________ >>>>> >>>>>On Sun, Jan 13, 2013 at 1:09 AM, <pw(a)themail.co.uk> wrote: >>>>>> Hello List >>>>>> >>>>>> I am experimenting with statistical data ( >>>>>> http://www.semantechs.co.uk/ ) and found that organising 2.5Gb >of >>>>xml >>>>>> data into 12 unevenly sized collections ranging from 40 to >400Mb >>>>>> performs much more slowly than 36 collections each containing >>>>>> approximately 75Mb of data. >>>>>> >>>>>> What rules of thumb are there to guide me in designing the most >>>>>> performant database? >>>>>> >>>>>> Many thanks >>>>>> >>>>>> Peter >>>>>> >>>>>> _______________________________________________ >>>>>> BaseX-Talk mailing list >>>>>> BaseX-Talk(a)mailman.uni-konstanz.de >>>>>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>>> >>> >>> >>

1 0

Re: [basex-talk] best way to partition large data sets among collections
by pw＠themail.co.uk 14 Jan '13

14 Jan '13

Hi Christian Do you have any information to guide me here; what sorts of XQuery expressions should I match with large numbers of collections, and which with small numbers of collections? If I am using data-rich XML there is a high ration of nodes to content. What are the rules of thumb for this type of content? Are there any recommendations specifically for GML? Many thanks Peter > > > >---- Original Message ---- >From: christian.gruen(a)gmail.com >To: pw(a)themail.co.uk >Subject: Re: [basex-talk] best way to partition large data sets among >collections >Date: Mon, 14 Jan 2013 13:05:00 +0100 > >>Hi Peter, >> >>thanks for the link. Theres no general answer for your question, as >>an application may both run flawlessly with a single or hundreds of >>databases, depending on how your XQuery expressions look like. If >you >>do regular updates, I suggest to split your data into fixed >instances >>that will never change, and use all indexes, and updating instances >>that may eventually be merged with the fixed instances if no more >>changes are expected. >> >>Christian >>___________________________ >> >>On Sun, Jan 13, 2013 at 1:09 AM, <pw(a)themail.co.uk> wrote: >>> Hello List >>> >>> I am experimenting with statistical data ( >>> http://www.semantechs.co.uk/ ) and found that organising 2.5Gb of >xml >>> data into 12 unevenly sized collections ranging from 40 to 400Mb >>> performs much more slowly than 36 collections each containing >>> approximately 75Mb of data. >>> >>> What rules of thumb are there to guide me in designing the most >>> performant database? >>> >>> Many thanks >>> >>> Peter >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

7.5.1 beta XMLDB api issues
by Jason Harrop 14 Jan '13

14 Jan '13

https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/bas… is not working for me. I'm using source code from github (7.5.1 beta). Is it just me? Also, any update on the status of March 2011 regarding the questions: Is it possible to create child Collections within a database through the XMLDB API? And is it possible to insert XMLResources in a child Collection through the XMLDB API? thanks .. Jason XMLDBInsert returns: === XMLDBInsert === * Get collection. * Create new resource. Exception in thread "main" java.lang.NullPointerException at com.plutext.basex.samples.XMLDBInsert.main(XMLDBInsert.java:52) ============== [basex-talk] xmldb api and child collections Andreas Weiler andreas.weiler at uni-konstanz.de Sun Mar 27 14:10:31 CEST 2011 Previous message: [basex-talk] xmldb api and child collections Next message: [basex-talk] BaseX 6.6 Update - Soft-Files.com Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Sorry, it seems that this is not supported right now. We will have a look into this. In the meantime, you could use our own API for your scenario. Kind regards, Andreas Am 26.03.2011 um 17:59 schrieb frans at planet: > Is it possible to create child Collections within a database through the XMLDB API? And is it possible to insert XMLResources in a child Collection through the XMLDB API? > > Regards, > Frans > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk