- BaseX-Talk - mailman.uni-konstanz.de

BaseX Antivirus Scan Report on FileCluster.com
by FileCluster Security 17 Nov '11

17 Nov '11

Dear Developer,BaseX has been downloaded and tested by Filecluster Labs on 17 Nov 2011 with today's best antivirus engines: BitDefender Antivirus, Kaspersky Antivirus, NOD32 and Avira Antivirus. The report shows that BaseX has been found clean of any viruses or malware and has been flagged as safe to install. You can find the detailed antivirus scan reports at this page: http://www.filecluster.com/Antivirus-Report/BaseX.html. (if the page isn't working, please allow between 5-30 mins for the cache to refresh) Please keep in mind that we will constantly scan your software with each version update or on-demand, if necessary. We have also displayed a clean award within FileCluster to assure our visitors that BaseX is clean and safe to install.If you feel that this report could add value to your visitors as well, you can use any of the banners presented within the report page. Future development of this feature will depend on feedback, so any suggestions or general feedback are greatly appreciated. Also, mentioning this report onto your blog, news page or press releases is highly appreciated, encouraging us to improve the scanner and extend it even further. If you wish to unsubscribe from receiving such notifications, please contact us via email and let us know. Keep in mind that by unsubscribing, you will never receive any emails from FileCluster again, such as promotions, security reports, community feedback and so on. Don't hesitate to contact us for any issues regarding our security service. Best Regards, Jamie P. - FileCluster Security www.filecluster.com

1 0

APIs continued
by Christian Grün 16 Nov '11

16 Nov '11

Hi Dave, > Well, I've polluted this message board enough for a couple days. If I'm starting to be a nuisance, please by all means tell me to cool off for a while - it won't hurt my feelings :) not at all, thanks for your elaborate considerations! I just had no time yet to give you some adequate feedback yet.. And I'd still be interested to get some more feedback from everyone else: do you like the status quo of our API, or do you feel there's something more that could be done? Regarding some of our commercial use cases, we know quite well what we would like to do first, but it's up to all of you to influence our priorities.. Christian

1 0

Streaming Updates (Embedded API)
by Dave Glick 16 Nov '11

16 Nov '11

So I've come across another important use case for an embedded API: direct streaming updates. What I mean by this is the ability to perform update operations (such as insertion) on the database using a stream that contains input content. As you know, I'm in the process of updating our own embedded API from an old version 5.X of BaseX to version 7 (which is what started this whole embedded API train of thought). We had code in place to accomplish this in our old version by reading from a stream (a .NET XmlReader, which is SAX-like), applying direct Data update calls (such as Data.elem() or Data.text()) for each event, and then flushing the Data object when the stream ran out. This has two problems - first, the code is very complex (I.e., to keep track of the current insertion point, resolve namespaces, etc.) and is difficult to maintain and second, it probably isn't conformant since it doesn't do things like check for adjacent text nodes at the insertion point. I would love to replace this code with something much simpler and let BaseX handle the details so I can be sure it works right. Which brings me to some more immediate questions than the more theoretical API discussion. I'm looking for ways to use existing BaseX functionality to manage database updates, particularly for very large content streams. I considered just creating appropriate queries and parsing/evaluating them, but some of the input content could be very large (as in, might overrun a single string buffer) and it doesn't seem totally efficient to read from a tokenizing SAX-like stream, convert to a string, then read into a second tokenizing string inside the query processor. So then I started looking at the internals of the query processor, particularly as it relates to XQuery updates. I found the DatabaseUpdates class and in particular the DatabaseUpdates.apply() method which looks to provide some direct control over the process. I could manually create update primitives, add them to a DatabaseUpdates instance, and then apply that to the database. I think that will accomplish what I'm looking for. The main area where I'm hung up right now is how to specify the input stream/content for the update operation. My current thinking is to use the stream to create an in-memory temporary database that holds the content. This shouldn't be tough since I already have a .NET XmlReader to BaseX Parser bridge in place and I can use that to create a MemData instance pretty easily. However, I could use some help getting from the MemData to a NodeCache that can be used in the constructor for update primitives like InsertBefore. So, another long post made short, here are my questions: - How can a NodeCache be constructed from a MemData? - It appears as though InputInfo is mainly used for error reporting - does it have other uses that I need to watch out for (I'm thinking I'll just initialize it to default values)? - Does this whole strategy sound reasonable or is there a better way to conduct update operations on the database given an input stream? BTW - I know it's been a long time coming, but we have every intention of releasing our .NET API once we get it updated and documented/packaged (which has been our main hang-up in the past - never enough time for polish). I'm not sure what kind of licensing model we're going to use. I'm personally pushing for open source (preferably BSD like BaseX), but we might end up with something like a GPL with an available commercial option or non-open source freemium model (unfortunately, got to please the accountants). If any of the community members out there are interested in the project and have some input (what you might be willing to pay, what kind of .NET-centric additional functionality would be valuable, etc.) I'd love to hear it. Thanks, Dave -----Original Message----- From: Dave Glick Sent: Tuesday, November 15, 2011 11:28 AM To: 'Christian Grün' Cc: BaseX-Talk(a)mailman.uni-konstanz.de; Eric Murphy; Charles Foster Subject: RE: [basex-talk] Embedded API Christian, Thank you, as always, for your thoughtful and thorough response. I'm excited to hear that this is already being discussed around BaseX world headquarters. Several of your possible approaches raise interesting questions or capabilities that I hadn't considered, especially given my somewhat myopic view of the situation based on our own requirements. As you guessed, my own interest is in directly exposing the underlying data structures in a more stable and cohesive way. I've looked in-depth at the existing APIs (DOM, XML:DB, and XQJ) and while the development idealist in me wants to use and champion standard protocols, the pragmatist that needs to get a job done accepts that such standards tend to support the lowest common denominator and thus fail to capture more complex or specific use cases (though I still applaud you guys for providing and supporting such a broad range of APIs, which are certainly valuable in many situations). While the existing APIs do provide a lot of functionality, they're not consistent (use DOM for traversing nodes, XQJ for querying, etc.) and there are gaps in functionality (creation and manipulation of databases, direct updating, etc.). In any case, the existence of these other APIs has provided a lot of sample code that we've adapted in our own embedded API. @ Eric - you mentioned that you're working on an Android port and have taken the viewpoint that BaseX is a framework on which other tools can be built. That's very much in line with how we've been treating it. For a little background, we've been using BaseX to support a variety of projects, the largest of which is a generic RCP for the manipulation of modeling and simulation data (think Eclipse for M&S). We take inputs to models and simulations, parse them up into XML, and then store, query, and manipulate them in BaseX with lots of frontend visualizations and other graphical interfaces. The challenging part of our work is that we use .NET and Mono, so we cross-compile BaseX using IKVM and then wrap it to make it easier to work with in .NET. Since all of our work uses BaseX as an embedded database, we've found that we're basically developing something akin to an embedded API in our wrapper. Rather than create such an API inside the .NET wrapper, I'd much prefer the API existed in BaseX itself and the wrapper could be "thinner". The biggest problem we have is keeping up with changes to the classes in BaseX. It's being released and improved on such a rapid cycle (great job!) that we're constantly reworking portions of our API to match, and frequently get multiple releases behind. Perhaps it would help if I shared some of our own use cases. Obviously we're not the only potential users of an embedded API, but some of the issues we've had using it in that mode may help refine requirements for an eventual implementation. * Node referencing and persistence We interact with the database a lot. The general pattern is that a given visualization or view in our tool will execute a query to return a result set and will then store references to the resultant nodes. The application provides a multiple document interface and each view on a given document can potentially manipulate the database. This is why we store references to each node. Some of the queries are rather time consuming and rather than re-evaluate them each time the data is changed in other views, we simply persist the node references through changes. This was one of the harder areas to get right in our API. Because BaseX is primarily client/server driven and focused on returning serialized results, the notion of node references and persistence just isn't there. Internally, BaseX stores a pre value inside it's node objects. This is fine until the database changes and something gets inserted or deleted and the pre values change. At that point, the previous pre value no longer refers to the same node and any objects that rely on it are invalidated. This works great for all the current use cases because there's no need for persistence in the node references following database updates. However, in an embedded environment, there are lots of reasons why node references should be persisted. To solve this we used the following algorithm: - We initialize node objects with a Data and pre value, just like the internal DBNode class. - We immediately fetch the associated (and immutable) index for the pre on construction. - We also store the timestamp for the most recent database modification. - Every time an operation is performed on a node object, it first checks to see if the saved timestamp is different than the database one, and if so it updates the pre value by using the stored index value and the new timestamp is saved. - If the index no longer exists (the node was removed) the object is invalidated. This ensures that a reference to a node object continues to reference the "same" node no matter what happens to the database. * Node traversal We traverse nodes in our application. A lot. There are many cases where an operation needs to be performed dynamically and it's much easier to manipulate the database directly than attempt to construct an XQuery that captures the desired behavior and execute that. Not to mention that we've realized significant performance gains by bypassing the query engine and relying on direct manipulation for operations that perform large numbers of traversal and modification operations (sometimes in the 10,000 or 100,000 node order of magnitude). In fact, this is already captured in the current DOM API implementation. I would just suggest that any future embedded API also make sure to provide direct and efficient node traversal operations. * Direct updating If you can store a reference to nodes, and you can traverse them to get to other nodes, it stands to reason that you should be able to change them as well. This is another area where we've spent significant effort creating some capability that didn't already exist. In our own node class we provide operations for setting inner text (or atomic value if you prefer), inner XML, and outer XML (though that last one is kind of a hack since we just remove the node in question and insert new content as a child of the parent at the appropriate location). Again, this is an area we noticed performance gains over using XQuery update when appropriate. That and we implemented a lot of the logic before XQuery update was even supported in BaseX (though we've had to change it significantly to keep up). This seems like another area where a complete embedded API should provide some capability. * Query manipulation I think XQJ is on the right track here. Ideally, the notion of a query should be captured in an object and exposed so that the context can be manipulated, collections can be created, variables can be created/initialized, etc. I'd just like to see the eventual embedded API include this functionality directly integrated with the other parts of the embedded API. I would be happy to download and start looking at QT3TS - I respect Michael's work on Saxon and am interested to see what his test suite looks like. I'd also be happy to run some benchmarking and performance comparisons on direct manipulation vs. XQuery evaluation for various operations in order to get some hard numbers. Anecdotally, my general experience has been that directly manipulating the database through primitives is much more efficient for large numbers of operations or situations where the logic is very dynamic or complex and an XQuery statement would be long or difficult to derive. This email ended up being a bit longer than I had intended - but that always seems to happen with me :). I hope of my stream-of-consciousness was valuable. Let's keep up this discussion...I'll write more as I think of it. Dave -----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Monday, November 14, 2011 7:56 PM To: Dave Glick Cc: BaseX-Talk(a)mailman.uni-konstanz.de; Eric Murphy; Charles Foster Subject: Re: [basex-talk] Embedded API Hi Dave, hi all, better Java APIs for BaseX - yes, that's a very relevant topic nowadays, something that we've frequently been discussing for the last weeks in our team. And the main challenge we are struggling with is that there are just too many ways how such an API could look like - and too many incoming requests that can hardly be bundled in one single API.. Here are some of the requirements we're dealing with, and the approaches that could be pursued (..and I already know which of them you would prefer ;) : * a new Command and Query/Result API could enhance/replace the existing light-weight client Java API, and the representation of results would be separated from the low-level data structures in BaseX. This API could be used in the client/server architecture as well, but it would introduce some overhead, as all the data structures would have to be replicated by the client. * The new Command, Query and Result objects could also be made serializable. This way, they could be easily transfered over the network, and there would be no need to develop custom binary protocols. * a real embedded API could ensure that developers do not suffer from frequent changes in our query and storage backend. Instead, we would ensure that the API does not change as long as the major version is not updated. This API would be much more efficient than a client/server API, but we might have to put more work into transactional issues. * the existing XML:DB and XQJ APIs could be revised and updated to support the client/server architecture. This could reduce the need for any other client/server-based API with a richer functionality. Everyone who is interested in more powerful APIs.. Please speak out! The more feedback we get, the better we'll be able to design our APIs. And of course we're interested in volunteers out there... Last but not least, this is an Open Source and community project ;) @Dave: I've recently added a minimum query API for the QT3TS, Michael Kay's new W3 XQuery Test Suite. Both the test suite driver and the mini API (qt3api) is still work in progress: https://github.com/BaseXdb/basex-tests/tree/master/src/main/java/org/basex/… It it not low-level enough to directly support any axis or update operations; instead new QueryProcessor instances are created to perform queries on intermediate nodes. It would be great if you could have a look at this API, and it would then be interesting to know more about your performance requirements: do you think that the overhead for parsing and compiling query expressions (which usually does not takes longer than some microseconds, and is often faster than the actual axis traversals) will be too expensive in your scenario? If you believe that this framework would be sufficient, we could start to enhance it, make it safe for concurrent access, document it, etc. If you need to work with the PRE and ID values of database nodes, e.g., you could take advantage of the db: functions of BaseX [1]: Output: db:node-id($node) Input: db:open-id($db, $id) Hope this helps, Christian [1] http://docs.basex.org/wiki/Database_Functions On Mon, Nov 14, 2011 at 6:26 PM, Dave Glick <dglick(a)dracorp.com> wrote: > Hi all, > > We've been using BaseX for several years now and have constantly been > skirting around our primary use case: using BaseX in an embedded mode. > What I mean by this is using BaseX in-process in an application > without running any kind of client/server communication bridge and > with very direct access to BaseX primitives. There are several reasons > for wanting to do this including performance (which seems to be the > subject of recent discussions, I.e., running the server in "local" > mode). My own primary reason is to gain more direct access to the > database objects. For example, we routinely have a need to: > > - Directly access and traverse database nodes by climbing, descending, > following, etc. > > - Insert or remove content at a specific database node > > - Store references to individual nodes (I.e., using its "pre" and "index" > value) > > - Fine-tune queries in order to set context, external functions, etc. > > While many of these operations can indeed be performed through the > existing client/server interface, it's less friendly - especially when > doing things like asking for the next sibling of a given node. With a > direct embedded API you just get the next node, bypassing the XQuery > processor altogether. From my current work in this area, I think BaseX > is already "primed" for this kind of API - 90% or more of the code is > already in place since most of the primitives already expose common > methods for use by database commands, XQuery processor, etc. All that > should be needed is to expose this functionality in a stable and complete API. > > Good examples of applications that may need this kind of API include > media players (I.e., for storage of the media library data), simple > stand-alone database applications, etc. Until recently, we've been > able to adapt BaseX to fit our needs by writing a thin wrapper layer > that interfaces with the appropriate BaseX classes. However, with the > rapid pace of BaseX development these days it's becoming increasingly > difficult to track each release since we rely on aspects of the BaseX > codebase that are not really intended for public consumption and thus > keep changing. This brings up a couple > questions: > > - Are we the only ones interested in a direct embedded interface? > > - Does the BaseX team have any plans to implement such an interface? > > - Would such an interface be better implemented by the BaseX team (as > opposed to a third party)? > > I don't mind doing some work in this area, however, I have some > concerns about doing so. Primarily, given that the whole idea would be > to make direct integration easier and more stable it seems like the > structure and layout of the classes in the embedded API and the ways > that they interact with the underlying BaseX objects should probably be determined by the BaseX team. > The danger is that someone outside the team spends effort creating > such an interface only to do things in a way that's either not > preferred or difficult to maintain as the core team continues to > improve the overall product. > > Hopefully this was clear... Thoughts? > > Dave

1 0

Re: [basex-talk] DXQ
by Lukas Lewandowski 16 Nov '11

16 Nov '11

Hi Eric, Thanks for the links to the papers about distributed XQuery. Currently we are evaluating different approaches to distribute and query XML data. Basically, we are interested in using such an approach directly out of XQuery. The complexity for XQuery developers should be as small as possible, therefore we prefer solutions which do not extend XQuery specification with complex hierarchical structures. The "Distributed XQuery-based integration and visualization of multimodality brain mapping data" approach looks interesting and straightforward. We do more research in distribution approaches within the next weeks. regards Lukas On Monday, November 14, 2011 17:32 CET, Christian Grün <christian.gruen(a)uni-konstanz.de> wrote: > Hi Eric, > > nice to have your update! > > > I've been seeing quite a bit of chatter about cloud computing and > > distributed systems at work of recent. I think over the next few years, > > there is going to be a dire need for Distributed XQuery with our government > > customers. They have massive amounts of XML data and it keeps growing. > > That's good to know; it's also reflected in the feedback we're getting > on BaseX. We have some students working on getting BaseX in the cloud; > it will be interesting to get their opinion on DXQ.. @Lukas, Dirk, > Lukas: could you have a look at the two papers referenced by Eric, and > give some feedback? > > > http://www.cs.washington.edu/homes/kmorton/paper.pdf > > http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2636687/ > > Regarding the Android port: do you think your code base is already > complete enough to be published, or to be given away? > > Thanks, > Christian

2 3

documentation question
by C. M. Sperberg-McQueen 16 Nov '11

16 Nov '11

For complicated reasons, one of the copies of BaseX I am working with at the moment is (as far as I can tell) version 6.7; is there a convenient way to find the 6.7 documentation on the BaseX site? Or should one just read the current documentation and be prepared to ignore all the material marked as new in 7.0? I'm particularly interested in reading up on the http interface, which I know has changed a lot in 7.0. At some point, I expect that that copy of BaseX will be upgraded to version 7, but for now if I want to work with that copy, I need to address it in 6.7 terms. Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

2 5

basexError
by blue 15 Nov '11

15 Nov '11

Hello. I am Isao. I'm japanese. Sorry, I can't speak english. When I use basex, SessionInfo output error log. I sent query to BaseXServer from ClientApplication(C#) repeatedly. -------error log------- Potential bug? Improper use? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.0.1 Java: Oracle Corporation, 1.7.0_01 OS: Linux, i386 Stack Trace: java.lang.RuntimeException: Not expected: Data Access out of bounds [pre:7081866, indexSize:16184, access:16184 > 16183]. org.basex.util.Util.notexpected(Util.java:67) org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:394) org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:107) org.basex.data.Data.kind(Data.java:319) org.basex.query.item.DBNode.parent(DBNode.java:218) org.basex.query.item.DBNode$9.next(DBNode.java:410) org.basex.query.path.IterStep$1.next(IterStep.java:46) org.basex.query.iter.NodeIter.next(NodeIter.java:1) org.basex.query.path.IterPath$1.next(IterPath.java:67) org.basex.query.iter.NodeIter.next(NodeIter.java:1) org.basex.query.expr.For$1.next(For.java:91) org.basex.query.expr.FLWR$1.next(FLWR.java:62) org.basex.core.cmd.AQuery.query(AQuery.java:101) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:328) org.basex.core.Command.exec(Command.java:309) org.basex.core.Command.execute(Command.java:77) org.basex.server.ClientListener.run(ClientListener.java:200)

3 3

'replace node' with new XML namespace prefix
by Laurent Chevalier 14 Nov '11

14 Nov '11

Hi. I've found a strange behaviour with the xquery "replace node" command. It seems that new namespace declarations within the XML tree added are not taken into account. Please, find hereafter a JUnit test code. Laurent package com.cyim.basex.test.ns; import org.junit.After; import org.junit.Assert; import org.junit.Before; import org.junit.Test; import org.basex.server.ClientSession; import org.basex.util.Util; /** * Test support of XML namespaces. * * @author Laurent Chevalier l.chevalier(a)cyim.com */ public final class NamespaceTest { /** Client session. */ private ClientSession session; /** Database name. */ protected static final String DB = Util.name(NamespaceTest.class); /** * Opens a client session, creates a database and loads documents. * @throws java.io.IOException IO exception */ @Before public void setUp() throws java.io.IOException { session = new ClientSession("localhost", 1984, "admin", "admin"); session.execute(String.format("create db %s <default/>", DB)); // Loads documents. addDoc("doc/1"); addDoc("doc/2"); addDoc("doc/3"); } /** * Adds a document. * @param uri * @throws java.io.IOException */ private void addDoc(String uri) throws java.io.IOException { // Document content. java.lang.StringBuilder qs = new java.lang.StringBuilder(); qs.append("<rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\" xmlns:core=\"http://rdfs.cyim.com/CoreContent#\">\n"); qs.append(String.format(" <core:Container rdf:about=\"%s\"/>\n", uri)); qs.append("</rdf:RDF>"); // Adds it. java.io.InputStream in = new java.io.ByteArrayInputStream(qs.toString().getBytes("UTF-8")); try { session.add(uri, in); } finally { in.close(); } } /** * Drops the database and close remaining client sessions. * @throws java.io.IOException IO exception */ @After public void tearDown() throws java.io.IOException { session.execute("drop db " + DB); session.close(); } /** * Inserts a node. * * @param uri Document uri. * @throws java.io.IOException IO exception */ private void insertNode(String uri) throws java.io.IOException { java.lang.StringBuilder qs = new java.lang.StringBuilder("xquery\n"); qs.append("declare namespace rdf = \"http://www.w3.org/1999/02/22-rdf-syntax-ns#\";\n"); qs.append("declare namespace core = \"http://rdfs.cyim.com/CoreContent#\";\n"); qs.append("declare namespace wfdata = \"http://rdfs.cyim.com/WorkflowData#\";\n"); qs.append("insert node ") .append("<wfdata:Workflow xmlns:wfdata=\"http://rdfs.cyim.com/WorkflowData#\">\n") .append(" <wfdata:raiseOn>2011-11-01T11:34:08Z</wfdata:raiseOn>\n") .append("</wfdata:Workflow>\n") .append("into /rdf:RDF/*[@rdf:about=\"" + uri + "\"] "); session.execute(qs.toString()); } /** * Replaces a node and declares the 'wfdata' prefix on the 'core:Container' node. * * @param uri Document uri. * @throws java.io.IOException IO exception */ private void replaceNode(String uri) throws java.io.IOException { java.lang.StringBuilder qs = new java.lang.StringBuilder("xquery\n"); qs.append("declare namespace rdf = \"http://www.w3.org/1999/02/22-rdf-syntax-ns#\";\n"); qs.append("declare namespace core = \"http://rdfs.cyim.com/CoreContent#\";\n"); qs.append("declare namespace wfdata = \"http://rdfs.cyim.com/WorkflowData#\";\n"); qs.append("replace node /rdf:RDF/*[@rdf:about=\"" + uri + "\"] with <core:Container rdf:about=\"" + uri + "\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\" xmlns:core=\"http://rdfs.cyim.com/CoreContent#\" xmlns:wfdata=\"http://rdfs.cyim.com/WorkflowData#\">\n") .append(" <wfdata:Workflow>\n") .append(" <wfdata:raiseOn>2011-11-01T11:34:08Z</wfdata:raiseOn>\n") .append(" </wfdata:Workflow>\n") .append("</core:Container>\n"); session.execute(qs.toString()); } /** * Replaces a node and declares the 'wfdata' prefix on the 'wfdata:Workflow' node. * * @param uri Document uri. * @throws java.io.IOException IO exception */ private void replaceNode2(String uri) throws java.io.IOException { java.lang.StringBuilder qs = new java.lang.StringBuilder("xquery\n"); qs.append("declare namespace rdf = \"http://www.w3.org/1999/02/22-rdf-syntax-ns#\";\n"); qs.append("declare namespace core = \"http://rdfs.cyim.com/CoreContent#\";\n"); qs.append("declare namespace wfdata = \"http://rdfs.cyim.com/WorkflowData#\";\n"); qs.append("replace node /rdf:RDF/*[@rdf:about=\"" + uri + "\"] with <core:Container rdf:about=\"" + uri + "\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\" xmlns:core=\"http://rdfs.cyim.com/CoreContent#\">\n") .append(" <wfdata:Workflow xmlns:wfdata=\"http://rdfs.cyim.com/WorkflowData#\">\n") .append(" <wfdata:raiseOn>2011-11-01T11:34:08Z</wfdata:raiseOn>\n") .append(" </wfdata:Workflow>\n") .append("</core:Container>\n"); session.execute(qs.toString()); } /** * Query. Select depending on added node. * * @param expected Number of expected results. * @throws java.io.IOException IO exception */ private void query(int expected) throws java.io.IOException { // Query using wildcards. java.lang.StringBuilder qs = new java.lang.StringBuilder("xquery\n"); qs.append("declare namespace rdf = \"http://www.w3.org/1999/02/22-rdf-syntax-ns#\";\n"); qs.append("for $c in /rdf:RDF/*[//*:raiseOn<\"2011-11-02T14:19:07Z\"]\n"); qs.append("return <item about=\"{data($c/@rdf:about)}\" raiseOn=\"{data($c//*:raiseOn)}\"/>\n"); String results = session.execute(qs.toString()); System.out.println("\n" + qs.toString() + "\n" + results); Assert.assertEquals("Missing results!", expected, results.isEmpty()?0:results.split("\r\n|\r|\n").length); // Query using namespace. qs = new java.lang.StringBuilder("xquery\n"); qs.append("declare namespace rdf = \"http://www.w3.org/1999/02/22-rdf-syntax-ns#\";\n"); qs.append("declare namespace wfdata = \"http://rdfs.cyim.com/WorkflowData#\";\n"); qs.append("for $c in /rdf:RDF/*[//wfdata:raiseOn<\"2011-11-02T14:19:07Z\"]\n"); qs.append("return <item about=\"{data($c/@rdf:about)}\" raiseOn=\"{data($c//wfdata:raiseOn)}\"/>\n"); results = session.execute(qs.toString()); System.out.println("\n" + qs.toString() + "\n" + results); Assert.assertEquals("Missing results!", expected, results.isEmpty()?0:results.split("\r\n|\r|\n").length); } /** * Test entry point. * @throws java.io.IOException IO exception */ @Test public void run() throws java.io.IOException { System.out.println("Query using unchanged documents."); query(0); System.out.println("\nInserts a node to doc/1."); insertNode("doc/1"); System.out.println("\nQuery. Should get 1 document."); query(1); System.out.println("\nReplaces a node in doc/2 and declares the 'wfdata' prefix on the 'core:Container' node.\n."); replaceNode("doc/2"); System.out.println("\nQuery. Should get 2 documents."); query(2); System.out.println("\nReplaces a node in doc/3 and declares the 'wfdata' prefix on the 'wfdata:Workflow' node.\n."); replaceNode2("doc/3"); System.out.println("\nDatabase content:"); System.out.println(session.execute("xquery /")); System.out.println("\nQuery. Should get 3 documents."); System.out.println("This test is going to fail without wildcards as if the new node linked to 'wfdata' prefix was not seen."); query(3); } }

3 6

Revised command-line API
by Christian Grün 13 Nov '11

13 Nov '11

Dear all, Starting with Version 7.0.3 (which might end up in 7.1), I have spent some time in reworking our command-line API. The latest revision is found at: http://files.basex.org/releases/latest/ Some of the new features: * a single dash ("-") is now supported to parse from standard input * multiple inputs, query files, queries and database commands can now be specified and will be evaluated in sequential order * spaces and commas are now alllowed in the declaration of variable bindings * variable bindings have been extended to support the Clark and EQName notations * arguments may now contain spaces if they are enclosed in quotes The new features are also documented in our Wiki: http://docs.basex.org/wiki/Startup_Options_(Snapshot) Everyone's feedback is welcome! Have fun, Christian ___________________________ BaseX Team Christian Grün Uni KN, Box 188 D-78457 Konstanz http://www.basex.org

1 0

output option with filenames with space in windows
by Randy reed 13 Nov '11

13 Nov '11

This is probably an elementary question but I am using basex on windows and trying to invoke the program with -o option, but it keeps choking on the space in the file name. I've tried escaping the name with \ but to no avail. Heres the line: C:\>java -cp "C:\Program Files\Basex\basex.jar" org.basex.BaseX -o"C:\Documents and Settings\reedrw\My Documents\audit.xml" "C:/Documents and Settings/reedrw/My Documents/My Dropbox/student_ind_detail_byname_no_ext_var.xq" The error is: Error: File or directory "and Settings\reedrw\My Documents\audit.xml C:/Document s and Settings/reedrw/My Documents/My Dropbox/student_ind_detail_byname_no_ext_var.xq" not found. So it looks to me that the -o has stopped at the space and then it hast taken the rest as the full path of the actual query. Just to be clear I have tried it java -cp "C:\Program Files\Basex\basex.jar" org.basex.BaseX -o"C:/Documents and Settings/reedrw/My Documents/audit.xml" "C:/Documents and Settings/reedrw/My Documents/My Dropbox/student_ind_detail_byname_no_ext_var.xq" as well and that generates the same error thanks in advance -- Dr. Randall Reed Assistant Professor, Dept. of Philosophy and Religion Appalachian State University

2 2

BaseXGUI Execution Problem (Windows 7 - 64bit)
by N.Ilker ERCIN 12 Nov '11

12 Nov '11

Hi there, I've downloaded BaseX installation file (BaseX703-20111111.185755.exe) and installed it on my PC, running "Windows 7 - *64bit* - English" version. No problems occured during installation. However when I tried to run Basex GUI (using the link file on the desktop / using the actual BaseX.exe / using BasexGui.bat / using basex.jar), GUI does not run. *BaseX.exe* : When I run it, a "javaw.exe" appears in the Task Manager but the GUI is not displayed. *BaseX.bat* : When I run it, the exception below is printed in the command line, but no further action appears. ----- Not found: /img/cmd-²nfo.png Uncaught error fetching image: java.lang.NullPointerException at sun.awt.image.URLImageSource.getConnection(Unknown Source) at sun.awt.image.URLImageSource.getDecoder(Unknown Source) at sun.awt.image.InputStreamImageSource.doFetch(Unknown Source) at sun.awt.image.ImageFetcher.fetchloop(Unknown Source) at sun.awt.image.ImageFetcher.run(Unknown Source) ----- *Basex.jar* : When I run it, the splash with "Launching BaseX - Please Wait" appears but nothing more. "javaw.exe" appears in the Task Manager. *NOTE-1* : There is NO PROBLEM with basexserver.bat and basexclient.bat, I can do what i want to do, using the command line. *NOTE-2* : I've the latest version of JavaRE (jre-6u29-windows-x64.exe) installed on my PC. *NOTE-3* : When I tried the same BaseX installation on my friends PC (Windows 7 - *32 bit *- English) "Basex.exe" works just fine, GUI is displayed successfully. *NOTE-4* : Before this version, I repeated same steps with BaseX-701.exe installation, but the problems were just the same. Please help me :( Am I doing something wrong? -- N. Ilker ERCIN

2 3

BaseX has been updated on Filecluster.com
by FileCluster Support 12 Nov '11

12 Nov '11

Dear Developer, You have received this email because we have updated BaseX and this was the contact email for the application. Your application update status BaseX has been updated on FileCluster.com to the 7.0.2 version. You can check out your application on our site at: http://www.filecluster.com/Business-Office/Database-Management/Download-Bas… If you want to let your users know about your software page on FileCluster.com, to rate it and review it, you can also use this banner to link it: Promote your software As a software website, it`s our duty to help the developers promote their software. Fortunatelly, our software promotions are cost free, for more informations please visit our promotion page. Submit your press releases We are also accepting press releases from software developers. All you have to do is to submit your press release and we will take care of the rest. Useful developer links All the essential links you need as a developer on FileCluster.com: Link to us - download banners or text links for linking to FileCluster.com Submit your software - Submit your software via PAD or manually Submit your press release - submit your press releases using our online form Contact Us - contact our editors Promote your software - our promotions page If you do not wish to receive any further notifications or confirmations from FileCluster.com, please send us an email and request a contact removal (email removal). Keep in mind that this means you will not receive ANY contact from us, including: software promotions, update confirmations or security alerts (antivirus and spyware test results). Regards, Renta Cezar - Senior Editor FileCluster Labs www.filecluster.com

1 0

FileGuru: BaseX 7.0.2 Published Today at FileGuru
by FileGuru 12 Nov '11

12 Nov '11

Dear BaseXWe are www.fileguru.com Editors. We have published your software BaseX 7.0.2 today in our site. Watch your listing here: Location: http://fileguru.com/BaseX/info Category: Business \ Information Database Your software has been reviewed and received 5 of 5 stars award. We like to congratulate you for this great useful software. Please feel free to display the award on your website and promotional materials. We recommend you to link your software, because this link will increase your ranking in the search engine and categories. And your software should be visible on the top! So, to provide a link to FileGuru from your site, you can find nice buttons at http://www.fileguru.com/linkus You can link to your software info page http://fileguru.com/BaseX/info Sincerely yours, FileGuru Team - http://www.fileguru.com © 2000-2011, FileGuru.

1 0

BaseX 7.0.2: 11-11-11 Edition
by Christian Grün 11 Nov '11

11 Nov '11

Dear list, we have just released a new maintenance release of BaseX, which contains the following features and updates: FULL TEXT - Stemming support for Japanese text corpora (thanks to Toshio HIRAI!) STARTUP - Updated start scripts (thanks to Ralf Jung!) - System property "org.basex.path" added to specify project's home directory (thanks to malamut2!) XQUERY - Numerous minor XQuery 3.0 enhancements and fixes - Fix for db:optimize() calls (thanks to Martin Hillert!) STORAGE - Fix to retain newly introduced namespaces (thanks to Laurent Chevalier!) USERS - Default privileges for new users set to "none" (thanks to Pascal Heus!) REST - query base URI for evaluated queries is now $HTTPPATH (thanks to Florent Georges!) We recommend all of you to switch to the latest version.. Have fun, Christian BaseX Team

1 0

Re: [basex-talk] BaseX 7.0.1: Release early, release often..
by Ralf Jung 10 Nov '11

10 Nov '11

Hi again, I saw you committed this - thanks a lot :) Kind regards, Ralf On Sunday 06 November 2011 22:23:03 Dimitar Popov wrote: > Am Sonntag, 6. November 2011, 21:26:03 schrieb Ralf Jung: > > Hi, > > > > this will break if the link is absolute, since it will still prepend the > > $(dirname "$FILE") to the value we got from readlink. So if I call > > /home/r/bin/basexhttp, which is a link to > > /home/r/bin/basex/bin/basexhttp, it will set FILE to > > /home/r/bin//home/r/bin/basex/bin/basexhttp. > > Yes, I saw that, but a second after I sent the email :) > > > Also, I wonder why you are splitting $SOURCE into basename and dirname, > > just to concat it again? > > Yep, that was not necessary. Here is a version, which I hope works > correctly: > > FILE="${BASH_SOURCE[0]}" > while [ -h "$FILE" ] ; do > SOURCE="$(readlink "$FILE")" > FILE="$( cd -P "$(dirname "$FILE")" && \ > cd -P "$(dirname "$SOURCE")" && pwd )/$(basename "$SOURCE")" > done > BASEX_DIR="$( cd -P "$(dirname $FILE)" && pwd )" > > Dimitar

1 0

What's the diff between Database and collection
by Edward Fong 09 Nov '11

09 Nov '11

Hi, What's the diff between Database and collection? I think there are same in baseX, am I right? And how to query by a document name? Following are steps of my testing: 1: loaded some xml files form a directory: d:\data\org-*.xml 2: query by doc('org-100525.xml') and got File or directory "D:/Program Files (x86)/BaseX701/bin/org-100525.xml" not found message. 3: because after loaded xml, directory: d:\data\org-*.xml was removed. I can't query by doc('d:\data\org-100525.xml') Thanks, Edward

4 9

xquery update impact on server?
by Pascal Heus 09 Nov '11

09 Nov '11

All: I'm running an long "xquery delete node" on one of my database and this pretty much seem to make my server unavailable for other operations. I expect this query to run for a few hours (affecting a few million nodes, database is around 800Mb). I'm running this on a Amazon EC2 small instance (1 CPU, 1.7Gb RAM) and the CPU has been stuck at 100%. If I attempt to query any database on the server (not only the one being updated), I basically don't get anything back. So do heavy xquery update requests basically lock down the server? Anyone else having similar experience? Feedback/suggestions? best *P

4 5

Bugreport
by Hillert, Martin, le-tex 08 Nov '11

08 Nov '11

Ziel der Query ist es, nach der Manipulation der DB selbige zu optimieren. Query: let $w := (//work)[position() = 17]//impression[1] return ( if ($w/@has_archive_data) then delete node $w/@has_archive_data else (), db:optimize("index") ) Error: Potential bug? Improper use? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.0.1 Java: Sun Microsystems Inc., 1.6.0_07 OS: Linux, i386 Stack Trace: java.lang.ArrayIndexOutOfBoundsException Beste Grüße, Martin Hillert -- Martin Hillert Senior Analyst Enterprise Applications le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 130, Fax +49 341 355356 530 martin.hillert(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard Vöckler

3 3

Re: [basex-talk] BaseX 7.0.1: Release early, release often..
by Ralf Jung 07 Nov '11

07 Nov '11

Hi, > Yep, that was not necessary. Here is a version, which I hope works > correctly: That's looking great, I just added a missing " " around the last $FILE. FILE="${BASH_SOURCE[0]}" while [ -h "$FILE" ] ; do SOURCE="$(readlink "$FILE")" FILE="$( cd -P "$(dirname "$FILE")" && \ cd -P "$(dirname "$SOURCE")" && pwd )/$(basename "$SOURCE")" done BASEX_DIR="$( cd -P "$(dirname "$FILE")" && pwd )" Of course, in the remainder of the script, $PWD has to be replaced by BASEX_DIR... or, to make things slightly more readable, BASEX_DIR could be the actual dir with the main jar, not the bin dir. While I was at it, I also fixed the "list all jars in lib/" part to work properly if the path basex lives in contains a ' ' (which would previously have been replaced by a ':'). You can find the result attached. It works here, but should be tested on some different setups :D Kind regards, Ralf

1 0

Re: [basex-talk] very common attribute values/custom indexing
by Christian Grün 07 Nov '11

07 Nov '11

Ross, > The DSI Utilities (http://dsiutils.dsi.unimi.it/) are a very> interesting set of classes. Thanks; I'll soon have a look at that package. Just some quick feedback, regarding the extension of index entries: we'd get even better results by extending the index entries with id's to the unique location paths.. e.g.: <a> <b>dig</b> <c>dig</c> <c>diz</c> </a> Path Index: 0: a/b 1: a/c Value Index: dig.0 -> /a/b/text() dig.1 -> /a/c/text()[.='bla'] diz.1 -> /a/c/text()[.='blu'] This way, we could completely give up path inversions, at least in a number of trivial cases. In both cases, I guess that most effort has to be put into the adaptation of the query optimizer. Volunteers are welcome.. ;) Christian___________________________ On Wed, Nov 2, 2011 at 11:29 PM, Ross Judson <rossjudson(a)gmail.com> wrote: > You guys are wizards at making the JVM run fast, without a doubt. I > haven't looked to see how namespaces are handled right now, within the > indexing scheme. In my particular case I do not have namespaces, but > clearly they need to be handled, generally. > > The DSI Utilities (http://dsiutils.dsi.unimi.it/) are a very > interesting set of classes. I have previously used these to construct > indexes of RDF-type triples. They are _extremely_ high performance, > able to process at average rates of hundreds of thousands of tuples > per second. > > The ImmutableExternalPrefixMap was very useful. I created compound > strings (subject-property-object) and stored them. I could then do a > very fast lookup on any prefix, like subject, or subject-property. The > prefix map would supply the applicable interval. By creating another > index compounding (object-property-subject), I could lookup all usages > of a given property value (object), restricting by property name, or > all the way to a specific subject. > > The ImmutableExternalPrefixMap uses an in-memory structure to compute > approximate intervals on any given query; it then reads disk to refine > and answer the query. The net result is that you generally have access > to the results of any given query in one sector read (more or less > like a hash structure), but the data is _ordered_, which means you can > perform range queries, at the cost of reading additional sectors as > necessary. > > Within BaseX you have already converted tokens to byte arrays. > ImmutableExternalPrefixMap does this as well. > > A useful experiment would be to add a new type of index to BaseX that > incorporates one of these prefix maps. > > Creating the compound index looks something like this (altering > ValueBuilder.build()). Replace: > > index.index(data.text(pre, text), pre); > > with: > > byte [] attrName = data.name(pre, Data.ATTR); > int parentElement = data.parent(pre, Data.ATTR); > byte [] elementName = data.name(parentElement, Data.ELEM); > TokenBuilder tb = new TokenBuilder(data.text(pre, text)); > tb.add(';'); > tb.add(attrName); > tb.add(';'); > tb.add(elementName); > index.index(tb.finish(), pre); > > Not sure if I have all of that right, but it shows the idea. When I > run with this the indexes on my data do expand a little, but not too > much (may a 10% growth?). Of course, it will be different for > everyone, depending on their content. > > By checking the database properties you can see that the index has > more keys in it. Next step would be to add a prefix-scanning mode to > the index iteration, in addition to equality scanning. In the example > above I use the ';' character to separate the composite token; (byte)0 > might be more appropriate.

2 1

Error: "xbug80.dtd" could not be parsed.
by Florent Georges 06 Nov '11

06 Nov '11

Hi, When using the BaseX-7.0.2pre-20111030 GUI, and try to create a new database, I fill in the form with the name and location, then press OK. Some files seem to be created, then after a few seconds I get the following error: "TestInputs/bug/bug80.xml" (Line 3): "xbug80.dtd" could not be parsed. I don't know what it means in this context. Any idea? Regards, -- Florent Georges http://fgeorges.org/ http://h2oconsulting.be/

2 7

BaseX 7.0.1: Release early, release often..
by Christian Grün 06 Nov '11

06 Nov '11

Hi all, we are glad to announce BaseX 7.0.1, which is basically the result of your initial feedback on 7.0. Those are the most important changes: DISTRIBUTIONS: - Windows installer was updated to support latest features - ZIP file was updated (initial config & directories added) - Short directory names are chosen if config file resides in app.dir. - Start scripts have been improved XQUERY: - much faster execution of count() when applied to opened databases SERVER (http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server): - Flag -c connects to an existing database server - Flag -s specifies a port for stopping the HTTP server (Jetty) - Flag -S starts the HTTP server as a service - running write operations will be completed before server is stopped API: - Ruby, Python, PHP, Java: clients updated Please check out the well-known links: - Homepage: http://basex.org/ - Documentation: http://docs.basex.org/ - Snapshots: http://files.basex.org/releases/latest/ - Sources: https://github.com/BaseXdb Enjoy the update! Christian BaseX Team

5 17

Simple Xquery '(/org)[1 to 5]' was failed in GUI
by 方立厅 05 Nov '11

05 Nov '11

Hi Michael, Thanks for your replying. So this BaseX does not compatible to XQuery 1.0 by this feature? "(/org)[$start to $end]" query excuted well in marklogic and bdbxml. And I need a solution to make my existing xquery modules could be exuted in BaseX without modifying the codes. Any suggestion? Thank you, Edward >> Dear Edward, >> >> the correct syntax is: >> (2 to 100)[position() = 5 to 20] > (sequence)[5] is only a shorthand for (sequence*)[position() eq 5] wher 'eq' compares two _single_ items for equality. > > With [positon() = 5 to 20] we perform a general comparison, such that the predicate evaluates to true if comparison of any of the items inside both sequences > evaluates to true. > Thus position() = (5,6,7,?,20) evaluates to true if position() is between 5 and 20, while > position() eq (5 to 20) throws an error, as eq is not allowed to compare single items with sequences. > > Hope this helped clear things up a little, s/o please correct me if my explanation was faulty :-) > > Thanks > Michael > Am 05.11.2011 um 03:54 schrieb ???: > >> Hi, >> I'm using BaseX first time. and using 7.0.1. >> about 5000 org.xml was loaded into the db in my test enviroment. >> >> But I got an error msg instead of the result i want when I was excuting an simple xquery '(/org)[1 to 5]' in GUI: >> Query: (/org)[1 to 5] >> Error: [FORG0006] (1 to 5) not allowed as condition type. >> >> The xquery '(/org)[position()> 2 and position()< 5]' excuted very well. >> >> Thanks, >> Edward >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

3 2

Simple Xquery '(/org)[1 to 5]' was failed in GUI
by 方立厅 05 Nov '11

05 Nov '11

Hi, I'm using BaseX first time. and using 7.0.1. about 5000 org.xml was loaded into the db in my test enviroment. But I got an error msg instead of the result i want when I was excuting an simple xquery '(/org)[1 to 5]' in GUI: Query: (/org)[1 to 5] Error: [FORG0006] (1 to 5) not allowed as condition type. The xquery '(/org)[position() > 2 and position() < 5]' excuted very well. Thanks, Edward

2 1

Re: [basex-talk] Importing a document using xquery statements?
by Lukas Kircher 04 Nov '11

04 Nov '11

So far there is no XQuery function that allows you to create or delete a database. I remember this being a conscious design decision, as creating/deleting a database requires CREATE permissions, but I might be wrong... Maybe one of my team mates can help me out here? Cheers, Lukas On Fri, Nov 4, 2011 at 10:22 AM, Erdal Karaca < erdal.karaca.de(a)googlemail.com> wrote: > Thanks for the quick reply! > Unfortunately, this function needs an existing database. I could not > find a function like db:create or the like... How would i create the > database, first? > > > 2011/11/4, Lukas Kircher <lukaskircher1(a)googlemail.com>: > > Hi Erdal, > > > > you could have a look at our BaseX-specific database functions - > > db:add() [1] might be what you are looking for. > > > > Hope I could help, regards, > > Lukas > > > > > > [1] http://docs.basex.org/wiki/Database_Functions#db:add > > > > > > > > On Fri, Nov 4, 2011 at 9:54 AM, Erdal Karaca < > erdal.karaca.de(a)googlemail.com > >> wrote: > > > >> I have set up eclipse XQDT to use basex as interpreter... Now, I need > >> to import a document into a database. Is that possible without using > >> the server commands in a command line? For example some basex specific > >> functions? > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk(a)mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> > > > > > -- > 1 + 1 = 10 >

1 0

Importing a document using xquery statements?
by Erdal Karaca 04 Nov '11

04 Nov '11

I have set up eclipse XQDT to use basex as interpreter... Now, I need to import a document into a database. Is that possible without using the server commands in a command line? For example some basex specific functions?

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk