- BaseX-Talk - mailman.uni-konstanz.de

ft:search and map-function
by kleist 05 Jan '16

05 Jan '16

Hi all, in the documentation for the ft:search-function is the following example: ft:search("db", ("A", "B"), map { "mode": "all words", "distance": { "max": "5", "unit": "words" } }) It's throwing this error for line 3: "Invalid value, simple expression expected." I'm using BaseX Version 8.3.1. Did the notation change or what's going wrong? Best regards, Günter

2 1

GUI: Horizontal splitting needed
by Alex R. Loayza 05 Jan '16

05 Jan '16

Hello, I'd like to suggest an improvement to the GUI: the ability to split the window horizontally, as you can do in IDEA or Visual Studio. I'm finding myself all too frequently scrolling up and down between functions or variables. After a while, it wears you out. A horizontal split would make BaseX's already great GUI, even better. Hope you agree and we can find it in BaseX 8.4! Best regards, Alex

2 1

Problems with option XINCLUDE
by Hans-Juergen Rennau 04 Jan '16

04 Jan '16

Dear BaseX team, I face unexpected problem when trying to switch off xinclude processing. From the documentation I understand that what I have to do is set the option XINCLUDE to the value "false". But how? Attempt #1========= I added to the config file .basex the line: XINCLUDE = false Result: no effect, and these messages: C:/Program Files (x86)/BaseX/.basex: Unknown option 'XINCLUDE'. C:/Program Files (x86)/BaseX/.basex: writing new configuration file. Did I misunderstand the documentation? Attempt #2======== I inserted into the query this initial line:declare option db:XINCLUDE 'false'; Result: Program crash with this message:[FODC0002] "file:///C:/projects/otds/otds-config/h.xml" (Line 1): Vorzeitiges Dateiende. where the named file (h.xml) is the output (!) file specified either like this:basex foo.xq > h.xmlor like this:basex -o h.xml foo.xq Note. Using neither > ... nor -o ..., I do get the correct output, written to stdout. But I must capture it, and the capturing fails! Attempt #3======== Patching the basex.bat file, inserting option -D: java -Dorg.basex.XINCLUDE=false -cp "%CP%" %BASEX_JVM% org.basex.BaseX %* Result:Same as attempt #2 So I'm at a loss. I need to switch off xinclude because my input files use xinclude in combination with xpointer attributes, like this: <xi:include href="../../../globals/notifications.xml" xpointer="xpointer(/config/*)" parse="xml" /> Applying doc(...) to such a file, yields this result: [FODC0002] "file:///C:/projects/otds/otds-config/branches/path/to/bar.xml" (Line 11): "include" mit href "../../../globals/notifications.xml" nicht erfolgreich, und es wurde kein "fallback"-Element gefunden. Can you help me? Hans-Jürgen PS: Using version:BaseX 8.3 beta 474994c [Standalone]

2 2

AWS Deployment, .war-file writing
by kleist 04 Jan '16

04 Jan '16

Hi all, I want to deploy my baxex-App to Amazon Web Services (AWS). Therefore I need a .war file of my existing application, not only the war-file of basex. How can I do it? There is the excellent tutorial of Paul Swennenhuis about Deployment on AWS (Part II). But it's a little bit outdated. AWS is now using tomcat 8 and it is no more stored at /var/lib/tomcat... Does anybody know, where I can find the new root-directory? Thanks for any help, Günter

1 0

Ignore option for full-text-search
by kleist 04 Jan '16

04 Jan '16

Hi Christian, my long-time project (kleist-digital.de) is nearly only based on documents (TEI-xml), so I'm working mostly with mixed content. Are there any plans for future versions to implement the ignore option for full-text-search? It would help a lot. Best, Günter

2 2

can't make Geo Module work
by Eric Boisvert 03 Jan '16

03 Jan '16

Greetings I just downloaded BaseX 8.3.1 .war and installed it on Tomcat 7 (jre1.8.0_66) running Windows 7. enterprise (SP1). Did nothing (just loaded a single XML file) throught the web based admin interface and tried this XQuery import module namespace geo = "http://expath.org/ns/geo"; declare namespace gml='http://www.opengis.net/gml'; let $point := <gml:Point><gml:coordinates>1,2</gml:coordinates></gml:Point> return geo:geometry-type($point) (it's from Geo Module documentation at http://docs.basex.org/wiki/Geo_Module#geo:geometry-type) all I get is *[XPST0017] Unknown function 'geo:geometry-type'.* in the logs, I have [POST] http://localhost:8080/BaseX831/dba/eval-query?name=&resource=&sort=&loglist=; Stopped at C:/Program Files/Apache Software Foundation/Tomcat 7.0/webapps/BaseX831/dba/modules/util.xqm, 94/17: [XPST0017] Unknown function 'geo:geometry-type'. The java Geo.class are present (webapps\BaseX831\WEB-INF\classes\org\expath\ns) Other trivial XQuery do work Any idea what's wrong ? Thanks and Happy New Year

3 6

GUI feature request "Add as import"
by Andy Bunce 03 Jan '16

03 Jan '16

Hi, I often find myself adding module imports in the GUI and I wonder if the following suggestion might speed up that process. Add a new menu item to the filelist right click context menu "Add as import" when the file selection is a XQuery module. The action would be to add a line to the current edit window source with import module namespace {$prefix} = '{$namespace}' at {$path}; Where $prefix and $namespace} are read from the source of the selected file, and $path is calculated from the relative path from the edit file source to the selected filelist source. The line would be inserted at a suitable place in the prologue or at the current location if that is too tricky. /Andy

2 4

archive:extract:-binary can cause OutOfMemoryException
by E. Wray Johnson 02 Jan '16

02 Jan '16

I have written an XQuery that properly extracts binary(raw) archive resource contents to binary(raw) resources. However since archive:extract:-binary returns the contents as xs:base64Binary and not streamable items, large archive contents are loaded completely into memory and will often cause an OutOfMemoryException. I have increased the BaseX JVM heap size and the problem goes away, but this is not acceptable in a system where many users may be doing this concurrently. archive:extract-to seems to only extract to an external file-system location and not to binary(raw) resources. Example code is provided below that extracts to the same database folder as the archive, however the target location of the contents is not pertinent to the problem. let $database := 'db' let $archivePath := 'myfolder/myarchive.zip' let $archive := db:retrieve($database, $archivePath) let $basePath := replace($archivePath, '[^/]+$', '') let $entries := archive:entries($archive) let $contents := archive:extract-binary($archive) for $entry at $pos in $entries let $content := $contents[$pos] let $target := $basePath || '/' || $entry/text() return db:store($database, $target, $content)

2 1

Java module to add documents to database
by Kevin S. Clarke 01 Jan '16

01 Jan '16

Hi, I'm writing a Java module that will do a couple of things, in addition to adding documents to a BaseX database. I've gotten the other things done, but I'm stuck a bit about how to add docs to the database from a module (I've seen the example that returns xml fragments that can be used from the xquery side). I see on http://docs.basex.org/wiki/Java_Examples that there are local/embedded and client-api ways to interact, but a module would want to be able to work in either situation. Are there any module examples that write to the database that someone would like to share (or just point me in the right direction)? Thanks, Kevin

2 4

Versioning
by Laurent Pellegrino 31 Dec '15

31 Dec '15

Hello all, I discovered BaseX recently and I find it really powerfull. Unfortunately, I read that versioning was not supported out of the box in 2014 [1]. Is it still the case ? Is there no simple workaround ? My usecase is the following. I have thousands of XML files which are indexed using BaseX in a specific database, lets say "mydb". At some point of time, a user may want to add a new XML file which is a slightly modified version of one already indexed by BaseX. I would like to be able to: - store this new XML resource in "mydb" database without loosing the XML from which it is issued - to retrieve the latest version or a previous one. - filter resources based on the latest version A simple way to achieve such a behaviour would be to be able to index a XML resource in BaseX with some additional meta information such as an ID and a timestamp, and then offer the possibility to use these meta information in an XQuery to filter resources. Is it possible with BaseX 8.3 ? I am looking for a solution that does not a require another database. [1] https://mailman.uni-konstanz.de/pipermail/basex-talk/2014-March/006535.html Kind Regards, Laurent Pellegrino

4 11

Full-text search with ft:mark (linking back)
by kleist 31 Dec '15

31 Dec '15

Dear members, given for example: ft:mark(//p[.//text() contains text 'real']). This will yield some results. Is there any way, to link back to the documents, where p was found? I know, it would work without ft:mark, but I need the <mark>-tag inside of the results, especially for fuzzy-search-results. Any help would be appreciated. - Günter

2 1

Store command with InputStream
by buddyonweb-software＠yahoo.com 29 Dec '15

29 Dec '15

I am writing a Java program that creates some text data that I want to store as a 'raw' document in a database.In the JavaDocs for the Store command () it states for the constructor public Store(java.lang.String path): "Constructor, specifying a target path. The input needs to be set via Command.setInput(InputStream)." Since the input is not a file, but rather an InputStream of data in memory, I do indeed call setInput. However, I cannotget this to work. Am I doing something wrong? Must the input only be from a file (and not from data in memory)? Here is snippet of code and thanks in advance. Note: Using the client/server mode. ClientSession session = new ClientSession(host, port, user, pwd); session.open("MyDb"); String theData = someFunctionThatReturnsALargeStringOfData(); ByteArrayInputStream bais = new ByteArrayInputStream(theData.getBytes()); Store cmd = new Store("newFile.txt"); cmd.setInput(bais); session.execute(cmd); session.close(); Note sure if this helps, but this is the stack trace I get. org.basex.core.BaseXException: Stopped at , 1/63: Syntax: STORE (TO [path]) [input] Store raw data. Stores raw data to the specified [path]. at org.basex.api.client.ClientSession.receive(ClientSession.java:193) at org.basex.api.client.ClientSession.execute(ClientSession.java:162) at org.basex.api.client.ClientSession.execute(ClientSession.java:167) at org.basex.api.client.Session.execute(Session.java:36) ......

2 1

Change context of QueryProcessor to query item
by Chris Speare 23 Dec '15

23 Dec '15

Good afternoon, I have a QueryProcessor to grab the top level items which works, but then I want to iterate over these items to query sub-items. My XML is currently (sub-section, excuse any typos however it is well formed) <MappingRequest> <Mappings destination="invoice_sftp" map="pdf_invoice"> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231749.pdf" xpath="//CommercialDocument[@id='231749']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231750.pdf" xpath="//CommercialDocument[@id='231750']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231751.pdf" xpath="//CommercialDocument[@id='231751']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231752.pdf" xpath="//CommercialDocument[@id='231752']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231753.pdf" xpath="//CommercialDocument[@id='231753']"/> </Mappings> <Mappings destination="invoice_sftp" map="pdf_device_report"> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375680.pdf" xpath="//Device[@report_id='8375680']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375681.pdf" xpath="//Device[@report_id='8375681']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375679.pdf" xpath="//Device[@report_id='8375679']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375682.pdf" xpath="//Device[@report_id='8375682']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375686.pdf" xpath="//Device[@report_id='8375686']"/> </Mappings> </MappingRequest> My code in brief is: final Context context = new Context(); new CreateDB("LocalDB", "/test.xml").execute(context); QueryProcessor proc = new QueryProcessor("/MappingRequest/Mappings", context); // Loop over the mappings, get the map name then iterate over the underlying mapping nodes for(Item item; (item = iter.next()) != null;) { Object elem = BXNode.get((ANode)item); BXElem node1 = (BXElem)elem; System.out.println(node1.getAttribute("map")); QueryProcessor proc2 = proc.context(item); proc2.??("/Mapping"); Iter iter2 = proc.iter(); for(Item item2; (item2 = iter2.next()) != null;) { elem = BXNode.get((ANode)item2); node1 = (BXElem)elem; System.out.println(node1.getNodeName()); } } Having worked with Saxon previously I was able to pass the XdmItem from one query to the context of another query - allowing further queries to be carried out, therefore I assumed I would be able to something similar with BaseX? Feel I'm almost there with various ideas but nothing seems to change the context to the an item, rather than sticking with the originally loaded document. Secondly to this I need to workout how to perform the queries via a ClientSession - however that only seems to return a ClientQuery where .next() returns a String, rather than a workable Item or other object. Kind regards Chris Speare The information contained in the email and any files transmitted with it is confidential and intended for the addressee only. If you have received this email in error please accept our apologies and notify the originator. Any disclosure, copying, distribution or any other use of this communication is strictly prohibited and may be unlawful. We reserve the right to monitor and intercept communications for lawful business purposes. Wireless Innovation Ltd , Unit D2, Churcham Business Park, Churcham, Gloucestershire, GL2 8AX, Great Britain. Company Registration Number 5240202, Place of Registration Companies House Cardiff.

2 2

Problem with 3rd party jars in repository
by E. Wray Johnson 22 Dec '15

22 Dec '15

I have a 3rd party jar which is open source, which has no startup eligible classes. The classes I need to use have class names (not package name) that are all lower case. This seems to be a problem for BaseX. So, I tried to create a wrapper class, and the wrapper class loads but BaseX cannot load the dependent 3rd party jar no matter what I try. I have even tried putting the jar in the basex\lib folder. I have been able to install my wrapper class and run it successfully, but when I include the 3rd party jar, I get "Module not initialized" for the classes in the 3rd party jar because they are in another package. I have tried packaging everything into one jar, but that did not help. I have added a main class to the manifest of the 3rd party jar and it loaded fine but I cannot include the namespace in my xquery. My example is similar to the FOP.jar example on the file server, however I do not see query ussage example of this example. I have not yet tried the expath-pkg.xml route.

2 2

index:attribute-names empty
by Benjamin Gathmann 18 Dec '15

18 Dec '15

Hi Christian & Co., I have created a DB with the following characteristics: - Size: 37 GB -Nodes: 1'532'608'670 - Documents: 5205 - Up-to-date: true - TEXTINDEX: false - ATTRINDEX: false If I run "index:attribute-names" in the GUI command line (as described here: http://docs.basex.org/wiki/Indexes), I get an empty result and the following Query Info: ¨Query: index:attribute-names Result: - Hit(s): 0 Items - Updated: 0 Items - Printed: 0 Bytes - Read Locking: local [database] - Write Locking: none Timing: - Parsing: 1.08 ms - Compiling: 0.2 ms - Evaluating: 52.83 ms - Printing: 0.02 ms - Total Time: 54.13 ms Query plan: <QueryPlan compiled="true"> <CachedPath> <IterStep axis="child" test="index:attribute-names"/> </CachedPath> </QueryPlan> Why is the result empty? Yours, Benjamin

2 1

GUI Textindex: false although true
by Benjamin Gathmann 16 Dec '15

16 Dec '15

I use the GUI of the current release 8.3.1. I have played around with the options in BaseX to test the speed of CREATE and ADD operations. Now the GUI "Open and manage" box shows me for any DB I create: Up-to-date: true TEXTINDEX: false ATTRINDEX: false However, I have never set TEXTINDEX or ATTRINDEX to false, and the files in the "data" directory suggest that the indexes are there, as does a "GET TEXTINDEX" (returns "true"). I only made the following changes: set updindex true; set autooptimize true; set autoflush false; -- joe-security-logo Joe Security LLC Benjamin Gathmann, Developer & Support business parc Reinach, Christoph Merian-Ring 11, 4153 Reinach, Switzerland benjamin.gathmann(a)joesecurity.org <mailto:benjamin.gathmann@joesecurity.org>, http://www.joesecurity.org ------------------------------------------------------------------------ The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.

2 2

Node Maximum 2^31
by Benjamin Gathmann 16 Dec '15

16 Dec '15

I read here that the current maximum number of nodes in a BaseX DB is 2^31 (2'147'483'648): http://docs.basex.org/wiki/Statistics Is this related to 32bit CPU architecture? And are there plans for a 64bit version of BaseX that would allow larger databases? I am asking because I am testing BaseX with large XML files (usually 5 to 15MB each) with a very deep nested structure and many small elements. Right now, I have created a DB with 3979 such documents. DB Size: 29,697 MB Nodes: 1,172,761,158 This means that I will reach the max. #nodes already with around 5500 reports and a DB size of 55 GB. This in turn will mean that I will have to split my documents on rather many database instances if I import e.g. 50,000 documents, which worries me a bit. Or do I worry with no reason?

2 1

Using external variable to perform node insert
by Mike Engledew 16 Dec '15

16 Dec '15

We are attempting to insert some new elements with attributes into an XML document from a string. The elements have a name space prefix such as asy: If I use an external variable containing XML to perform a node insert example :- declare namespace asy="http://xml.acme.com/asy"; declare variable $part external; insert nodes $part as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] The characters in the xs:string $path reserved for XML e.g. < > etc are converted to < > etc and the insert silently fails. if I change the insert statement to : insert nodes fn:parse-xml($part) as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] I get the message FODC0006 (Line 1): The prefix "asy" for element "asy:part" is not bound. If I add declare namespace asy="http://xml.acme.com/asy"; at the beginning of the $path variable contents before the XML I get FODC0006 (Line 1): Content is not allowed in prolog. If I then change the statement to : insert nodes xquery:eval($part) as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] The insert works as expected. Note that if the same XML is defined within the query using let $path := < new xml stuff > The insert works fine just using $path with no functions applied. Is this behavior correct and is using xquery:eval($part) the right approach or am I just lucky it works and I should have been doing something different? Thanks for any insight you can offer.

2 3

Re: [basex-talk] Extracting binary stored zip into database as binary stored elements not working
by Christian Grün 11 Dec '15

11 Dec '15

> In the meantime can you shed some light into the > [FODC0002] Resource > 'delta.zip' does not exist.' error? (cc to the list): It simply says that a database resource does not exist. Maybe you have more than one BaseX database directory in your environment? Could you try db:list('...db...') and check what results are returned in the GUI and via the client? > On Fri, Dec 11, 2015 at 9:53 AM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> > The raw stored file was slightly larger than the original zip. >> >> Do you have an example for that? >> >> > But when I submit the exact same query using the Java client I get the >> > following error: >> > >> > Stopped at ., 1/216: >> > [FODC0002] Resource 'delta.zip' does not exist. >> >> It shouldn’t make a difference. I’ll probably need more info to track this >> down. > >

2 6

Extracting binary stored zip into database as binary stored elements not working
by E. Wray Johnson 11 Dec '15

11 Dec '15

My zip file has files nested in subfolders. First I used db:store to store delta.zip into the database. My first problem was I got an error " *invalid entry CRC: expected 0x0 but got..."* The raw stored file was slightly larger *than the original zip.* So I replaced the zip file in the raw folder with the original one and the first error went away. Now I don't get any error bu the extracted file are not stored as binaries What is wrong with this code? I removed the updating annotation and returned $target instead of the try/store/catch inside the loop and got the expected result path name let $database := 'db' let $extractTo := 'target.zip' (: path to raw stored zip :) let $archivePath := 'alpha/bravo/charlie/delta.zip' let $archive := db:retrieve($database, $archivePath) let $basePath := replace($archivePath, '[^/]+$', '') || $extractTo let $entries := archive:entries($archive) let $contents := archive:extract-binary($archive) return for-each-pair($entries, $contents, %updating function($entry, $content) { let $target := $basePath || '/' || $entry/text() return try { db:store($database, $target, $content) } catch * { () } })

2 2

Concurrency model
by E. Wray Johnson 11 Dec '15

11 Dec '15

What is the best way to manage multiple users in a multi-threaded environment like servlets?

2 1

DB Contention
by buddyonweb-software＠yahoo.com 11 Dec '15

11 Dec '15

I have a Java process that continually scans for incoming .xml files deposited into a system folder on the OS. When it finds a file (say a.xml), it will create a DB for that file (call it DB-A) and load the XML file into that database (DB-A).Note: DB names will be guaranteed to be unique when created, so there will never be 2 DB-A databases created. As this file is processed by a seondar, other XML files will be added into DB-A - but all documents will relate only to the processing of the a.xml file (for example, various statistics, etc.). Now, there is a secondary/separate java process, that scans the database instances that have been created with the XML file loaded. This secondary process does some querying on that file and adds additional XML documents to that database. That is only this process will add new documents to the database. So my question is this: 1. With process 1 creating and inserting the original .xml file, is there a chance for database contention or is this architecture pretty safe from contention? Note: both Java process are simply using the BaseX.jar file. 2. If I added a third Java process in the future, that would a) only access existing documents in read only mode b) could add new documents to the database that no other process would read or update, is this safe from contention? Thanks in advance.

5 8

Question about baseX add "/path/to/filesystem/directory"
by ansel.jm＠free.fr 11 Dec '15

11 Dec '15

Hello I'm experiencing a trouble with the add "/path/to/filesystem/directory" on a large directory structure with thousands of files. The result is a strange error : > add "/data/PARTAGE/DOSSIERS EC&T/PC" "Configurations2/accelerator/current.xml" (Line 1): No text allowed before root element. And in the directory /data/PARTAGE/DOSSIERS EC&T/PC, no Configurations2 directory... I think a file is having a problem but with a thousand of files, how to know which of them ? On another directory, the add function works but finds xml files that aren't in the directory structure (find . -name "xxx" doesn't find them as basex finds !!!)... Is the "add function" fragile on a directory ? Many thanks in advance JMA

2 1

Broken Links on website
by E. Wray Johnson 07 Dec '15

07 Dec '15

On: http://docs.basex.org/wiki/Java_Examples Links in the following text seem to be broken: XQJ API > > The implementation of the BaseX XQJ API <http://xqj.net/basex/> > (closed-source) has been written by Charles Foster. It uses the > client/server architecture. The basex-examples repository contains various > examples > <https://github.com/BaseXdb/basex-examples/tree/master/src/main/java/org/bas…> > on how to use XQJ. > The broken links are: - http://xqj.net/basex/ - https://github.com/BaseXdb/basex-examples/tree/master/src/main/java/org/bas…

2 2

[bxerr:BXDB0002] Too many open files
by Mansi Sheth 07 Dec '15

07 Dec '15

Hello, I am importing BaseX, with tons of XML files. Currently I have roughly 1600 databases, I am starting basexhttp service, to access it over a web service endpoint, thru a xquery file. Using BaseX 8.2.3. I am receiving below error: [bxerr:BXDB0002] Resource "/veracode/msheth/BaseXDB/bi_output_713/inf.basex (Too many open files)" not found. basexhttp, is running with 10240M virtual memory. I can share the xquery file, if thats needed. Has anyone experienced this before ? Is there a limit on no of databases supported by BaseX ? Is there some configuration option, which I can use to close already queried database ? Thanks, - Mansi

4 5

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk