BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5269 discussions

Change context of QueryProcessor to query item
by Chris Speare 23 Dec '15

23 Dec '15

Good afternoon, I have a QueryProcessor to grab the top level items which works, but then I want to iterate over these items to query sub-items. My XML is currently (sub-section, excuse any typos however it is well formed) <MappingRequest> <Mappings destination="invoice_sftp" map="pdf_invoice"> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231749.pdf" xpath="//CommercialDocument[@id='231749']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231750.pdf" xpath="//CommercialDocument[@id='231750']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231751.pdf" xpath="//CommercialDocument[@id='231751']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231752.pdf" xpath="//CommercialDocument[@id='231752']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/invoice/2/3/1/231753.pdf" xpath="//CommercialDocument[@id='231753']"/> </Mappings> <Mappings destination="invoice_sftp" map="pdf_device_report"> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375680.pdf" xpath="//Device[@report_id='8375680']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375681.pdf" xpath="//Device[@report_id='8375681']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375679.pdf" xpath="//Device[@report_id='8375679']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375682.pdf" xpath="//Device[@report_id='8375682']"/> <Mapping path="/var/lib/michrosatbilling/billingrunoutput_testing/dummy/report/8/3/7/8375686.pdf" xpath="//Device[@report_id='8375686']"/> </Mappings> </MappingRequest> My code in brief is: final Context context = new Context(); new CreateDB("LocalDB", "/test.xml").execute(context); QueryProcessor proc = new QueryProcessor("/MappingRequest/Mappings", context); // Loop over the mappings, get the map name then iterate over the underlying mapping nodes for(Item item; (item = iter.next()) != null;) { Object elem = BXNode.get((ANode)item); BXElem node1 = (BXElem)elem; System.out.println(node1.getAttribute("map")); QueryProcessor proc2 = proc.context(item); proc2.??("/Mapping"); Iter iter2 = proc.iter(); for(Item item2; (item2 = iter2.next()) != null;) { elem = BXNode.get((ANode)item2); node1 = (BXElem)elem; System.out.println(node1.getNodeName()); } } Having worked with Saxon previously I was able to pass the XdmItem from one query to the context of another query - allowing further queries to be carried out, therefore I assumed I would be able to something similar with BaseX? Feel I'm almost there with various ideas but nothing seems to change the context to the an item, rather than sticking with the originally loaded document. Secondly to this I need to workout how to perform the queries via a ClientSession - however that only seems to return a ClientQuery where .next() returns a String, rather than a workable Item or other object. Kind regards Chris Speare The information contained in the email and any files transmitted with it is confidential and intended for the addressee only. If you have received this email in error please accept our apologies and notify the originator. Any disclosure, copying, distribution or any other use of this communication is strictly prohibited and may be unlawful. We reserve the right to monitor and intercept communications for lawful business purposes. Wireless Innovation Ltd , Unit D2, Churcham Business Park, Churcham, Gloucestershire, GL2 8AX, Great Britain. Company Registration Number 5240202, Place of Registration Companies House Cardiff.

2 2

Problem with 3rd party jars in repository
by E. Wray Johnson 22 Dec '15

22 Dec '15

I have a 3rd party jar which is open source, which has no startup eligible classes. The classes I need to use have class names (not package name) that are all lower case. This seems to be a problem for BaseX. So, I tried to create a wrapper class, and the wrapper class loads but BaseX cannot load the dependent 3rd party jar no matter what I try. I have even tried putting the jar in the basex\lib folder. I have been able to install my wrapper class and run it successfully, but when I include the 3rd party jar, I get "Module not initialized" for the classes in the 3rd party jar because they are in another package. I have tried packaging everything into one jar, but that did not help. I have added a main class to the manifest of the 3rd party jar and it loaded fine but I cannot include the namespace in my xquery. My example is similar to the FOP.jar example on the file server, however I do not see query ussage example of this example. I have not yet tried the expath-pkg.xml route.

2 2

index:attribute-names empty
by Benjamin Gathmann 18 Dec '15

18 Dec '15

Hi Christian & Co., I have created a DB with the following characteristics: - Size: 37 GB -Nodes: 1'532'608'670 - Documents: 5205 - Up-to-date: true - TEXTINDEX: false - ATTRINDEX: false If I run "index:attribute-names" in the GUI command line (as described here: http://docs.basex.org/wiki/Indexes), I get an empty result and the following Query Info: ¨Query: index:attribute-names Result: - Hit(s): 0 Items - Updated: 0 Items - Printed: 0 Bytes - Read Locking: local [database] - Write Locking: none Timing: - Parsing: 1.08 ms - Compiling: 0.2 ms - Evaluating: 52.83 ms - Printing: 0.02 ms - Total Time: 54.13 ms Query plan: <QueryPlan compiled="true"> <CachedPath> <IterStep axis="child" test="index:attribute-names"/> </CachedPath> </QueryPlan> Why is the result empty? Yours, Benjamin

2 1

GUI Textindex: false although true
by Benjamin Gathmann 16 Dec '15

16 Dec '15

I use the GUI of the current release 8.3.1. I have played around with the options in BaseX to test the speed of CREATE and ADD operations. Now the GUI "Open and manage" box shows me for any DB I create: Up-to-date: true TEXTINDEX: false ATTRINDEX: false However, I have never set TEXTINDEX or ATTRINDEX to false, and the files in the "data" directory suggest that the indexes are there, as does a "GET TEXTINDEX" (returns "true"). I only made the following changes: set updindex true; set autooptimize true; set autoflush false; -- joe-security-logo Joe Security LLC Benjamin Gathmann, Developer & Support business parc Reinach, Christoph Merian-Ring 11, 4153 Reinach, Switzerland benjamin.gathmann(a)joesecurity.org <mailto:benjamin.gathmann@joesecurity.org>, http://www.joesecurity.org ------------------------------------------------------------------------ The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.

2 2

Node Maximum 2^31
by Benjamin Gathmann 16 Dec '15

16 Dec '15

I read here that the current maximum number of nodes in a BaseX DB is 2^31 (2'147'483'648): http://docs.basex.org/wiki/Statistics Is this related to 32bit CPU architecture? And are there plans for a 64bit version of BaseX that would allow larger databases? I am asking because I am testing BaseX with large XML files (usually 5 to 15MB each) with a very deep nested structure and many small elements. Right now, I have created a DB with 3979 such documents. DB Size: 29,697 MB Nodes: 1,172,761,158 This means that I will reach the max. #nodes already with around 5500 reports and a DB size of 55 GB. This in turn will mean that I will have to split my documents on rather many database instances if I import e.g. 50,000 documents, which worries me a bit. Or do I worry with no reason?

2 1

Using external variable to perform node insert
by Mike Engledew 16 Dec '15

16 Dec '15

We are attempting to insert some new elements with attributes into an XML document from a string. The elements have a name space prefix such as asy: If I use an external variable containing XML to perform a node insert example :- declare namespace asy="http://xml.acme.com/asy"; declare variable $part external; insert nodes $part as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] The characters in the xs:string $path reserved for XML e.g. < > etc are converted to < > etc and the insert silently fails. if I change the insert statement to : insert nodes fn:parse-xml($part) as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] I get the message FODC0006 (Line 1): The prefix "asy" for element "asy:part" is not bound. If I add declare namespace asy="http://xml.acme.com/asy"; at the beginning of the $path variable contents before the XML I get FODC0006 (Line 1): Content is not allowed in prolog. If I then change the statement to : insert nodes xquery:eval($part) as first into db:open("PARTDB")/asy:assembly[@name="ZB09010"] The insert works as expected. Note that if the same XML is defined within the query using let $path := < new xml stuff > The insert works fine just using $path with no functions applied. Is this behavior correct and is using xquery:eval($part) the right approach or am I just lucky it works and I should have been doing something different? Thanks for any insight you can offer.

2 3

Re: [basex-talk] Extracting binary stored zip into database as binary stored elements not working
by Christian Grün 11 Dec '15

11 Dec '15

> In the meantime can you shed some light into the > [FODC0002] Resource > 'delta.zip' does not exist.' error? (cc to the list): It simply says that a database resource does not exist. Maybe you have more than one BaseX database directory in your environment? Could you try db:list('...db...') and check what results are returned in the GUI and via the client? > On Fri, Dec 11, 2015 at 9:53 AM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> > The raw stored file was slightly larger than the original zip. >> >> Do you have an example for that? >> >> > But when I submit the exact same query using the Java client I get the >> > following error: >> > >> > Stopped at ., 1/216: >> > [FODC0002] Resource 'delta.zip' does not exist. >> >> It shouldn’t make a difference. I’ll probably need more info to track this >> down. > >

2 6

Extracting binary stored zip into database as binary stored elements not working
by E. Wray Johnson 11 Dec '15

11 Dec '15

My zip file has files nested in subfolders. First I used db:store to store delta.zip into the database. My first problem was I got an error " *invalid entry CRC: expected 0x0 but got..."* The raw stored file was slightly larger *than the original zip.* So I replaced the zip file in the raw folder with the original one and the first error went away. Now I don't get any error bu the extracted file are not stored as binaries What is wrong with this code? I removed the updating annotation and returned $target instead of the try/store/catch inside the loop and got the expected result path name let $database := 'db' let $extractTo := 'target.zip' (: path to raw stored zip :) let $archivePath := 'alpha/bravo/charlie/delta.zip' let $archive := db:retrieve($database, $archivePath) let $basePath := replace($archivePath, '[^/]+$', '') || $extractTo let $entries := archive:entries($archive) let $contents := archive:extract-binary($archive) return for-each-pair($entries, $contents, %updating function($entry, $content) { let $target := $basePath || '/' || $entry/text() return try { db:store($database, $target, $content) } catch * { () } })

2 2

Concurrency model
by E. Wray Johnson 11 Dec '15

11 Dec '15

What is the best way to manage multiple users in a multi-threaded environment like servlets?

2 1

DB Contention
by buddyonweb-software＠yahoo.com 11 Dec '15

11 Dec '15

I have a Java process that continually scans for incoming .xml files deposited into a system folder on the OS. When it finds a file (say a.xml), it will create a DB for that file (call it DB-A) and load the XML file into that database (DB-A).Note: DB names will be guaranteed to be unique when created, so there will never be 2 DB-A databases created. As this file is processed by a seondar, other XML files will be added into DB-A - but all documents will relate only to the processing of the a.xml file (for example, various statistics, etc.). Now, there is a secondary/separate java process, that scans the database instances that have been created with the XML file loaded. This secondary process does some querying on that file and adds additional XML documents to that database. That is only this process will add new documents to the database. So my question is this: 1. With process 1 creating and inserting the original .xml file, is there a chance for database contention or is this architecture pretty safe from contention? Note: both Java process are simply using the BaseX.jar file. 2. If I added a third Java process in the future, that would a) only access existing documents in read only mode b) could add new documents to the database that no other process would read or update, is this safe from contention? Thanks in advance.

5 8

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk