- BaseX-Talk - mailman.uni-konstanz.de

upgrade of BaseX from 8.6.5 to 8.6.6 how to migrate the data?
by Stefania Axo 18 Sep '17

18 Sep '17

hi I want to upgrade from 8.6.5 to 8.6.6. How do i migrate the data? do i just copy over the Basex\data folder? thanks Stefania

3 2

Possible Bug in BaseX 8.2.3 when importing XML (Was RE: A few general questions about BaseX)
by Anastasiou A. 14 Sep '17

14 Sep '17

Hello everyone Many thanks to Alexander, Bridger, Fabrice, Michael for getting back to me with very detailed responses, these have been really helpful. A few notes: 1) The name is Athanasios :D. Sorry, just couldn’t help it, it seemed incredibly formal to be addressed via the surname in our communications. Our mail server advertises the “Surname. Initial” pattern, so I can see where the confusion came from. 2) I think that there is scope for adding some sort of “logging” to all actions of the server in general because I think I may have hit a bug but I cannot provide any more illuminating comments. Here is what is happening: a. During import, I get an error that file somethingsomething140.xml has an incredibly long element that cannot be imported at line (blahblah). The whole process just dies there. b. This is a bug, because if I simply imported JUST the offending file itself, a single file database is created without any problems and I can query it and all. So, maybe, the error is caused because of the previous file OR because of the way the files are loaded. But I have absolutely no way of knowing the “load history” of the files or the exception that was caught or anything else. In fact, once you press “OK” in the error dialog box, any database files that have been created are lost. In addition to this, the XML files to import are enumerated in a random order. So, I had to run the import again and stay there looking at each one of the files loading, to witness that the system “breaks” after 254 files (which is suspiciously close to 256). None of the files around the vicinity of the offending file caused any problems, so this may be a more difficult to catch bug (but it is thrown with both the internal and external parsers). Following this, I created smaller databases with 250 XML files and then got “predictable” errors on running out of memory and not creating indexes which I can solve more easily. 3) It’s good to know that I don’t need the original files because that’s a lot of space I can get rid of. Thank you. 4) Seems like the ADDCACHE would have saved me some trouble here, many thanks for that, but of course, if you don’t know the file enumeration order, you are still stuck in not knowing which files have already been imported. 5) Michael, logging won’t help with the internal import procedure, except of course if you were implying writing a quick script to do the import “manually”? 6) Michael, the fork-join and “client connect” are really interesting and worth a try before I start connecting things together via Hadoop. Are these modules already available to BaseX? Do I simply import their namespace or is it not even needed? Many thanks again. All the best From: Bridger Dyson-Smith [mailto:bdysonsmith@gmail.com] Sent: 12 September 2017 16:53 To: Anastasiou A. Cc: basex-talk(a)mailman.uni-konstanz.de Subject: Re: [basex-talk] A few general questions about BaseX Hi Anastasiou, Hopefully some of these answers are somewhat helpful. On Tue, Sep 12, 2017 at 4:54 AM, Anastasiou A. <a.anastasiou(a)swansea.ac.uk<mailto:a.anastasiou@swansea.ac.uk>> wrote: Hello everyone I am trying to load BaseX with a large number of XML files (~500), each one a few hundreds of MBs big. BaseX fails with a message along the lines “This is too big for one database”. Can I please ask: 1) Are there any logs, beyond the DB logs? If yes, where can I find them? a. The reason I am asking is because once basexgui gives the message, there is no indication about the error. Ideally, I would like to know if this is a limitation on memory amount or number of items (?). I'm not sure how to enable more verbose logging with the GUI -- hopefully one of the devs or power users can weigh in on this. 2) The parser options include reading XML files from archives, which is very convenient, but once the file has been parsed, does BaseX require the “originals” for queries / returning results? AFAIK, no it does not. BaseX will query and return results from the internal database(s). 3) Is it possible to do federation with BaseX? In other words, let’s say I split a database in two large parts (as per #1), is it possible to launch two baseX servers and then have them talk to each other so that ultimately I just query one of them and get back unified results? AFAIK, the preferred method is to split your files across many databases, then query multiple databases from a single expression[1]. Others will be able to speak to this better, but I don't think there's a straightforward way to run multiple BaseX servers in a single JVM. All the best Best, Bridger [1] http://docs.basex.org/wiki/Databases

2 3

Re: [basex-talk] BaseX 7.9 unstable
by Alexander Holupirek 13 Sep '17

13 Sep '17

> On 13. Sep 2017, at 11:28, Steinhauer, Dorothea <Dorothea.Steinhauer(a)proalpha.de> wrote: > >>> On 13. Sep 2017, at 10:06, Steinhauer, Dorothea <Dorothea.Steinhauer(a)proalpha.de> wrote: >>> >>> Hi, >>> >>> we use BaseX 7.9 and we have the problem that in some (non-replicable) situations no more data is written to the database. >>> The database is still readable. >>> Logfiles are currently not available. >>> >>> If the problem is known, can changes to properties help? >>> >>> For us, it is not an option to switch to a newer version of the baseX-DB because API changes would be a huge effort. >>> >>> >>> Thanks, >>> Dorothea >> >> I think, you would have to detect the pattern that leads to the unwanted behaviour. >> Maybe the documentation on Transaction Management [1] can help to reason about the problem? >> >> By the way, what do you mean with 'no more data is written to the database'. Never ever again after some situation or during a phase (and later on it possible again)? >> >> Thanks, >> Alex >> >> [1] http://docs.basex.org/wiki/Transaction_Management#Concurrency_Control > > > Hi, > > we use the database in production, so we can't wait if anything changed. You wanted to know with the question whether the database is permanently locked? I have problems to understand what 'no more data is written to the database' actually means. - Is it there a permanent write-lock on the database? - Or is the database corrupt after a write operation? - Or can't you write at some point in time and then, at later point, once again write to the database? > When the database is "write-locking", is it possible to back up our data? When a write operation alters the database you can not read (or write) the database [1], so a regular backup is not possible. What you nevertheless can do, is, on the OS-level and only for debugging and failure analysis, copy the database directory to another location and, for instance, inspect [2] the state of the database. Perhaps you can elaborate a bit on your whole scenario. How does your application operate? What do you do, if you encounter a situation in which you can not write to the database? Can you make a guess what is happening in your application when the situation occurs? Cheers, Alex [1] http://docs.basex.org/wiki/Transaction_Management#Concurrency_Control [2] http://docs.basex.org/wiki/Commands#INSPECT

1 0

BaseX 7.9 unstable
by Steinhauer, Dorothea 13 Sep '17

13 Sep '17

Hi, we use BaseX 7.9 and we have the problem that in some (non-replicable) situations no more data is written to the database. The database is still readable. Logfiles are currently not available. If the problem is known, can changes to properties help? For us, it is not an option to switch to a newer version of the baseX-DB because API changes would be a huge effort. Thanks, Dorothea

2 1

How do create XML for a one byte hex field that is used as a flag byte
by Dave Day 08 Sep '17

08 Sep '17

Greetings list, I don't know if I am asking this question correctly. I'm new to the XML world. I can google this, get some hits, but when I read the posts, it is not really clear to me. Any help is appreciated. On an IBM mainframe, I've got a one byte hex field defined as follows: PCDTFLAG DS XL1 FLAG TO INDICATE ORIGIN OF INFO DTFLENTP EQU B'10000000' INFO CAME FROM MODULE ENTRY POINT DTFLMEPS EQU B'01000000' DB2 MEPL LIST HAS BEEN SEARCHED DTFLMEPL EQU B'00100000' INFO CAME FROM DB2 MEPL LIST DTFLDB2S EQU B'00010000' DSNMODS HAS BEEN SEARHCED DTFLDSNM EQU B'00001000' INFO CAME FROM DSNMODS SEARCH DTFLSYST EQU B'00000100' INFO CAME FROM SYSTEM TABLE DTFLSYSS EQU B'00000010' SYSTEM TABLE HAS BEEN SEARCHED DTFLCMNC EQU B'00000001' COMMON AND NUC HAS BEEN SEARCHED I need to be able to create XML for the field PCDTFLAG. OK, define an element as hexBinary. But then how do I define the bits within the byte. On the mainframe side, with the equated values, I can OR the flag byte with the equated value, and this will set the correct bit on in the byte. OI PCDTFLAG,DTFLENTP will turn on bit 0 in the 1 byte field if it is not on already. I can test for the bit on or off. TM PCDTFLAG,DTFLENTP JZ some-other-area-of-code will branch if the bit is off JO another-area-of-code will branch if the bit is on So, I guess I am asking two questions. One, how to give definition to bits within a byte, and two, how to manipulate and test those defined bits. Regards, -- Dave Day

3 2

XML import issue (header related)
by Ron Katriel 07 Sep '17

07 Sep '17

Hi, I ran into a peculiar issue with importing an XML file from DrugBank into BaseX. It involves the file header, which looks like this: <?xml version="1.0" encoding="UTF-8"?> <drugbank xmlns="http://www.drugbank.ca" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.drugbank.ca http://www.drugbank.ca/docs/drugbank.xsd" version="5.0" exported-on="2017-08-31"> The XML file (which is quite large ~3GB) loads in cleanly but queries executed against it produce no output (no errors reported). When I change the header to the following and re-import the file everything works fine: <?xml version="1.0" encoding="UTF-8"?> <drugbank> I had this issue for a while (go thought this every month) so thought it might be good to share. I am currently using BaseX 8.6.4. Any idea why this is happening? Attached is the console trace for both cases (they show subtle differences in the query plans). Thanks, Ron Ron Katriel, Ph.D. | Principal Data Scientist | Medidata Solutions 350 Hudson Street, 7th Floor, New York, NY 10014 rkatriel(a)mdsol.com | direct: +1 201 337 3622 | mobile: +1 201 675 5598 | main: +1 212 918 1800

5 14

Re: [basex-talk] Fetching Images
by Günter Dunz-Wolff 06 Sep '17

06 Sep '17

Sorry about the question, I only had to put the url to <img/> source. Fetching wasn’t necessary.

1 0

Fetching Images
by Günter Dunz-Wolff 06 Sep '17

06 Sep '17

Hi all, in my RESTXQ-App I’m trying to fetch images (for example via „https://maps.googleapis.com/maps/api <https://maps.googleapis.com/maps/api>“): return fetch:binary("https://maps.googleapis.com/maps/api <https://maps.googleapis.com/maps/api>/blabla“) With fetch:content-type I get the right type: image/png. But with fetch:binary I only get the binary as large string but the browser doesn’t render the picture. If I load the url in browser directly, the image renders as expected. What am I doing wrong? Thanks for any help, Guenter

1 0

Re: [basex-talk] New basex-version -> broken paths
by Günter Dunz-Wolff 04 Sep '17

04 Sep '17

Hi Michael, thanks for the information. I changed let $tokens := doc("tokens.xml")//entry to let $tokens := fetch:xml("tokens.xml")//entry (tokens.xml is in the same directory as the xq-file.) The problem is still the same: [bxerr:BXFE0001] Resource "/Users/User/tokens.xml (No such file or directory)" not found. The path should be: /Users/User/Documents/project/BaseXWeb/tokens.xml Same problems with RESTPATH empty or path to project-folder. As I wrote, I have only problems with the REST-Service, not with RESTXQ. Best, Guenter

2 2

Simple Map Operator in oXygen XML (XQJ)
by Omar Siam 04 Sep '17

04 Sep '17

Hi! I just tried to use ther Simple Map Operator while writing an XQuery in oXygen XML and executing the query using a client/server BaseX 8.6.6 data source. I configured it as described here: http://docs.basex.org/wiki/Integrating_oXygen When I use the Simple Map Operator (!) I only get "Invalid XQuery syntax, syntax does not pass static validation.". For example using "(1 to 10) ! element node { . }" If I run that XQuery in DBA for example it works. Any ideas? Best regards Omar Siam

3 3