- BaseX-Talk - mailman.uni-konstanz.de

New basex-version -> broken paths
by Günter Dunz-Wolff 04 Sep '17

04 Sep '17

Hi all, I updated my aged basex-version (8.5.3) to 8.6.6. With the new version, my defined file-references are broken, for example: let $style := doc('../xq_diff/remove_elements_from_doc.xsl‘) Even files in the same directory with the XQuery-file (corrigenda.xq) like: let $tokens := doc(„tokens.xml") With 8.5.3 the REST-Requests with http://localhost:8984/rest?run=xqs/corrigenda.xq <http://localhost:8984/rest?run=xqs/corrigenda.xq> worked without problems. With 8.6.6 and with the same request I’ll get the following error: Stopped at ., 11/18: [FODC0002] Resource '/Users/xq_diff/remove_elements_from_doc.xsl' does not exist OR Stopped at ., 79/26: [FODC0002] Resource '/Users/xxx/kleist_tokens.xml' does not exist. It does work in BaseXGUI but not with the http-server. Thanks for any help, Guenter

3 2

file:name -> admin permission required
by Günter Dunz-Wolff 01 Sep '17

01 Sep '17

Hi all, I updated my older basex-version, now I get in my RESTXQ-App the error 'admin permission required‘ with let $collection := collection("data/stories") for $doc in $collection let $file_name := file:name($doc) return $file_name Whats going wrong? Thanks for any help. Guenter

2 2

Best practice for storing timestamp information in XML
by Dave Day 31 Aug '17

31 Aug '17

Greetings List, I've got an 8 byte value that is the output of a Store Clock instruction on a mainframe. The value created by the instruction is a 64 bit hex value where bit 51 is equal to one microsecond. To get a value that is microseconds, the original clock value is shifted to the right 12 bits. The clock started at all 0's on January 1st, 1900. I need to be able to store and do basic math(addition and subtraction) on the value in microseconds. And, these values will need to be displayed on a browser in HH:MM:SS.TTTTTT format. If I create the decimal value for the integer on the mainframe when creating the XML record, I will have to go thru some conversion logic. There could be possibly millions of these records in one XML database. It seems to me to make more sense to create the value as a hexBinary element, and then have some code running within BaseX do the conversion to create the correct time value. Is this the way to do something like this? Regards, -- Dave Day

4 4

Old dog trying to learn some new tricks.
by Dave Day 29 Aug '17

29 Aug '17

Greetings BaseX list, I've got some very basic questions I was hoping I could get the answers to on this list. Questions really break down into two categories. My background is strictly on IBM mainframes, so please bear with me. 1)Some XML schema questions, and 2)Questions regarding how to configure/use BaseX for what I want to do. XML schema. I've got a record definition created by IBM that changes somewhat depending upon the release of the system. I want to create XML schema for each release dependent definition, and then make that definition available to BaseX. I thought using namespaces was the way to go, but now am not sure. What I was hoping to do was to connect to a running BaseX, and send the schema definitions that would be used to validate the XML. In reading doc, I see it is possible to create namespaces and use them, but the format has to be either a URL or a URN. Is it possible to create a URN for a namespace on the fly, and then tell the code later on which namespace to use? Regards, -- Dave Day

4 3

Test message. Just to see if it works.
by Dave Day 28 Aug '17

28 Aug '17

1 0

XUDY0016 error
by Britney Fox 25 Aug '17

25 Aug '17

Hello, I get this error: [XUDY0016] Documents in path '...' can only be replaced once. Code : XUDY0016 Description : In a replace expression, a target is replaced more than once. Examples : let $a := <x>x</x>/node() return (replace node $a with <a/>, replace node $a with ) Is there a BaseX option to allow multiple "replace" in one query on the same document ?

2 1

Could not reserve enough space for object heap
by Bram Vanroy 25 Aug '17

25 Aug '17

Hi all I'm running into an issue with many databases. I.e. one server instance with millions of databases. When creating all of these, I found that the more databases are included on the instance, the slower further database generation got. For instance, I could see in the logs that in the first < 10.000 databases the creation happened smoothly with around 50ms per file of 1-4kB. However, when having more and more databases for this server instance, things got very slow: for an XML file of 1-4kB the logs show ~600ms. This is terribly slow, as you can imagine. At first I thought something was wrong with my hardware, but I checked on another system and the same issues arises. Then I thought maybe Java is doing something strange, so I figured I'd reboot and see if that cleared some stuff up. But now when I try to launch 'basex' or 'basexserver', I get the following message: Could not reserve enough space for 1433600KB object heap I googled the issue, and it was suggested that I added a JAVA option to my system's variable (I'm on Windows 10 64 bit, BaseX 8.6.4) indicating the memory it could use. I set that to 2048MB. But still the same issue persists. I have contacted the list before, with issues of generating millions of database with the same server instance, and this seems another one related to the problem. I am no expert AT ALL, but isn't it possible there is some sort of micro memory leak that only becomes apparent when creating an amount of databases of this magnitude? If not, other ideas are welcome as well. At least on how to get rid of the Java error mentioned above. Kind regards Bram Vanroy

5 10

BaseX 8.6.6
by Christian Grün 24 Aug '17

24 Aug '17

Hi there, BaseX 8.6.6 is out. Among other things, we fixed two bugs in the map:merge function and where clauses with predicates, so we recommend you to update to the latest version if you work with BaseX 8.6.x. Apart from that, binary data handling was tweaked a lot: The Binary and Conversion Modules will now be much faster if you iterate over single bytes. Have fun everyone, Christian PS: I’ll be offline most of the time until Sep 15. Thanks everyone for keeping the mailing list alive!

1 0

RESTXQ custom response
by Tim Thompson 22 Aug '17

22 Aug '17

Hello, Is it possible to overwrite default HTTP headers in RESTXQ? I am running BaseX in Tomcat 7 and have set the authmethod param set to "Custom." I wanted to define my own WWW-Authenticate header for Digest authentication. I tried something like this: <http:header name="WWW-Authenticate" value="{ ``[Digest realm="BaseX", qop="auth, auth-int", algorithm=MD5, nonce=`{hash:md5(random:uuid())}`]`` }"/> (in a custom response, as specified in http://docs.basex.org/wiki/RESTXQ#Custom_Response). However, the WWW-Authenticate header only returns a value of Custom realm="BaseX" no matter what I do in the custom RESTXQ response. Thanks in advance, Tim -- Tim A. Thompson Discovery Metadata Librarian Yale University Library

2 13

Re: [basex-talk] The BaseX REPLACE command and archives
by Christian Grün 22 Aug '17

22 Aug '17

I have remastered the way how paths of command calls are parsed: Relative path segments (".", "..") will now be correctly resolved as well. I hope you can now specify '/' as replacement path [1]. Cheers, C. [1] http://files.basex.org/releases/latest/ On Tue, Aug 22, 2017 at 4:21 PM, Britney Fox <britfox42(a)hotmail.com> wrote: > i have already tried but I have an error: > > Path '/' is invalid.

1 0

Re: [basex-talk] The BaseX REPLACE command and archives
by Christian Grün 22 Aug '17

22 Aug '17

...even better! Can you specify '/' as path? Am 22.08.2017 4:02 nachm. schrieb "Britney Fox" <britfox42(a)hotmail.com>: Hi, REPLACE command already works correctly with .tar.gz :) It just add a path at the start of the database tree. This for example: REPLACE path "/Users/DB/file.tar.gz" Add path for every directories found in the archive to the database... if path could be the database root or optional it would be perfect :)

1 0

The BaseX REPLACE command and archives
by Britney Fox 22 Aug '17

22 Aug '17

Hi, I would like to use BaseX for a database and the daily updates are .tar.gz files which inlude .xml inside directories. Of course, the directories need to match the ones inside the database. Unfortunately, the BaseX REPLACE command absolutly need a path so it makes the update fail since the path is added. So, is there a solution using BaseX ? Thanks.

2 1

bug in XPath evaluation
by nikos dimitrakas 22 Aug '17

22 Aug '17

Hi! I discovered a strange behavior i BaseX (8.6.5): I evaluate //S/@k/parent::* or //S/@k/.. and it works I evaluate //S[.//@m="V"] and it works But when I evaluate //S/@k/parent::*[.//@m="V"] or //S/@k/..[.//@m="V"] it does not work. There is no error, just no result. At first I thought that there was an error with going to a parent of an attribute node, but that works. Then I thought the problem was with ".//@" in the predicate, but that also works. It appears that it is the combination that confuses the engine. Here is an xml to test the above expressions: <Root> <S k="AAA"> </S> <S k="AA"> </S> <S k="D"> </S> <S k="C"> </S> <S k="AAA"> </S> <S k="AA"> </S> <S k="C"> </S> <S k="AAA"> </S> <S k="AA"> </S> </Root> I hope you can find the problem and fix it in a new version soon. Regards /nikos

3 3

BaseX results window
by Koh Khai Huat 22 Aug '17

22 Aug '17

Hi, I would like to ask, everyone here knows if its possible to turn off XML Namespace attributes in the results window when we query? Any suggestion or solutions are welcomed. Regards Alex Koh

2 1

RESTXQ %input:csv annotation
by Kendall Shaw 21 Aug '17

21 Aug '17

The example produces an error saying it can’t convert string to document node: declare %rest:path("/store.csv") %rest:POST("{$csv}") %input:csv("header=true,encoding=CP1252") function page:store-csv($csv as document-node()) { "Number of rows: " || count($csv/csv/record) }; I can just change the function signature and parse the string with csv:parse, but is the example missing a detail (or am I missing a detail)? I used this to store CSV data in the default format. curl –XPOST –H’Content-Type: text/csv’ –data @some.csv http://example.com/basex/store.csv Kendall

2 3

Keep the comments when parsing HTML?
by Graydon Saunders 21 Aug '17

21 Aug '17

Hello -- So I have a pile of near-XML HTML with semantically significant comments to deal with. (I must have been sinning much more than I realized!) Using BaseX866-20170818.124137, BaseX will parse the content but all the comments go away. This is with passing the "lexical" option on the parser tab where it asks for TagSoup options, which I understand from https://github.com/orbeon/tagsoup/blob/master/trunk/README to pass through comments (and DOCTYPE declarations). How do I parse HTML and keep the comments? Thanks! Graydon

2 4

%input:csv annotation
by Kendall Shaw 21 Aug '17

21 Aug '17

Trying to use the example, in the RESTXQ section of the documentation: declare %rest:path("/store.csv") %rest:POST("{$csv}") %input:csv("header=true,encoding=CP1252") function page:store-csv($csv as document-node()) { "Number of rows: " || count($csv/csv/record) }; Using: curl -i -XPOST –data @test.csv<mailto:data@test.csv> -H’Content-Type: text/csv’ http://blah/basex/store.csv The result is [XPTY0004] Cannot cast xs:string to document-node() I can just invoke csv:parse but, is the example missing a detail? Kendall

1 0

Java 9 support?
by Chris West 19 Aug '17

19 Aug '17

Debian and Ubuntu are looking to move to Java 9 for their next releases, for all possible packages. basex currently fails to build with Java 9 due to some rather boring errors: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project basex: Compilation failure: Compilation failure: [ERROR] /build/basex-8.5.1/basex-core/src/main/java/org/basex/query/func/inspect/Inspect.java:[34,3] reference to Module is ambiguous [ERROR] both class org.basex.query.Module in org.basex.query and class java.lang.Module in java.lang match [ERROR] /build/basex-8.5.1/basex-core/src/main/java/org/basex/gui/GUIInput.java:[253,13] incompatible types: javax.swing.JComboBox<java.lang.String> cannot be converted to javax.swing.JComboBox<java.lang.Object> [ERROR] /build/basex-8.5.1/basex-core/src/main/java/org/basex/query/func/crypto/MyKeySelector.java:[51,44] incompatible types: java.util.List<javax.xml.crypto.XMLStructure> cannot be converted to java.util.List<java.lang.Object> Would there be any interest in fixing these? The website suggests that any version of Java is supported, and Java 9 will be out soon (~31 days). I can have a go at a pull request, but I'd not heard of basex before today, so maybe someone else would like a stab at it. I had a quick poke through the mailing list archives, and can't see any other discussion about this; sorry if it's been brought up before. Chris.

2 1

Implement read lock and write lock
by DK Singh 18 Aug '17

18 Aug '17

Hi All, How can i implement read lock and write lock, any functions to do this Regards Dharmendra Kumar Singh

5 5

Reading JSON
by Giuseppe Celano 18 Aug '17

18 Aug '17

Hi, I have noticed different speeds when running the following functions (from slowest to fastest): parse-json(unparsed-text('example.txt')) json-doc("example.txt") parse-json(file:read-text('example.txt')) similarly for documents on the web: parse-doc('http://example.com/text') parse-json(fetch:text('http://example.com/text')) Does this make sense to you? Is there any recommendation to follow? Despite the runtime speed difference, I admit I love writing one single function (json-doc) to get map conversion. Everything is so immediate :) Best, Giuseppe Universität Leipzig Institute of Computer Science, Digital Humanities Augustusplatz 10 04109 Leipzig Deutschland E-mail: celano(a)informatik.uni-leipzig.de E-mail: giuseppegacelano(a)gmail.com Web site 1: http://www.dh.uni-leipzig.de/wo/team/ Web site 2: https://sites.google.com/site/giuseppegacelano/

2 6

value compare in namespace
by radim-havlicek＠post.cz 18 Aug '17

18 Aug '17

Hello, I want to count failed reports in DB, it means where outcome value of report is "Failed". declare namespace TestResults='urn:IEEE-1671:2009.04:TestResults'; declare namespace Common='urn:IEEE-1671:2010:Common'; let $DocumentNames := (db:list("ASC", "Report")) for $DocumentName in $DocumentNames let $Document := db:open('ASC',$DocumentName) let $FailedTests := $Document/TestResults:TestResults/TestResults:ResultSet/ TestResults:TestGroup/TestResults:TestGroup/TestResults:TestGroup/ TestResults:TestGroup/TestResults:Test/TestResults:Outcome/data(@value) where $FailedTests="Failed" return count($FailedTests) However, it counts all reports, no matter if the value is "Passed" or "Failed". If I change the end to: return $FailedTests, it displays both "Failed" and "Passed" values. If I change it to: where $FailedTests="Test", it displays correctly 0 reports. What am I doing wrong? Thank you in advance for your reply. Radim Havlicek

3 3

Should xml-to-json ignore a "key" attribute of a "map" root element?
by Martin Honnen 18 Aug '17

18 Aug '17

When I try xml-to-json( <map xmlns="http://www.w3.org/2005/xpath-functions" key="root"> <number key="n">2</number> <string key="foo">a</string> <boolean key="checked">true</boolean> </map>, map { 'indent' : true() } ) with BaseX 8.6.5 I get an error "[FOJS0006] Element 'map' must have no key.". Saxon 9.8 however parses that input fine. Looking at https://www.w3.org/TR/xpath-functions-31/#func-xml-to-json there is some text saying "An element node E having a key attribute and/or an escaped-key attribute provided that E would satisfy one of the above conditions if the key and/or escaped-key attributes were removed." which sounds as if a "key" attribute should or could be ignored. Or is BaseX schema-validating the argument and that way ("Otherwise, the processor may attempt to validate the element against the schema, in which case it is treated as valid if and only if the outcome of validation is valid.") the "key" attribute is not allowed? It seems the spec that way allows some confusing lack of interoperability.

2 3

db:replace modified date
by Kendall Shaw 14 Aug '17

14 Aug '17

After debugging a query for a while, I figured out that db:replace isn’t changing the modification date of an XML resource, when the content hasn’t changed. The case is like this: db:replace(‘db’, ‘abc.xml’, doc(‘file:/tmp/abc.xml’)) Is that normal, or do I have an option set that results in that behavior? ADDCACHE is false. Kendall

2 1

Differences in serialization of arrays with JSON vs. adaptive methods
by Joe Wicentowski 13 Aug '17

13 Aug '17

Hi all, First, I'm just back from DH2017, where Clifford Anderson and I taught two workshops on XQuery using BaseX, along with eXist and Saxon. BaseX performed like a champ. We were able to configure the GUI window to show just the query and results windows—perfect when you're projecting the screen in a large room and want everyone to see. Many thanks for such a great teaching tool! (Our materials are at https://github.com/CliffordAnderson/XQuery4Humanists.) Back to the topic of this post, though, I noticed a slight difference between BaseX's serialization of arrays when using JSON vs. adaptive methods: with JSON, the array's items are separated by newlines, whereas with adaptive, the items are separated by spaces. This is interesting since the serialization spec notes that the adaptive method delegates the handling of the "indent" parameter to JSON. Some code to reproduce this is below. I'm curious to know - is there a particular reason for this difference? Thanks, Joe serialization-test.xq ```xquery xquery version "3.1"; declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization"; let $array := ["Cheapside","London","Dean Prior","Devon"] for $method in ("json", "adaptive") let $serialization-parameters := <output:serialization-parameters> <output:method>{$method}</output:method> <output:indent>yes</output:indent> </output:serialization-parameters> return fn:serialize($array, $serialization-parameters) ``` serialization-test_results.txt ```txt [ "Cheapside", "London", "Dean Prior", "Devon" ] ["Cheapside", "London", "Dean Prior", "Devon"] ```

3 11

Shouldn't CHOP = false make xml:space="preserve" the default behavior?
by Ottid Mes 10 Aug '17

10 Aug '17

(Sorry for the noise, but because my previous mail was shown under trimmed content in my Sent Mail, I will resend it without format just in case it was not shown properly to others as well) >From the documentation about the CHOP option I assumed that since xml:space="preserve" sets CHOP = false for that part of the document, that if I set CHOP = false in my configuration file, that the behavior you get when you use xml:space="preserve" would be applied to the whole database (I created the database after setting the option). However the only way I have ever been able to get this behavior, has been to set xml:space="preserve" at the root element. Am I missing something, or is this a bug? How could I get this behavior by default in my databases? I thought this would not warrent a thorough example given the clear conditions which cause the above situation, but I was asked for it anyway, so here it goes: A little bit of context, although it should not matter, I have had this issue for years (at least 4) under Arch Linux. At the time I just assumed I did something wrong and went with xml:space="preserve" workaround everywhere. > uname -a Linux phoenix 4.9.38 #1-NixOS SMP Sat Jul 15 10:17:55 UTC 2017 x86_64 GNU/Linux The default situation: No config file (.basex) > basex BaseX 8.6.4 [Standalone] > create db chop-test Database 'chop-test' created in 123.01 ms. > open chop-test Database 'chop-test' was opened in 1.19 ms. > replace /a <root><a>foo</a> bar</root> 0 resource(s) replaced in 103.68 ms. > xquery /root <root> <a> foo </a>bar</root> Query executed in 106.05 ms. I never use the REPL other than to create and drop databases, so I was a bit suprised that this did not work: > replace /a <root xml:space="preserve"><a>foo</a> bar</root> "a.xml" (Line 1): Open quote is expected for attribute "xml:space" associated with an element type "root". While this does: > replace /a <root xml:space='preserve'><a>foo</a> bar</root> 1 resource(s) replaced in 4.06 ms. > xquery /root <root xml:space="preserve"><a>foo</a> bar</root> Query executed in 0.95 ms. > quit Have fun. The resource with xml:space="preserve" is the behavior I want to have within my database, because all my documents are mixed content. On the wiki (http://docs.basex.org/wiki/Options#CHOP) this is also mentioned: It explicitly states that in my use case I should set CHOP to false: "The flag should be turned off if a document contains mixed content." It also states that setting the xml:space="preserve" attribute is the same as having CHOP = false: "If the xml:space="preserve" attribute is attached to an element, chopping will be turned off for all descendant text nodes." So lets do that: Let us first confirm that the config file is correctly read: > echo 'FOO = 0' > /some/path/.basex > BASEX_JVM='-Dorg.basex.path=/some/path' basex /some/path/.basex: Unknown option 'FOO'. /some/path/.basex: writing new configuration file. Now we set the option CHOP = false in our config: > echo 'CHOP = false' >> /some/path/.basex So lets see what this changes in the basex REPL: > BASEX_JVM='-Dorg.basex.path=/some/path' basex BaseX 8.6.4 [Standalone] > drop db chop-test Database 'chop-test' was dropped. > create db chop-test Database 'chop-test' created in 106.42 ms. > open chop-test Database 'chop-test' was opened in 0.05 ms. > replace /a <root><a>foo</a> bar</root> 0 resource(s) replaced in 39.24 ms. > xquery /root <root> <a> foo </a> bar</root> Query executed in 97.09 ms. > quit Have fun. This is not what I expect, it should have been: > xquery /root <root><a>foo</a> bar</root> And hence my question: Shouldn't CHOP = false make xml:space="preserve" the default behavior? I even tried this: > BASEX_JVM='-Dorg.basex.path=/some/path' basex BaseX 8.6.4 [Standalone] Try 'help' to get more information. > open chop-test Database 'chop-test' was opened in 90.65 ms. > set chop off CHOP: false > replace /a <root><a>foo</a> bar</root> 1 resource(s) replaced in 45.58 ms. > xquery /root <root> <a> foo </a> bar</root> Query executed in 96.78 ms. > quit See you. Am I making some mistake in the above? Is the wiki simply outdated and should this be configured differently? Is having having mixed content in basex so rare that this bug has gone unnoticed for years?

3 5

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk