Omar,

Sure—if it’s possible to avoid duplication and redundancy. Contact me at my ServiceNow email and we can work out the logistics.

Cheers,

E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow

From: Omar Siam Omar.Siam@oeaw.ac.at Date: Tuesday, May 17, 2022 at 10:31 AM To: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de, Eliot Kimber eliot.kimber@servicenow.com Subject: Re: [basex-talk] Adding another document from XQuery evaluation [External Email]

Hi Eliot,

Am 17.05.2022 um 16:34 schrieb Eliot Kimber:

This needs to be done in an updating function or module as a separate transaction.

If you’re looking for general code on managing sequences of transactions using BaseX’s jobs feature, I created this github project:

https://github.com/ekimbernow/basex-orchestrationhttps://urldefense.com/v3/__https:/github.com/ekimbernow/basex-orchestration__;!!N4vogdjhuJM!CQ8WsLqkcH-RhXb-Cnb03D6sdkTxcRMuN6moe_AS0qE-NhKpEf08qVzRLJO2OJ-bEOrMLuyZ-mCGWqJZiLnL7BFJ1Hc$

This is a distant relative of my utils module https://github.com/acdh-oeaw/vleserver_basex/blob/main/vleserver/util.xqmhttps://urldefense.com/v3/__https:/github.com/acdh-oeaw/vleserver_basex/blob/main/vleserver/util.xqm__;!!N4vogdjhuJM!CQ8WsLqkcH-RhXb-Cnb03D6sdkTxcRMuN6moe_AS0qE-NhKpEf08qVzRLJO2OJ-bEOrMLuyZ-mCGWqJZiLnLuykyC0I$ .

Would you be interested in developing such functions together?

Best regards

--

Mag. Ing. Omar Siam

Austrian Center for Digital Humanities and Cultural Heritage

Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences

Stellvertretende Behindertenvertrauensperson | Deputy representative for disabled persons

Wohllebengasse 12-14, 1040 Wien, Österreich | Vienna, Austria

T: +43 1 51581-7295

omar.siam@oeaw.ac.atmailto:omar.siam@oeaw.ac.at | www.oeaw.ac.at/acdhhttps://urldefense.com/v3/__http:/www.oeaw.ac.at/acdh__;!!N4vogdjhuJM!CQ8WsLqkcH-RhXb-Cnb03D6sdkTxcRMuN6moe_AS0qE-NhKpEf08qVzRLJO2OJ-bEOrMLuyZ-mCGWqJZiLnLEzvLfhc$

Markus,

There are a couple of syntax issues with your code shown below:

1. You have a “,” (comma) at the end of the $let $my_data line. 2. The assignment following the “,” on that line needs to be the return statement

This aspect of XQuery syntax (when to use commas, when to construct explicit sequences with wrapping parens) can be a bit challenging to internalize. If you haven’t already, I’d highly recommend reading _XQuery for Humanists_ from A&M press [2]. I found it to be a really good introductory text that covers all the details with practical examples.

This rework runs for me in the BaseX GUI:

(: Start of query :) declare %updating function local:add_document($items as item()*) as empty-sequence() { db:add("Test_DB", $items, "Test.xml") };

let $my_data as element() := <root><contents>My document</contents></root> return local:add_document($my_data) (: end of query :)

You can also use the BaseX MIXEDUPDATES option to turn off the XQuery update restrictions to allow you to mix updating and non-updating expressions in a single query, although you should only do this if you fully understand the implications, in particular, that updates may not be applied synchronously with non-updating instructions, as explained here at [1].

[1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List [2] https://www.tamupress.com/book/9781623498290/xquery-for-humanists/ _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow

From: Markus Elfring Markus.Elfring@web.de Date: Wednesday, May 18, 2022 at 2:23 AM To: Eliot Kimber eliot.kimber@servicenow.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Adding another document from XQuery evaluation [External Email]

You can add documents to the database without first writing them to the file system[1]:

Thanks for such information.

This needs to be done in an updating function or …

I am looking for further advices also according to XQuery script variants like the following.

declare %updating function local:add_document($items as item()*) as empty-sequence() { let $result := db:add("Test_DB", $items, "Test.xml") return () };

let $my_data as element() := <root><contents>My document</contents></root>, $result := local:add_document($my_data) return ()

Test result: [XUST0001] let: no updating expression allowed.

let $my_data as element() := <root><contents>My document</contents></root>, $add_document := %updating function($items) {db:add("Test_DB", $items, "Test.xml")} return ($add_document($my_data))

Test result: [XPTY0004] Function is updating: $add_document.

https://urldefense.com/v3/__https://docs.basex.org/wiki/XQuery_Update*User-D...https://urldefense.com/v3/__https:/docs.basex.org/wiki/XQuery_Update*User-Defined_Functions__;Iw!!N4vogdjhuJM!Fy9Cp90xL1ZsEl9TqGgJQ3PgRIIfCx5mNMBfC2fXspz6fJUXIGjWforMVJhv_358UEriAcxn-cjW7Xe8aPUq1ukLx0i2Zbw$

How should the affected components be adjusted further according to the development software “BaseX 9.7.2”?

Regards, Markus

Hi Markus,

I don't think the messages are misleading, however the rules here can be hard to follow at first.

My understanding of the updating rules is: All the updating must be done together as the very last thing to happen. This means ONLY in the final 'return' statement of the main expression or defined in the final 'return' statement in functions. These updating function definitions can only be *called *in the final statement of the main expression.

The other XQuery rule that applies to your examples is that: if there is only one expression in main or a function body then the keyword "return" is omitted

So your first example becomes:

declare %updating function local:add_document($items as item()*) as empty-sequence() { db:add("Test_DB", $items, "Test.xml") };

let $my_data as element() := <root><contents>My document</contents></root> return local:add_document($my_data)

In your second example, you can *define *the updating function with let, but *calling *it requires the use of the *updating *keyword as described in the link you give.

let $my_data as element() := <root><contents>My document</contents></root>, $add_document := %updating function($items) {db:add("Test_DB", $items, "Test.xml")} return (*updating *$add_document($my_data))

Regards /Andy

On Wed, 18 May 2022 at 16:05, Markus Elfring Markus.Elfring@web.de wrote:

...

This aspect of XQuery syntax (when to use commas, when to construct

explicit

...

sequences with wrapping parens) can be a bit challenging to internalize.

Thanks for your constructive feedback.

I hope that further clarifications will adjust known construction challenges.

...

This rework runs for me in the BaseX GUI:

I observed also software behaviour for this adjusted script variant which is acceptable finally.

• How much were the error messages (which I mentioned for the discussed use case) misleading?

• Should the desired document addition work also by the application of a corresponding inline function expression? https://docs.basex.org/wiki/XQuery_Update#User-Defined_Functions

Regards, Markus

Hi Markus,

I became curious if this development situation will be improved further.

Who would you expect to improve that? Are you thinking of the developers of the official W3 recommendations or the developers of BaseX?

...

The other XQuery rule that applies to your examples is that: if there is only one expression in main or a function body then the keyword "return" is omitted

Which specification from the standard does support this implementation detail?

Please note that, for the sake of simplicity, you can regard the term "specification" as synonym for "standard" (or "recommendation", as the W3 calls its documents).

The "return" keyword is part of the so-called FLWOR expression [1,2]. As Andy stated, "return" won’t be accepted if you have a single expression. The following expressions are both valid XQuery expressions:

(: FLWOR expression :) let $a := 'abcde' return $a

(: Simple expression :) 'abcde'

The following expression …

"return 'abcde'

… is illegal, as the grammar of the XQuery language does not accept this construct. The same applies to semicolons: They are not defined in XQuery.

Should the shown annotation be replaced by a standard keyword for the function declaration? https://www.w3.org/TR/2011/REC-xquery-update-10-20110317/#id-function-declar...

Both "updating" and "%updating" can be used. The annotation was introduced with XQuery Update 3.0 [3], but BaseX still supports the "updating" keyword as legacy feature.

Is this a tool-specific setting? https://docs.basex.org/wiki/XQuery_Update#User-Defined_Functions

The support for the invocation of dynamic functions is another feature that is only supported since version 3 of the specification. To be strict, the finalized version of the spec woud require the "updating" keyword to be prepended by an additional "invoke" keyword [4]. We should add that in BaseX as well. I believe, though, that there is actually no implementation out there yet that can handle the "invoke" keyword, and that provides the full range of the latest version.

Function items are a rather advanced feature of XQuery. If you are still a beginner, it’s advisable to stick with conventional functions and function declarations (the ones you used in your first example), as they are easier to use and understand.

Best, Christian

[1] https://www.w3.org/TR/xquery-31/#id-flwor-expressions [2] https://docs.basex.org/wiki/XQuery_3.0#Enhanced_FLWOR_Expressions [3] https://www.w3.org/TR/xquery-update-30/ [4] https://www.w3.org/TR/xquery-update-30/#id-dynamic-updating-function-invocat...

I became curious if this development situation will be improved further.

Who would you expect to improve that? Are you thinking of the developers of the official W3 recommendations or the developers of BaseX?

How will the available software evolve further also according to information

like “BaseX offers a complete implementation of the XQuery Update Facility (XQUF).”?

Version 1 is fully supported. We might add additional features of Version 3, depending on requirements of our users and clients.

Would you like to improve error reporting any more?

Maybe. We’d be thankful for:

a) a concrete query example; b) the error you got; and c) the error you’d have expected.

There are a couple of syntax issues with your code shown below:

How often would you dare to pass a detailed FLWOR expression directly as a function parameter instead of specifying an extra variable name or function call?

db:add("Test_DB", <root><contents>Another document example: {$X}</contents></root>, "my_test.xml")

Regards, Markus

Gruss Gott, die Herren! Good day, fans and developers of basex!

After lots of searching online, I turn to you.

I have text loaded into my basex database in the following format

<text id=“Rev”> ... <verse id="Rev.22.14"/> Beati, qui lavant stolas suas in sanguine Agni : ut sit potestas eorum in ligno vitæ, et per portas intrent in civitatem. <verse id="Rev.22.15"/> Foris canes, et venefici, et impudici, et homicidæ, et idolis servientes, et omnis qui amat et facit mendacium. <verse id="Rev.22.16”/> ... </text>

If I wished to query the all nodes between <verse id="Rev.22.14”/> and the following <verse/> tag, I imagined it would be as simple as this:

xquery /text[starts-with(@id, 'Rev')]//verse[@id='Rev.22.1']/following-sibling::node()[following-sibling::verse]

But alas, the query does not stop searching at the following sibling, but continues to the end.

What is the correct syntax of the xpath to get what I need without specifying the particular end tag? I know the following works, but I require the more general query without the id of the ending tag.

xquery /text[starts-with(@id, 'Bible.N')]//verse[@id='Rev.22.1']/following-sibling::node()[following-sibling::verse[@id='Rev.22.2’]].

Thanks for your help!

Mark Bordelon

No, this is not the general solution yet.

Suppose that there is more markup in the text that is between the verse markers.

(And suppose the starting ID is 'Rev.22.15' since this ID is contained in your example.)

If the verse folling the start marker were like this:

Foris canes, et venefici, et<tag/> impudici, et homicidæ, et idolis servientes, et omnis qui amat et facit mendacium.

only this text node would be returned:

Foris canes, et venefici, et

I propose this XQuery instead:

let $text := <text id="Rev"> ... <verse id="Rev.22.14"/> Beati, qui lavant stolas suas in sanguine Agni : ut sit potestas eorum in ligno vitæ, et per portas intrent in civitatem. <verse id="Rev.22.15"/> Foris canes, et venefici, et<tag/> impudici, et homicidæ, et idolis servientes, et omnis qui amat et facit mendacium. <verse id="Rev.22.16"/> ... </text>, $start := $text//verse[@id = 'Rev.22.15'], $end := $start/following-sibling::verse[1] return $start/following-sibling::node()[. << $end]

Result: Foris canes, et venefici, et<tag/> impudici, et homicidæ, et idolis servientes, et omnis qui amat et facit mendacium.

There may be more compact pure XPath expressions, but they will probably be less legible.

Gerrit

On 20.05.2022 08:40, Mark Bordelon wrote:

Perhaps I have just discovered my own answer. Would you all agree this is the best way?

xquery /text[starts-with(@id, 'Bible.N')]//verse[@id='Rev.22.5']/following-sibling::node()[following-sibling::verse][1]

(simply limitie the returning sequence with [1])

...

On May 20, 2022, at 09:34, Mark Bordelon <markcbordelon@yahoo.com mailto:markcbordelon@yahoo.com> wrote:

Gruss Gott, die Herren! Good day, fans and developers of basex!

After lots of searching online, I turn to you.

I have text loaded into my basex database in the following format

<text id=“Rev”> ... <verse id="Rev.22.14"/> Beati, qui lavant stolas suas in sanguine Agni : ut sit potestas eorum in ligno vitæ, et per portas intrent in civitatem. <verse id="Rev.22.15"/> Foris canes, et venefici, et impudici, et homicidæ, et idolis servientes, et omnis qui amat et facit mendacium. <verse id="Rev.22.16”/> ... </text>

If I wished to query the all nodes between <verse id="Rev.22.14”/> and the following <verse/> tag, I imagined it would be as simple as this:

...

xquery /text[starts-with(@id, 'Rev')]//verse[@id='Rev.22.1']/following-sibling::node()[following-sibling::verse]

But alas, the query does not stop searching at the following sibling, but continues to the end.

What is the correct syntax of the xpath to get what I need without specifying the particular end tag? I know the following works, but I require the more general query without the id of the ending tag.

...

xquery /text[starts-with(@id, 'Bible.N')]//verse[@id='Rev.22.1']/following-sibling::node()[following-sibling::verse[@id='Rev.22.2’]].

Thanks for your help!

Mark Bordelon

On Fri, May 20, 2022 at 09:40:49AM +0300, Mark Bordelon scripsit:

Perhaps I have just discovered my own answer. Would you all agree this is the best way?

xquery /text[starts-with(@id, 'Bible.N')]//verse[@id='Rev.22.5']/following-sibling::node()[following-sibling::verse][1]

Generally speaking, any time you're relying on understanding the implicit context of position() in a complex XPath expression, it's going to hurt. For example, following-sibling::node() can come to grief on a processing instruction or a comment.

XPath expressions are powerful but XQuery is more powerful and has some less-brain-melting options.

I have, in query below, wrapped the text of the verse in the verse element because that is much more natural to XML. In a production environment I'd adovcate for using something to pre-wrap all the data, but if you can't do that, that wrapper step ought to be smarter and handle processing instructions and comments creating multiple text nodes in a verse.

let $example as element(text) := <text id="Rev"> <verse id="Rev.22.14"/> Beati, qui lavant stolas suas in sanguine Agni : ut sit potestas eorum in ligno vit=C3=A6, et per portas intrent in civitatem. <verse id="Rev.22.15"/> Foris canes, et venefici, et impudici, et homicid=C3=A6, et idolis servientes, et omnis qui amat et facit mendacium. <verse id="Rev.22.16"/> </text>

(: we want to be able to return a range of verses relative to any particular verse :)

(: create map entries for the information we expect to want about the verses; relative position, id value, and the actual verse :) let $mapRefs as map(*)+ := for $verse at $index in $example/verse let $id as xs:string := $verse/@id/string() return (map:entry($index,$id), map:entry($id,$index), map:entry($id,element {name($verse)} {($verse/@*,normalize-space($verse/following-sibling::text()[1]))}) )

(: pull the sequence of map entries into three merged maps:) (: position to id :) let $posIdMap as map(xs:integer,xs:string) := $mapRefs[map:keys(.) instance of xs:integer] => map:merge() (: id to position :) let $idPosMap as map(xs:string,xs:integer) := $mapRefs[not(map:keys(.) instance of xs:integer) and not(.?* instance of element(verse))] => map:merge() (: id to verse :) let $idVerseMap as map(xs:string,element(verse)) := $mapRefs[.?* instance of element(verse)] => map:merge()

(: which verse do we start with? How many verses after this do we want? :) (: in production these would presumably be external variables/parameters :) let $initial as xs:string := "Rev.22.14" let $offset as xs:integer := 1

(: where is that intial verse in the positional sequence of verses? :) let $initialPos as xs:integer := $idPosMap($initial)

(: return the range of verses in order :) for $found in ($initialPos to $initialPos + $offset) return $posIdMap($found) => $idVerseMap()

Much longer, not itself production ready, but potentially reliable. Clever XPath is rarely reliable.