Maps are a derivative of the function type and highly general.

I got informed that the data structure “map” got a clear meaning according to XQuery programming interfaces.

You're in effect asking for examples of using a function with no other information provided.

I came along another clarification desire according to the mentioned standard data structure.

It's just about impossible to answer usefully.

The standard specification just shows examples for topics like “days” and “books” with map constructors based on literal data. How would the XQuery code look like if a detailed FLWOR expression would be applied instead?

The subsequent CSV result output should show finally that keys and associated values were properly determined before.

Regards, Markus

Hi Markus,

The standard specification just shows examples for topics like “days” and “books” with map constructors based on literal data.

If you are looking for functions to construct maps dynamically, maybe map:merge & map:entry are what you are after?

map:merge( for $i at $pos in 1 to 10 return map:entry($i, $i*$i) )

Best Michael

Von: BaseX-Talk basex-talk-bounces@mailman.uni-konstanz.de im Auftrag von Markus Elfring Markus.Elfring@web.de Datum: Mittwoch, 25. Mai 2022 um 17:35 An: Graydon Saunders graydonish@gmail.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Betreff: Re: [basex-talk] Constructing and querying map structures in a dynamic way

Maps are a derivative of the function type and highly general.

I got informed that the data structure “map” got a clear meaning according to XQuery programming interfaces.

You're in effect asking for examples of using a function with no other information provided.

I came along another clarification desire according to the mentioned standard data structure.

It's just about impossible to answer usefully.

The standard specification just shows examples for topics like “days” and “books” with map constructors based on literal data. How would the XQuery code look like if a detailed FLWOR expression would be applied instead?

The subsequent CSV result output should show finally that keys and associated values were properly determined before.

Regards, Markus

Note that the optional options argument in map:merge can change the way it handles duplicate keys. I’ve used this to create maps of normalized document ids from two different collections to de-dup, Where the ids SHOULD be uniq within a collection, but actually aren’t, or are missing.

https://www.w3.org/TR/xpath-functions-31/#func-map-merge https://www.w3.org/TR/xpath-functions-31/#func-map-merge

( Next step was to split the uniq and non-uniq ones to handle separately. )

declare variable $OAI := map:merge( for $d in $ORG_OAI return map:entry( local:normalize( local:unitid($d )), $d ), map{'duplicates':'combine'}); declare variable $VH := map:merge( for $d in $ORG_VH return map:entry( local:normalize( local:unitid($d )), $d ), map{'duplicates':'combine'});

However, after discussion on this list about SQL join equivalents, I discovered I could do something almost equivalent with a for/join, and that full text search tokenization gives me a similar way of normalizing IDs:

for $VH in collection( $R($ORG)('vh'))/ead/archdesc/did/unitid, $OAI in collection( $R($ORG)('oai'))/ead/archdesc/did/unitid[ . contains text {string($VH)}] order by $VH

I think the map version may be a bit faster, but I haven’t actually measured And they are doing things a bit differently, as I’m not removing the internal duplicates, But using the order by to group multiple matches together to make it visible in the report.

I would also note that dealing with messy data, often with hidden encoding errors in documents, Is the reason I was asking in a previous thread about using the result of one script as context for other queries. Sometimes this ends up being a rather exploratory process, so that has come in very handy.

— Steve M.

On May 25, 2022, at 12:13 PM, Hans-Juergen Rennau hrennau@yahoo.de wrote:

Hi Markus,

although I do not quite understand your description (why should I care for the representation of the map I construct?), it may be that one piece of information is useful to you, as it is not obvious from the spec: it's about how to construct a map dynamically, that is,

(1) when the keys are not known beforehand, (2) and/or when the entry values are assigned in "iterations" of a FLWOR expression

(Especially (2) might be what puzzles you.)

(1) and (2) can't be accomplished when using the map constructor expression, like this: basex "map{'x':1, 'y': false()}"

The trick consists of using a combination of the functions map:merge() [1] and map:entry() [2].

So summarize, the code constructing the map entries is the argument of function map:merge().This code may for example be (most often is) a FLWOR expression. In this code, each entry is constructed by a call of function map:entry(). For a tiny example see below.

Kind regards, Hans-Jürgen

PS: Tiny example demonstrating the construction of maps via map:merge() & map:entry().

(: Creates a map with one entry for each element name encountered in the input document. The entry value is the number of elements with that name. As the map keys are not known in advance, map:merge() is used. :) map:merge( for $elem in //* group by $name := $elem/local-name(.) return map:entry($name, count($elem)) )

[1] https://www.w3.org/TR/xpath-functions-31/#func-map-merge https://www.w3.org/TR/xpath-functions-31/#func-map-merge [2] https://www.w3.org/TR/xpath-functions-31/#func-map-entry https://www.w3.org/TR/xpath-functions-31/#func-map-entry

Am Mittwoch, 25. Mai 2022, 16:59:48 MESZ hat Markus Elfring markus.elfring@web.de Folgendes geschrieben:

Hello,

I would like to construct a map structure based on a dynamic selection by the means of a FLWOR expression. https://www.w3.org/TR/2017/REC-xquery-31-20170321/#id-maps https://www.w3.org/TR/2017/REC-xquery-31-20170321/#id-maps

The keys and associated values should be presented as a CSV-like file by a subsequent data processing step.

Can it become easier to find existing XQuery script examples for such an use case?

Regards, Markus

On Fri, May 27, 2022 at 10:50:57AM +0200, Markus Elfring scripsit:

...

This code may for example be (most often is) a FLWOR expression. In this code, each entry is constructed by a call of function map:entry().

Now I stumble on the error message “[XPTY0004] Item expected, empty sequence found.” from my software test according to a special data source.

So either the key or the value isn't in the source data.

How would you determine where unexpected gaps occur?

let $simple as element(bucket) := <bucket> <thingy name="oleander"/> <thingy name="primrose">likes well-drained partial shade</thingy> <thingy/> <thingy>zone 7</thingy> <thingy name="tulip">marsh plant, not a swamp plant</thingy> </bucket>

let $thingMap as map(xs:string,xs:string) := map:merge( for $thing in $simple/descendant::thingy let $key as xs:string? := $thing/@name/string() let $value as xs:string? := $thing/string() where normalize-space($key) and normalize-space($value) return map:entry($key,$value) )

return $thingMap

The map entry can be constructed only when it has meaning. Which means the code has to allow for finding neither the key nor the value, which is why the ? (0 or 1) quantifier on the xs:string types.

(The map-constructing FLOWR expression should also consider that there might be duplicate keys, but one thing at a time.)

If you really want to know where in the data something is missing

let $simple as element(bucket) := <bucket> <thingy name="oleander"/> <thingy name="primrose">likes well-drained partial shade</thingy> <thingy/> <thingy>zone 7</thingy> <thingy name="tulip">marsh plant, not a swamp plant</thingy> </bucket>

let $thingProblems as xs:string* := for $thing in $simple/descendant::thingy let $key as xs:string? := $thing/@name/string() let $value as xs:string? := $thing/string() where not(normalize-space($key)) or not(normalize-space($value)) return $thing ! concat(base-uri(.),path(.))

return $thingProblems

will tell you which nodes in the input can't create map entries.

This approach can be made more robust but this is the general idea.

Gaps? First, to avoid a misunderstanding: whereas map keys must not be the empty sequence, map values may.

Special data sources might contain challenges for further clarification of the desired referential integrity.

Regards, Markus

Well - but are such concerns related to using maps as the output? I do not think so, and therefore I suggest to avoid mixing specific how-to's with general challenges. By now you know the idiom how to construct maps dynamically, as you had already known how to construct node trees dynamically. I agree with Graydon - best to stick to specific questions. General questions are also welcome, but they should be phrased very carefully, in order to enable meaningful answers. A yardstick worth trying is to pose the question in the form "How can I ..."? or "How should one ..."? If you try, for example, to construct a question starting with "How can I" and then introducing referential integrity, you may find it difficult to come up with a question making real sense. Graydon wrote: "Understanding the whole comes from understanding an accumulation of parts and understanding the individual parts comes from making specific mistakes" Yes, but I would like to add something: PRINCIPLES. XQuery is a passionately principled language, and it is of prime importance to grasp the principles - a process that takes time, but one should be aware of it and welcome it. Most important is to understand the "everything-is-an-expression principle", which makes the language composable - expressions are defined in terms of operands which are expressions. Another key principle is the Effectvie Boolean Value, defined by a little mantra mapping value to boolean value. (With a couple of nasty pitfalls!) And, of course, the underlying data model - XDM - with its beautiful stream of consciousness "Information is a value. A value is a sequence of items. An item is a node, or an atom, or a mapping. A node has one of six kinds. An atom is a string plus a schema type. A mapping is a mapping of values to value, can be a map, or an array, or a function item." The firmer your understanding of the principles, the less your need to "try out" - you know. Kind regards,Hans-Jürgen Am Freitag, 27. Mai 2022, 15:38:12 MESZ hat Markus Elfring markus.elfring@web.de Folgendes geschrieben:

Gaps? First, to avoid a misunderstanding: whereas map keys must not be the empty sequence, map values may.

Special data sources might contain challenges for further clarification of the desired referential integrity.

Regards, Markus