Re: [basex-talk] Help with a Query/Performance

Hi Tom,

I think that trying to copy/modify a huge tree is definitely the bottleneck here. Why don’t you copy only your third Message element and then reconstruct the wrapping Container with ContainerMetaData?

Since the wanted result is a transformation, perhaps a typeswitch expression might be an alternative, if there is something that stops you from reconstructing.

Daniel

Von: Tom Rauchenwald (UNIFITS) tom.rauchenwald@unifits.com Gesendet: Montag, 20. Januar 2020 10:01 An: basex-talk@mailman.uni-konstanz.de Betreff: [basex-talk] Help with a Query/Performance

Hi list,

I'm struggling with a query.

We have XML documents with a structure similar to this:

<Container> <ContainerMetaData1>FOO</ContainerMetaData1> <ContainerMetaData2>FOO</ContainerMetaData2> <MessageA> <MessageAMetaData> <MessageMetaData1>FOO</MessageMetaData1> <MessageMetaData2>FOO</MessageMetaData2> </MessageAMetaData> <MessageADetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageADetail> <MessageADetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageADetail> </MessageA> <MessageB> <MessageBMetaData> <MessageMetaData1>FOO</MessageMetaData1> <MessageMetaData2>FOO</MessageMetaData2> </MessageBMetaData> <MessageBDetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageBDetail> </MessageB> <MessageC> <MessageCMetaData> <MessageMetaData1>FOO</MessageMetaData1> <MessageMetaData2>FOO</MessageMetaData2> </MessageCMetaData> <MessageCDetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageCDetail> </MessageC> </Container>

Messages are bundled in a container (up to n times for each message), and each message has details (also up to n times). Container, Message contain data that is the same for all details (it's basically a grouping). I'd like to retrieve a Detail with all corresponding data associated with it, so basically a MessageADetail, MessageA (without all the other MessageADetails), Container (without all the other Messages). I know the position of the message (i.e., I know that I want the second MessageA for example), and I know the position of the Detail (i.e., I know that I want the 3rd Detail). The use case is to show the detail in context in a UI.

The query to do this I came up with is (here I want to get the 2nd detail from the third MessageA):

let $fh := (copy $x := /*:Container modify ( delete node $x/*:MessageA[position() != 3] , delete node $x/*:MessageA[3]/*:MessageADetail[position() != 2] , delete node $x/*:MessageB , delete node $x/*:MessageC ) return $x) return $fh

This works well for small documents. For large documents it can take a couple of seconds to evaluate the query (our real-life documents do have more data/elements in Details and Message). I'm wondering if there's a better/more efficient way to do this. I tried formulating a query that doesn't do deletes, but I couldn't come up with a solution that performs better and is correct.

Any pointers would be very much appreciated.

Here's a function to generate sufficiently large test data:

declare function local:sample($numberOfMessages, $numberOfDetails) { <Container> <ContainerMetaData1>FOO</ContainerMetaData1> <ContainerMetaData2>FOO</ContainerMetaData2> {for $i in 1 to $numberOfMessages return <MessageA> <MessageAMetaData> <MessageMetaData1>FOO {$i}</MessageMetaData1> <MessageMetaData2>FOO {$i}</MessageMetaData2> </MessageAMetaData> {for $j in 1 to $numberOfDetails return <MessageADetail> <DetailData1>FOO {$j}</DetailData1> <DetailData2>FOO {$j}</DetailData2> </MessageADetail> } </MessageA> } <MessageB> <MessageBMetaData> <MessageMetaData1>FOO</MessageMetaData1> <MessageMetaData2>FOO</MessageMetaData2> </MessageBMetaData> <MessageBDetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageBDetail> </MessageB> <MessageC> <MessageCMetaData> <MessageMetaData1>FOO</MessageMetaData1> <MessageMetaData2>FOO</MessageMetaData2> </MessageCMetaData> <MessageCDetail> <DetailData1>FOO</DetailData1> <DetailData2>FOO</DetailData2> </MessageCDetail> </MessageC> </Container> };

db:create('tr-test', local:sample(20, 100000), 'test.xml')

Thanks, Tom Rauchenwald