Also, I would think that map:merge would be the typical FP idea where conceptually each step is a copy of the entire map, but it’s implemented as only what has changed being new. If that isn’t the case that would be important to know.

 

Kendall

 

From: <basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Kendall Shaw <kendall.shaw@workday.com>
Date: Monday, November 20, 2017 at 12:13 PM
To: BaseX <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Loading data

 

I hope that the tuple stream of a FLWOR expression does not accumulate heap memory. Is that the case, Christian, or anyone?

 

Kendall

 

From: "E. Wray Johnson" <wray.johnson@gmail.com>
Date: Monday, November 20, 2017 at 10:37 AM
To: Kendall Shaw <kendall.shaw@workday.com>
Cc: BaseX <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Loading data

 

I may need to load hundreds of thousands of items each with dozens of sub-nodes.

I am looking for a solution that will not be at risk of running out of heap memory.  

 

On Mon, Nov 20, 2017 at 12:44 PM, Kendall Shaw <kendall.shaw@workday.com> wrote:

Hi,

 

Just about building maps, in general: If you look at map:merge in http://docs.basex.org/wiki/Map_Module it shows a way to build a map with a for expression. If the question updating maps in a loop was about this, then it is probably preferable to have the result of the for expression be  a map, rather than updating a map, iteratively.

 

Kendall

 

From: <basex-talk-bounces@mailman.uni-konstanz.de> on behalf of "E. Wray Johnson" <wray.johnson@gmail.com>
Date: Sunday, November 19, 2017 at 5:15 AM
To: Michael Seiferle <ms@basex.org>
Cc: BaseX <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Loading data

 

Thanks. I tried doing #2 inside the iteration and that did not work.  I will try it your way and let you know if it works.

Wray Johnson


On Nov 19, 2017, at 6:30 AM, Michael Seiferle <ms@basex.org> wrote:

Dear Wray,

 

I tried to come up with a sketch of what–I think–might help you:

 

Basically it boils down to:

1. Fetch the JSON

2. Convert that JSON to an XQuery item

3. Iterate over each array entry and explicitly construct the XML representation you want

 

 

So in a nutshell, in XQuery 3.1, something like the following:

  => parse-json()  (: Convert to XQuery item representation :)

  => array:for-each(function($map){ (: For each entry in that array, do :)

    element item { (: Construct an XML element named item :)

      $map => map:keys()

      => for-each(function($key){  (: For each key in the map, do: :)

        element { $key } {         (: Return an element named $key :)

          $map($key)               (: …and the value of $map($key) :)

        }

      })

    }

  })

  => array:flatten() (: Converts the array to a sequence :)

 

 

I assume you are using BaseX, so maybe the json:parse() function might be another option, that is based on our own implementation and will create an XML representation right away:

 

As I can only guess what your XQuery code actually looks like I hope this comes somewhat close to what you want.

 

 

Hope this helps ;-)

 

Best from Konstanz

 

Michael

 

Am 19.11.2017 um 06:25 schrieb E. Wray Johnson <wray.johnson@gmail.com>:

 

I want to load it into a database in a different XML format where I can use gml:Point in place of longitude and latitude as separate elements.  However my for loop does not emit individual XML elements for each json object in the loaded array.