Thanks! I was getting hung up trying to do this in a command.

Jonathan

On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie jonathan.robie@gmail.com wrote:

On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith < bdysonsmith@gmail.com> wrote:

...

Hi Jonathan,

for $T in file:list($path-to-your-files) let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(), 'format': 'direct' }) return db:add($your-db, $TSV)

maybe? I'm probably getting something wrong, semantics-wise, w/ `db:add()`, but maybe it's close to get started? Best, Bridger

On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

I have a command file that configures a database, throwing a bunch of sources into various paths.

I have a set of TSV files. I want to do the equivalent of ADD TO, pointing to a directory of TSV files, so that they are parsed and converted to XML and placed in the appropriate path.

How do I do that?

Jonathan

Hi Bridger, hi Jonathan,

When using the graphical user interface, you can learn more about commands by

a) enabling the Info View panel b) performing a GUI operation, such as adding a directory with specific options, and c) checking the contents of the Info View.

Here’s a .bxs solution for adding TSV files:

SET CREATEFILTER *.tsv SET PARSER tsv SET CSVPARSER header=true,separator=tab CREATE DB test ADD csv-dir

As usual, CREATE and ADD can be combined whenever appropriate:

CREATE DB test tsv-dir

Hope this helps, Christian

On Tue, Oct 12, 2021 at 7:54 PM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

Jonathan - apologies for misreading your email! I'm not well-versed in the .bxs COMMAND syntax, but maybe this is closer? I couldn't quite get a handle on using the EXECUTE syntax for this, but I'm under-caffeinated :)

Another piece of uncertainty: can you pass variables from the .bxs to a script you are RUNning? Maybe Christian can help us both understand some things!

Best, Bridger

test-tsv.bxs

CREATE DB test-tsv
RUN /home/bridger/test-tsv.xq
LIST test-tsv

test-tsv.xq

for $T in file:children("/home/bridger/tsvs/")
let $TSV := csv:doc($T, map { "format": "direct", "separator": "tab", "header": true() })
let $TSV-NAME := file:name($T)
return db:add("test-tsv", $TSV, "/" || $TSV-NAME)

note: you could specify a db path here; e.g. db:add("test-tsv", $TSV, "/tsv-files/" || $TSV-NAME)

I've attached my two sample TSV files if that helps.

On Tue, Oct 12, 2021 at 11:45 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

Thanks! I was getting hung up trying to do this in a command.

Jonathan

On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

...

Hi Jonathan,

for $T in file:list($path-to-your-files) let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(), 'format': 'direct' }) return db:add($your-db, $TSV)

maybe? I'm probably getting something wrong, semantics-wise, w/ `db:add()`, but maybe it's close to get started? Best, Bridger

On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

I have a command file that configures a database, throwing a bunch of sources into various paths.

I have a set of TSV files. I want to do the equivalent of ADD TO, pointing to a directory of TSV files, so that they are parsed and converted to XML and placed in the appropriate path.

How do I do that?

Jonathan

Is there a similar incantation for JSON?

Absolutely; it’s *.json and JSONPARSER ;)

On Tue, Oct 12, 2021 at 6:22 PM Christian Grün christian.gruen@gmail.com wrote:

...

Hi Bridger, hi Jonathan,

When using the graphical user interface, you can learn more about commands by

a) enabling the Info View panel b) performing a GUI operation, such as adding a directory with specific options, and c) checking the contents of the Info View.

Here’s a .bxs solution for adding TSV files:

SET CREATEFILTER *.tsv SET PARSER tsv SET CSVPARSER header=true,separator=tab CREATE DB test ADD csv-dir

As usual, CREATE and ADD can be combined whenever appropriate:

CREATE DB test tsv-dir

Hope this helps, Christian

On Tue, Oct 12, 2021 at 7:54 PM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

...

Jonathan - apologies for misreading your email! I'm not well-versed in the .bxs COMMAND syntax, but maybe this is closer? I couldn't quite get a handle on using the EXECUTE syntax for this, but I'm under-caffeinated :)

Another piece of uncertainty: can you pass variables from the .bxs to a script you are RUNning? Maybe Christian can help us both understand some things!

Best, Bridger

test-tsv.bxs

CREATE DB test-tsv
RUN /home/bridger/test-tsv.xq
LIST test-tsv

test-tsv.xq

for $T in file:children("/home/bridger/tsvs/")
let $TSV := csv:doc($T, map { "format": "direct", "separator": "tab", "header": true() })
let $TSV-NAME := file:name($T)
return db:add("test-tsv", $TSV, "/" || $TSV-NAME)

note: you could specify a db path here; e.g. db:add("test-tsv", $TSV, "/tsv-files/" || $TSV-NAME)

I've attached my two sample TSV files if that helps.

On Tue, Oct 12, 2021 at 11:45 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

Thanks! I was getting hung up trying to do this in a command.

Jonathan

On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

...

Hi Jonathan,

for $T in file:list($path-to-your-files) let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(), 'format': 'direct' }) return db:add($your-db, $TSV)

maybe? I'm probably getting something wrong, semantics-wise, w/ `db:add()`, but maybe it's close to get started? Best, Bridger

On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie jonathan.robie@gmail.com wrote: > > I have a command file that configures a database, throwing a bunch of sources into various paths. > > I have a set of TSV files. I want to do the equivalent of ADD TO, pointing to a directory of TSV files, so that they are parsed and converted to XML and placed in the appropriate path. > > How do I do that? > > Jonathan

[ Unasked for comment from the gallery - this series of "parsers" - which hopefully will be further extended - is the reason because XQuery has a revolutionary character, because it turns the wealth of structured information surrounding us into a uniform and homogeneous substrate of information defined with mathematical precision, in which XPath/XQuery acts as a superconductor. ]

Am Mittwoch, 13. Oktober 2021, 20:35:35 MESZ hat Jonathan Robie jonathan.robie@gmail.com Folgendes geschrieben:

Thanks. And I guess this is the documentation I was looking for: https://docs.basex.org/wiki/Parsers

Jonathan On Wed, Oct 13, 2021 at 9:35 AM Christian Grün christian.gruen@gmail.com wrote:

Is there a similar incantation for JSON?

Absolutely; it’s *.json and JSONPARSER ;)

On Tue, Oct 12, 2021 at 6:22 PM Christian Grün christian.gruen@gmail.com wrote:

...

Hi Bridger, hi Jonathan,

When using the graphical user interface, you can learn more about commands by

a) enabling the Info View panel b) performing a GUI operation, such as adding a directory with specific options, and c) checking the contents of the Info View.

Here’s a .bxs solution for adding TSV files:

SET CREATEFILTER *.tsv SET PARSER tsv SET CSVPARSER header=true,separator=tab CREATE DB test ADD csv-dir

As usual, CREATE and ADD can be combined whenever appropriate:

CREATE DB test tsv-dir

Hope this helps, Christian

On Tue, Oct 12, 2021 at 7:54 PM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

...

Jonathan - apologies for misreading your email! I'm not well-versed in the .bxs COMMAND syntax, but maybe this is closer? I couldn't quite get a handle on using the EXECUTE syntax for this, but I'm under-caffeinated :)

Another piece of uncertainty: can you pass variables from the .bxs to a script you are RUNning? Maybe Christian can help us both understand some things!

Best, Bridger

test-tsv.bxs

CREATE DB test-tsv
RUN /home/bridger/test-tsv.xq
LIST test-tsv

test-tsv.xq

for $T in file:children("/home/bridger/tsvs/")
let $TSV := csv:doc($T, map { "format": "direct", "separator": "tab", "header": true() })
let $TSV-NAME := file:name($T)
return db:add("test-tsv", $TSV, "/" || $TSV-NAME)

note: you could specify a db path here; e.g. db:add("test-tsv", $TSV, "/tsv-files/" || $TSV-NAME)

I've attached my two sample TSV files if that helps.

On Tue, Oct 12, 2021 at 11:45 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

Thanks!  I was getting hung up trying to do this in a command.

Jonathan

On Tue, Oct 12, 2021 at 11:44 AM Jonathan Robie jonathan.robie@gmail.com wrote:

...

On Tue, Oct 12, 2021 at 10:33 AM Bridger Dyson-Smith bdysonsmith@gmail.com wrote:

...

Hi Jonathan,

for $T in file:list($path-to-your-files) let $TSV := csv:parse($T, map { 'separator': 'tab', 'header': true(), 'format': 'direct' }) return db:add($your-db, $TSV)

maybe? I'm probably getting something wrong, semantics-wise, w/ `db:add()`, but maybe it's close to get started? Best, Bridger

On Tue, Oct 12, 2021 at 9:31 AM Jonathan Robie jonathan.robie@gmail.com wrote: > > I have a command file that configures a database, throwing a bunch of sources into various paths. > > I have a set of TSV files. I want to do the equivalent of ADD TO, pointing to a directory of TSV files, so that they are parsed and converted to XML and placed in the appropriate path. > > How do I do that? > > Jonathan