Hello,
Using the store module, is it possible to read and merge multiple custom stores into main memory, or to set a particular store as context for a query? Or, do I need to serialize the stores, e.g., as JSON and then create databases for them?
Thanks in advance, Tim
Hi Tim,
Using the store module, is it possible to read and merge multiple custom stores into main memory
We could add an argument (or options) to file:read to allow entries to be added to an existing main-memory store.
set a particular store as context for a query?
The recommended approach is to invoke store:read before any other operation in the query. Please note, however, that the store will be globally available. If a second query runs on the same server and opens another store, it will change the behavior of the first query. For such use cases, it’s better to use databases.
Or, do I need to serialize the stores, e.g., as JSON and then create databases for them?
You could have a look at the new db:get-value and db:put-value functions [1]: These functions have particularly been designed to organize XQuery items in databases, making casts to JSON or XML obsolete.
Hope this helps, Christian
[1] https://docs.basex.org/wiki/Database_Module#db:get-value
That's perfect--I knew I was missing something :)
Are db:get-value and db:put-value side-effecting as well?
Thanks, Tim
You'll be safe: Updates with db:put-value will be managed via the Pending Update List [1], and future queries that operate on the same database will be queued until the update is finished.
[1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
Tim Thompson timathom@gmail.com schrieb am Di., 15. Nov. 2022, 18:50:
That's perfect--I knew I was missing something :)
Are db:get-value and db:put-value side-effecting as well?
Thanks, Tim
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( christian.gruen@gmail.com) escribió:
Hi Tim,
Using the store module, is it possible to read and merge multiple
custom stores into main memory
We could add an argument (or options) to file:read to allow entries to be added to an existing main-memory store.
set a particular store as context for a query?
The recommended approach is to invoke store:read before any other operation in the query. Please note, however, that the store will be globally available. If a second query runs on the same server and opens another store, it will change the behavior of the first query. For such use cases, it’s better to use databases.
Or, do I need to serialize the stores, e.g., as JSON and then create
databases for them?
You could have a look at the new db:get-value and db:put-value functions [1]: These functions have particularly been designed to organize XQuery items in databases, making casts to JSON or XML obsolete.
Hope this helps, Christian
[1] https://docs.basex.org/wiki/Database_Module#db:get-value
Thanks! I just tried to dump the contents of the store to a database value:
db:put-value( "entries", array { store:keys() ! store:get(.) (: sequence of arrays :) }, "x" )
But when I query with db:get-value("entries", "x"), I get a "Not Expected" error. Should this kind of operation be possible?
Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.3 beta Java: Azul Systems, Inc., 11.0.16 OS: Mac OS X, aarch64 Stack Trace: java.lang.RuntimeException: Not Expected. at org.basex.util.Util.notExpected(Util.java:54) at org.basex.query.value.type.AtomType.read(AtomType.java:983) at org.basex.core.Store.read(Store.java:315) at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:48) at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.iter(QueryContext.java:354) at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.cache(QueryContext.java:598) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) at org.basex.core.cmd.AQuery.run(AQuery.java:82) at org.basex.core.Command.run(Command.java:233) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.execute(GUI.java:415) at org.basex.gui.GUI.lambda$5(GUI.java:360) at java.base/java.lang.Thread.run(Thread.java:829)
Wow. I’ll look into that soon. Do you think you can provide me with a little test case (e.g., a BaseX command script) to reproduce this?
Tim Thompson timathom@gmail.com schrieb am Di., 15. Nov. 2022, 19:16:
Thanks! I just tried to dump the contents of the store to a database value:
db:put-value( "entries", array { store:keys() ! store:get(.) (: sequence of arrays :) }, "x" )
But when I query with db:get-value("entries", "x"), I get a "Not Expected" error. Should this kind of operation be possible?
Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.3 beta Java: Azul Systems, Inc., 11.0.16 OS: Mac OS X, aarch64 Stack Trace: java.lang.RuntimeException: Not Expected. at org.basex.util.Util.notExpected(Util.java:54) at org.basex.query.value.type.AtomType.read(AtomType.java:983) at org.basex.core.Store.read(Store.java:315) at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:48) at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.iter(QueryContext.java:354) at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.cache(QueryContext.java:598) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) at org.basex.core.cmd.AQuery.run(AQuery.java:82) at org.basex.core.Command.run(Command.java:233) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.execute(GUI.java:415) at org.basex.gui.GUI.lambda$5(GUI.java:360) at java.base/java.lang.Thread.run(Thread.java:829)
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 13:07, Christian Grün ( christian.gruen@gmail.com) escribió:
You'll be safe: Updates with db:put-value will be managed via the Pending Update List [1], and future queries that operate on the same database will be queued until the update is finished.
[1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
Tim Thompson timathom@gmail.com schrieb am Di., 15. Nov. 2022, 18:50:
That's perfect--I knew I was missing something :)
Are db:get-value and db:put-value side-effecting as well?
Thanks, Tim
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( christian.gruen@gmail.com) escribió:
Hi Tim,
Using the store module, is it possible to read and merge multiple
custom stores into main memory
We could add an argument (or options) to file:read to allow entries to be added to an existing main-memory store.
set a particular store as context for a query?
The recommended approach is to invoke store:read before any other operation in the query. Please note, however, that the store will be globally available. If a second query runs on the same server and opens another store, it will change the behavior of the first query. For such use cases, it’s better to use databases.
Or, do I need to serialize the stores, e.g., as JSON and then create
databases for them?
You could have a look at the new db:get-value and db:put-value functions [1]: These functions have particularly been designed to organize XQuery items in databases, making casts to JSON or XML obsolete.
Hope this helps, Christian
[1] https://docs.basex.org/wiki/Database_Module#db:get-value
Sure, attached are two command scripts: one that fails (when dumping store contents) and one that passes (when adding a value directly).
Thanks, Tim
…thanks; a new snapshot is available [1]. Best, Christian
[1] https://files.basex.org/releases/latest/
On Tue, Nov 15, 2022 at 7:40 PM Tim Thompson timathom@gmail.com wrote:
Sure, attached are two command scripts: one that fails (when dumping store contents) and one that passes (when adding a value directly).
Thanks, Tim
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 13:21, Christian Grün ( christian.gruen@gmail.com) escribió:
Wow. I’ll look into that soon. Do you think you can provide me with a little test case (e.g., a BaseX command script) to reproduce this?
Tim Thompson timathom@gmail.com schrieb am Di., 15. Nov. 2022, 19:16:
Thanks! I just tried to dump the contents of the store to a database value:
db:put-value( "entries", array { store:keys() ! store:get(.) (: sequence of arrays :) }, "x" )
But when I query with db:get-value("entries", "x"), I get a "Not Expected" error. Should this kind of operation be possible?
Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.3 beta Java: Azul Systems, Inc., 11.0.16 OS: Mac OS X, aarch64 Stack Trace: java.lang.RuntimeException: Not Expected. at org.basex.util.Util.notExpected(Util.java:54) at org.basex.query.value.type.AtomType.read(AtomType.java:983) at org.basex.core.Store.read(Store.java:315) at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:48) at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.iter(QueryContext.java:354) at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.cache(QueryContext.java:598) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) at org.basex.core.cmd.AQuery.run(AQuery.java:82) at org.basex.core.Command.run(Command.java:233) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.execute(GUI.java:415) at org.basex.gui.GUI.lambda$5(GUI.java:360) at java.base/java.lang.Thread.run(Thread.java:829)
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 13:07, Christian Grün ( christian.gruen@gmail.com) escribió:
You'll be safe: Updates with db:put-value will be managed via the Pending Update List [1], and future queries that operate on the same database will be queued until the update is finished.
[1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
Tim Thompson timathom@gmail.com schrieb am Di., 15. Nov. 2022, 18:50:
That's perfect--I knew I was missing something :)
Are db:get-value and db:put-value side-effecting as well?
Thanks, Tim
-- Tim A. Thompson (he, him) Librarian for Applied Metadata Research www.linkedin.com/in/timathompson timothy.thompson@yale.edu
El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( christian.gruen@gmail.com) escribió:
Hi Tim,
> Using the store module, is it possible to read and merge multiple custom stores into main memory
We could add an argument (or options) to file:read to allow entries to be added to an existing main-memory store.
> set a particular store as context for a query?
The recommended approach is to invoke store:read before any other operation in the query. Please note, however, that the store will be globally available. If a second query runs on the same server and opens another store, it will change the behavior of the first query. For such use cases, it’s better to use databases.
> Or, do I need to serialize the stores, e.g., as JSON and then create databases for them?
You could have a look at the new db:get-value and db:put-value functions [1]: These functions have particularly been designed to organize XQuery items in databases, making casts to JSON or XML obsolete.
Hope this helps, Christian
[1] https://docs.basex.org/wiki/Database_Module#db:get-value
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Tim Thompson