Hello, Imagine you have data on 100000 test subjects. For every test subject there are 2 data files. For example, one might be a data file describing the actions of a test subject as he went through an obstacle course and the other data file may be time-stamped readings of the subject's heartbeat and breathing rates during the time. These are kept in separate files because the data differs in structure and paradigm (one is event-based the other is a time log).
Queries run against such a data base will very frequently want to match up events in one data file to timestamped biometric data in the other for the same student.
My question is whether there is a way to set up a persistent pairing of these two files so that any query run against the database knows of this natural connection without it having to be explicitly built each time.
For example, it would be nice to create two persistent maps $events and $bio so that the keys of the $events map are document nodes of the data files recording actions students make in the obstacle course and the values are document nodes for associated biometric data, and then $bio is the reverse lookup.
Is it possible in BaseX to create a persistent, user-defined hash that is stored and available without needing to be rebuilt to all queries run against a database?
Thanks, Davd
Hello David,
short answer: No, I do not think this is possible.
There is currently no such automatic linking and I also don't think it is feasible. However, there are a number of opportunities to achieve what you want.
The closest thing I guess would be to use IDREFs and IDs in your documents and associated xsds. However, depending on what you will do exactly you will most likely still have to apply that in some way or another in your queries. Certainly you can not automatically build maps with this.
Maybe you prefer to build from your two databases always a third one after an update, construction the desired XML fragment and combining all the required information. You could then query this combined database.
Also, please be aware that keys to a map have to be a xs:anyAtomicType, i.e. document() is not a valid map key type.
Also, a short explanation why I don't think this kind of functionality would be feasible in BaseX. This sounds very much like a business logic functionality and after all, BaseX is a database and XQuery processor. It also sounds like a very specific requirement and to generalize the concept with all corner cases seems to be a rather complex problem. Hence, I think this problem is better tackled at the business logic layer.
Cheers, Dirk
On 01/05/14 20:25, David Rudel wrote:
Hello, Imagine you have data on 100000 test subjects. For every test subject there are 2 data files. For example, one might be a data file describing the actions of a test subject as he went through an obstacle course and the other data file may be time-stamped readings of the subject's heartbeat and breathing rates during the time. These are kept in separate files because the data differs in structure and paradigm (one is event-based the other is a time log).
Queries run against such a data base will very frequently want to match up events in one data file to timestamped biometric data in the other for the same student.
My question is whether there is a way to set up a persistent pairing of these two files so that any query run against the database knows of this natural connection without it having to be explicitly built each time.
For example, it would be nice to create two persistent maps $events and $bio so that the keys of the $events map are document nodes of the data files recording actions students make in the obstacle course and the values are document nodes for associated biometric data, and then $bio is the reverse lookup.
Is it possible in BaseX to create a persistent, user-defined hash that is stored and available without needing to be rebuilt to all queries run against a database?
Thanks, Davd
Thanks, Dirk, for your response.
Quick question: in my particular case the paths of these pairs have a strongly related structure. For example, for user XXXYYY the path of one file might be /XXXYYY/File_A.xml and the path to the second might be /XXXYYY/File_B.xml .
Perhaps the simplest thing would be to use the Resource Index and regular expressions?
For $user in db:open('my_obstacle_data') Let $biometric := db:open('my_biometric_data')[base-uri(.)=replace(base-uri($user),'A.xml','B.xml')] ... ...
Not sure exactly how I would write that in a way to ensure that the index was actually being used, but I think you see my point: use a replace() command to get the path of paired file and then access it immediately via the Resource Index.
Does that seem workable option?
On Fri, May 2, 2014 at 12:42 PM, Dirk Kirsten dk@basex.org wrote:
Hello David,
short answer: No, I do not think this is possible.
There is currently no such automatic linking and I also don't think it is feasible. However, there are a number of opportunities to achieve what you want.
The closest thing I guess would be to use IDREFs and IDs in your documents and associated xsds. However, depending on what you will do exactly you will most likely still have to apply that in some way or another in your queries. Certainly you can not automatically build maps with this.
Maybe you prefer to build from your two databases always a third one after an update, construction the desired XML fragment and combining all the required information. You could then query this combined database.
Also, please be aware that keys to a map have to be a xs:anyAtomicType, i.e. document() is not a valid map key type.
Also, a short explanation why I don't think this kind of functionality would be feasible in BaseX. This sounds very much like a business logic functionality and after all, BaseX is a database and XQuery processor. It also sounds like a very specific requirement and to generalize the concept with all corner cases seems to be a rather complex problem. Hence, I think this problem is better tackled at the business logic layer.
Cheers, Dirk
On 01/05/14 20:25, David Rudel wrote:
Hello, Imagine you have data on 100000 test subjects. For every test subject there are 2 data files. For example, one might be a data file describing the actions of a test subject as he went through an obstacle course and the other data file may be time-stamped readings of the subject's heartbeat and breathing rates during the time. These are kept in separate files because the data differs in structure and paradigm (one is event-based the other is a time log).
Queries run against such a data base will very frequently want to match up events in one data file to timestamped biometric data in the other for the same student.
My question is whether there is a way to set up a persistent pairing of these two files so that any query run against the database knows of this natural connection without it having to be explicitly built each time.
For example, it would be nice to create two persistent maps $events and $bio so that the keys of the $events map are document nodes of the data files recording actions students make in the obstacle course and the values are document nodes for associated biometric data, and then $bio is the reverse lookup.
Is it possible in BaseX to create a persistent, user-defined hash that is stored and available without needing to be rebuilt to all queries run against a database?
Thanks, Davd
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
basex-talk@mailman.uni-konstanz.de