Thanks to Graydon, Tamara, and Christian for responding!
I figured out a pretty fast way to exploit the infrastructure I had built (the files allocated out into many databases and a single index database generated from the databases).
Here is a sample record from my index database:
*<entry> <dbname>pmed_updates_b</dbname> <pmid>34239076</pmid> <version>1</version> <path>pubmed22n1145.xml</path> <date_revised>2022-01-09</date_revised></entry>*
As it happens, there are eight versions of this record scattered across 7 of the component databases and located in 8 input files (two of the input files were allocated to one of the databases). Each of these instances has an entry in the index database.
My approach has four steps:
1. retrieve all entries from the index database that have the desired PMID; 2. convert the sequence of XML entries into a sequence of maps with the same data, ordering by filename descending, so that the most recent file is the first element of the sequence; 3. take the first item/map of the sequence; 4. look up all occurrences of records with that PMID in the database specified in the first item and call *db:path()* on each item and compare it to the filename specified in the most recent record; the record whose *db:path()* matches the item/map taken in step three is the most recent version of the record with that PMID.
Files are allocated by modulo to the different databases, so it is conceivable that a database will have more than one record with a given PMID, hence the necessity of comparing each record's path with the one given in the map from step three to determine which is the most recent.
Given the above PMID (for which there are eight versions of the record, as noted above) it took less than half a second to retrieve the most recent instance of that record out of over 35 million records.
I can post the XQuery if anyone wants to see it. It would take longer to document how I build the content & index databases, and I still have to work out the best way to keep it all up to date.
All the best, Chuck -- Sr Systems Analyst University of Texas M.D. Anderson Cancer Center
On Thu, May 19, 2022 at 12:39 AM Christian Grün christian.gruen@gmail.com wrote:
Hi Charles,
Is there a way in XQuery I can identify a record in this database not
only by its PubMed ID but also limit the result set by source document name?
If the source document name equals the document path in the database, you can supply it as second argument to db:open [1]:
db:open($db, $path)
You could combine this query with an id check:
db:open($db, $path)//...[pmid = $pmid]
I have 26 such databases, each of which contains 43-44 source files,
each of which contains 30000 records. I have built one index database that, for each record, includes PubMed ID, source filename, and database name. For a given PMID I can very quickly retrieve the db name and the file name, but I need to search the db not only by PMID but also by source filename in order to select the most recent version of a record.
If all document versions occur in the same database, and if the number of versions is limited, a query could look as follows:
let $path := '...' let $pmid := '...' let $db := (: ...retrieved from index database :) let db:open($db, $path)
Hope this helps, Christian