On Tue, Jul 24, 2012 at 11:20 AM, Christian Grün <christian.gruen@gmail.com> wrote:
> I am very confused. I never explicitly "open" a database. I use sessions
> that issue a series of queries & commands, such as 'XQUERY', 'CREATE DB',
> and the like. I don't issue any 'OPEN' or 'CLOSE' commands.
As you already guessed, the XQUERY commands is also capable of opening
databases (otherwise, there would be no chance to access data inside
those instances).
> Suppose one client run an XQUERY against a db, which involves (internally)
> one or more 'OPEN' calls, then accesses, then 'CLOSE' commands --- how do
> you synchronize this client with others? Do you have a master lock so that
> the server processes only one query/command at a time?
The pin counter is used to find out how often a particular database is
opened. Our architecture supports multiple read and single write
operations at the same time [1].
> if not, your assumption that you can throw an error in CreateDB.run if the db is pinned is wrong. Instead, you'll need a 'wait_until_unpinned' method [...]
Well, it's tedious to guess what might be the problem as long as we
can't reproduce your particular use case. If you believe you have some
concrete ideas what might be the problem, though, feel free to check
out the sources and give it a try.
I cannot make a reproducible test case, at least not yet, because of the nature of the failure, so I'm trying to help you make sure your design is correct.
If, indeed, you have a multiple-reader, single-writer design, *AND* if you correctly treat 'CREATE DB' as a write operation requiring exclusive access, then your design would be correct, and the failure I'm seeing shouldn't occur in client/server mode. (I can't easily find where you implement this, though, considering that 'pin' doesn't have a flag read-only or read/write - so I'm assuming the multiple-reader, single-writer lock is not done on a per-db basis.) Like I said, given the way we're accessing our db, this is a less likely mode of failure.
My guess is that you permanently fail to close a db after an XQUERY operation, leaving it in the pinned state. I'm not able to quickly find your code where you ensure this - presumably, you'd need some thread-local list of open dbs you'll have to close once a transaction completes or aborts.
- Godmar
ps: on that topic; on your wiki, you write:
"To avoid starvation of any transaction and wrong execution orders the waiting queue works with the FIFO principle ('First-In First-Out'), which states that the first process that arrives at the server will be the first one that will be executed. The FIFO principle cannot be adhered in a group of reading transactions, as they run in different threads and thus can overtake each other."
That claim is wrong - if you allow unrestricted overlapping read transactions, you don't guarantee the absence of starvation. You need to limit overlap of read transactions if you want to be starvation free, else pending writers may never get access.