Re: [basex-talk] locking problem

Hi Christian,

I've attempted to reproduce the problem on another machine, but could not get the error to occur on the other machine. Using the latest BaseX my program did not encounter any problems when importing about 44000 XML files. Also, there seems to be no correlation in regard to concurrent reads on the database while the import is running (from oXygen, no problem); some of the errors I saw earlier occurred while the import ran overnight.

On searching the internet for the error message "The requested operation cannot be performed on a file with a user-mapped section open" I found some suggestions that this might be due to another program, possibly antivirus, locking the needed file. The problem might not be with BaseX but something else trying to access files in the basex/data folder, although I'm not sure what that something else could be. If I can narrow it down to a self-contained example I'll let you know.

Thank you, Vincent

-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Sunday, October 19, 2014 6:05 PM To: Lizzi, Vincent Cc: BaseX Subject: Re: [basex-talk] locking problem

Hi Vincent,

Thanks for reporting the bug. It could be related to the one that was reported by Gerald de Jong just recently.

Do you think you could provide us with a self-contained example that allows us to reproduce the error?

Thanks in advance Christian

On Fri, Oct 17, 2014 at 12:56 AM, Lizzi, Vincent Vincent.Lizzi@taylorandfrancis.com wrote:

This is an update, and further problem, on a problem I posted here a few months ago. I've written a program to import XML documents into BaseX in one database, and populate a second database with a small amount of metadata about each document. This works but crashes every so often, and I have to restore the databases from backup and restart the import. To get through a large set of documents I now have the program create a backup (using the CREATE BACKUP command) after every 1000 documents imported.

Since my earlier post, I found that the connection between the client and server was occasionally being dropped. I've switched to opening and closing a connection for each document instead of using one long-running connection. The client and server are running on the same machine.

I'm using the latest version of BaseX (BaseX80-20141013.213520.zip). Here is the error message reported by my program when it crashes. The error occurs on the Query.execute() method of org.basex.examples.api.BaseXClient.java.

Improper use? Potential bug? Your feedback is welcome:

Contact: basex-talk@mailman.uni-konstanz.de

Version: BaseX 8.0 beta a91f5c2

Java: Oracle Corporation, 1.7.0_67

OS: Windows 7, amd64

Stack Trace:

java.nio.channels.OverlappingFileLockException

            at sun.nio.ch.SharedFileLockTable.checkList(Unknown 

Source)

            at sun.nio.ch.SharedFileLockTable.add(Unknown Source)

            at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source)

            at

org.basex.io.random.TableDiskAccess.lock(TableDiskAccess.java:139)

            at

org.basex.io.random.TableDiskAccess.<init>(TableDiskAccess.java:87)

            at org.basex.data.DiskData.init(DiskData.java:119)

            at org.basex.data.DiskData.<init>(DiskData.java:80)

            at org.basex.core.cmd.Open.open(Open.java:70)

            at org.basex.core.cmd.Open.run(Open.java:36)

            at org.basex.core.Command.run(Command.java:360)

            at org.basex.core.Command.execute(Command.java:94)

            at

org.basex.server.ClientListener.run(ClientListener.java:145)

The server instance reports the error with this stack trace:

C:\Programs\basex\basex\bin>basexhttp

BaseX 8.0 beta a91f5c2 [Server]

Server was started (port: 1984)

HTTP Server was started (port: 8984)

java.io.FileNotFoundException: C:\Programs\basex\basex\data\article_xml_meta\doc.basex (The requested operation cannot be performed on a file with a user-mapped section open

)

    at java.io.FileOutputStream.open(Native Method)

    at java.io.FileOutputStream.<init>(Unknown Source)

    at java.io.FileOutputStream.<init>(Unknown Source)

    at org.basex.io.out.DataOutput.<init>(DataOutput.java:47)

    at org.basex.io.out.DataOutput.<init>(DataOutput.java:36)

    at org.basex.index.resource.Docs.write(Docs.java:73)

    at org.basex.index.resource.Resources.write(Resources.java:51)

    at org.basex.data.DiskData.write(DiskData.java:141)

    at org.basex.data.DiskData.flush(DiskData.java:249)

    at org.basex.data.DiskData.finishUpdate(DiskData.java:239)

    at

org.basex.query.up.ContextModifier.apply(ContextModifier.java:124)

    at org.basex.query.up.Updates.apply(Updates.java:129)

    at org.basex.query.QueryContext.iter(QueryContext.java:351)

    at org.basex.query.QueryProcessor.iter(QueryProcessor.java:80)

    at org.basex.server.ServerQuery.execute(ServerQuery.java:128)

    at 

org.basex.server.ClientListener.query(ClientListener.java:492)

    at 

org.basex.server.ClientListener.run(ClientListener.java:109)

Here is the update statement where the error occurs, along with a bit of the surrounding Java.

BaseXClient.Query queryIndex = client.query("declare variable $indexDb external; declare variable $hash external; declare variable $database external; declare variable $path external; let $p := <properties><database>{$database}</database><path>{$path}</path><sha1> {$hash}</sha1><loadedDate>{current-dateTime()}</loadedDate></propertie s> return db:replace('" + indexDatabaseName + "', $path, $p)");

try {

            queryIndex.bind("$indexDb", indexDatabaseName);

            queryIndex.bind("$database", managedDatabaseName);

            queryIndex.bind("$path", path);

            queryIndex.bind("$hash", hash);

            queryIndex.execute();

} finally {

            queryIndex.close();

}

Previously this code used a variable for the first parameter of db:replace() but I changed it to "hard code" the database name into the query after reading about transactions and locking in the wiki. This change doesn't appear to have helped.

To put this in context, the above update is run after the XML document is imported using BaseXClient.replace(). The next thing that happens in the program is an XQuery db:replace to add more properties for the document to the metadata database.

I've added a Thread.sleep(300), to create a small delay between loading each document and this has helped the program to import more documents before crashing. Adjusting the delay doesn't seem to completely stop the crashes though.

I'm not sure if this is a bug or improper use. Any help would be appreciated.

Thanks,

Vincent