Hi Jeremy,
The most common place for threads to be blocking is at org.basex.io.random.TableDiskAccess.read*. The threads are waiting for other threads to finish, since these method calls are synchronized. Is it possible to concurrently read from disk?
Concurrent reads are indeed supported, but the most lower-level operations need to be synchronized in order to avoid two operations changing the file offset pointer at the same time. It’s interesting to hear that this could be a potential bottleneck for your use case, as it hasn’t been of importance in our benchmarks and profilings so far.
Another place threads appear to be blocking is while waiting for the lock to call org.basex.http.RestXqModules.cache (another synchronized method). It seems like this should only be called once to parse the modules, and then all requests afterwards should be retrieved from the cache, but from the thread dumps it appears threads are still re-parsing the xquery modules. Is this expected behavior?
With each RESTXQ call, the restxq directory will be parsed for changes. If XQuery Modules are modified or added, they will be parsed before the actual request is answered. – Usually, this check shouldn’t represent any bottleneck, but we could think about an option that disallows changes to the RESTXQ directory while the server is running. – Do you have many files in your RESTXQ directory?
Hope this helps, Christian