Slave scripts and jobs are scripts to me I generate on the fly to hard code the DB name so locking and optimzing know what they deal with.
Some of the code I'm working with right now (now including an error so this kind of lock up does not happen:
declare%private functionwde:eval($queryasxs:string, $bindingsasmap(*)?, $jobNameasxs:string, $dontCheckQueryasxs:boolean) asitem()*{ let$too-many-jobs:=if(count(jobs:list())
=xs:integer(db:system()//parallel)) then
error(xs:QName('wde:too-many-parallel-requests'),'Too many parallel requests!') else(), $query-is-sane:=$dontCheckQueryorwde:query-is-sane($query), $j:=jobs:eval($query, $bindings, map { 'cache': true(), 'id':'wde:'||$jobName||'-'||jobs:current(), 'base-uri': $wde:basePath||'/wde_'||$jobName||'.xq'}), $_:=jobs:wait($j) returnjobs:result($j) };
declare %rest:path("wde/{$dict}/{$index}") %rest:query-param("q","{$q}") %rest:query-param("type","{$type}") %rest:GET functionwde:query-index($dictasxs:string, $indexasxs:string, $qasxs:string*, $typeasxs:string*) { let$query-index-script:=``[ importmodulenamespacewde ="http://acdh.oeaw.ac.at/wde%22at%27rest.xqm'; declarevariable$qexternal; declarevariable$indexexternal; declarevariable$typeexternal; wde:do-query-index(collection("`{$dict}`"), $index, $q, $type) ]`` returnwde:eval($query-index-script, map { 'q': $q, 'index': $index, 'type': $type },'query-index', true()) };
And the last part you might need is something that is faster than a human when it comes to start requests like node with a bunch of superagent/supertest request promises:
Promise.all(requests) .then(docs=>{})
14:24:56.545 127.0.0.1:52592 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.546 127.0.0.1:52593 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.546 127.0.0.1:52595 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.546 127.0.0.1:52594 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.546 127.0.0.1:52596 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.547 127.0.0.1:52597 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.547 127.0.0.1:52598 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.548 127.0.0.1:52599 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.548 127.0.0.1:52600 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.549 127.0.0.1:52601 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.549 127.0.0.1:52602 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.550 127.0.0.1:52603 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.551 127.0.0.1:52604 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.551 127.0.0.1:52605 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.552 127.0.0.1:52606 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.552 127.0.0.1:52607 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.552 127.0.0.1:52608 admin REQUEST [GET] http://localhost:8984/wde/... 14:24:56.554 127.0.0.1:52609 admin REQUEST [GET] http://localhost:8984/wde/...
Best regards Omar Siam
Am 29.11.2017 um 15:42 schrieb Christian Grün:
Hi Omar,
Thanks for reporting your observation.
What do you exactly mean by ”slave jobs”? Do you have some guidance for us to make this reproducible?
Best, Christian
On Wed, Nov 29, 2017 at 3:20 PM, Omar Siam Omar.Siam@oeaw.ac.at wrote:
Hi!
I just observed the following: If I start more RESTXQ requests than the PARALLEL settings allows jobs, which start slave jobs and wait for them, the RESTXQ jobs never finish, hang.
The RESTXQ jobs don't lock anything but start a slave job each which does lock some DB for reading.
It seems to me that the RESTXQ requests are started regardless of the number of parallel jobs already running but the slave jobs never start because they are over the limit of parallel running jobs.
Is this a known situation? Is there some way to stop accepting RESTXQ requests or not starting slave jobs until the parallel limit permits it?
Best regards
Omar Siam