Hi Omar,
Beforehand: You mentioned that your RESTXQ always cause global locks. Do you have an example for that (see my last mail)?
- Updating jobs don't seem to run in parallel even if they don't work with
and lock the same database. Or do they start much later than I expected?
It should be possible indeed to run updating jobs in parallel. An example would be welcome.
- It seems to me there is an upper limit of jobs that can be queued (about
100?).
There is actually no limit; I have successfully queued more than 10.000 jobs in the past.
Is this perhaps the wrong way to tackle this scaling problem?
Concurrent updates can slow down execution a lot (because of random IO access patterns). However, from a scalability point of view, it can be the right way to go.
Cheers, Christian