I wanted to revisit this discussion in the new year.
In the context of a different internal initiative, I’ve been learning more about traditional web sites architecture and implementation (i.e., apache httpd plus statically generated sites using node.js).
This has gotten me to wondering whether the general architecture for a large-user-count, long-query-handling web site is to have one BaseX HTTP server to serve the web site and handle requests and a second server (basexserver) that does all the query work and accepts requests from the HTTP server?
That’s based on the assumption that a basexserver instance can handle a large number of concurrent requests as it must handle its own internal threading etc. This also presumes that a single server instance (one JVM) can use multiple cores on a multi-core server (my production server is an 8-CPU server). A little reading on jetty suggests that it should be able to handle the concurrent load I’m likely to have with no problem.
This would still require a mechanism to manage long-lived HTTP requests from the client, but there are various ways to handle that, including using web sockets to alert a client that a long request has completed and be more sophisticated with HTTP request details, as well as storing query results in a cache location from they are served back to the client.
It seems like my architectural mistake is having a BaseX HTTP server that both serves the interactive web site and makes queries and then trying to scale horizontally by having multiple HTTP servers to which requests are delegated by the primary server.
By having a single baseserver instance handling all queries, it can manage read and write locking appropriately.
Is this an appropriate architecture?
Thanks,
Eliot
_____________________________________________
Eliot Kimber
Sr. Staff Content Engineer
O: 512 554 9368
servicenow
LinkedIn | X | YouTube | Instagram