Good morning (in Oregon), Eric,
You've brought up SQLite many times as an example of how you want BaseX to behave. The "embeddable" nature of SQLite is not a feature the developer decided to add to an existing project, but the very purpose of its existence. SQLite was designed by a US military contractor specifically to be included in applications on naval ships without requiring the installation of a MySQL or PostgreSQL server.
In contrast, BaseX uses a server/client architecture like MySQL and PostgreSQL. It needs to be installed and configured system-wide and started as a service, so there's no point in allowing users to define multiple different DBPATHs. This isn't an unusual or limiting structure, just different from what you've envisioned for your particular application.
For example, anyone who wants to put WordPress on a web server needs to first install and configure the MySQL server package on the system. WordPress doesn't ship with embedded databases, just SQL statements to build the necessary tables. Similarly, to transfer an existing WordPress website to a new server, the database needs to be dumped as SQL and reconstructed from that source, the same way I reconstruct BaseX databases from the XML documents when I move the project around. And if the server admin wants to run other applications that use MySQL databases, they will all live in the same central service, not from files read from user-specified directories. I've rarely encountered projects that use SQLite, but I often find ones that require MySQL, so this workflow is not a significant problem for users.
In this thread, we've offered suggestions of ways you could structure your application to use BaseX, and how each of your concerns could be addressed. You can either consider them and experiment, or you can find a different tool you could use to fully embed your data and access it without a system-wide service.
-Tamara
On Mon, Feb 21, 2022 at 5:01 PM Eric Levy contact@ericlevy.name wrote:
On Mon, 2022-02-21 at 15:50 -0800, Tamara Marnell wrote:
Assuming every distribution of your project will contain the original files, or work based off of user-supplied files, there is no limitation to putting all databases under one directory. Your instructions for installation only need to include specifying that directory in the .basex file, or in a custom file of definitions if you want to set it programmatically; and running a script that constructs the BaseX database on that computer from the documents. Whenever I move our project to a new machine--which I did frequently during development--our scripts rebuilt our databases of ~42K documents, including full-text and custom indexes, in about five minutes.
I think you're not understanding the use, as your general framing is quite far from what I have tried to explain. I'm not sure it would be helpful for me to address each point you raised in detail.
I feel my explanations are generally clear, especially in light of the context I offered, and I don't want to keep repeating myself. Perhaps the SQLite documentation, especially the section "Appropriate Uses For SQLite" [1], would provide a more robust or accessible representation of embedded database applications than has emerged so far in the discussion.
On Tue, 2022-02-22 at 01:16 +0100, Imsieke, Gerrit, le-tex wrote:
Or to say it more bluntly: Eric, start using BaseX productively before making feature requests that seem out-of-sync with BaseX’s architecture to core developers and to long-time users alike. The fact that no one has raised such a request in the last decade – no one has come forth to say “wow, if I could specify each database’s own directory, I could do things with less friction” – speaks volumes. This is not a request that a BaseX user would make, even if they thought about ways to embed the database engine in an application.
I made a request, and explained the use case, drawing from examples of established design patterns. The request may not be one you see as aligned to the project objectives, which is fine, but the analysis meets me as rather circular.