If you want to achieve concurrently access on your databases, you need to use the client-server architecture.

Which sample did you mean? This one works for sure...

package org.basex.examples.server;

import org.basex.BaseXServer; import org.basex.core.BaseXException; import org.basex.server.ClientSession;

/** * This class demonstrates database access via the client/server architecture. * * @author Workgroup DBIS, University of Konstanz 2005-10, ISC License * @author BaseX Team */ public final class ServerCommands { /** Session reference. */ static ClientSession session;

/** * Runs the example code. * @param args (ignored) command-line arguments * @throws Exception exception */ public static void main(final String[] args) throws Exception {

System.out.println("=== ServerExample ===");

// ------------------------------------------------------------------------ // Start server on default port 1984 BaseXServer server = new BaseXServer();

// ------------------------------------------------------------------------ // Create a client session with host name, port, user name and password System.out.println("\n* Create a client session.");

session = new ClientSession("localhost", 1984, "admin", "admin");

// ------------------------------------------------------------------------ // Create a database System.out.println("\n* Create a database.");

session.execute("CREATE DB input etc/xml/input.xml");

// ------------------------------------------------------------------------ // Run a query System.out.println("\n* Run a query:");

System.out.println(session.execute("XQUERY //li"));

// ------------------------------------------------------------------------ // Faster version: specify an output stream and run a query System.out.println("\n* Run a query (faster):");

session.setOutputStream(System.out); session.execute("XQUERY //li");

// Reset output stream session.setOutputStream(null);

// ------------------------------------------------------------------------ // Run a query System.out.println("\n* Run a buggy query: ");

try { session.execute("XQUERY ///"); } catch(final BaseXException ex) { System.out.println(ex.getMessage()); }

// ------------------------------------------------------------------------ // Drop the database System.out.println("\n* Close and drop the database.");

session.execute("DROP DB input");

// ------------------------------------------------------------------------ // Close the client session System.out.println("\n* Close the client session.");

session.close();

// ------------------------------------------------------------------------ // Stop the server System.out.println("\n* Stop the server.");

server.stop(); } }

Andreas

Am 06.01.11 15:46, schrieb Charles F. Munat:

The latest version.

That's good to know. I haven't changed the query yet because I've been struggling to fix other bugs, but I should be fixing that today.

Do you think that performance would be better running client/server instead? If so, do you know any example code that is up-to-date with the latest version. The sample on the website doesn't work as it's currently written (at least it didn't for me).

Thanks,

Chas.

On 01/06/2011 8:08 AM, Andreas Weiler wrote:

...

Hi Chas,

what BaseX version are you running?

As a first hint, you don't have to open the database, when you are using the doc('database') function in your queries. So you don't have to close it, too.

Kind regards, Andreas

Am 06.01.11 05:20, schrieb Charles F. Munat:

...

So I went live with my embedded Base-X website and that has been a very painful learning experience.

I had been opening the DB on each request and running various queries, but assumed that the process would close it when it exited. That was bad, bad, bad. About an hour after I uploaded the new site (and after I'd gone to bed for a nine hour snooze), the site started serving a 500 error code because too many files were open.

OK, figured that out. So I created a singleton and with a method that opens the database in a try/catch block, and closes it in the finally block. Problem solved.

Sort of.

Apologies for the Scala, but here is the singleton code:

object DB { private val context = new Context() new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context)

def executeQuery(query: String): String = { try { new Open("data").execute(context) new XQuery(query).execute(context) } catch { case _ => new CreateDB("data", "<data/>").execute(context) new XQuery(query).execute(context) } finally { new Close().execute(context) } } }

Try blocks in Scala return a value, so this method returns the output of the passed-in XQuery. Works like a charm.

When a request is received, my servlet (using Circumflex, a Scala web framework) creates a new RequestRouter class. A "get" method is called and passed the URL. Here is that class and method, which shows how I am calling the above DB object:

class Main extends RequestRouter { get(url) = { val query = Rx.getQuery(url) val output = DB.executeQuery(qry)

"<!DOCTYPE html>\n" + output } }

This passes the URL (url) to a method on another singleton (Rx.getQuery) which returns an XQuery based on the URL. That query is passed to DB.executeQuery which stores the output in the immutable variable "output." I then prepend a doctype and serve it as text/html.

This runs great... for about an hour. Then the server stops responding altogether. I've searched the logs, and I found this error:

java.io.IOException: Stream Closed java.io.RandomAccessFile.seek(Native Method) org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) org.basex.data.Data.kind(Data.java:305) org.basex.query.item.DBNode$4.next(DBNode.java:273) org.basex.query.path.IterStep$1.next(IterStep.java:45) org.basex.query.path.AxisPath.iter(AxisPath.java:437) org.basex.query.path.AxisPath.iter(AxisPath.java:406) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.For$1.init(For.java:127) org.basex.query.expr.For$1.next(For.java:92) org.basex.query.expr.FLWR$1.next(FLWR.java:63) org.basex.query.expr.Constr.<init>(Constr.java:53) org.basex.query.expr.CElem.item(CElem.java:112) org.basex.query.expr.CElem.item(CElem.java:1) org.basex.query.expr.CFrag.item(CFrag.java:1) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FLWR$1.next(FLWR.java:67) org.basex.query.iter.Iter.finish(Iter.java:65) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.Func.iter(Func.java:80) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FuncCall.iter(FuncCall.java:76) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.TypeCase.iter(TypeCase.java:95) org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.Constr.<init>(Constr.java:52) org.basex.query.up.Replace.item(Replace.java:50) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.up.Transform.iter(Transform.java:86) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FLWR$1.next(FLWR.java:67) org.basex.query.iter.Iter.finish(Iter.java:65) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.Func.iter(Func.java:80) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FuncCall.iter(FuncCall.java:76) org.basex.query.QueryContext.iter(QueryContext.java:306)

I'm guessing that this is some sort of race condition?

What is the trick to using Base-X in embedded mode? Do I need to be doing some sort of threading here? (I have managed to avoid learning anything about threads for 15 years now. Hate to ruin that record.)

Suggestions? Advice? At this point I'm seriously thinking of just running Base-X as a server and connecting through a connection pool, but I have to figure that out tonight, so if there's an easier answer, I'd sure love to know it.

Thanks again for all your help!

Chas. _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

I would suggest to you, that you just start a basexserver on your server once and create the database there. Then you can connect to it using one of the several language bindings [1]. So just get the programming language of your choice and call the execute method with your command , or write a client in scala? :-)

All clients will be handled by the server so you don't have to learn anything about threads.

We are happy that you have such plans with BaseX and we will help you whenever you need it.

Kind regards, Andreas

[1] http://www.inf.uni-konstanz.de/dbis/basex/bindings

Am 06.01.11 18:40, schrieb Charles F. Munat:

Where did you get that? I was looking at

http://www.inf.uni-konstanz.de/dbis/basex/code/ServerExample

which does not work. The code below looks right. I'll try it. Moving the db access code out of the singleton definitely fixed the threading issue, but I think for performance I'd probably be better off in client/server mode. (Using the embedded database was mostly a learning exercise for me -- never done it before. I am not wedded to it, and not even sure I like it.)

This helps a lot!

I notice that these are all set up to access the server from an application (thus the "main" method). But everything I'm doing is in a servlet. There don't seem to be any examples of best practices for how to set up a connection pool or access the db from within a servlet, which has a very distinctive pattern of request/response with objects being created and then destroyed. I always get confused about what should be in a singleton and what should be created/destroyed on each request.

Do you have any advice on that or links to code examples?

BTW, if I ever get this working and we determine that it's a good practice, I'd be happy to provide Scala examples that you could put on the website for anyone else using Base-X with Scala. Having to convert everything over from Java is a real pain, especially since I'm not that great a Java programmer to begin with, having done much more work in C#, Ruby, and other languages. I'd also be willing to provide a basic framework using Circumflex (a Scala web framework), Maven, and Base-X. I'd even be willing to set up a Maven archetype for Circumflex for it, so people could just generate a Circumflex project with Base-X ready to go. I can talk to the folks at Circumflex about it.

Thanks again for all your kind assistance. I am very pleased with Base-X and am now about to integrate it into another big project and use it as the basis for all my future dynamic web sites -- a dream I've had for more than a decade. XML databases make the most sense on the Web! I'm outputting XML! Why in the world would I want to keep converting it back and forth to and from objects? It makes so much more sense to work in an FP paradigm and simply manipulate the data that is ALWAYS in XML format. That was obvious back in 1999. Finally, there is a product that allows me to very easily accomplish this (and works in Maven, which is the killer for me with eXist).

(Another big issue is that much of the data I work with is tree-structured. Why in the hell am I having to flatten this stuff out to stick it in an RDBMS and then rebuild the tree? Finally, PostgreSQL has recursive queries, which make that a bit easier. But for years I had to do all this work in the business layer. Why, why, why? XML *is* a tree structure. Problem not just solved, but eliminated.)

Chas.

On 01/06/2011 11:55 AM, Andreas Weiler wrote:

...

If you want to achieve concurrently access on your databases, you need to use the client-server architecture.

Which sample did you mean? This one works for sure...

package org.basex.examples.server;

import org.basex.BaseXServer; import org.basex.core.BaseXException; import org.basex.server.ClientSession;

/**

• This class demonstrates database access via the client/server

architecture.

• @author Workgroup DBIS, University of Konstanz 2005-10, ISC License
• @author BaseX Team

*/ public final class ServerCommands { /** Session reference. */ static ClientSession session;

/**

• Runs the example code.
• @param args (ignored) command-line arguments
• @throws Exception exception

*/ public static void main(final String[] args) throws Exception {

System.out.println("=== ServerExample ===");

//

// Start server on default port 1984 BaseXServer server = new BaseXServer();

//

// Create a client session with host name, port, user name and password System.out.println("\n* Create a client session.");

session = new ClientSession("localhost", 1984, "admin", "admin");

//

// Create a database System.out.println("\n* Create a database.");

session.execute("CREATE DB input etc/xml/input.xml");

//

// Run a query System.out.println("\n* Run a query:");

System.out.println(session.execute("XQUERY //li"));

//

// Faster version: specify an output stream and run a query System.out.println("\n* Run a query (faster):");

session.setOutputStream(System.out); session.execute("XQUERY //li");

// Reset output stream session.setOutputStream(null);

//

// Run a query System.out.println("\n* Run a buggy query: ");

try { session.execute("XQUERY ///"); } catch(final BaseXException ex) { System.out.println(ex.getMessage()); }

//

// Drop the database System.out.println("\n* Close and drop the database.");

session.execute("DROP DB input");

//

// Close the client session System.out.println("\n* Close the client session.");

session.close();

//

// Stop the server System.out.println("\n* Stop the server.");

server.stop(); } }

Andreas

Am 06.01.11 15:46, schrieb Charles F. Munat:

...

The latest version.

That's good to know. I haven't changed the query yet because I've been struggling to fix other bugs, but I should be fixing that today.

Do you think that performance would be better running client/server instead? If so, do you know any example code that is up-to-date with the latest version. The sample on the website doesn't work as it's currently written (at least it didn't for me).

Thanks,

Chas.

On 01/06/2011 8:08 AM, Andreas Weiler wrote:

...

Hi Chas,

what BaseX version are you running?

As a first hint, you don't have to open the database, when you are using the doc('database') function in your queries. So you don't have to close it, too.

Kind regards, Andreas

Am 06.01.11 05:20, schrieb Charles F. Munat:

...

So I went live with my embedded Base-X website and that has been a very painful learning experience.

I had been opening the DB on each request and running various queries, but assumed that the process would close it when it exited. That was bad, bad, bad. About an hour after I uploaded the new site (and after I'd gone to bed for a nine hour snooze), the site started serving a 500 error code because too many files were open.

OK, figured that out. So I created a singleton and with a method that opens the database in a try/catch block, and closes it in the finally block. Problem solved.

Sort of.

Apologies for the Scala, but here is the singleton code:

object DB { private val context = new Context() new org.basex.core.cmd.Set("dbpath", "/var/www/db").execute(context)

def executeQuery(query: String): String = { try { new Open("data").execute(context) new XQuery(query).execute(context) } catch { case _ => new CreateDB("data", "<data/>").execute(context) new XQuery(query).execute(context) } finally { new Close().execute(context) } } }

Try blocks in Scala return a value, so this method returns the output of the passed-in XQuery. Works like a charm.

When a request is received, my servlet (using Circumflex, a Scala web framework) creates a new RequestRouter class. A "get" method is called and passed the URL. Here is that class and method, which shows how I am calling the above DB object:

class Main extends RequestRouter { get(url) = { val query = Rx.getQuery(url) val output = DB.executeQuery(qry)

"<!DOCTYPE html>\n" + output } }

This passes the URL (url) to a method on another singleton (Rx.getQuery) which returns an XQuery based on the URL. That query is passed to DB.executeQuery which stores the output in the immutable variable "output." I then prepend a doctype and serve it as text/html.

This runs great... for about an hour. Then the server stops responding altogether. I've searched the logs, and I found this error:

java.io.IOException: Stream Closed java.io.RandomAccessFile.seek(Native Method) org.basex.io.TableDiskAccess.readBlock(TableDiskAccess.java:349) org.basex.io.TableDiskAccess.cursor(TableDiskAccess.java:326) org.basex.io.TableDiskAccess.read1(TableDiskAccess.java:92) org.basex.data.Data.kind(Data.java:305) org.basex.query.item.DBNode$4.next(DBNode.java:273) org.basex.query.path.IterStep$1.next(IterStep.java:45) org.basex.query.path.AxisPath.iter(AxisPath.java:437) org.basex.query.path.AxisPath.iter(AxisPath.java:406) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.For$1.init(For.java:127) org.basex.query.expr.For$1.next(For.java:92) org.basex.query.expr.FLWR$1.next(FLWR.java:63) org.basex.query.expr.Constr.<init>(Constr.java:53) org.basex.query.expr.CElem.item(CElem.java:112) org.basex.query.expr.CElem.item(CElem.java:1) org.basex.query.expr.CFrag.item(CFrag.java:1) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FLWR$1.next(FLWR.java:67) org.basex.query.iter.Iter.finish(Iter.java:65) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.Func.iter(Func.java:80) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FuncCall.iter(FuncCall.java:76) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.TypeCase.iter(TypeCase.java:95) org.basex.query.expr.TypeSwitch.iter(TypeSwitch.java:74) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.Constr.<init>(Constr.java:52) org.basex.query.up.Replace.item(Replace.java:50) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:49) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:298) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:291) org.basex.query.expr.GFLWOR.iter(GFLWOR.java:266) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.up.Transform.iter(Transform.java:86) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FLWR$1.next(FLWR.java:67) org.basex.query.iter.Iter.finish(Iter.java:65) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.expr.Func.iter(Func.java:80) org.basex.query.QueryContext.iter(QueryContext.java:306) org.basex.query.expr.FuncCall.iter(FuncCall.java:76) org.basex.query.QueryContext.iter(QueryContext.java:306)

I'm guessing that this is some sort of race condition?

What is the trick to using Base-X in embedded mode? Do I need to be doing some sort of threading here? (I have managed to avoid learning anything about threads for 15 years now. Hate to ruin that record.)

Suggestions? Advice? At this point I'm seriously thinking of just running Base-X as a server and connecting through a connection pool, but I have to figure that out tonight, so if there's an easier answer, I'd sure love to know it.

Thanks again for all your help!

Chas. _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk