Hi all,
I was thinking about the opportunity of writing a basic linear algebra module in XQUERY (matrix / vector elementary computations, maybe up to cholesky decomposition), since I did not find any at present time. However, there are some little things that seems difficult to me. That could be nice to have opinions of more experimented XQUERY developers before starting trying to write it.
Indeed, XQUERY might provide the two basics needs to write a good linear algebra modulus :
- The first one is template programming. template programming is straightforward with XQUERY. - The second one is template expressions (see http://en.wikipedia.org/wiki/Expression_templates for a definition). I guess that such patterns are not too difficult to implement with XQUERY.
However, the bad points seems to be the following ones. Maybe someone have some insight over these questions ?
- I don't know how to allocate and address directly contiguous memory blocks in XQUERY. That is not a good point for performance issues, as working directly with such memory blocks drastically improves basic operations in copying objects (in linear algebra implementation, this is a key point to performance). - Immutability might result in a big overload in memory? This might not be true, see a recent thread over this topic (XQUERY for noobs). - I don't know how to overload the basic operators (*,+,-) with XQUERY. This may result in quite heavy notations like alg:prod($mat1,alg:minus($mat2,$mat3)) instead of $mat1*($mat2-$mat3).
Notice however that there are alternatives to XQUERY, since some very good linear algebra modules exists, but written in imperative languages. I list some alternatives, however, it might not be a good idea to wrap a functional language like XQUERY to an imperative, not thread safe, external module (this point has been noticed to me by BaseX teams members).
At my knowledge, UBLAS is very performant, flexible, and almost standardized one, but written in C++ http://www.boost.org/doc/libs/1_54_0/libs/numeric/ublas/doc/index.htm. I don't know how to bind C++ to JAVA, but it is certainly possible. However, I am wondering if this could be a good idea.
Eigen is another quite interesting C++ implementation (less performant, but more complete). Indeed, it provides a JAVA wrapper, named JEIGEN. Thus it could suit most of my needs.
Any suggestions welcome
Regards,
Jean-Marc
Hi again,
- I don't know how to allocate and address directly contiguous memory blocks
in XQUERY.
Operations on this level cannot be realized in XQuery. Instead, the query processor is responsible for all memory management. If operations turn out to really be too cpu/memory consuming, Java bindings (and annotations like @Deterministic) may be the better choice.
However, this doesn’t necessarily mean that XQuery is the wrong language for a linear algebra. Even a raytracer has already been implemented in XQuery (from the developers of XMLPrime; it seems to be offline). Some other examples on what can be done with XQuery is found in [1].
- Immutability might result in a big overload in memory? This might not be
true, see a recent thread over this topic (XQUERY for noobs).
That’s difficult to say, as memory consumption largely depends on the specific problem, and the concrete implementation.
- I don't know how to overload the basic operators (*,+,-) with XQUERY. This
may result in quite heavy notations like alg:prod($mat1,alg:minus($mat2,$mat3)) instead of $mat1*($mat2-$mat3).
Function implementations can be assigned to variables, which makes the notation more compact:
$prod($mat1, $minus($mat2, $mat3))
Direct overloading may be supported in a future version of XQuery. Many things are happening; for example, arrays will be introduced with XQuery 3.1. You are welcome to post feature requests in the W3 Bug Tracker [2].
Christian
[1] http://cs.uef.fi/~kilpelai/RDK11/exercises/Ex8Files/xqueryProblems.pdf [2] https://www.w3.org/Bugs/Public/
However, this doesn’t necessarily mean that XQuery is the wrong language for a linear algebra. Even a raytracer has already been implemented in XQuery (from the developers of XMLPrime; it seems to be offline). Some other examples on what can be done with XQuery is found in [1].
Indeed, I think that XQUERY might be very well suited to linear algebra (except for dynamic memory allocation). Most of the linear algebra algorithmic is of "functional" (in the sense recursive then evaluate expression) nature. I bet that a linear algebra module written in xquery could end up with 10% amount lines of code than its C++ or JAVA equivalent.
Direct overloading may be supported in a future version of XQuery. Many things are happening; for example, arrays will be introduced with XQuery 3.1. You are welcome to post feature requests in the W3 Bug Tracker [2].
That's interesting. Specially if further version of xquery support constant-sized arrays, meaning that xquery can allocate contiguous block of memory (I will not have to bind JAVA anymore !). I was trying to look to a site listing the new features of xquery 3.1 but did not succeed. Could you be kind enough to provide a link if you have one in mind ?
Thanks for yours answers,
cheers,
Jean-Marc
2013/11/18 Christian Grün christian.gruen@gmail.com
Hi again,
- I don't know how to allocate and address directly contiguous memory
blocks
in XQUERY.
Operations on this level cannot be realized in XQuery. Instead, the query processor is responsible for all memory management. If operations turn out to really be too cpu/memory consuming, Java bindings (and annotations like @Deterministic) may be the better choice.
However, this doesn’t necessarily mean that XQuery is the wrong language for a linear algebra. Even a raytracer has already been implemented in XQuery (from the developers of XMLPrime; it seems to be offline). Some other examples on what can be done with XQuery is found in [1].
- Immutability might result in a big overload in memory? This might not
be
true, see a recent thread over this topic (XQUERY for noobs).
That’s difficult to say, as memory consumption largely depends on the specific problem, and the concrete implementation.
- I don't know how to overload the basic operators (*,+,-) with XQUERY.
This
may result in quite heavy notations like alg:prod($mat1,alg:minus($mat2,$mat3)) instead of $mat1*($mat2-$mat3).
Function implementations can be assigned to variables, which makes the notation more compact:
$prod($mat1, $minus($mat2, $mat3))Direct overloading may be supported in a future version of XQuery. Many things are happening; for example, arrays will be introduced with XQuery 3.1. You are welcome to post feature requests in the W3 Bug Tracker [2].
Christian
[1] http://cs.uef.fi/~kilpelai/RDK11/exercises/Ex8Files/xqueryProblems.pdf [2] https://www.w3.org/Bugs/Public/
That's interesting. Specially if further version of xquery support constant-sized arrays, meaning that xquery can allocate contiguous block of memory (I will not have to bind JAVA anymore !).
This will depend on the way how XQuery arrays are implemented; a constant-sized array could be one possible option. Depending on the actual data, there may even be cheaper solutions. For example, the XQuery expression (1 to 5) will never be materialized to 5 integers in BaseX; instead, a range value is created that contains the min and max value.
I was trying to look to a site listing the new features of xquery 3.1 but did not succeed. Could you be kind enough to provide a link if you have one in mind ?
It’s too early to go public, because many possible features are still being discussed.
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
jean-marc Mercier