Hi Alex,
I decided to add another function signature to the convert:binary-to-string function [1]: If you set the third parameter to true(), invalid XML characters will automatically be replaced with \uFFFD (the so-called Unicode replacement character):
declare %rest:path("/upload") %rest:POST("{$body}") function local:store-csv($body as xs:base64Binary) { let $string := convert:binary-to-string($input, 'CP1251', true()) return csv:parse($string) };
A new stable snapshot is available [2]; the final version of BaseX 8.5 will be released in June or July.
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Conversion_Module#convert:binary-to-string [2] http://files.basex.org/releases/latest/
On Sun, Jun 12, 2016 at 11:06 PM, Alexander Shpack shadowkin@gmail.com wrote:
Oh.. right. I forgot about it
On Mon, Jun 13, 2016 at 12:02 AM, Christian Grün christian.gruen@gmail.com wrote:
This line tells you the difference:
let $content := file:read-text($fName, "cp1251")
You are reading the file contents as text (i.e., string). You could also use file:read-binary. Using the browser, you can also upload images or other binary contents.
let $csv := csv:parse($content, map{ 'separator': ';', 'header': false() } ) return $csv
What the difference between post data and raw file?
On Sun, Jun 12, 2016 at 11:45 PM, Alexander Shpack shadowkin@gmail.com wrote:
Did you always upload the same file?
Yeah
As the error indicates, it seems that your CSV input seems to contain characters that are not valid in XML. There are various ways to tackle this; one looks as follows:
(: interpret client data as Base64 :) declare %rest:path("/upload") %rest:POST("{$body}") function local:store-csv($body as xs:base64Binary) { (: replace invalid characters with a question mark :) let $input := bin:from-octets( bin:to-octets($body) ! (if(. >= 32 or . = (9, 10, 13)) then . else 63) ) (: convert to XQuery Unicode string; convert to XML :) let $string := bin:decode-string($input, 'CP1251') return csv:parse($string) };
There may be easier solutions as well (I’ll give you an update once I remember them ;).
Strange solution... Ok, I'll wait for a better one.
-- s0rr0w
-- s0rr0w
-- s0rr0w