Hi.
Although I was trying implements of Japanese Lexer to 6.8 beta, I have noticed that the result of a ft:mark() function shifts. 55 line of https://github.com/BaseXdb/basex/blob/master/src/main/java/org/basex/query/u...
final FTLexer lex = new FTLexer(ftopt).sc().init(nd.atom());
FTLexer will generate the tokenizer of the default language instead of an original language, if it does not perform in this way.
How is it? In code which I am correcting, ft:mark() operated normally by carrying out like this.
thanks.
-- Toshio HIRAI