[…] is gone.

Good to hear.

Overall, it was a such breaking change, a big scare! IMHO, it is much better and cleaner to stay away from the environment variable.

It would certainly be nice to be able to set this property differently, but the JDK does not offer a cleaner way to do so (suggestions are welcome, though). The temporary solution is to only assign the property if it the corresponding fn:doc option is actually supplied in the function call. In the long term, the fn:doc function signature may be subject to further changes until XQuery 4 is finalized.

It will still be interesting to understand which specific XML parser was used in your setup, and what would be the way to set the property for this parser accordingly.