Hi Christian,

IMO, it is just the number of requests. I attached the .log file.

-----Original Message----- From: Christian Grün christian.gruen@gmail.com Sent: Monday, April 3, 2023 10:32 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

Hi Yitzhak,

have you checked the resulting log files in the data/.logs directory? Are there specific requests that take too much time, or is it the plain number of incoming requests that eventually slows down the system?

Best, Christian

On Mon, Apr 3, 2023 at 4:29 PM ykhabins@bellsouth.net wrote:

Hello,

We are using BaseX 10.5 via its HTTP service in a corporate environment.

We have an automated Qualys Agent that does a vulnerability scan of that server with the BaseX.

Qualys Agent scan process includes web sites related tests such as Cross-Site Scripting, SQL Injection, etc.

The rapid nature of the Qualys Agent requests effectively gives us a DoS attack on the eclipse.jetty.server.

It cannot process so many requests and goes down.

In the meantime, our solution is to restart BaseX HTTP service manually via basexhttp.bat.

Question: is it possible to somehow configure the eclipse.jetty.server so it will be able to sustain the Qualys Agent vulnerability scan?

Regards, Yitzhak Khabinsky

...

"You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload?"

It doesn't accept any further requests. Just launching the basexhttp.bat revives it.

...

"Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?"

We never tried to disable anything. The Qualys Agent runs once every two weeks on a schedule. So, it is not easy to run on demand for testing.

...

" Which BaseX services are enabled in your web.xml?"

We never modified anything in the web.xml. Please see it below.

<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://www.oracle.com/webfolder/technetwork/jsc/xml/ns/javaee/web-app_4_0.xsd" version="4.0">

<display-name>BaseX: The XML Database and XQuery Processor</display-name> <description>HTTP Services</description>

<!-- A BaseX option can be overwritten by prefixing the key with "org.basex." and specifying it in <context-param/> elements, as shown below. Check out https://docs.basex.org/wiki/Options for a list of all options.

<context-param> <param-name>org.basex.restxqpath</param-name> <param-value>.</param-value> </context-param> <context-param> <param-name>org.basex.dbpath</param-name> <param-value>WEB-INF/data</param-value> </context-param> <context-param> <param-name>org.basex.repopath</param-name> <param-value>WEB-INF/repo</param-value> </context-param>

<context-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </context-param> <context-param> <param-name>org.basex.authmethod</param-name> <param-value>Digest</param-value> </context-param>

<context-param> <param-name>org.basex.httplocal</param-name> <param-value>true</param-value> </context-param> <context-param> <param-name>org.basex.timeout</param-name> <param-value>5</param-value> </context-param> <context-param> <param-name>org.basex.log</param-name> <param-value>false</param-value> </context-param> -->

<!-- Global session and servlet listener --> <listener> <listener-class>org.basex.http.SessionListener</listener-class> </listener> <listener> <listener-class>org.basex.http.ServletListener</listener-class> </listener>

<!-- CORS in Jetty: Access-Control-Allow-Origin: * <filter> <filter-name>cross-origin</filter-name> <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class> <init-param> <param-name>allowedOrigins</param-name> <param-value>*</param-value> </init-param> </filter> <filter-mapping> <filter-name>cross-origin</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> -->

<!-- RESTXQ Service (can be disabled by removing this entry) --> <servlet> <servlet-name>RESTXQ</servlet-name> <servlet-class>org.basex.http.restxq.RestXqServlet</servlet-class> <init-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>RESTXQ</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping>

<!-- WebSocket Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebSocket</servlet-name> <servlet-class>org.basex.http.ws.WsServlet</servlet-class> <!-- Limits of the WebSocket connection <init-param> <param-name>maxIdleTime</param-name> <param-value>100000</param-value> </init-param> <init-param> <param-name>maxTextMessageSize</param-name> <param-value>3000</param-value> </init-param> <init-param> <param-name>maxBinaryMessageSize </param-name> <param-value>3000</param-value> </init-param> --> </servlet> <servlet-mapping> <servlet-name>WebSocket</servlet-name> <url-pattern>/ws/*</url-pattern> </servlet-mapping>

<!-- REST Service (can be disabled by removing this entry) --> <servlet> <servlet-name>REST</servlet-name> <servlet-class>org.basex.http.rest.RESTServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>REST</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping>

<!-- WebDAV Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebDAV</servlet-name> <servlet-class>org.basex.http.webdav.WebDAVServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>WebDAV</servlet-name> <url-pattern>/webdav/*</url-pattern> </servlet-mapping>

<!-- Mapping for static resources (may be restricted to a sub path) --> <servlet> <servlet-name>default</servlet-name> <init-param> <param-name>useFileMappedBuffer</param-name> <param-value>false</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>/static/*</url-pattern> </servlet-mapping>

</web-app>

-----Original Message----- From: Christian Grün christian.gruen@gmail.com Sent: Monday, April 3, 2023 11:27 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

The logs look inconspicuous indeed. Some more questions:

• You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload? • Which BaseX services are enabled in your web.xml? Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?

Best, Christian

On Mon, Apr 3, 2023 at 4:44 PM ykhabins@bellsouth.net wrote:

Hi Christian,

IMO, it is just the number of requests. I attached the .log file.

-----Original Message----- From: Christian Grün christian.gruen@gmail.com Sent: Monday, April 3, 2023 10:32 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

Hi Yitzhak,

have you checked the resulting log files in the data/.logs directory? Are there specific requests that take too much time, or is it the plain number of incoming requests that eventually slows down the system?

Best, Christian

On Mon, Apr 3, 2023 at 4:29 PM ykhabins@bellsouth.net wrote:

...

Hello,

We are using BaseX 10.5 via its HTTP service in a corporate environment.

We have an automated Qualys Agent that does a vulnerability scan of that server with the BaseX.

Qualys Agent scan process includes web sites related tests such as Cross-Site Scripting, SQL Injection, etc.

The rapid nature of the Qualys Agent requests effectively gives us a DoS attack on the eclipse.jetty.server.

It cannot process so many requests and goes down.

In the meantime, our solution is to restart BaseX HTTP service manually via basexhttp.bat.

Question: is it possible to somehow configure the eclipse.jetty.server so it will be able to sustain the Qualys Agent vulnerability scan?

Regards, Yitzhak Khabinsky

I don’t such call, i.e. on the port 8081, in the *.log file.

From: Johan Mörén johan.moren@gmail.com Sent: Monday, April 3, 2023 11:56 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de; Christian Grün christian.gruen@gmail.com Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

Is the agent calling the stop port?

https://docs.basex.org/wiki/Options#STOPPORT

On Mon, 3 Apr 2023 at 17:38, <ykhabins@bellsouth.net mailto:ykhabins@bellsouth.net > wrote:

...

"You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload?"

It doesn't accept any further requests. Just launching the basexhttp.bat revives it.

...

"Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?"

We never tried to disable anything. The Qualys Agent runs once every two weeks on a schedule. So, it is not easy to run on demand for testing.

...

" Which BaseX services are enabled in your web.xml?"

We never modified anything in the web.xml. Please see it below.

<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://www.oracle.com/webfolder/technetwork/jsc/xml/ns/javaee/web-app_4_0.xsd" version="4.0">

<display-name>BaseX: The XML Database and XQuery Processor</display-name> <description>HTTP Services</description>

<!-- A BaseX option can be overwritten by prefixing the key with "org.basex." and specifying it in <context-param/> elements, as shown below. Check out https://docs.basex.org/wiki/Options for a list of all options.

<context-param> <param-name>org.basex.restxqpath</param-name> <param-value>.</param-value> </context-param> <context-param> <param-name>org.basex.dbpath</param-name> <param-value>WEB-INF/data</param-value> </context-param> <context-param> <param-name>org.basex.repopath</param-name> <param-value>WEB-INF/repo</param-value> </context-param>

<context-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </context-param> <context-param> <param-name>org.basex.authmethod</param-name> <param-value>Digest</param-value> </context-param>

<context-param> <param-name>org.basex.httplocal</param-name> <param-value>true</param-value> </context-param> <context-param> <param-name>org.basex.timeout</param-name> <param-value>5</param-value> </context-param> <context-param> <param-name>org.basex.log</param-name> <param-value>false</param-value> </context-param> -->

<!-- Global session and servlet listener --> <listener> <listener-class>org.basex.http.SessionListener</listener-class> </listener> <listener> <listener-class>org.basex.http.ServletListener</listener-class> </listener>

<!-- CORS in Jetty: Access-Control-Allow-Origin: * <filter> <filter-name>cross-origin</filter-name> <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class> <init-param> <param-name>allowedOrigins</param-name> <param-value>*</param-value> </init-param> </filter> <filter-mapping> <filter-name>cross-origin</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> -->

<!-- RESTXQ Service (can be disabled by removing this entry) --> <servlet> <servlet-name>RESTXQ</servlet-name> <servlet-class>org.basex.http.restxq.RestXqServlet</servlet-class> <init-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>RESTXQ</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping>

<!-- WebSocket Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebSocket</servlet-name> <servlet-class>org.basex.http.ws.WsServlet</servlet-class> <!-- Limits of the WebSocket connection <init-param> <param-name>maxIdleTime</param-name> <param-value>100000</param-value> </init-param> <init-param> <param-name>maxTextMessageSize</param-name> <param-value>3000</param-value> </init-param> <init-param> <param-name>maxBinaryMessageSize </param-name> <param-value>3000</param-value> </init-param> --> </servlet> <servlet-mapping> <servlet-name>WebSocket</servlet-name> <url-pattern>/ws/*</url-pattern> </servlet-mapping>

<!-- REST Service (can be disabled by removing this entry) --> <servlet> <servlet-name>REST</servlet-name> <servlet-class>org.basex.http.rest.RESTServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>REST</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping>

<!-- WebDAV Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebDAV</servlet-name> <servlet-class>org.basex.http.webdav.WebDAVServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>WebDAV</servlet-name> <url-pattern>/webdav/*</url-pattern> </servlet-mapping>

<!-- Mapping for static resources (may be restricted to a sub path) --> <servlet> <servlet-name>default</servlet-name> <init-param> <param-name>useFileMappedBuffer</param-name> <param-value>false</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>/static/*</url-pattern> </servlet-mapping>

</web-app>

-----Original Message----- From: Christian Grün <christian.gruen@gmail.com mailto:christian.gruen@gmail.com > Sent: Monday, April 3, 2023 11:27 AM To: ykhabins@bellsouth.net mailto:ykhabins@bellsouth.net Cc: BaseX <basex-talk@mailman.uni-konstanz.de mailto:basex-talk@mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

The logs look inconspicuous indeed. Some more questions:

• You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload? • Which BaseX services are enabled in your web.xml? Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?

Best, Christian

On Mon, Apr 3, 2023 at 4:44 PM <ykhabins@bellsouth.net mailto:ykhabins@bellsouth.net > wrote:

Hi Christian,

IMO, it is just the number of requests. I attached the .log file.

-----Original Message----- From: Christian Grün <christian.gruen@gmail.com mailto:christian.gruen@gmail.com > Sent: Monday, April 3, 2023 10:32 AM To: ykhabins@bellsouth.net mailto:ykhabins@bellsouth.net Cc: BaseX <basex-talk@mailman.uni-konstanz.de mailto:basex-talk@mailman.uni-konstanz.de > Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

Hi Yitzhak,

have you checked the resulting log files in the data/.logs directory? Are there specific requests that take too much time, or is it the plain number of incoming requests that eventually slows down the system?

Best, Christian

On Mon, Apr 3, 2023 at 4:29 PM <ykhabins@bellsouth.net mailto:ykhabins@bellsouth.net > wrote:

...

Hello,

We are using BaseX 10.5 via its HTTP service in a corporate environment.

We have an automated Qualys Agent that does a vulnerability scan of that server with the BaseX.

Qualys Agent scan process includes web sites related tests such as Cross-Site Scripting, SQL Injection, etc.

The rapid nature of the Qualys Agent requests effectively gives us a DoS attack on the eclipse.jetty.server.

It cannot process so many requests and goes down.

In the meantime, our solution is to restart BaseX HTTP service manually via basexhttp.bat.

Question: is it possible to somehow configure the eclipse.jetty.server so it will be able to sustain the Qualys Agent vulnerability scan?

Regards, Yitzhak Khabinsky

Is it possible that the Jetty stop call is not logged in the *.log files?

I would be surprised (even more if there’s no request to the stop port). Just run basexhttp and basexhttp and check out the result:

16:49:45.431 SERVER admin OK HTTP Server was started (port: 80). 16:49:57.003 SERVER admin OK HTTP Server was stopped (port: 80).

Does the problem persist with RESTXQ, WebSocket, and WebDAV removed?

If you manage to create an environment in which you can also disable the BaseX REST service, and if Jetty goes down as well, it may be a problem that doesn’t relate to BaseX.

Any other ideas what could be done?

Does the problem persist with RESTXQ, WebSocket, and WebDAV removed?

If you manage to create an environment in which you can also disable the BaseX REST service, and if Jetty goes down as well, it may be a problem that doesn’t relate to BaseX.

-Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.StdErrLog -D{classref}.LEVEL=DEBUG

But the basexhttp.bat file contains just SET and JAVA commands. Question: how to activate Jetty verbose logging?

-D is a standard Java option, so you can e.g. assign it to the BASEX_JVM variable. You can also directly edit the script.

We are using just the REST service. So, based on the recommendation, I disabled, by commenting them out, not needed services: RESTXQ, WebSocket, and WebDAV.

-----Original Message----- From: ykhabins@bellsouth.net ykhabins@bellsouth.net Sent: Monday, April 3, 2023 11:38 AM To: 'Christian Grün' christian.gruen@gmail.com Cc: 'BaseX' basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

...

"You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload?"

It doesn't accept any further requests. Just launching the basexhttp.bat revives it.

...

"Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?"

We never tried to disable anything. The Qualys Agent runs once every two weeks on a schedule. So, it is not easy to run on demand for testing.

...

" Which BaseX services are enabled in your web.xml?"

We never modified anything in the web.xml. Please see it below.

<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://www.oracle.com/webfolder/technetwork/jsc/xml/ns/javaee/web-app_4_0.xsd" version="4.0">

<display-name>BaseX: The XML Database and XQuery Processor</display-name> <description>HTTP Services</description>

<!-- A BaseX option can be overwritten by prefixing the key with "org.basex." and specifying it in <context-param/> elements, as shown below. Check out https://docs.basex.org/wiki/Options for a list of all options.

<context-param> <param-name>org.basex.restxqpath</param-name> <param-value>.</param-value> </context-param> <context-param> <param-name>org.basex.dbpath</param-name> <param-value>WEB-INF/data</param-value> </context-param> <context-param> <param-name>org.basex.repopath</param-name> <param-value>WEB-INF/repo</param-value> </context-param>

<context-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </context-param> <context-param> <param-name>org.basex.authmethod</param-name> <param-value>Digest</param-value> </context-param>

<context-param> <param-name>org.basex.httplocal</param-name> <param-value>true</param-value> </context-param> <context-param> <param-name>org.basex.timeout</param-name> <param-value>5</param-value> </context-param> <context-param> <param-name>org.basex.log</param-name> <param-value>false</param-value> </context-param> -->

<!-- Global session and servlet listener --> <listener> <listener-class>org.basex.http.SessionListener</listener-class> </listener> <listener> <listener-class>org.basex.http.ServletListener</listener-class> </listener>

<!-- CORS in Jetty: Access-Control-Allow-Origin: * <filter> <filter-name>cross-origin</filter-name> <filter-class>org.eclipse.jetty.servlets.CrossOriginFilter</filter-class> <init-param> <param-name>allowedOrigins</param-name> <param-value>*</param-value> </init-param> </filter> <filter-mapping> <filter-name>cross-origin</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> -->

<!-- RESTXQ Service (can be disabled by removing this entry) --> <servlet> <servlet-name>RESTXQ</servlet-name> <servlet-class>org.basex.http.restxq.RestXqServlet</servlet-class> <init-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>RESTXQ</servlet-name> <url-pattern>/*</url-pattern> </servlet-mapping>

<!-- WebSocket Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebSocket</servlet-name> <servlet-class>org.basex.http.ws.WsServlet</servlet-class> <!-- Limits of the WebSocket connection <init-param> <param-name>maxIdleTime</param-name> <param-value>100000</param-value> </init-param> <init-param> <param-name>maxTextMessageSize</param-name> <param-value>3000</param-value> </init-param> <init-param> <param-name>maxBinaryMessageSize </param-name> <param-value>3000</param-value> </init-param> --> </servlet> <servlet-mapping> <servlet-name>WebSocket</servlet-name> <url-pattern>/ws/*</url-pattern> </servlet-mapping>

<!-- REST Service (can be disabled by removing this entry) --> <servlet> <servlet-name>REST</servlet-name> <servlet-class>org.basex.http.rest.RESTServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>REST</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping>

<!-- WebDAV Service (can be disabled by removing this entry) --> <servlet> <servlet-name>WebDAV</servlet-name> <servlet-class>org.basex.http.webdav.WebDAVServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>WebDAV</servlet-name> <url-pattern>/webdav/*</url-pattern> </servlet-mapping>

<!-- Mapping for static resources (may be restricted to a sub path) --> <servlet> <servlet-name>default</servlet-name> <init-param> <param-name>useFileMappedBuffer</param-name> <param-value>false</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>/static/*</url-pattern> </servlet-mapping>

</web-app>

-----Original Message----- From: Christian Grün christian.gruen@gmail.com Sent: Monday, April 3, 2023 11:27 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

The logs look inconspicuous indeed. Some more questions:

• You mentioned that the Jetty server “goes down”. What does that mean? Does it simply block any further requests? Do you have a 100% CPU workload? • Which BaseX services are enabled in your web.xml? Does Jetty stall if you disable all REST, RESTXQ, and/or WebDAV?

Best, Christian

On Mon, Apr 3, 2023 at 4:44 PM ykhabins@bellsouth.net wrote:

Hi Christian,

IMO, it is just the number of requests. I attached the .log file.

-----Original Message----- From: Christian Grün christian.gruen@gmail.com Sent: Monday, April 3, 2023 10:32 AM To: ykhabins@bellsouth.net Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX HTTP service goes down due to Qualys Agent

Hi Yitzhak,

have you checked the resulting log files in the data/.logs directory? Are there specific requests that take too much time, or is it the plain number of incoming requests that eventually slows down the system?

Best, Christian

On Mon, Apr 3, 2023 at 4:29 PM ykhabins@bellsouth.net wrote:

...

Hello,

We are using BaseX 10.5 via its HTTP service in a corporate environment.

We have an automated Qualys Agent that does a vulnerability scan of that server with the BaseX.

Qualys Agent scan process includes web sites related tests such as Cross-Site Scripting, SQL Injection, etc.

The rapid nature of the Qualys Agent requests effectively gives us a DoS attack on the eclipse.jetty.server.

It cannot process so many requests and goes down.

In the meantime, our solution is to restart BaseX HTTP service manually via basexhttp.bat.

Question: is it possible to somehow configure the eclipse.jetty.server so it will be able to sustain the Qualys Agent vulnerability scan?

Regards, Yitzhak Khabinsky