Hi Chirstian, I tried what Martin suggested. Please find the below is the sample code what I am using for creating collection.
/* This method takes the xml files list as input and process the list Create the xml collection DB, adds xml to the collection if already exists replace it */
public static void createCollection(ArrayList<String> listFiles){ ClientSession clientSession = null; String strIndexedFile=""; try { clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, "<<USER>>", "<<PWD>>"); String strDbName = "Sample_Collection"; String strKeysFileLocation = "D:\TEMP\7GB_Collection_TestingTeam\Files";
String res = clientSession.execute("XQUERY db:exists('" + strDbName + "')"); //DB existence checking if (!res.equalsIgnoreCase("true")) { clientSession.execute("CREATE DB " + strDbName); } else { clientSession.execute("OPEN " + strDbName); }
for (int i = 0; i < listFiles.size(); i++) { strIndexedFile = listFiles.get(i); String tempFlag=clientSession.execute("XQUERY db:exists('" + strDbName + "','" + strIndexedFile + "')"); clientSession.execute("ADD " + strKeysFileLocation + File.separator + strIndexedFile); } System.out.println("InFormation:: "+clientSession.execute("INFO")); //optimizing the current opend DB using OPTIMIZE ALL command clientSession.execute("OPTIMIZE ALL"); System.out.println("InFormation:: "+clientSession.execute("INFO"));
} catch (Exception e) { System.out.println("Exception files DB Updating: " + e.getMessage()); e.printStackTrace(); } finally { if (clientSession != null) { try { clientSession.close(); } catch (IOException ex) { ex.printStackTrace(); } } } }
Thanks & Regards, Adi
On Tue, Sep 15, 2015 at 4:52 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Adi,
I may not be up-to-date, as I expected you to send me an updated version of your XQuery code. Did you try what Martín suggested?
Christian
On Tue, Sep 15, 2015 at 1:19 PM, Adi Babu adibabu.b@gmail.com wrote:
Hai Chistian, Please find the attachment sample piece of code.
Thanks & Regards Adi
On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Adi,
How does your new code look like? How did you rewrite the collection function?
Christian
On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu adibabu.b@gmail.com wrote:
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than
7GB)
thats why I am sharing sample query(with xml doc but actually I am
using
collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards, Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari ferrari_martin@hotmail.com wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a
BaseX
db first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must
be
fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default).
You
might need to run the OPTIMIZE ALL command after uploading the xml file,
you
can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de
wrote:
Hi Adi,
some general pointers:
- Server version 7.9 is missing all the latest perf improvements,
current version is 8.2.3
- You are accessing the files in the file system - try adding them
to
the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu adibabu.b@gmail.com:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide
me
to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics :
Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever :
Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi