Dear all, We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly. Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation). How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run. The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096). Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14 Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67). Have you had similar problems? Are you parsing real sentences, how long are your sentences? Do you have any suggestions what we could do so as to reduce the number of timed out sentences? Best, Agnieszka
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum. My suggestions would be the following: 1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary). 2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high. As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above. 3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it. I hope this can help. Best Bamba On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
Hi Agnieszka, We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007. We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming. I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens. Also, XLE crashed on rare occasions (less than 0.002% of sentences). More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225) 2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/ Best regards, Joachim On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram -- Dr. Joachim Wagner Postdoctoral Researcher Centre for Next Generation Localisation (CNGL) Dublin City University Dublin 9 ph: +353 (0)1 700 6915 http://www.cngl.ie/
Hi, to add to this discussion from my experience... Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show. One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot. The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want. Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important. As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon. The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency. Best, Miriam On 10/8/13 1:02 PM, Joachim Wagner wrote:
Hi Agnieszka,
We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115 miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt "Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28) *************************************************************
Briefly (being on vacation with only iPhone access): as Miriam says, moving distinctions into the c-structure helps a lot. The Norwegian grammar doesn't use complex categories, but we (equivalently) distinguish between many subtypes of syntactic categories by means of subscripts (VPmain, VPfin, VPinf, PRONexpl, etc.), which clearly helps a lot. In order to keep f-structure constraints local, use the COMPLETE template. Avoid restricted unification, especially in combination with functional uncertainty. Helge Sendt fra min iPhone
Den 8. okt. 2013 kl. 17:12 skrev Miriam Butt <miriam.butt@uni-konstanz.de>:
Hi,
to add to this discussion from my experience...
Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show.
One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot.
The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want.
Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important.
As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon.
The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency.
Best,
Miriam
On 10/8/13 1:02 PM, Joachim Wagner wrote: Hi Agnieszka,
We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115
miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt
"Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28)
*************************************************************
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/parsem
Dear all, Bamba, Joachim, Miriam and Helge: many thanks for all the advice, I haven't even hoped to get so much feedback! I'm going to try all these, hopefully the timeout rate will decrease substantially. If there is anybody else who has some experience with solving performance problems in XLE, timeouts especially, I'd like to encourage you to write to the list. Miriam mentioned Tracy and John – English LFG grammar is said to achieve great results with regard to performance. It would be very nice to hear some details about how it was achieved. Also, I'm very curious about your valence dictionaries – how ambiguous are your entries, how many frames per predicate do you have on average? Again, many thanks for your help and suggestions! Best, Agnieszka On 8 October 2013 18:36, Helge Dyvik <helge.dyvik@lle.uib.no> wrote:
Briefly (being on vacation with only iPhone access): as Miriam says, moving distinctions into the c-structure helps a lot. The Norwegian grammar doesn't use complex categories, but we (equivalently) distinguish between many subtypes of syntactic categories by means of subscripts (VPmain, VPfin, VPinf, PRONexpl, etc.), which clearly helps a lot. In order to keep f-structure constraints local, use the COMPLETE template. Avoid restricted unification, especially in combination with functional uncertainty.
Helge
Sendt fra min iPhone
Den 8. okt. 2013 kl. 17:12 skrev Miriam Butt <miriam.butt@uni-konstanz.de>:
Hi,
to add to this discussion from my experience...
Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show.
One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot.
The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want.
Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important.
As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon.
The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency.
Best,
Miriam
On 10/8/13 1:02 PM, Joachim Wagner wrote:
Hi Agnieszka,
We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115
miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt
"Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28)
*************************************************************
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/parsem
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/parsem
Hi, Subscripting is computationally equivalent to using complex categories, but it may not make it as easy to capture certain kinds of generalizations. The parameters of a complex category allow various expansions to share the same specification, and the parameters than specialize that to the various different cases. This can propagate through a chain of lower rules, where everything in the chain is shared but specialized. You can express some generalizations among alternative subscripts of the same basic category, but I think then you have to do that by invoking the same macro with different parameters. It may be more difficult to get the special cases to propagate through a chain of rules. With respect to the use of optimality marks as in the English grammar, this has its own dangers. The marks in English were very carefully arranged (by Tracy, Martin, and maybe others) to get certain effects, both in terms of preferences and performance. In the absence of Tracy and Martin, I have been trying to make only minor adjustments to the marking and ranking to tune the grammar in slightly different ways (e.g. to give priority to imperatives instead of sequences of keywords, to downgrade passive reduced relatives in some contexts), and I have found it very difficult to introduce new marks or reorder old ones, even just a little bit, to get the relatively straightforward outcomes that I am after. The indirect consequences of even minor adjustments are hard to understand and predict. And much less documentation is provided for the marks and their ranking compared to the comments that are distributed in the structural rules. I think this is itself a reflection of the fact that the OT specifications have global interactions and are thus more difficult to describe. --Ron On Oct 8, 2013, at 9:36 AM, Helge Dyvik <Helge.Dyvik@lle.uib.no> wrote:
Briefly (being on vacation with only iPhone access): as Miriam says, moving distinctions into the c-structure helps a lot. The Norwegian grammar doesn't use complex categories, but we (equivalently) distinguish between many subtypes of syntactic categories by means of subscripts (VPmain, VPfin, VPinf, PRONexpl, etc.), which clearly helps a lot. In order to keep f-structure constraints local, use the COMPLETE template. Avoid restricted unification, especially in combination with functional uncertainty.
Helge
Sendt fra min iPhone
Den 8. okt. 2013 kl. 17:12 skrev Miriam Butt <miriam.butt@uni-konstanz.de>:
Hi,
to add to this discussion from my experience...
Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show.
One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot.
The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want.
Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important.
As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon.
The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency.
Best,
Miriam
On 10/8/13 1:02 PM, Joachim Wagner wrote:
Hi Agnieszka,
We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115
miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt
"Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28)
*************************************************************
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/parsem
XLE mailing list XLE@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/xle
Yes, indeed, I didn't mean to suggest that using subscripts is equivalent to using complex categories from the point of view of the grammar writer. Helge Sendt fra min iPhone
Den 9. okt. 2013 kl. 12:13 skrev Ron Kaplan <ron.kaplan@post.harvard.edu>:
Hi,
Subscripting is computationally equivalent to using complex categories, but it may not make it as easy to capture certain kinds of generalizations. The parameters of a complex category allow various expansions to share the same specification, and the parameters than specialize that to the various different cases. This can propagate through a chain of lower rules, where everything in the chain is shared but specialized.
You can express some generalizations among alternative subscripts of the same basic category, but I think then you have to do that by invoking the same macro with different parameters. It may be more difficult to get the special cases to propagate through a chain of rules.
With respect to the use of optimality marks as in the English grammar, this has its own dangers. The marks in English were very carefully arranged (by Tracy, Martin, and maybe others) to get certain effects, both in terms of preferences and performance. In the absence of Tracy and Martin, I have been trying to make only minor adjustments to the marking and ranking to tune the grammar in slightly different ways (e.g. to give priority to imperatives instead of sequences of keywords, to downgrade passive reduced relatives in some contexts), and I have found it very difficult to introduce new marks or reorder old ones, even just a little bit, to get the relatively straightforward outcomes that I am after. The indirect consequences of even minor adjustments are hard to understand and predict. And much less documentation is provided for the marks and their ranking compared to the comments that are distributed in the structural rules. I think this is itself a reflection of the fact that the OT specifications have global interactions and are thus more difficult to describe.
--Ron
On Oct 8, 2013, at 9:36 AM, Helge Dyvik <Helge.Dyvik@lle.uib.no> wrote:
Briefly (being on vacation with only iPhone access): as Miriam says, moving distinctions into the c-structure helps a lot. The Norwegian grammar doesn't use complex categories, but we (equivalently) distinguish between many subtypes of syntactic categories by means of subscripts (VPmain, VPfin, VPinf, PRONexpl, etc.), which clearly helps a lot. In order to keep f-structure constraints local, use the COMPLETE template. Avoid restricted unification, especially in combination with functional uncertainty.
Helge
Sendt fra min iPhone
Den 8. okt. 2013 kl. 17:12 skrev Miriam Butt <miriam.butt@uni-konstanz.de>:
Hi,
to add to this discussion from my experience...
Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show.
One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot.
The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want.
Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important.
As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon.
The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency.
Best,
Miriam
On 10/8/13 1:02 PM, Joachim Wagner wrote: Hi Agnieszka,
We parsed the BNC http://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper: http://doras.dcu.ie/15214/ 2012 thesis: http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115
miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt
"Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28)
*************************************************************
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/parsem
XLE mailing list XLE@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/xle
_______________________________________________ XLE mailing list XLE@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/xle _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
On 10/9/13 12:13 PM, Ron Kaplan wrote:
Hi,
Subscripting is computationally equivalent to using complex categories, but it may not make it as easy to capture certain kinds of generalizations. The parameters of a complex category allow various expansions to share the same specification, and the parameters than specialize that to the various different cases. This can propagate through a chain of lower rules, where everything in the chain is shared but specialized.
You can express some generalizations among alternative subscripts of the same basic category, but I think then you have to do that by invoking the same macro with different parameters. It may be more difficult to get the special cases to propagate through a chain of rules.
With respect to the use of optimality marks as in the English grammar, this has its own dangers. The marks in English were very carefully arranged (by Tracy, Martin, and maybe others) to get certain effects, both in terms of preferences and performance. In the absence of Tracy and Martin, I have been trying to make only minor adjustments to the marking and ranking to tune the grammar in slightly different ways (e.g. to give priority to imperatives instead of sequences of keywords, to downgrade passive reduced relatives in some contexts), and I have found it very difficult to introduce new marks or reorder old ones, even just a little bit, to get the relatively straightforward outcomes that I am after. The indirect consequences of even minor adjustments are hard to understand and predict. And much less documentation is provided for the marks and their ranking compared to the comments that are distributed in the structural rules. I think this is itself a reflection of the fact that the OT specifications have global interactions and are thus more difficult to describe.
I have also been experimenting with OT marks in the Urdu grammar and they are indeed EXTREMELY tricky. But on the whole I think they are very useful. What is missing, in some way, is a documentation of 'best practice" or "tips and tricks" with respect to these. Miriam
--Ron
On Oct 8, 2013, at 9:36 AM, Helge Dyvik <Helge.Dyvik@lle.uib.no <mailto:Helge.Dyvik@lle.uib.no>> wrote:
Briefly (being on vacation with only iPhone access): as Miriam says, moving distinctions into the c-structure helps a lot. The Norwegian grammar doesn't use complex categories, but we (equivalently) distinguish between many subtypes of syntactic categories by means of subscripts (VPmain, VPfin, VPinf, PRONexpl, etc.), which clearly helps a lot. In order to keep f-structure constraints local, use the COMPLETE template. Avoid restricted unification, especially in combination with functional uncertainty.
Helge
Sendt fra min iPhone
Den 8. okt. 2013 kl. 17:12 skrev Miriam Butt <miriam.butt@uni-konstanz.de <mailto:miriam.butt@uni-konstanz.de>>:
Hi,
to add to this discussion from my experience...
Bamba's tips sound very good. Tracy spent a great deal of time getting the English grammar to be efficient and I guess with a good success rate, as Joachims statistics show.
One thing to do is to have as much as possible be dealt with via the c-structure rather than using "expensive" f-structure equations/disjunctions. In the German and English grammars, the use of complex categories helped quite a lot.
The English grammar also makes heavy use of the OT marks --- this took some careful adjusting and experimenting so as to cut out the analyses one didn't want without losing the ones one does want.
Also, as Bamba says, the elimination of (spurious) ambiguities is extremely important.
As far as I remember, John Maxwell also spent some time implementing XLE so that where possible, local f-structure constraints would be resolved speedily. So one thing to try to do is to have f-structure constraints resolved as locally as possible, rather than passing them up and up. Of course, this depends on the phenomenon.
The Urdu grammar at the moment is horribly inefficient. We have very few c-structure rules and tons of disjunctions with much of the work being done via f-structure constraints. And we have many spurious ambiguities, whereby there is more than one way to arrive at the correct solution. We are currently reimplementing it to remove this inefficiency.
Best,
Miriam
On 10/8/13 1:02 PM, Joachim Wagner wrote:
Hi Agnieszka,
We parsed the BNChttp://www.natcorp.ox.ac.uk/ with XLE and the ParGram English LFG and got only 0.55% timeouts on machines from around 2007.
We used the XLE command parse-testfile with parse-literally set to 1, max_xle_scratch_storage set to 1,000 MB, a time-out of 60 seconds and no skimming.
I remember that I had strange problems when trying higher values for max_xle_scratch_storage. Maybe try 1000 MB instead of 4096 just to see what happens.
Also, XLE crashed on rare occasions (less than 0.002% of sentences).
More details in my phd thesis: * Breakdown of events (time out, out of memory, no parse etc): Table 5.1 (page 136) * Sentence length distribution of the BNC: curve 1.000 in Figure 7.13 (page 225)
2007 paper:http://doras.dcu.ie/15214/ 2012 thesis:http://doras.dcu.ie/16776/
Best regards, Joachim
On Tuesday 08 October 2013 12:01:22 Bamba Dione wrote:
Hi Agnieszka, This looks like a classic XLE problem. I face similar problems when parsing Wolof data (real sentences with a similar length as yours). @Jani: By the way, I think this is a good topic to discuss in the XLE forum.
My suggestions would be the following:
1) Try to eliminate all kind of ambiguities as much as possible, in particular spurious ambiguities. XLE provides some tools that help to inspect spurious ambiguities; e.g. the xle command print-ambiguity-sources may be particularly useful. A similar trick that helps a lot is check non-exclusive disjunctions and clearly define disjunctions in the grammar (rules, templates, etc.) to be sure that these are mutually exclusive (use check features if necessary).
2) A second possibility would be to try to put XLE in the skimming mode (see the XLE documentation). You could then ``play'' with the skimming variables start_skimming_when_scratch_storage_exceeds, start_skimming_when_total_events_exceed, max_new_events_per_graph_when_skimming and skimming_nogoods. Also, check if the value you are using for max_xle_scratch_storage is not a bit too high.
As stated in the documentation, in this mode, XLE does a bounded amount of work per subtree. This guarantees that it will finish processing the sentence in a polynomial amount of time. Though, this does not guarantee to get the sentences parsed correctly, it may help you to have at least some output and to reduce the number of timeouts. In my experience, the main problem with skimming is to find the best value for the variables mentioned above.
3) A third option is to use the optimality mark stoppoint. With this mark, you can design your grammar so that it will consider expensive and rare constructions only when no other analysis is available. You can combine this option with skimming: for instance you put those OT marks denoting to expensive and rare constructions in the variable skimming_nogoods. This can help reduce the timeouts and speed up the parser. Here also, there is the drawback that you might not get the analysis you're looking for just because stoppoint has blocked it.
I hope this can help.
Best Bamba
On Oct 8, 2013, at 11:01 AM, Agnieszka Patejuk wrote:
Dear all,
We've been developing POLFIE, an LFG grammar of Polish, at ICS PAS for over 3 years. We'd like to concentrate on extending the empirical scope of the grammar, but we're having performance issues which affect the results quite badly.
Currently we're testing the grammar on sentences extracted from Składnica, a treebank of Polish. There are 8333 sentences, average sentence length is 10 (all segments are counted, including punctuation).
How are these sentences parsed? – an individual dictionary is created for each sentence (so the words are already disambiguated morphosyntactically) – each sentence is parsed on its own, in one XLE run.
The following performance variables are used when parsing: – 100 seconds (set timeout 100) – 4096 MB memory (set max_xle_scratch_storage 4096).
Current results (out of 8333 sentences): – parsed: 6926 – failed: 154 – out of memory: 11 – timeout: 1228 – unknown error: 14
Almost 15% of sentences are timed out, which is very worrying. The average length of a parsed sentence is almost 9 (8.74), while the average length of a timed out sentence is almost 19 (18.67).
Have you had similar problems? Are you parsing real sentences, how long are your sentences?
Do you have any suggestions what we could do so as to reduce the number of timed out sentences?
Best, Agnieszka _______________________________________________ ParGram mailing list ParGram@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/pargram
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115
miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt
"Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28)
*************************************************************
_______________________________________________ ParSem mailing list ParSem@mailman.uni-konstanz.de <mailto:ParSem@mailman.uni-konstanz.de> https://mailman.uni-konstanz.de/mailman/listinfo/parsem
XLE mailing list XLE@mailman.uni-konstanz.de <mailto:XLE@mailman.uni-konstanz.de> https://mailman.uni-konstanz.de/mailman/listinfo/xle
-- ************************************************************* Miriam Butt FB Sprachwissenschaft Universitaet Konstanz Fach 184 Tel: +49 7531 88 5109 78457 Konstanz Fax: +49 7531 88 4865 Germany +49 7531 88 5115 miriam.butt@uni-konstanz.de http://ling.uni-konstanz.de/pages/home/butt "Leo looked scandalized. 'Before lunch? No, my dear boy. Poker. Wouldn't play bridge before lunch.' Margery Allingham "The Case of the Late Pig" (p. 28) *************************************************************
participants (6)
-
Agnieszka Patejuk -
Bamba Dione -
Helge Dyvik -
Joachim Wagner -
Miriam Butt -
Ron Kaplan