ParGram February 2016

pargram@mailman.uni-konstanz.de

2 participants
2 discussions

Question about setting up regression test/benchmarking with xle
by Antske Fokkens 09 Feb '16

09 Feb '16

Hi all, Sorry to bother you with a probably trivial question, but I haven't managed to figure this out following the xle documentation (probably missing something). I'd like to set up regression test and/or small benchmark testsuites. I've made a file with test sentences which I can evaluate using run-syn-testsuite or parse-testfile. I'd now like to run these and make xle register the differences so that I can monitor progress/regression in the grammar's performance. Basically, I'd like to run the testsuite on new data, be notified of changes in the f-structure and decide whether these are an improvement (update the standard to measure again) or a regression (retain old standard). Can anyone tell me how to do this? I've tried to play around with the -outputPrefix and -goldPrefix options, but it is not clear to me how to use them exactly. I've also tried to look into benchmarking, but wasn't able to figure out how to apply this to creating a benchmark for syntactic parsing (i.e. parse sentences and select the right f-structure, store this for evaluation). Any help would be greatly appreciated! Thanks! Antske

1 0

Tense/Aspect Testsuite & A Request
by Sebastian Sulger 05 Feb '16

05 Feb '16

Dear fellow ParGrammers, In our current project[1], we are investigating Tense/Aspect systems with the ultimate goal of coming up with a cross-linguistically applicable annotation scheme for these systems (morphosyntax and semantics). To that end, we'd like to collect data from as many ParGram grammars as possible in the form of translations and XLE parses. I am attaching a file with 45 sentences that aim at testing a couple of Tense/Aspect configurations. We don't want to prime you too much, so I won't say what these configurations are exactly. You will see that the file is formatted as an XLE testsuite; words in parentheses constitute supplementary information which you don't have to translate, but which nevertheless may be important to fully understand the desired semantics. Commented lines include contexts for the sentences; some sentences make use of the same contexts, as indicated in the file. If you would like to participate and help us in gathering data, you can do the following: 1. Answer to this email by saying "Yes, I'm in!" 2. Translate the sentences into the language you are working on. Use the most natural way of saying a sentence. If there are several equally natural ways, indicate all of them. 3. Parse all the translations, disambiguate them and save them as prolog files (these are very short sentences, so that we do not expect them to be major challenges to your grammars). 4. Send us your testsuite with the translations as well as your prolog files with the structures. We are in dire need of your support for this, so please consider submitting translations/structures to us! If you have any further questions at all, please let us know. Cheers, Sebastian & the PARTMA team [1] http://ling.uni-konstanz.de/pages/home/partma/ -- Sebastian Sulger FB Sprachwissenschaft Universität Konstanz http://ling.uni-konstanz.de/pages/home/sulger

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

ParGram February 2016