Evaluation discussion

inari.listenmaa

Multilingual Online Translation

Add flagships to trac

Grammar evaluation in terms of best practices

Inari will make a list of questions for all grammar writers to answer, and write about the results in D9.2, the final evaluation report.

D9.2 is due March 2013, let's set a deadline in February 2013.

D2.3 Best Practices

All MOLTO WPs that have grammars:

WP4 : query grammar -- Ontotext; Maria
WP6 : MGL -- Jordi, Ares
WP7 : patents grammar -- Ramona, John
WP8 : museum grammar -- Ramona, Dana, Aarne
WP10 : Phrasebook -- Inari etc.
WP11 : ACE grammar -- Kaarel, John
- source code: https://github.com/Attempto/ACE-in-GF
- documentation: http://www.molto-project.eu/biblio/deliverable/ace-grammar-library
- evaluation wiki: https://github.com/Attempto/ACE-in-GF/wiki/Evaluation
WP12 : Monnet grammars -- Christina

Developer's tools

Eclipse plugin
Test management tools
Grammar diagnostic tools

Ongoing projects: Monnet/Be Informed, Query(?),

Already completed grammars: Phrasebook, Museum, ACE(?), Patents(?), MGL(?)

Things to report:

use of developer's tools
treebanks to automate the testing

Good grammar practices

Things to report:

Abstract syntax design

Module structure: base grammar and domain extensions (query grammar as a good example)
Language-independentness; good categories and abstractions (how to quantify that?)
Manual or automatically derived from ontology (Monnet, Museum)

Concrete syntax implementation

Use of functors and modules
Linearization types according to Best practices document: no lexical categories
Only API functions (mkC), no RGL internal functions or manual concatenation

This applies for all grammars, in progress or complete. Phrasebook was the first MOLTO grammar, in 2010 spring. We could compare it to newer grammars. MGL has been rewritten since WebALT; some reflections on that maybe. How about the evolution of Ontotext query grammar? First it was a bunch of rules to transform SPARQL-GF, only later transformed into a grammar where English, Swedish and SPARQL are all concrete syntaxes.