The deliverables promised from WP2:
|
ID |
|
Due date |
Dissemination level |
Nature |
Publication |
|
D2.1 |
1 March, 2011 |
Public |
Prototype |
|
|
|
D2.2 |
1 September, 2011 |
Public |
Prototype |
|
|
|
D2.3 |
1 March, 2012 |
Public |
Regular Publication |
|
[this comes from the MOLTO website:]
The objective is to develop a tool for building domain-specific grammar-based multilingual translators. This tool will be accessible to users who have expertise in the domain of translation but only limited knowledge of the GF formalism or linguistics. The tool will integrate ontologies with GF grammars to help in building an abstract syntax. For the concrete syntax, the tool will enable simultaneous work on an unlimited number of languages and the addition of new languages to a system. It will also provide linguistic resources for at least 15 languages, among which at least 12 are official languages of the EU.
The top-level user tool is an IDE (Integrated Development Environment) for the GF grammar compiler. This IDE provides a test bench and a project management system. It is built on top of three more general techniques: the GF Grammar Compiler API (Application Programmer’s Interface), the GF-Ontology mapping (from WP4), and the GF Resource Grammar Library. The API is a set of functions used for compiling grammars from scratch and also for extending grammars on the fly. The Library is a set of wide-coverage grammars, which is maintained by an open source project outside MOLTO but will be via MOLTO efforts made accessible for programmers on lower levels of linguistic expertise. Thus we rely on the available GF resource grammar library and its documentation, available through digitalgrammars.com/gf/lib. The API is also used in WP3, as a tool for limited grammar extension, mostly with lexical information but also for example-based grammar writing. UGOT designs APIs and the IDE, coordinates work on grammars of individual languages, and compiles the documentation. UHEL contributes to terminology management and work on individual languages. UPC contributes to work on individual languages. Ontotext works on the Ontology-Grammar interface and contributes to the ontology-related part of the IDE.
Here we try to make a bit clearer what the functionalities of the WP2 tools are, and how they relate to the translator's tool.
We surmise that the grammar compiler's IDE is meant primarily for grammarian/engineer roles, i.e. for extending the system to new domains and languages. But it may contain facilities or components which are also relevant for the translation tool. In many scenarios, we must allow the translator to extend the system, i.e. switch to some of the last four roles. Just how the translation tool is linked to the grammar IDE needs specifying.
What the average user can do to fix the translation depends on how user friendly we can get. Minimally, a translator only supplies a missing translation on the fly, and all necessary adaptation is handled by the system. Maximally, an ontology or grammar needs extending as a separate chore by hand, using the grammar IDE.
An author/editor/translator can be expected to translate with the given lingware. The next level of involvement is extending the translation. This may cause entries or rules to be added to a text, company, or domain specific ontology/lexicon/grammar. If the tool is used in an organization, roles may be distributed to different people and questions of division of labor and quality control (as addressed in TF) already arise.
For it is not only, even in the first place, a question of being able to change the grammar technically, but managing the changes. A change in the source may cause improvement in some languages, deterioration in others. The author can't possibly check the repercussions in all languages. Assume each user site makes its own local changes. How many different versions of MOLTO lingware will there be? One for each website maintained with MOLTO? – how can sites share problems and solutions? A picture of a MOLTO community not unlike the one envisaged for multilingual ontology management TF starts to form. The challenge is analogous to ontology evolution. There are hundreds of small university ontologies in Swoogle. Quality can be created in the crowd, but there must be an organisation for it (cf. Wikipedia).
The MOLTO worfklow and role play must be spelled out in the grammar tool manual (D 2.3) and the MOLTO translation tools / workflow manual (D 3.3). We should start writing these manuals now, to fix and share our ideas about the user interfaces.
The way disambiguation now works is that translation of a vague source against a finer grained target generates the alternative translations with disambiguating metatext to help choose the intended meaning. (try I love you in http://www.grammaticalframework.org/demos/phrasebook/. Compare to Boitet et al.'s 1993 dialogue based MT system Lidia e.g. http://www.springerlink.com/content/kn8029t181090028/)
This facility could link to the ontology as a source of disambiguating metatext, either from meta comments or directly verbalised from ontology).
Some of the GF 3.2 features, like parse ranking and example based grammar generation, have consequences to front end design, as enabling technology.
[11 Productivity and usability]
Our case studies should show that it is possible to build a completely functional high-quality translation system for a new application in a matter of months—for small domains in just days.
The effort to create a system dynamically applicable to an unlimited number of documents will be essentially the same as the effort it currently takes to manually translate a set of static documents.
The expertise needed for producing a translation system will be low, essentially amounting to the skills of an average programmer who has practical knowledge of the targeted language and of the idiomatic vocabulary and syntax of the domain of translation.
1. localization of systems: the MOLTO tool for adding a language to a system can be learned in less than one day, and the speed of its use is in the same order of magnitude as translating an example text where all the domain concepts occur
The role requirements for extending the system remain quite high, not because of the requirements on the individual skills, but because it is less common to find their combination in one person.
The user requirements entail an important evaluation criterion: the guidance provided by MOLTO. It should also lead to system requirements, like online help, examples, profiling capabilities.
One part of MOLTO adaptivity is meant to come from the grammar IDE. Another part should come from ontologies. While the former helps extending GF “internally”, the latter should allow bringing in semantics and vocabulary from OWL ontologies. We discuss these two parts in this order.
[8 Grammar engineering for new languages in DoW]
In the MOLTO project, grammar engineering in GF will be further improved in two ways:
• An IDE (Integrated Development Environment), helping programmers to use the RGL and manage large projects.
• Example-Based Grammar Writing, making it possible to bootstrap a grammar from a set of example translations.
The former tool is a standard component of any library-based software engineering methodology. The latter technique uses the large-coverage RGL for parsing translation examples, which leads to translation rule suggestions.
The task of building a new language resource from scratch currently is described in http://grammaticalframework.org/doc/gf-lrec-2010.pdf. As this is largely a one-shot language engineering task outside of MOLTO (MOLTO was supposed to have its basic lingware done ahead of time), it should not call for evaluation here.
Building a multilingual application for a given abstract domain grammar by way of applying and extending concrete resource grammars can use a lighter process. The proposed example-based grammar writing process is described in the Phrasebook deliverable (http://www.molto-project.eu/node/1040). The tentative conclusions were:
• The grammarian need not be a native speaker of the language. For many languages, the grammarian need not even know the language, native informants are enough. However, evaluation by native speakers is necessary.
• Correct and idiomatic translations are possible.
• A typical development time was 2-3 person working days per language.
• Google translate helps in bootstrapping grammars, but must be checked. In particular, we found it unreliable for morphologically rich languages.
• Resource grammars should give some more support e.g. higher-level access to constructions like negative expressions and large-scale morphological lexica.
Effort and Cost
Based on this case study, we roughly estimated the effort used in constructing the necessary sources for each new language and compiled the following summarizing chart.
|
Language |
Language skills |
GF skills |
Informed development |
Informed testing |
Impact of external tools |
RGL Changes |
Overall effort |
|
Bulgarian |
### |
### |
- |
- |
? |
# |
## |
|
Catalan |
### |
### |
- |
- |
? |
# |
# |
|
Danish |
- |
### |
+ |
+ |
## |
# |
## |
|
Dutch |
- |
### |
+ |
+ |
## |
# |
## |
|
English |
## |
### |
- |
+ |
- |
- |
# |
|
Finnish |
### |
### |
- |
- |
? |
# |
## |
|
French |
## |
### |
- |
+ |
? |
# |
# |
|
German |
# |
### |
+ |
+ |
## |
## |
### |
|
Italian |
### |
# |
- |
- |
? |
## |
## |
|
Norwegian |
# |
### |
+ |
- |
## |
# |
## |
|
Polish |
### |
### |
+ |
+ |
# |
# |
## |
|
Romanian |
### |
### |
- |
- |
# |
### |
### |
|
Spanish |
## |
# |
- |
- |
? |
- |
## |
|
Swedish |
## |
### |
- |
+ |
? |
- |
## |
The phrasebook deliverable is one simple example what can be done to evaluate the grammar workpackage's promises. The results from the Phrasebook experiment may be positively biased because the test subjects were very well qualified. But this and similar tests can be repeated with more “ordinary people”, and changes in the figures followed as the grammar IDE is developing.
It could be instructive to repeat the exact same test with different subjects and compare the solutions, to see how much creativity was involved in the solutions. The less there is variation the better the chances to automate the process. Even failing that, analysis of the variant solutions could help suggest guidelines and best practices to the manual. Possible variation here also raises the issue of managing changes in a community of users.