1. The GF Cloud Service API

The GF Cloud Service API exposes any PGF compiled grammar as a web service via the PGF web service API, it provides additional functionality of some commands in the GF shell and some services for grammar compilation and persistent storage of files in the cloud. These features are used for instance in the implementation of the GF Simple Editor (http://cloud.grammaticalframework.org/gfse/) developed as a translators' tool during MOLTO.

1.1 Availability and Protocol

The service is available from http://cloud.grammaticalframework.org/. The source code for hosting a local version of the webservice is distributed in the GF distribution, hence users that have GF installed on their own computer can also run the service locally by starting GF with the parameter

--server[=port] Run in HTTP server mode on given port (default 41296).

Requests are made via HTTP with the GET or POST method. (The examples below show GET requests, but POST is preferred for requests that change the state on the server.) Data in requests is in the application/x-www-form-urlencoded format (the format used by default by web browsers when submitting form data). Data in responses is usually in JSON format. The HTTP response code is usually 200, but can also be 204 (after file upload), 404 (file to download or remove was not found), 400 (for unrecognized commands or missing/unacceptable parameters in requests) or 501 (for unsupported HTTP request methods). Unrecognized parameters in requests are silently ignored.

More details on how to run the service are given in Deliverable 2.3 on page 7 under "Building a web application".

1.2 PGF Service Requests

The GF Cloud Service supports a set of PGF service requests, for example, a request like

http://cloud.grammaticalframework.org/grammars/Foods.pgf?command=random

might return a result like

[{"tree":"Pred (That Pizza) (Very Boring)"}]

The PGF Service in the GF Cloud is the application which exposes the PGF API as Web Service. The application uses FastCGI as communication protocol to talk with the web server. The data protocol that we use is JSON. Information for how to compile and install the service could be found here.

A compiled GF grammars could be used in web applications in the same way as JSP, ASP or PHP pages are used. The compiled PGF file is just placed somewhere in the web site directory. When there is a request for access to a .pgf file then the web server redirects the request to the GF web service. The service knows how to load the grammar and interpret the parameters given in the URL.

If my_grammar.pgf is a grammar placed in the root folder of the web site for localhost then the grammar could be accessed using this URL:

http://localhost/my_grammar.pgf

Since there aren't any parameters passed in this case, the web service will respond with some general information about the grammar, encoded in JSON format. To perform specific command you have to tell what command you want to perform. The command is encoded in the parameter command i.e.:

http://localhost/my_grammar.pgf?command=cmd

where cmd is the name of the command. Usually every command also requires specific list of other arguments which are encoded as parameters as well. The list of all supported commands follows:

Commands

Grammar

This command provides some general information about the grammar. This command is also executed if no command parameter is given.

Input

Parameter	Description	Default
command	`grammar`	-

Output

Object with three fields:

Field	Description
name	the name of the abstract syntax in the grammar
userLanguage	the concrete language in the grammar which best matches the default language, set in the user's browser
categories	list of all abstract syntax categories defined in the grammar
functions	list of all abstract syntax functions defined in the grammar
languages	list of concrete languages available in the grammar

Every language is described with object having this two fields:

Field	Description
name	the name of the concrete syntax for the language
languageCode	the two character language code according to the ISO standard i.e. en for English, bg for Bulgarian, etc.

The language codes should be specified in the grammar because they are used to identify the user language. The web service receives the code of the language set in the browser and compares it with the codes defined in the grammar. If there is a match then the service returns the corresponding concrete syntax name. If no match is found then the first language in alphabetical order is returned.

Parsing

This command parses a string and returns a list of abstract syntax trees.

Input

Parameter	Description	Default
command	`parse`	-
cat	the start category for the parser	the default start category for the grammar
input	the string to be parsed	empty string
from	the name of the concrete syntax to use for parsing	all languages in the grammar will be tried
limit	limit how many trees are returned (gf>3.3.3)	no limit is applied

Output

List of objects where every object represents the analyzes for every input language. The objects have three fields:

Field	Description
from	the concrete language used in the parsing
brackets	the bracketed string from the parser
trees	list of abstract syntax trees
typeErrors	list of errors from the type checker

The abstract syntax trees are sent as plain strings. The type errors are objects with two fields:

Field	Description
fid	forest id which points to a bracket in the bracketed string where the error occurs
msg	the text message for the error

The current implementation either returns a list of abstract syntax trees or a list of type errors. By checking whether the field trees is not null we check whether the type checking was successful.

Linearization

The command takes an abstract syntax tree and produces string in the specified language(s).

Input

Parameter	Description	Default
command	`linearize`	-
tree	the abstract syntax tree to linearize	-
to	the name of the concrete syntax to use in the linearization	linearizations for all languages in the grammar will be generated

Output

Field	Description
to	the concrete language used for the linearization
tree	the output text

Translation

The translation is a two step process. First the input sentence is parsed with the source language and after that the output sentence(s) are produced via linearization with the target language(s). For that reason the input and the output for this command is the union of the input/output of the commands for parsing and the one for linearization.

Input

Parameter	Description	Default
command	`translate`	-
cat	the start category for the parser	the default start category for the grammar
input	the input string to be translated	empty string
from	the source language	all languages in the grammar will be tried
to	the target language	linearizations for all languages in the grammar will be generated
limit	limit how many parse trees are used (gf>3.3.3)	no limit is applied

Output

The output is a list of objects with these fields:

Field	Description
from	the concrete language used in the parsing
brackets	the bracketed string from the parser
translations	list of translations
typeErrors	list of errors from the type checker

Every translation is an object with two fields:

tree	abstract syntax tree
linearizations	list of linearizations

Every linearization is an object with two fields:

Field	Description
to	the concrete language used in the linearization
text	the sentence produced

The type errors are objects with two fields:

Field	Description
fid	forest id which points to a bracket in the bracketed string where the error occurs
msg	the text message for the error

The current implementation either returns a list of translations or a list of type errors. By checking whether the field translations is not null we check whether the type checking was successful.

Random_Generation

This command generates random abstract syntax tree where the top-level function will be of the specified category. The categories for the sub-trees will be determined by the type signatures of the parent function.

Input

Parameter	Description	Default
command	should be `random`	-
cat	the start category for the generator	the default start category for the grammar
limit	maximal number of trees generated	1

Output

The output is a list of objects with only one field:

Field	Description
tree	the generated abstract syntax tree

The length of the list is limited by the limit parameter.

Word_Completion

Word completion is a special case of parsing. If there is an incomplete sentence then it is first parsed and after that the state of the parse chart is used to predict the set of words that could follow in a grammatically correct sentence.

Input

Parameter	Description	Default
command	`complete`	-
cat	the start category for the parser	the default start category for the grammar
input	the string to the left of the cursor that is already typed	empty string
from	the name of the concrete syntax to use for parsing	all languages in the grammar will be tried
limit	maximal number of trees generated	all words will be returned

Output

The output is a list of objects with two fields which describe the completions.

Field	Description
from	the concrete syntax for this word
text	the word itself

Abstract Syntax Tree Visualization

This command renders an abstract syntax tree into image in PNG format.

Input

Parameter	Description	Default
command	`abstrtree`	-
tree	the abstract syntax tree to render	-
format	output format (gf>3.3.3)	PNG

Output

Byy default, the output is an image in PNG format. The content-type is set to image/png, so the easiest way to visualize the generated image is to add HTML element <img> which points to URL for the visualization command i.e.:

<img src="http://localhost/my_grammar.pgf?command=abstrtree&tree=..."/>

The output can also be in GIF ('image/gif'), SVG ('image/svg+xml') or GV (graphviz) format by setting the 'format' option

Parse Tree Visualization

This command renders the parse tree that corresponds to a specific abstract syntax tree. The generated image is in PNG format.

Input

Parameter	Description	Default
command	`parsetree`	-
tree	the abstract syntax tree to render	-
from	the name of the concrete syntax to use in the rendering	-
format	output format (gf>3.3.3)	png
options	additional rendering options (gf>3.4)	-

The additioal rendering options are: noleaves, nofun and nocat (booleans, false by default); nodefont, leaffont,nodecolor, leafcolor, nodeedgestyle and leafedgestyle (strings, have builtin defaults).

Output

By default, the output is an image in PNG format. The content-type is set to 'image/png', so the easiest way to visualize the generated image is to add HTML element <img> which points to URL for the visualization command i.e.:

<img src="http://localhost/my_grammar.pgf?command=parsetree&tree=..."/>

The output can also be in GIF ('image/gif'), SVG ('image/svg+xml') or gv (graphviz) format by setting the format option

Word Alignment Diagram

This command renders the word alignment diagram for some sentence and all languages in the grammar. The sentence is generated from a given abstract syntax tree.

Input

Parameter	Description	Default
command	`alignment`	-
tree	the abstract syntax tree to render	-
format	output format (gf>3.3.3)	PNG
to	list of languages to include in the diagram (gf>3.4)	all languages supported by the grammar

Output

By default, the output is an image in PNG format. The content-type is set to 'image/png', so the easiest way to visualize the generated image is to add HTML element `` which points to URL for the visualization command i.e.:

<img src="http://localhost/my_grammar.pgf?command=alignment&tree=..."/>

The output can also be in GIF ('image/gif'), SVG ('image/svg+xml') or GV (graphviz) format by setting the 'format' option

1.3 GF Shell Service

This service lets you execute arbitrary GF shell commands. Before you can do this, you need to use the /new command to obtain a working directory (which also serves as a session identifier) on the server, see below.

/gfshell?dir=...&command=i+Foods.pgf
/gfshell?dir=...&command=gr: Pred (That Pizza) (Very Boring)
/gfshell?dir=...&command=ps+-lextext+%22That+pizza+is+very+boring.%22: that pizza is very boring .

For documentation of GF shell commands, see:

GF Shell Reference

Additional cloud service

/new: This generates a new working directory on the server, e.g. /tmp/gfse.123456. Most of the cloud service commands require that a working directory is specified in the dir parameter. The working directory is persistent, so clients are expected to remember and reuse it. Access to previously uploaded files requires that the same working directory is used.
/parse?path=source: This command can be used to check GF source code for syntax errors. It also converts GF source code to the JSON representation used in GFSE (the cloud-based GF grammar editor).
/cloud?dir=...&command=upload&path1=source1&path2=source2&...: Upload files to be stored in the cloud. The response code is 204 if the upload was successful.
/cloud?dir=...&command=make&path1=source1&path2=source2&...: Upload grammar files and compile them into a PGF file. Example response: { "errorcode":"OK", // "OK" or "Error" "command":"gf -s -make FoodsEng.gf FoodsSwe.gf FoodsChi.gf", "output":"\n\n" // Warnings and errors from GF }
/cloud?dir=...&command=remake&path1=source1&path2=source2&...: Like command=make, except you can leave the sourcei parts empty to reuse previously uploaded files.
/cloud?dir=...&command=download&file=path: Download the specified file.
/cloud?dir=...&command=ls&ext=.pgf: List files with the specified extension, e.g. ["Foods.pgf","Letter.pgf"].
/cloud?dir=...&command=rm&file=path: Remove the specified file.
/cloud?dir=...&command=link_directories&newdir=...: Combine server directores. This is used by GFSE to share grammars between multiple devices.

1.4 Examples

GF can be used interactively from the GF Shell. Some of the functionality availiable in the GF shell is also available via the GF web services API.

The GF Web Service API page describes the calls supported by the GF web service API. Below, we illustrate these calls by examples, and also show how to make these calls from JavaScript using the API defined in <a href="js/pgf_online.js" rel="nofollow">pgf_online.js</a>.

Note that pgf_online.js was initially developed with one particular web application in mind (the minibar), so the server API was incomplete. It was simplified and generalized in August 2011 to support the full API.

These boxes show what the calls look like in the JavaScript API defined in pgf_online.js. These boxes show the corresponding URLs sent to the PGF server. These boxes show the JSON (JavaScript data structures) returned by the PGF server. This will be passed to the callback function supplied in the call.

Initialization

// Select which server and grammars to use:

var server_options = {
                             grammars_url: "http://www.grammaticalframework.org/grammars/",
                             grammar_list: ["Foods.pgf"] // It's ok to skip this
                             }
var server = pgf_online(server_options);

Examples

// Get the list of available grammars

server.get_grammarlist(callback)
http://localhost:41296/grammars/grammars.cgi
["Foods.pgf","Phrasebook.pgf"]

// Select which grammar to use

server.switch_grammar("Foods.pgf")

// Get list of concrete languages and other grammar info

server.grammar_info(callback)
http://localhost:41296/grammars/Foods.pgf
       {"name":"Foods",
        "userLanguage":"FoodsEng",
        "startcat":"Comment",
        "categories":["Comment","Float","Int","Item","Kind","Quality","String"],
        "functions":["Boring","Cheese","Delicious","Expensive","Fish","Fresh",
                           "Italian","Mod","Pizza","Pred","That","These","This","Those","Very",
                           "Warm","Wine"],
        "languages":[{"name":"FoodsBul","languageCode":""},
                            {"name":"FoodsEng","languageCode":"en-US"},
                            {"name":"FoodsFin","languageCode":""},
                            {"name":"FoodsSwe","languageCode":"sv-SE"},
                            ...]
          }

// Get a random syntax tree

server.get_random({},callback)
http://localhost:41296/grammars/Foods.pgf?command=random
       [{"tree":"Pred (That Pizza) (Very Boring)"}]

// Linearize a syntax tree

server.linearize({tree:"Pred (That Pizza) (Very Boring)",to:"FoodsEng"},callback)
http://localhost:41296/grammars/Foods.pgf?command=linearize&amp;tree=Pred+(That+Pizza)+(Very+Boring)&amp;to=FoodsEng
       [{"to":"FoodsEng","text":"that pizza is very boring"}]
       server.linearize({tree:"Pred (That Pizza) (Very Boring)"},callback)

 http://localhost:41296/grammars/Foods.pgf?command=linearize&amp;tree=Pred+(That+Pizza)+(Very+Boring)
     [{"to":"FoodsBul","text":"онази пица е много еднообразна"},
     {"to":"FoodsEng","text":"that pizza is very boring"},
     {"to":"FoodsFin","text":"tuo pizza on erittäin tylsä"},
     {"to":"FoodsSwe","text":"den där pizzan är mycket tråkig"},
     ...
     ]

// Parse a string

server.parse({from:"FoodsEng",input:"that pizza is very boring"},callback)
http://localhost:41296/grammars/Foods.pgf?command=parse&amp;input=that+p...
       [{"from":"FoodsEng",
         "brackets":{"cat":"Comment","fid":10,"index":0,
         "children":[{"cat":"Item","fid":7,"index":0,
         "children":[{"token":"that"},{"cat":"Kind","fid":6,"index":0,
         "children":[{"token":"pizza"}]}]},    
        {"token":"is"},{"cat":"Quality","fid":9,"index":0,
          "children":[{"token":"very"},{"cat":"Quality","fid":8,"index":0,
          "children":[{"token":"boring"}]}]}]},
          "trees":["Pred (That Pizza) (Very Boring)"]}]

// Translate to all available languages

server.translate({from:"FoodsEng",input:"that pizza is very boring"},callback)
...

// Translate to one language

server.translate({input:"that pizza is very boring", from:"FoodsEng", to:"FoodsSwe"}, callback)
http://localhost:41296/grammars/Foods.pgf?command=translate&amp;input=th...
      [{"from":"FoodsEng",
        "brackets":{"cat":"Comment","fid":10,"index":0,
        "children":[{"cat":"Item","fid":7,"index":0,
        "children":[{"token":"that"},{"cat":"Kind","fid":6,"index":0,
        "children":  [{"token":"pizza"}]}]},{"token":"is"},{"cat":"Quality","fid":9,"index":0,
        "children":[{"token":"very"},{"cat":"Quality","fid":8,"index":0,"children":[{"token":"boring"}]}]}]},
        "translations":
        [{"tree":"Pred (That Pizza) (Very Boring)",
          "linearizations":
           [{"to":"FoodsSwe",
              "text":"den där pizzan är mycket tråkig"}]}]}]

// Get completions (what words could come next)

server.complete({from:"FoodsEng",input:"that pizza is very "},callback)
http://localhost:41296/grammars/Foods.pgf?command=complete&amp;input=tha...
      [{"from":"FoodsEng", "brackets":{"cat":"_","fid":0,"index":0,
        "children":[{"cat":"Item","fid":7,"index":0,
        "children":[{"token":"that"},{"cat":"Kind","fid":6,"index":0,
        "children":[{"token":"pizza"}]}]},{"token":"is"},{"token":"very"}]},
        "completions":["boring","delicious","expensive","fresh","Italian","very","warm"],
        "text":""}]

// Get info about a category in the abstract syntax

server.browse({id:"Kind"},callback)
http://localhost:41296/grammars/Foods.pgf?command=browse&amp;id=Kind&amp...
      {"def":"cat Kind", "producers":["Cheese","Fish","Mod","Pizza","Wine"],
       "consumers":["Mod","That","These","This","Those"]}

// Get info about a function in the abstract syntax

server.browse({id:"This"},callback)
http://localhost:41296/grammars/Foods.pgf?command=browse&amp;id=This&amp...
       {"def":"fun This : Kind -&gt; Item","producers":[],"consumers":[]}

// Get info about all categories and functions in the abstract syntax

server.browse({},callback)
http://localhost:41296/grammars/Foods.pgf?command=browse&amp;format=json
       {"cats":{"Kind":{"def":"cat Kind",
             "producers":["Cheese","Fish","Mod","Pizza","Wine"],
             "consumers":["Mod","That","These","This","Those"]},
     ...},
         "funs":{"This":{"def":"fun This : Kind -&gt; Item","producers":[],"consumers":[]},
     ...}
     }

// Convert an abstract syntax tree to JSON

server.pgf_call("abstrjson",{tree:"Pred (That Pizza) (Very Boring)"},callback)

http://localhost:41296/grammars/Foods.pgf?command=abstrjson&amp;tree=Pred+(That+Pizza)+(Very+Boring)
       {"fun":"Pred","fid":4,
        "children":[{"fun":"That","fid":1,
          "children":[{"fun":"Pizza","fid":0}]},
         {"fun":"Very","fid":3,
          "children":[{"fun":"Boring","fid":2}]}]}