What does extension programming language mean?
There is increasing demand for customizable applications. As applications became more complex, customization with simple parameters became impossible: users now want to make configuration decisions at execution time; users also want to write macros and scripts to increase productivity. In response to these needs, there is an important trend nowadays to split complex systems in two parts: kernel and configuration.
The kernel implements the basic classes and objects of the system, and is usually written in a compiled, statically typed language, like C. The configuration part, usually written in an interpreted, flexible language, connects these classes and objects to give the final shape to the application.
Configuration languages come in several flavors, ranging from
a. simple languages for selecting preferences usually implemented as parameter lists in command lines or
b. as variable-value pairs read from configuration files (Windows .ini files, X11 resource files) or
c. as embedded languages, for extending applications with user defined functions based on primitives provided by the applications. Embedded languages can be quite powerful, being sometimes simplified variants of mainstream programming languages such as C.
Such configuration languages are also called extension languages, since they allow the extension of the basic kernel semantics with new, user defined capabilities. Lua is such a language.
What does powerful data description facilities mean?
Associative arrays are a powerful language construct for describing data used in an application.
Many algorithms are simplified to the point of triviality because the required data structures and algorithms for searching them are implicitly provided by the language.
Lua uses tabled for data description. Most typical data containers, like ordinary arrays, sets, bags, and symbol tables, can be directly implemented by tables.
Tables can also simulate records by simply using field names as indices. Lua supports this representation by providing
a.name as syntactic sugar for a["name"].Lua provides a number of interesting ways for creating a table. The simplest form is the expression {}, which returns a new empty table. A more descriptive way, which creates a table and initializes some fields, is shown below; the syntax is somewhat inspired in the BibTeX [13] database format:
window1 = {x = 200, y = 300, foreground = "blue"}
This command creates a table, initializes its fields x, y, and foreground, and assigns it to the variable window1. Note that tables need not be homogeneous; they can simultaneously store values of all types. A similar syntax can be used to create lists:
colors = {"blue", "yellow", "red", "green", "black"}
This statement is equivalent to: colors = {}
colors[1] = "blue"; colors[2] = "yellow"; colors[3] = "red"
colors[4] = "green"; colors[5] = "black"
Sometimes, more powerful construction facilities are needed. Instead of trying to provide everything, Lua provides a simple constructor mechanism. Constructors are written name{...}, which is just syntactic sugar for name({...}). Thus, with a constructor, a table is created, initialized, and passed as parameter to a function. This function can do whatever initialization is needed, such as (dynamic) type checking, initialization of absent fields, and auxiliary data structures update, even in the host program. Typically, the constructor function is pre-defined, in C or in Lua, and often configuration users are not aware that the constructor is a function; they simply write something like:
window1 = Window{ x = 200, y = 300, foreground = "blue" }
and think about ``windows'' and other high level abstractions. Thus, although Lua is dynamically typed, it provides user controlled type constructors. Because constructors are expressions, they can be nested to describe more complex structures in a declarative style, as in the code below:
d = dialog{
hbox{
button{ label = "ok" },
button{ label = "cancel" }
}
}
What makes extension languages different?
They only work embedded in a host client, called the host program. Moreover, the host program can usually provide domain-specific extensions to customize the embedded language for its own purposes, typically by providing higher level abstractions. For this, an embedded language has both a syntax for its own programs and an application program interface (API) for communicating with hosts. Unlike simpler configuration languages, which are used to supply parameter values and sequences of actions to hosts, there is a two-way communication between embedded languages and host programs.
What are the requirements of extension languages?
- extension languages need good data description facilities, since they are frequently used as configuration languages;
- extension languages should have a clear and simple syntax, because their main users are not professional programmers;
- extension languages should be small, and have a small implementation. Otherwise, the cost of adding the library to an application may be too high;
- extension languages are not for writing large pieces of software, with hundreds of thousands lines. Therefore, mechanisms for supporting programming-in-the large, like static type checking, information hiding, and exception handling, are not essential;
- finally, extension languages should also be extensible. Unlike conventional languages, extension languages are used in a very high abstraction level, adequate for interfacing with users in quite diverse domains.
In its design, the addition of many different features has been replaced by the creation of a few meta mechanisms that allow programmers to implement those features themselves. These meta mechanisms are: dynamic associative arrays, reflexive facilities, and fallbacks.
Dynamic associative arrays directly implement a multitude of data types, like ordinary arrays, records, sets, and bags. They also lever the data description power of the language, by means of constructors.
Reflexive facilities allow the creation of highly polymorphic parts. Persistence and multiple name spaces are examples of features not directly present in Lua, but that can be easily implemented in Lua itself using reflexive facilities.
Finally, although Lua has a fixed syntax, fallbacks can extend the meaning of many syntactical constructions. For instance, fallbacks can be used to implement different kinds of inheritance, a feature not present in Lua.
Where can we use Lua?Currently, Lua is being extensively used in production for several tasks, including user configuration, general-purpose data-entry, description of user interfaces, storage of structured graphical metafiles, and generic attribute configuration for finite element meshes.
It took me 1 Hour and 10 mins to write this up on Nov 20th 2007 (9:30 am - 10:40 am). Just after bi-weekly HDK meeting.
More Information
