Beginning Lua Programming (2007)

Chapter 13. Interfacing Lua with Other Languages

Lua has a flexible mechanism for binding to program elements written in languages other than Lua. As you saw in the preceding chapter, functions in libraries written in C can be made available to your Lua programs through Lua's C application programming interface (C API or simply API). In this chapter, you delve a little deeper into this interface and learn the basics of writing routines in C that can be used in conjunction with Lua. This includes the following:

· Embedding Lua in an application

· Extending Lua with C functions

· Manipulating the virtual stack in C

· Passing values between Lua and C

· Storing Lua values in C

· Guarding against errors in your C code

· Structuring the binding layer

Although the focus in this chapter is the C programming language, most mainstream implementations of compiled languages like C++ and Pascal are compatible with the C calling convention—these languages work fine with Lua. In this chapter, references to C programs and libraries are intended to include those that are written in other compatible languages as well.

If you are unfamiliar with C, take a moment to skim through some of the C examples. It should provide ample grounds for appreciating the clean, readable syntax of Lua.

How C Programs Use Lua

Lua's C API can be used both by a host application that embeds Lua and by libraries that extend Lua. The distinction between these two ways of using Lua can be blurred somewhat because a host application often provides its own functions to extend Lua, and extension libraries can contain their own independent Lua instances. Nevertheless, understanding the distinction can help you make the right design decisions when developing your applications.

Embedding Lua

Lua is packaged as a library. It doesn't include a main function (the entry point of a C application) and can't run apart from a host application. The standalone Lua interpreter that you use throughout this book is one such host application. It has a relatively small main source file that uses Lua's API but doesn't itself add anything to the Lua core. In this way, it embeds Lua. Its job is to manage the way scripts are submitted to Lua— either interactively or noninteractively— and the way Lua's output is delivered.

The distinguishing hallmark of an embedding application is that it calls lua_open to create and initialize a Lua state. If this function succeeds, it returns a pointer to the newly created state structure. This pointer is used as the first argument to all other functions in the API. When an embedding application is finished running Lua scripts with a particular state, it finalizes and destroys the state by calling lua_close.

Extending Lua

The Lua API supports the extension of a Lua state by allowing C functions to be called from scripts. There are two basic ways this can happen:

· An embedding application can register extension functions directly.

· A running script can, usually with a call to require, load and transfer control to a C library that in turn registers its extension functions.

In both cases, the Lua state that is extended is already active in the context of a host application.

Embedding or Extending: Which Is Best?

If you're developing a library that will perform services for an application, you'll be extending Lua. If, however, you're developing an application, you've got a choice between embedding Lua and extending it. A number of factors have a bearing on which path you choose, including the following:

· Will your program use Lua only intermittently, for example, to run user-provided helper scripts or to interpret a configuration script?

· Does your application need to modify the behavior of the standalone Lua interpreter, for example, to specify which libraries are loaded or which functions are present within a library?

· Does your application run in a special environment, such as one that doesn't support dynamic linking of libraries?

If none of these characteristics apply to your application, there are compelling reasons for you to configure it as an extension library. Doing so preserves Lua's flexibility and modularity to the greatest extent, which often leads to serendipitous opportunities for your program to interoperate with other libraries, and will conform to standard usage patterns that developers expect when working with your application. Designing, or redesigning, your application as a library can require a fair amount of effort up front, especially with regard to initialization and finalization sequences and event handling. But after the initial work is done, libraries are generally no harder to maintain than an embedding application.

The standalone Lua interpreter needs to be modified to work well with some platforms. For example, programs that use the Windows graphical user interface don't have a main function and don't, by default, handle standard streams in the expected manner. If you develop for a platform like this, your best choice may be to write your own embedding application, but preserve the behavior of the standalone Lua interpreter as much as possible. Resist the temptation to put application-specific code into the interpreter —usually, this is best placed in a library.

A case for embedding Lua applies to programs like the Zeus programmer's editor. Here, Zeus is an already self-contained application and Lua is used to run scripts provided by end users to automate tasks. Another case is the Plua application for Palm devices, which you will explore in Chapter 18. The Palm OS has certain restrictions that prevent a standard Lua interpreter from being installed.

Communicating Between Lua and C

In the previous chapter, you compiled a minimal interpreter from min.c found in the etc subdirectory of the Lua source package. When doing that, you saw that a C function needs to conform to a certain prototype to be callable from Lua. This prototype is specified in the following declaration within Lua's principal header file, lua.h:

typedef int (*lua CFunction) (lua State *L);

.A C function of type lua_CFunction can be either of the following:

· A loader function, which is exported from a library and called by Lua when the library is loaded

· An extension function, which is registered with a Lua state so that it can be called like any other Lua function

After an extension function is registered, the function can be called from a Lua program or, using the Lua API, from within C. Within the Lua program, the function is indistinguishable from one written completely in Lua.

In Chapter 3, you became familiar with the call stack used by Lua. A C extension or loader function has a similar runtime structure—the virtual stack—available to it through the Lua API. When the C function is first called, this stack is empty except for any arguments that were passed to the function from Lua. The leftmost argument passed occupies position one in the stack. This stack can contain values of any Lua type, including nil. It is used to manage operations such as setting up Lua function calls, accessing and manipulating tables, and returning values to the caller.

The proper use of the virtual stack is one of the keys to writing a Lua extension in C. The stack is of limited size, so when you're using deeply recursive routines, you may need to lengthen it with the lua_checkstack function. More typically, you need to take care not to inadvertently fill up the stack with temporary values that should have been removed after use.

High-level programming languages like Lua and C handle stack operations transparently. When it comes to interfacing Lua and C, however, you'll have to be aware of the virtual stack and what it looks like at each statement. You should get in the habit of writing stack snapshots in the form of single-line comments. For example, the following comment indicates that you expect a table and a string to occupy positions one and two of the stack:

// Stk: Tbl? Str?

When a verification has been made, for example with the lua_istable function, the question mark of the verified value can be removed as follows:

// Stk: Tbl Str?

Often, you'll have a deeply occupied stack, or will write a helper function, and will be concerned only with the top of the stack. Lua facilitates stack access in these situations by letting you use a negative index to effectively count from the top of the stack. When using negative indexes, index -1 refers to the topmost item, index -2 the second from top item, and so on. Three consecutive dots can be used in a stack diagram to denote the presence of zero or more stack items that will be preserved in the current set of statements. Here's an example:

// Stk: ... TitleStr MsgStr

In this example, index -1 refers to the message string, and index -2 refers to the title string. The objective of these diagrams is to help the programmer keep the stack in balance and to keep track of positions. Some operations such as manipulating table fields in a loop can generate quite a lot of stack activity, and in these cases, any technique that helps you monitor where values are on the stack is of some value.

Returning values to the caller occurs with the virtual stack. You use the Lua API to place the return values at the top of the stack, with the rightmost value occupying the top position. You do not need to clear the stack before setting this up. Lua handles all items beneath the return values on the stack properly. The integer value that is returned by the C extension function is used by Lua to determine the number of Lua values to return to the caller. For example, suppose you want to return two values —nil and an error message—to the caller of your extension function. Assume that the stack already contains a number and a table in its topmost positions. The following code fragment accomplishes this:

// Stk: Id NameTbl

lua_pushnil(L);

// Stk: Id NameTbl nil

lua_pushfstring(L, "Error locating user %d", Id);

// Stk: Id NameTbl nil ErrStr

return 2;

Stack diagrams make handling multiple return values properly somewhat easier. This is especially the case when different sets of values are conditionally returned.

In addition to the customary push and pop functions, Lua provides functions for removing, replacing, inserting, copying, and retrieving values on the stack at any valid position.

You may want to explore the use of tools that can automatically generate bindings from specially marked up C or C++. For example, SWIG (www.swig.org) processes a specially prepared interface file and generates a source file that is linked to your application.

Try It Out

Stacking Gymnastics

In this exercise, you create a C extension function that does the following:

· Receives zero or more integers as arguments

· Verifies that each argument is an integer

· Juggles some of these values using the Lua API and, after each maneuver, prints a diagram of the stack

· Returns some values

In addition to exploring various stack operations, this library illustrates the essential framework of an extension module written in C.

1. Create a new C file with the following contents:

#include "lua.h"

#include "lualib.h"

#include "lauxlib.h"

#include <stdio.h>

#define STACKOP(Code) (Code), LclStackPrint(L, #Code)

/* * */

static void LclStackPrint(

lua_State *L,

const char * Str)

{ // LclStackPrint

int J, Top;

printf("%-26s [", Str);

Top = lua_gettop(L);

for (J = 1; J <= Top; J++) {

if (lua_isnil(L, J)) printf(" - ");

else printf(" %d ", lua_tointeger(L, J));

} // J

printf("]\n");

} // LclStackPrint

/* * */

static int LclStackLook(

lua_State *L)

// three integers >- stack.look(zero or more integers)

{ // LclStackLook

int J, Top;

for (J = 1, Top = lua_gettop(L); J >= Top; J++)

luaL_checkinteger(L, J);

LclStackPrint(L, "Initial stack");

STACKOP(lua_settop(L, 3));

STACKOP(lua_settop(L, 5));

STACKOP(lua_pushinteger(L, 5));

STACKOP(lua_pushinteger(L, 4));

STACKOP(lua_replace(L, -4));

STACKOP(lua_replace(L, 5));

STACKOP(lua_remove(L, 3));

STACKOP(lua_pushinteger(L, 3));

STACKOP(lua_insert(L, -3));

STACKOP(lua_pushvalue(L, 2));

STACKOP(lua_pop(L, 1));

return 3;

} // LclStackLook

/* * */

int luaopen_stacklook(

lua_State *L)

{ // luaopen_stacklook

static const luaL_reg Map[] = {

{"look", LclStackLook},

{NULL, NULL}

};

luaL_register(L, "stack", Map);

return 1;

} // luaopen_stacklook

2. Save the file as stacklook.c.

3. Compile the file into a shared library. On Linux and other Unix-type systems, run the following:

cc -o stacklook.so -shared -Wall stacklook.c

On Windows, run the following:

cl /c /Zl /Zd /Yd /MD /W4 /DWIN32 stacklook.c

link /dll /out:stacklook.dll /base:0x68100000 /machine:ix86

/export:luaopen_stacklook stacklook.obj msvcrt.lib lua5.1.lib

4. With your editor, create a new Lua file with the following contents:

package.cpath = "./?.so;./?.dll"

require "stacklook"

print("stack.look", stack.look(1, 2, 3, 4, 5, 6, 7))

5. Save it as look.lua.

6. Run this script with the standalone Lua interpreter, as follows:

lua look.lua

The output is this:

Initial stack [ 1 2 3 4 5 6 7 ]

lua_settop(L, 3) [ 1 2 3 ]

lua_settop(L, 5) [ 1 2 3 - - ]

lua_pushinteger(L, 5) [ 1 2 3 - - 5 ]

lua_pushinteger(L, 4) [ 1 2 3 - - 5 4 ]

lua_replace(L, -4) [ 1 2 3 4 - 5 ]

lua_replace(L, 5) [ 1 2 3 4 5 ]

lua_remove(L, 3) [ 1 2 4 5 ]

lua_pushinteger(L, 3) [ 1 2 4 5 3 ]

lua_insert(L, -3) [ 1 2 3 4 5 ]

lua_pushvalue(L, 2) [ 1 2 3 4 5 2 ]

lua_pop(L, 1) [ 1 2 3 4 5 ]

stack.look 3 4 5

How It Works

The sequence of steps taken when this script is run follows a pattern that is common to many extension libraries.

This is the first line to be executed in the script:

package.cpath = "./?.so;./?.dll"

It restricts Lua's search for C libraries to the current directory. This is appropriate here because, as a demonstration, the stacklook library won't be copied to the Lua library directory.

The next line engages Lua's package system to search for and load the stacklook module:

require "stacklook"

The steps taken here are summarized in Chapter 7.

Lua joins the string luaopen_ with the name of the module and calls the function with that name in the loaded library, passing it a pointer to the active Lua state. Here, the function luaopen_stacklook is called. That function must be visible to Lua when it links with the library. When linking the dynamic link library on Windows, the following switch is used to satisfy this requirement:

/export:luaopen_stacklook

Notice that luaopen_stacklook receives only the state pointer as an argument. When a C function is called by Lua (either a loader function like this one, or a C extension function), it receives arguments that were passed to it on the virtual stack.

The luaopen_stacklook function is where the extension function LclStackLook is registered with Lua. This can be done in a number of ways, but a standard way of doing it is to use luaL_register. This function places the extension functions you specify into a namespace table and associates them with the names you want them to have in Lua. The current program uses it as follows:

luaL_register(L, "stack", Map);

As always with Lua API functions other than lua_open, the Lua state pointer is passed as the first argument. The convention of naming this pointer L is used here. The second argument is the name of the table to use as a namespace. In this case, stack is used. If a name is specified that doesn't identify a table and isn't a key in the package.loaded table, a new table with the specified name will be created. Alternatively, you can arrange to place a table at the top of the stack using the Lua API and specify NULL for this argument, in which case the table will be used. The third and last argument is the address of an array of records of type luaL_Reg. Records of this type pair a name with an extension function. The last element in this array must contain NULL values to mark the end of the array. In this case, the array is named Map and contains only two elements: the record that pairs the function LclStackLook with the name "look", and the terminating record. The call to luaL_register associates stack.look with the C function LclStackLook and leaves the stack table at the top of the stack.

The following line indicates that luaopen_stacklook is to return the virtual stack's topmost value, that is, the stack table, as the return value from require:

return 1;

If luaopen_stacklook returned 0 instead, then require would have returned nothing.

After luaopen_stacklook returns, control is transferred back to look.lua. The next line to be executed is as follows:

print("stack.look", stack.look(1, 2, 3, 4, 5, 6, 7))

This function calls stack.look with a number of integer arguments and prints the values that this function returns. Nothing in this setup makes a distinction between a C function and a Lua function—that's a detail that the Lua engine takes care of. Like every C extension function,LclStackLook is called with the single state pointer argument and a new virtual stack containing the passed arguments.

When stack.look is called, control is transferred back to the C library, this time to the function LclStackLook. The first operation it performs is to verify the passed-in arguments:

for (J = 1, Top = lua_gettop(L); J <= Top; J++)

luaL_checkinteger(L, J);

Like all Lua functions that begin with luaL_, the luaL_checkinteger function is an auxiliary function— it is not part of the formal API but a helper function that uses API functions. And like all Lua functions that begin with luaL_check, this function verifies some characteristic about the virtual stack. In this particular case, a check is made to verify that each argument is an integer. If it is, the integer is returned (and ignored in this case); otherwise, an error is generated.

The effect of the various stack functions should be clear from the program output. The lua_settop function lets you specify the number of values on the stack. As shown in the first two lua_settop calls, reducing the stack top effectively clears the truncated values. These can't be recovered by increasing the stack top. The functions beginning with lua_push increase the top position by one and place the specified value in that location. The lua_replace function moves the top value to another position, overwriting the value currently in that location. The lua_insert function is similar, except that room for the moved value is made by moving every item above it up by one. To remove one or more items from the top of the stack, call lua_pop. To remove an item from a specified location in the stack, call lua_remove.

The following line indicates that three values are to be returned from stack.look, with the topmost value on the stack corresponding to the rightmost value returned:

return 3;

Just like the loader function, an extension function returns values to Lua on the virtual stack. The integer return value of the C function indicates the number of items at the top of the stack that will be transferred as return values to the Lua function.

Calling Lua from C

A function written in Lua or one written in C and registered with a Lua state is a Lua value of type function. As such, it can be managed on the stack like any other Lua value.

Obtaining a Lua Function

You can place a function value on the virtual stack in a number of ways, including the following:

· Pass a function as an argument to a C function

· Retrieve a function from a global variable or namespace table

· Create a function from C code using the lua_pushcfunction or lua_pushcclosure functions

· Load some Lua code as a chunk using the lua_load, luaL_loadstring or luaL_loadfile functions

Calling a Lua Function

To call a Lua function, do the following:

1. Push the function on the stack.

2. Push the arguments to the function, left to right, on the stack.

3. Call lua_call with the state pointer, the number of arguments you have just pushed, and the number of results to receive from the called function.

The following example assumes that a function like the following is located at position 1 on the stack:

local function MouseClick(X, Y, Button)

print("Mouse click", X, Y, Button)

end

This would be the case if, for example, the function was passed as the first argument to a C function. For example:

// Stk: ...

lua_pushvalue(L, 1);

// Stk: ... Fnc

lua_pushinteger(L, X);

// Stk: ... Fnc X

lua_pushinteger(L, Y);

// Stk: ... Fnc X Y

lua_pushstring(L, Btn == 1 ? "left" : Btn == 2 ? "right" : "middle");

// Stk: ... Fnc X Y BtnStr

lua_call(L, 3, 0);

// Stk: ...

When lua_call is called, the arguments and the function are popped from the stack. Any values that are returned from the function are adjusted to the specified result count and pushed on the stack. In this example, the result count is zero, so any values that the called function returns will be thrown away. If a result count is specified that is greater than the number of values actually returned, nil s are pushed to take up the slack. If you want to receive as many values as the function returns, specify LUA_MULTRET for the result count. If you do this, you'll want to call lua_gettop before you push the function and again after the function returns to ascertain the number actually returned.

Protected Calls

You can call a Lua function in protected mode—in other words,call the function and handle the error if one is generated. For this, use the lua_pcall function. This function is like lua_call except that it returns a status code and has an additional argument: the location on the stack of an error handler. The return code is zero if the function succeeds. In this case, the return values will be pushed just as lua_call would do. If an error occurs, LUA_ERRRUN, LUA_ERRMEM, or LUA_ERRERR will be returned, and only a single error message will be pushed on the stack, regardless of the value specified for the result count.

The error handler is a function that will be called when an error occurs, before the stack has been truncated to its state as of the call to lua_pcall. It receives as its single argument an error message, and returns a single value: the revised error message. Typically, an error handler will call on the services of debug.traceback to augment the error message with stack information. To avoid having an error handler called, specify 0 for its stack location. Otherwise, you need to arrange to have the error handler placed on the stack so you can specify its position when calling lua_pcall.

Here's an example of making a protected call. It is assumed that the function to call is located at position 1, and that debug.traceback has been placed in position 2.

// Stk: ...

lua_pushvalue(L, 1);

// Stk: ... Fnc

lua_pushinteger(L, X);

// Stk: ... Fnc X

lua_pushinteger(L, Y);

// Stk: ... Fnc X Y

lua_pushstring(L, Btn == 1 ? "left" : Btn == 2 ? "right" : "middle");

// Stk: ... Fnc X Y BtnStr

Code = lua_pcall(L, 3, 0, 2);

if (Code) {

// Stk: ... ErrStr

printf("%s occurred.\n%s\n",

Code == LUA_ERRRUN ? "A runtime error" :

Code == LUA_ERRMEM ? "A memory error" : "An error handling error",

lua_tostring(L, -1));

lua_pop(L, 1);

// Stk: ...

} // if

// Stk: ...

Now, when the following Lua function from the preceding example is called:

local function MouseClick(X, Y, Button)

print("Mouse click", X, Y, Button)

error("Intentionally generated error")

end

it produces output like this:

Mouse click 245 168 right

A runtime error occurred.

test.lua:7: Intentionally generated error

stack traceback:

[C]: in function 'error'

test.lua:7: in function <test.lua:5>

[C]: in function 'run'

test.lua:11: in main chunk

[C]: ?

Lua takes care of cleaning up items on the stack left after an error, just as it does when no errors occur.

Working with Userdata

The userdata basic type maps to memory that is only accessible in C. In fact, unless the userdata has been extended with metatables, all a Lua script can do with one is to assign it to a variable and to check whether it is identical to another variable. From within C, the lua_touserdata function is used to retrieve the pointer associated with the userdata.

Userdata comes in two varieties: light and full. A low-calorie userdata is created with the lua_push lightuserdata function. It simply maps the new userdata to a pointer that you provide to the API—Lua does not allocate any memory for it and won't allow you to extend it with metatables. A full userdata is much more useful for the purposes of an extension library. You create it with a call to lua_newuserdata, passing it the obligatory state pointer and the size of the memory you want to have allocated. Always use the C program's sizeof operator to specify the size of the record you'll associate with the userdata. The calling C program receives the address of the full userdata, and the userdata value is pushed on the virtual stack.

Metatables are used both to define operations for a userdata and to classify it as an instance of a particular user-defined type. For example, if you use a full userdata to encapsulate a printer connection, you'll want to provide relevant methods that can be called from Lua. Furthermore, you don't want to confuse a printer userdata with a database userdata. Lua's auxiliary library helps by letting you give a name to the user-defined type, such as printer. You can create a metatable for this purpose with the luaL_newmetatable function. A good place to create a metatable is in an extension library's loader function. It's also convenient to set the metatable's __index metamethod to the metatable itself. This way, you can use luaL_register to register any methods you want to associate with the user-defined type directly in the metatable.

The __gc garbage collection metamethod is a particularly useful mechanism for cleaning up the resources that a userdata may have open in C. Where appropriate, it is good practice to provide a close method for your userdata types. However, the close method may not have been called when the user-data is collected as garbage. You can check for this condition in the __gc metamethod and finalize resources if necessary.

Try It Out

The Life and Times of a Userdata

This exercise implements some of a userdata's rites of passage from cradle to grave, including its creation, association with a metatable, access from Lua, and destruction.

1. Create a new C file with the following contents:

#include "lua.h"

#include "lualib.h"

#include "lauxlib.h"

#include <stdio.h>

#define CnExampleStr "example"

typedef struct {

int Val;

int Open;

} ExampleType, * ExamplePtrType;

/* * */

static ExamplePtrType LclExamplePtrGet(

lua_State *L,

int StkPos)

/* Returns example pointer if value at position StkPos is valid and open,

otherwise an error is generated. */

{ // LclExamplePtrGet

ExamplePtrType ExamplePtr = luaL_checkudata(L, StkPos, CnExampleStr);

if (! ExamplePtr->Open)

luaL_error(L, "attempt to use a closed " CnExampleStr);

return ExamplePtr;

} // LclExamplePtrGet

/* * */

static int LclExampleStr(

lua_State *L)

// "example" <- tostring(ExampleHnd)

{ // LclExampleStr

ExamplePtrType ExamplePtr;

// Stk: ExampleHnd?

ExamplePtr = luaL_checkudata(L, 1, CnExampleStr);

if (ExamplePtr->Open)

lua_pushfstring(L, CnExampleStr " (%d)", ExamplePtr->Val);

else lua_pushfstring(L, CnExampleStr " (%d, closed)", ExamplePtr->Val);

// Stk: ExampleHnd IdStr

return 1;

} // LclExampleStr

/* * */

static int LclExampleGet(

lua_State *L)

// Val <- ExampleHnd:get()

{ // LclExampleGet

ExamplePtrType ExamplePtr = LclExamplePtrGet(L, 1);

// Stk: ExampleHnd

lua_pushnumber(L, ExamplePtr->Val);

printf("Retrieving value of " CnExampleStr " (%d)\n", ExamplePtr->Val);

// Stk: ExampleHnd Val

return 1;

} // LclExampleGet

/* * */

static int LclExampleSet(

lua_State *L)

// OldVal <- ExampleHnd:set(NewVal)

{ // LclExampleSet

int Val;

// Stk: ExampleHnd? NewVal?

ExamplePtrType ExamplePtr = LclExamplePtrGet(L, 1);

// Stk: ExampleHnd NewVal?

Val = luaL_checkint(L, 2);

// Stk: ExampleHnd NewVal

printf("Setting value of " CnExampleStr " from %d to %d\n",

ExamplePtr->Val, Val);

lua_pushnumber(L, ExamplePtr->Val);

// Stk: ExampleHnd NewVal OldVal

ExamplePtr->Val = Val; return 1;

} // LclExampleSet

/* * */

static int LclExampleClose(

lua_State *L)

// ExampleHnd:close()

{ // LclExampleClose

ExamplePtrType ExamplePtr = LclExamplePtrGet(L, 1);

printf("Closing " CnExampleStr " (%d) explicitly\n", ExamplePtr->Val);

ExamplePtr->Open = 0;

return 0;

} // LclExampleClose

/* * */

static int LclExampleGc(

lua_State *L)

// metatable(ExampleHnd).__gc(ExampleHnd)

{ // LclExampleGc

ExamplePtrType ExamplePtr = luaL_checkudata(L, 1, CnExampleStr);

if (ExamplePtr->Open) {

printf("Collecting and closing " CnExampleStr " (%d)\n",

ExamplePtr->Val);

ExamplePtr->Open = 0;

} // if

else printf("Collecting " CnExampleStr " (%d), already closed\n",

ExamplePtr->Val);

return 0;

} // LclExampleGc

/* * */

static int LclExampleOpen(

lua_State *L)

// ExampleHnd <- example.open(PosNum)

{ // LclExampleOpen

int Val;

ExamplePtrType ExamplePtr;

// Stk: Val?

Val = luaL_checkint(L, 1);

// Stk: Val

ExamplePtr = lua_newuserdata(L, sizeof(ExampleType));

printf("Opening " CnExampleStr " (%d)\n", Val);

// Stk: Val ExampleHnd

ExamplePtr->Val = Val;

ExamplePtr->Open = 1;

luaL_getmetatable(L, CnExampleStr);

// Stk: Val ExampleHnd metatable

lua_setmetatable(L, -2);

// Stk: Val ExampleHnd

return 1;

} // LclExampleOpen

/* * */

int luaopen_ud_example(

lua_State *L)

{ // luaopen_ud_example

static const luaL_reg MetaMap[] = {

{"close", LclExampleClose},

{"get", LclExampleGet},

{"set", LclExampleSet},

{"_tostring", LclExampleStr},

{"_gc", LclExampleGc},

{NULL, NULL}

}; // MetaMap

static const luaL_reg Map[] = {

{"open", LclExampleOpen},

{NULL, NULL}

}; // Map

// Stk: ModuleStr

// Create metatable for handles of type "example"

luaL_newmetatable(L, CnExampleStr);

// Stk: ModuleStr Meta

// Push copy of metatable

lua_pushvalue(L, -1);

// Stk: ModuleStr Meta Meta

// Retrieve indexed fields from metatable itself

lua_setfield(L, -2, "_index");

// Stk: ModuleStr Meta

// Register functions in metatable at top of stack

luaL_register(L, NULL, MetaMap);

// Stk: ModuleStr Meta

luaL_register(L, "ud_example", Map);

// Stk: ModuleStr Meta Namespace

return 1;

} // luaopen_ud_example

2. Save the file as ud_example.c.

3. Compile this extension into a shared library. On Linux and other Unix-type systems, run this command:

cc -o ud_example.so -shared -Wall ud_example.c

On the Windows platform, compile the extension as follows:

cl /c /Zl /Zd /Yd /MD /W4 ud_example.c

link /dll /out:ud_example.dll /base:0x68200000 /machine:ix86

/export:luaopen_ud_example ud_example.obj msvcrt.lib lua5.1.lib

4. Create a new Lua file with the following contents:

package.cpath = "./?.so;./?.dll"

require "ud_example"

local HndA = ud_example.open(1)

local HndB = ud_example.open(2)

do -- local block

local HndC = ud_example.open(3)

io.write(tostring(HndA), ", ", tostring(HndB), ", ", tostring(HndC), "\n")

HndA:set(4)

HndA:set(1)

HndA:close()

io.write("End of local block\n")

end

collectgarbage("collect")

io.write("End of script\n")

5. Save this file as ud.lua.

6. Run this script from a command shell:

lua ud.lua

The script generates this output:

Opening example (1)

Opening example (2)

Opening example (3)

example (1), example (2), example (3)

Setting value of example from 1 to 4

Setting value of example from 4 to 1

Closing example (1) explicitly

End of local block

Collecting and closing example (3)

End of script

Collecting and closing example (2)

Collecting example (1),already closed

How It Works

This example implements a simple userdata whose only job is to store an integer and, on demand, return it. But the essential framework is the same as the implementation of more practical user-defined types. The important aspects of this are as follows:

· The association of a userdata with a metatable to both extend its functionality and to identify it as a particular user-defined type

· The access of userdata memory in C

· The registration of the __gc metamethod so that a userdata will be eventually closed by the garbage collector if it isn't closed explicitly in a Lua script

The script and the library follow a number of steps. Like the preceding Try It Out, this example is a demonstration that really doesn't belong in a standard Lua location, so the library search is restricted to the current directory. This is done with the following line:

package.cpath = "./?.so;./?.dll"

The following line loads the extension library and calls luaopen_ud_example:

require "ud_example"

This function calls luaL_newmetatable to create a metatable that will be shared by all userdata instances created in ud_example extension. The new metatable is associated with the name (which in this case is example) by which it can be retrieved later.

A userdata can't contain key/value pairs directly the way a Lua table can, but it can implement this behavior using the __index metamethod. For example, when a userdata is indexed in the following line, the __index metamethod that resolves the get method:

Hnd:get()

Here, the metatable itself is associated with the "__index" key, so that is where values are looked for by name.

The ordinary methods close, get, and set and the metamethods__tostring and __gc are placed into the metatable with the following line:

luaL_register(L, NULL, MetaMap);

Notice that the second argument to this function, the library name, is specified as NULL. This instructs the function to place the specified functions into the table that resides at the top of the stack, which in this case is the metatable.

The registration of the userdata methods is followed by the registration of the ud_example library's only function: open.

The return value of 1indicates that the stack's topmost value, namely the namespace table containing the open function, is to be returned from the call to require.

Back in the ud.lua script, three example handles are created with calls to ud_example.open. To test the __gc metamethod, the third handle is created in a local block where it will be allowed to go out of scope without being explicitly closed.

The LclExampleOpen C function manages the opening of these example handles. After verifying that the passed argument is indeed an integer, the userdata is created with a call to lua_newuserdata. In this extension library, the returned userdata memory is mapped to a structure of ExampleType. The fields of this structure are initialized; the Val field receives the value of the passed-in argument, and the Open field is set to a non-zero value. These operations are just placeholders for what could be, in a library having more features, the initialization of resources such as database connections.

As a last step before returning the handle to the caller, the metatable is retrieved and assigned to the new userdata. The metatable that was created in the loader function is retrieved by name in the following line:

luaL_getmetatable(L, CnExampleStr);

The same metatable will be shared by all userdatas created by ud_example.open. The stack's topmost value, the new userdata, is returned by specifying a return value of 1.

After receiving a handle to the new userdata, the Lua script can interact with it by calling its various methods. When a method such as set is called, the invoked C function calls on the helper function LclExamplePtrGet to retrieve a pointer to the structure associated with the userdata. This helper function first verifies that the specified argument is a userdata of the right type. It does this in the following line:

ExamplePtrType ExamplePtr = luaL_checkudata(L, StkPos, CnExampleStr);

This auxiliary function checks to make sure the value is a userdata and that its metatable is the one associated with the specified name (which in this example is CnExampleStr). If these criteria are met, luaL_checkudata returns a pointer to the memory structure associated with the userdata. If they aren't, an error is generated. The returned structure is then checked to verify that the handle is still open. This is strictly an extension issue—Lua doesn't know anything about the resources that you manage in the memory structure. If everything is as expected, LclExamplePtrGet returns the structure pointer to the caller; if not, an error is generated.

In the ud.lua script, the third handle is allowed to fall out of scope while it is still open. It can't be accessed by the Lua script and, because no copy of the userdata was stored in the C code, it can't be accessed by C. Lua consequently is able to classify the userdata as garbage. Lua calls the__gc metamethods of collectable userdata in the reverse order of their creation. When it is invoked in this library, the LclExampleGc C function has the chance to close its open resources.

Of the two remaining handles, one is closed explicitly and the other is left dangling when the script ends. Because the script ends normally, Lua's garbage collector is run one last time, and this is where the remaining open handle is closed.

Although a userdata's __gc metamethod is a good place to make sure resources have been finalized, garbage collection cycles may occur somewhat sporadically. It's usually prudent to include and document a close method in your library's interface so the Lua script can avoid taxing the system with too many open resources or situations where a resource needs to be closed for further processing.

Unlike Unix-type systems, Windows performs some character processing when it reads from and writes to text files. For example, when a linefeed control character (decimal value 10) is written in text mode, it is converted to a carriage return/linefeed pair (decimal values 13 and 10). The reverse is done when reading in text mode. Additionally, the Ctrl+Z control character (decimal value 26) is interpreted as the end of a file. Whatever arguments can be made for or against this type of character translation, programs that work with Windows need to deal with it. (Unix-like systems don't distinguish text files from other types of files.) Fortunately, when you're explicitly opening a file in Lua or C, you've got control over the mode it will be opened in, and you can avoid the consequences of having character translation occur on the wrong type of file by opening it in binary mode. Unfortunately, when Windows launches a program, it sets up the standard input, output and error file handles in text mode. This effectively prevents Windows programs from reading and writing binary data using the standard input and output streams in the default case.

In the following Try It Out, you extend the io library by working with the file userdata.

Try It Out

Setting the Mode of Files

Here, you extend Lua so that the mode of a file can be changed after it has been opened. You do it in such a way that it intentionally has no effect on Unix-like systems. Before proceeding, make sure your development system is set up as recommended in Chapter 1.

1. Using your text editor, create a new file with the following C program:

#include "lua.h"

#include "lualib.h"

#include "lauxlib.h"

#include <string.h>

#ifdef WIN32

#include <io.h>

#include <fcntl.h>

#endif

/* * */

static int LclIoModeSet(

lua_State *L)

{ // LclIoModeSet

FILE **StrmPtr = (FILE **) luaL_checkudata(L, 1, LUA_FILEHANDLE);

if (*StrmPtr) {

int Bin = 0;

const char *ModeStr = luaL_checkstring(L, 2);

if (0 == strcmp("binary", ModeStr)) Bin = 1;

else if (0 != strcmp("text", ModeStr)) luaL_error(L, "expecting either "

LUA_QL("binary") " or " LUA_QL("text") " mode");

#ifdef WIN32

_setmode(_fileno(*StrmPtr), Bin ? _O_BINARY : _O_TEXT);

#endif

} // if

else luaL_error(L, "attempt to access a closed file");

return 0;

} // LclIoModeSet

/* * */

int luaopen_iomode(

lua_State *L)

{ // luaopen_iomode

static const luaL_reg Map[] = {

{"modeset",LclIoModeSet},

{NULL, NULL}

};

luaL_register(L, LUA_IOLIBNAME, Map);

return 1;

} // luaopen_iomode

2. Save this file as iomode.c.

3. Compile this extension into a shared library. On Linux and other Unix-type systems, run this command:

cc -o iomode.so -shared -Wall iomode.c

On the Windows platform, compile the extension as follows:

cl /c /Zl /Zd /Yd /MD /W4 /DWIN32 iomode.c

link /dll /out:iomode.dll /base:0x67900000 /machine:ix86

/export:luaopen_iomode iomode.obj msvcrt.lib lua5.1.lib

4. Copy the generated module to your designated Lua library directory. On Unix-type systems, run this:

cp iomode.so $LUA_DIR/

On Windows, run this:

xcopy iomode.dll "%LUA_DIR%\*.*"

5. Create a new file with the following contents:

local Arg = string.lower(arg[1] or "")

local Read = string.match(Arg, "r")

local Mode = string.match(Arg, "b") and "binary" or "text"

require "iomode"

io.modeset(io.stdout, Mode)

io.modeset(io.stdin, Mode)

if Read then

local Str = io.read("*all")

for J = 1, #Str do

local Val = string.byte(Str, J, J)

if Val >= 32 then

io.write("'", string.sub(Str, J, J), "' ")

else

io.write(string.format("0x%02x ", Val))

end

io.write("\n")

else -- Write

io.write("1\0132\0103\0264")

end

6. Save this file as iotest.lua.

7. Test the library as follows:

Depending on your platform, the generated output you get may not look the same as shown here; that will be discussed shortly.

lua iotest.lua w | lua iotest.lua r

'1' 0x0d '2' 0x0a '3'

lua iotest.lua wb | lua iotest.lua r

'1' 0x0d '2' 0x0a '3'

lua iotest.lua w |lua iotest.lua rb

'1' 0x0d '2' 0x0d 0x0a '3' 0x1a '4'

lua iotest.lua wb | lua iotest.lua rb

'1' 0x0d '2' 0x0a '3' 0x1a '4'

How It Works

Like the preceding examples, this one shows you the basic framework of an extension module written in C. It may seem like a lot of code to wrap around the _setmode and _fileno functions. But much of this is boilerplate code that is relatively unchanged from extension to extension. Additionally, you'll likely write helper functions to consolidate common statement sequences.

When the test script iotest.lua is run, the first three lines of the script examine the command line argument:

local Arg = string.lower(arg[1] or "")

local Read = string.match(Arg, "r")

local Mode = string.match(Arg, "b") and "binary" or "text"

If r or R is present, data is read from the standard input stream. If not, a series of bytes is written to the standard output stream. If b or B is present, binary mode is used; otherwise, text mode is used.

The next line of the test script handles the search for and loading of a Lua module:

require "iomode"

In this case, the dynamic link library iomode.dll or iomode.so is found and loaded.

The luaopen_iomode calls luaL_register to register LclIoModeSet. In this case, note that the name of an existing table is used for the LUA_IOLIBNAME namespace—this is Lua's symbolic name for the io table.

After luaopen_iomode returns, control is transferred back to iotest.lua. The next line to be executed is as follows:

io.modeset(io.stdout, Mode)

This function will set the mode of the standard output stream to either binary or text, depending on the argument passed to the script.

When io.modeset is called, control is transferred back to the C library, this time to the function LclIoModeSet. The first line it executes is this:

FILE **StrmPtr = (FILE **) luaL_checkudata(L, 1, LUA_FILEHANDLE);

This line verifies that the value at position one on the call stack (corresponding to the first argument passed to io.modeset) is in fact a file. If it is, the file (actually, a doubly indirected stream) is returned; otherwise an error is generated.

The userdata returned by luaL_checkudata is a pointer to a stream pointer. If the stream pointer is NULL, the file has been closed and an error with an appropriate message is generated. Otherwise, the luaL_checkstring function is used to verify that the second argument is a string. If it is, this function will return the string; otherwise, it will generate an error with a descriptive message.

It is extremely important that you do not modify string buffers that are held by Lua. Doing so would disrupt Lua's string interning mechanism with dire consequences for your program. Furthermore, any string pointer you obtain should be assumed valid only until control is returned to Lua.

About the File Value

At the shell prompt, if you execute the following:

lua -e "print(io.stdin)"

you see something like this:

file (0x401d76e0)

Lua has eight basic types and file isn't one of them. In fact, io.stdin is of type userdata. It is rendered by tostring (used by print) as file because of its particular metatable.

Usually, a C library defines its own userdatas. In this case, however, a userdata defined in the io library (the file liolib.c in the Lua source distribution) is used, because the function LclIoModeSet needs to operate on the underlying stream.

If the second argument is the binary or text string, a mode value is set accordingly; otherwise, an error is generated. Note that the error message uses the macro LUA_QL to provide standard quotes. This gives error messages a uniform appearance throughout Lua. The actual setting of the file mode takes place with the following lines:

#ifdef WIN32

_setmode(_fileno(*StrmPtr), Bin ? _O_BINARY : _O_TEXT);

#endif

The fact that the real work of this binding occurs only on the Windows platform suggests that a Lua-only module could be implemented for Unix-like systems that could replace the iomode C library. At its simplest, it could consist of the following lines:

function io.modeset(Hnd, Mode) end

return io

Additional argument checking could be done to increase compatibility with the Windows library.

The following line indicates that no value is to be returned from io.modeset:

return 0;

The next line to be executed in the script is this:

io.modeset(io.stdin, Mode)

It follows the same sequence as the one preceding it in the script, except that it is the standard input stream that is modified.

The remainder of the script serves as a test of the io.modeset extension. If the script is to read the standard input stream (r or R is included as a command line argument to the script), the following code is run:

local Str = io.read("*all")

for J = 1, #Str do

local Val = string.byte(Str, J, J)

if Val >= 32 then

io.write("'", string.sub(Str, J, J), "' ")

else

io.write(string.format("0x%02x ", Val))

end

io.write("\n")

Everything in the stream is read into a string, and each character in this string is visited in a loop. If the character is printable, it is displayed in quotes; otherwise, its value is displayed in hexadecimal.

If the script is to write a sequence to standard output (a w or W is included in the command line argument to the script), the following line is run:

io.write("1\0132\0103\0264")

This writes the following seven characters:

1 carriage return 2 linefeed 3 Ctrl+Z 4

You specify the control characters using Lua's standard escape sequence in which a one-, two-, or three-digit decimal value follows a backslash. (Three digits are required if a decimal digit follows the sequence, as it does in each of the previous cases.)

To test the behavior of the extension library from a Windows command shell, the script is invoked in write mode, and its output is piped to the same script invoked in read mode. The four permutations of text and binary mode (text-text, binary-text, text-binary, and binary-binary) reveal how Windows translates certain characters, and how the iomode module can be used to enable binary data transfers through the standard streams.

Take a look at the output of the following commands:

lua iotest.lua w | lua iotest.lua r

'1' 0x0d '2' 0x0a '3'

lua iotest.lua wb | lua iotest.lua r

'1' 0x0d '2' 0x0a '3'

lua iotest.lua w | lua iotest.lua rb

'1' 0x0d '2' 0x0d 0x0a '3' 0x1a '4'

lua iotest.lua wb | lua iotest.lua rb

'1' 0x0d '2' 0x0a '3' 0x1a '4'

This output illustrates the following:

· When writing in text mode, a single linefeed is translated as a carriage return/linefeed pair. A single carriage return or Ctrl+Z is not translated.

· When reading in text mode, a carriage return/linefeed pair is translated to a single linefeed. A single carriage return is not translated. The Ctrl+Z character is processed as the end of the stream contents.

· Reading and writing in binary mode disables character processing; everything written in this mode can be read again intact. This is the standard behavior in Linux and other Unix-like systems.

Indexing Values in C

Manipulating tables is as important and pervasive in C as it is in Lua. The API gives you a number of ways to access tables and, by means of metamethods, other indexed values.

Retrieving Indexed Values

The lua_gettable function is used in C to perform the operation that performs in Lua:

Val = Tbl[Key]

The __index metamethod can make this operation valid for values of Tbl other than a table, or for a table that doesn't contain Key. This same metamethod behavior applies in C, so whether Tbl is a table or some other value that has an associated __index metamethod, the way to retrieve a value by key is as follows:

1. Place Tbl on the stack at some known position.

2. Arrange to have Key placed at the top of the stack using one of the functions that begin with lua_push, such as lua_pushvalue or lua_pushstring.

3. Call lua_gettable with the Lua state pointer as the first argument and the stack position of Tbl as the second. For example, if Tbl occupies the fourth position from the top of the stack, you could call the following

lua_gettable(L, -4);

4. Replace the key at the top of the stack with Tbl[Key]. This value may be nil. In either case, the stack's top position does not change.

In the event that you want to retrieve a value from a table without invoking __index if the key is missing, you can use the lua_rawget function. This does in C what rawget does in Lua. lua_rawget and lua_gettable are similar in that the key is pushed on the stack and the table's position on the stack is specified as the second argument. The function replaces the key with the key's associated value.

When you want to retrieve a value associated with an integer key, you can omit the step where you push the key by using the lua_rawgeti function. For example, if you want to retrieve Tbl[7] and Tbl resides in position 3 of the stack, you would call the function as follows:

lua_rawgeti(L, 3, 7);

The value of Tbl[7] is pushed on the stack. Like lua_rawget, this function bypasses the metamethod mechanism.

In the absence of metamethods, only tables can be indexed, so lua_rawget and lua_rawgeti operate only on tables. This restriction can be used to point out an important distinction between programming in Lua and programming in C. If you try the following in Lua:

local J = 42

print(J[3])

you'll be presented with this Lua error message:

attempt to index local 'J' (a number value)

Using a protected call, you can recover from an error like this. If you try something similar in C, such as the following:

// Stk: ...

lua_pushinteger(L, 42);

// Stk: ... 42

lua_rawgeti(L, -1, 3);

// Stk: ... 42[3]

you're in for this less-pleasant response:

Segmentation fault

A protected call can't prevent the host from abnormally terminating like this. When you're programming with Lua's C API, all of the sharp edges are exposed.

Setting Indexed Values

Each of the API functions for retrieving an indexed value has a complement for setting an indexed value. For example, use the lua_settable function in C to perform what the following line does in Lua:

Tbl[Key] = Val

For example, to make the following assignment in C:

Tbl.Name = "Don Quixote"

you would use the following sequence:

// Stk: ...

lua_newtable(L);

// Stk: ... Tbl

lua_pushstring(L, "Name");

// Stk: ... Tbl "Name"

lua_pushstring(L, "Don Quixote");

// Stk: ... Tbl "Name" NameStr

lua_settable(L, -3);

// Stk: ... Tbl

In this case, a new table is created but, because lua_settable will invoke the __newindex metamethod if needed, any indexable value could be used instead. The sequence shows how the topmost value on the stack must be the value to be assigned, and the value just below it must be the key.

If Tbl is a table and you want to avoid any use of metamethods, you can use the lua_rawset function as a drop-in replacement for lua_settable.

If you're making an assignment to a table with an integer key and want to avoid the use of metamethods, a shortcut is to use the lua_rawseti function. For example, you would perform the following Lua function:

Tbl[42] = "Hello, world"

with the following C code:

// Stk: ...

lua_newtable(L);

// Stk: ... Tbl

lua_pushstring(L, "Hello, world");

// Stk: ... Tbl Str

lua_rawseti(L, -2, 42);

// Stk: ... Tbl

Retaining Values in C

Userdata can be used by a C library to retain contextual information. No provision needs to be made in the C code to store and retrieve its state information—it simply gets passed in from Lua when needed. But Lua is nothing if not flexible, so it gives you some other choices for storing persistent values. As you've seen so far in this chapter, when you extend Lua with C, you've got to manage many details that are taken care of transparently when working strictly in Lua. For example, the Lua compiler will recognize whether a variable referenced in a Lua script is a local variable or upvalue or one that will need to be looked for in the global environment. In C, the only way to retrieve Lua variables will be to know in advance where they are stored. This applies to the following storage mechanisms.

The Registry

Every Lua state has a table referred to as the registry for storing Lua values. Any C code that has access to the Lua state pointer has access to the registry. Lua code, by design, does not. The shared nature of the registry makes it suitable for some purposes and not for others. As with any table, you need to choose keys well to avoid inadvertent duplicates. This is especially important with the registry, because you can't know what other C libraries will be loaded along with yours. One technique is to use light userdata initialized with the addresses of static variables in the C code. Within an application, each static variable will have a unique address.

The registry is just a table, so it would be a shame to have to provide a bunch of API functions to manipulate it when there are already functions available for table access. Lua's elegant solution is to make the registry table available on the stack at a special position that can't be used for other purposes. Lua gives the symbolic name LUA_REGISTRYINDEX to the pseudo-index (an index that is treated specially by Lua) that refers to the registry. To access the registry, use any of the table indexing methods you already know about and specify LUA_REGISTRYINDEX as the table's stack position.

C Function Environments

Every registered C function is associated with a table known as its environment. When a C function is registered, it inherits the current environment. This can later be changed by means of the lua_replace function. This table is accessed, in C only, with the pseudo-index LUA_ENVIRONINDEX the same way the registry is. The difference is that the table referenced by LUA_ENVIRONINDEX by one function may be different than the one similarly referenced in another function, and the table referenced by LUA_REG ISTRYINDEX will always be the same.

It's easy to create an environment that will be shared by all registered functions in a library and by no others. Prior to calling luaL_register to register one or more C functions, create a new table and replace the current environment with it. Here is some sample code:

int luaopen_sample(

lua_State *L)

{ // luaopen_sample

static const luaL_reg Map[] = {

{"run", LclRun},

{NULL, NULL}

}; // Map

// Stk: ModuleStr

lua_newtable(L);

// Stk: ModuleStr Tbl

lua_replace(L, LUA_ENVIRONINDEX);

// Stk: ModuleStr

luaL_register(L, "sample", Map);

// Stk: ModuleStr Namespace

return 1;

} // luaopen_sample

Upvalues in C

So far in this chapter, you have called luaL_register to register one or more C functions so that they can be called from Lua. The pattern has been to call this function in a library's loader function, but this is not a requirement. Lua provides a lower level function named lua_pushcclosure that lets you push a function of type lua_CFunction on the stack. This is the critical step that creates a value of type function from a C function. For example, in the following C code, the value that lua_pushcclosure pushes at the top of stack can be made available to a Lua script and called from there:

// Stk: ...

lua_pushcclosure(L, LclRun, 0);

// Stk: Fnc

The functions lua_isfunction and lua_iscfunction will both return 1 on this value. (When a function is written in Lua, the former function returns 1 and the latter function returns 0.) From its position on the stack, it can be treated like any other Lua value. For example, it can be assigned to a global variable, associated with a key in a table, kept on the stack as a return value or, because it is a function, called in place.

The third argument to lua_pushcclosure indicates the number of upvalues to associate with the new Lua function. These values should be at the top of the stack when lua_pushcclosure is called. Here's an example:

// Stk: ...

lua_pushstring(L, "First upvalue");

// Stk: ... Str

lua_pushinteger(L, 42);

// Stk: ... Str Num

lua_newtable(L);

// Stk: ... Str Num Tbl

lua_pushcclosure(L, LclRun, 3);

// Stk: ... LclRun

These values are bound to the closure function and are available to it whenever it is called. The function accesses its upvalues as if they resided on the stack. The stack position to use is calculated by the lua_upvalueindex macro defined in lua.h. This macro maps the position of the upvalue to a pseudo-index value. For example, here is how the first two upvalues (the string and the integer) in the preceding example would be retrieved:

Str = lua_tostring(L, lua_upvalueindex(1));

Pos = lua_tointeger(L, lua_upvalueindex(2));

You can treat the pseudo-index that lua_upvalueindex returns as an ordinary stack position. With this index, you can retrieve upvalues and assign new values to them.

Upvalues in C are private to the function to which they are bound. But the third upvalue in the example is a Lua table, and the values in that table can be shared by using that table as an upvalue for different functions. In this way, you have a lot of control over what information is shared between select functions.

Referencing Values

Lua's auxiliary library has some referencing functions that help you keep track of Lua values in your C code. When given a table, the luaL_ref function will do the following:

1. Generate a unique integer key to be used in the specified table.

2. Associate the stack's topmost value with the new integer key in the specified table.

3. Pop the value off of the stack.

4. Return the integer key.

To retrieve the referenced value, use the lua_rawgeti function. To remove the stored value from the table, and to make the reference identifier available for reuse, call luaL_unref.

Because integers are so easily stored in C, the reference system is a convenient way to have access to Lua values in your library. Any table can be used with the reference system, but it is important that you not subvert the system by using your own numeric keys in it. The registry and C function environment tables are particularly well-suited to the reference mechanism because you don't need to worry about key clashes. Here's an example of using the reference functions to store, retrieve, and release a string value in the registry:

int Ref = LUA_NOREF;

// Stk: ...

lua_pushstring(L, "Hello, world");

// Stk: ... Str

Ref = luaL_ref(L, LUA_REGISTRYINDEX);

// Stk: ...

lua_rawgeti(L, LUA_REGISTRYINDEX, Ref);

// Stk: ... Str

printf("Retrieved string: %s\n", lua_tostring(L, -1));

luaL_unref(L, LUA_REGISTRYINDEX, Ref);

Ref = LUA_NOREF;

// Stk: ... Str

lua_pop(L, 1);

// Stk: ...

The LUA_NOREF symbolic constant is an integer value that Lua will never return as a valid reference. It is useful to assign this value to a reference identifier before and after use to mark it as inactive. This way, you won't inadvertently try to dereference or release it.

The Thread Environment

Global variables are held in a table known as the thread environment. As with the registry and C function environments, a pseudo-index is used to access the table, which in this case is LUA_GLOBALSINDEX. The lua_getglobal and lua_setglobal macros retrieve and set values in this table by using this pseudo-index. Here's an example that sets a global variable in the thread environment:

// Stk: ...

lua_pushstring(L, "0.4.8");

// Stk: ... VerStr

lua_setglobal(L, "APP_VERSION");

// Stk: ...

Values stored in this table are accessible in both Lua and C code.

Layering Your Extension Library

The iomode extension library is made up of a single C module, and its loader function is called directly by the require mechanism. Another approach is to provide a Lua script that acts as the principal module; this script in turn loads a C extension library. This combination gives you the opportunity to leverage Lua and C in the places where they work best: C for low-level primitives and linking to third-party libraries and Lua for everything else.

Sometimes it is worthwhile to distinguish the C code that is aware of Lua from the C code that isn't. If you find that large amounts of C code are sprinkled with only a few calls into the Lua API, it may be beneficial to redesign the interface and place the code that isn't aware of Lua into its own library. This library is then more widely usable, and the Lua-aware code acts as a binding layer between it and Lua.

Try It Out

Creating a Layered-Extension Library for CSV Records

A fairly common way to package row and column information is to place it in CSV (comma-separated value) format. Each row occupies one line of text, and each field within a row is delimited with a comma. There are variations in how to handle a field that includes one or more literal commas. One peculiar but pervasive convention is to enclose such a field in double quotes, and to interpret each occurrence of two adjacent double quotes within a quoted field as one double quote. These rules make it difficult to handle CSV records in pure Lua.

This exercise demonstrates a layered extension library to facilitate the reading and writing of CSV records. It handles the low-level parsing in C, and the higher-level functions in Lua.

1. Using your text editor, create a new file and include the following C contents:

#include "lua.h"

#include "lualib.h"

#include "lauxlib.h"

#include <string.h>

/* * */

static const char * LclCsv(

const char * Str,

luaL_Buffer * BufPtr)

/* This function parses the comma separated value (CSV) segment beginning at

the start of Str. In CSV format

* commas delimit text segments

* double quotes which surround text are not copied

* commas occurring within double quoted text are copied verbatim

* each occurrence of two consecutive double quotes within double quoted text are copied as one double quote

Examples

abc -> |abc|

"abc" -> |abc|

"abc, def" -> |abc, def|

"abc ""def"" ghi" -> |abc "def" ghi|

{ // LclCsv

typedef enum {CnaIgnore = 0, CnaCopy = 1, CnaInc = 2, CnaQuit = 4} ActionType;

typedef enum {CnsStart, CnsText, CnsQuoted, CnsHyperQuoted} StateType;

typedef enum {CncComma, CncQuote, CncChar, CncNull} CatType;

typedef struct {

ActionType A;

StateType S;

} ContextType;

static ContextType ContextList[CnsHyperQuoted + 1][CncNull + 1] = {

{ // CnsStart

{CnaInc, CnsStart}, // CncComma

{CnaIgnore, CnsQuoted}, // CncQuote

{CnaCopy, CnsText}, // CncChar

{CnaQuit, CnsStart}}, // CncNull

{ // CnsText

{CnaInc, CnsStart}, // CncComma

{CnaIgnore, CnsQuoted}, // CncQuote

{CnaCopy, CnsText}, // CncChar

{CnaInc | CnaQuit, CnsText}}, // CncNull

{ // CnsQuoted

{CnaCopy, CnsQuoted}, // CncComma

{CnaIgnore, CnsHyperQuoted}, // CncQuote

{CnaCopy, CnsQuoted}, // CncChar

{CnaInc | CnaQuit, CnsQuoted}}, // CncNull

{ // CnsHyperQuoted

{CnaInc, CnsStart}, // CncComma

{CnaCopy, CnsQuoted}, // CncQuote

{CnaCopy, CnsText}, // CncChar

{CnaInc | CnaQuit, CnsHyperQuoted}}}; // CncNull

char Ch;

ContextType Context;

CatType Cat;

Context.S = CnsStart;

do {

Ch = *(Str++);

if (! Ch) Cat = CncNull;

else if (Ch == 34) Cat = CncQuote;

else if (Ch == ',') Cat = CncComma;

else {

Cat = CncChar;

if (Ch < ' ') Ch = ' ';

} // else

Context = ContextList[Context.S][Cat];

if (CnaCopy & Context.A) luaL_addchar(BufPtr, Ch);

if (CnaInc & Context.A) Ch = 0;

} while (Ch);

return Str;

} // LclCsv

/* * */

static int LclCsvParse(

lua_State *L)

// str_segment, pos <- csv.parse(str [, pos])

{ // LclCsvParse

const char *Str, *EndStr;

int Len, Pos;

luaL_Buffer Buf;

Str = luaL_checkstring(L, 1);

if (lua_isnil(L, 2)) Pos = 1;

else Pos = luaL_checkinteger(L, 2);

Len = strlen(Str);

if ((Pos >= 1) && (Pos == Len)) {

luaL_buffinit(L, &Buf);

EndStr = LclCsv(Str + Pos - 1, &Buf);

luaL_pushresult(&Buf);

Pos = EndStr - Str;

Pos = Pos > Len ? -1 : Pos + 1;

lua_pushinteger(L, Pos);

} // if

else luaL_error(L, "pos is out of range");

return 2;

} // LclCsvParse

/* * */

int luaopen_csvparse(

lua_State *L)

{ // luaopen_csvparse

static const luaL_reg Map[] = {

{"parse", LclCsvParse},

{NULL, NULL}

};

luaL_register(L, "csv", Map);

return 1;

} // luaopen_csvparse

2. Save the file as csvparse.c.

3. Compile this extension into a shared library. On Linux and other Unix-type systems, run this command:

cc -o csvparse.so -shared -Wall csvparse.c

On the Windows platform, compile the extension as follows:

cl /c /Zl /Zd /Yd /MD /W4 /DWIN32 csvparse.c

link /dll /out:csvparse.dll /base:0x67C00000 /machine:ix86

/export:luaopen_csvparse csvparse.obj msvcrt.lib lua5.1.lib

4. Copy the generated module to your designated Lua library directory. On Unix-type systems, run this:

cp csvparse.so $LUA_DIR/

On Windows, run this:

xcopy csvparse.dll "%LUA_DIR%\*.*"

5. Create another new file with the following Lua contents:

require "csvparse"

-- Return a string which has been properly quoted for inclusion in a

-- comma-separated value file.

function csv.escape(str)

local wrap = ""

str = tostring(str)

if string.find(str, '"') then

str = string.gsub(str, '"', '""')

wrap = '"'

end

if string.find(str, ',') then

wrap = '"'

end

return wrap .. str .. wrap

end

-- Iterator to allow traversal of CSV cells

function csv.cells(str)

local pos = 1

local function nextcell()

local cellstr

if pos > 0 then

cellstr, pos = csv.parse(str, pos)

else

cellstr = nil

end

return cellstr

end

return nextcell

end

6. Save this file as csv.lua.

7. Copy this module to your designated Lua library directory. On Unix-type systems, run this:

cp csv.lua $LUA_DIR/

On Windows, run this:

xcopy csv.lua "%LUA_DIR%\*.*"

8. Create another new Lua file with these contents:

require "csv"

local Str = 'Natty Bumppo,"Natty Bumppo, Pathfinder","Natty ""Hawkeye"" Bumppo"'

local SubStr, Pos

io.write("--- csv.parse ---\n")

Pos = 1

io.write(Str, "\n")

for J = 1, 10 do

if Pos > 0 then

SubStr, Pos = csv.parse(Str, Pos)

io.write(string.format("Pos %3d, field [%s], escaped [%s]\n", Pos, SubStr,

csv.escape(SubStr)))

end

io.write("--- csv.cells ---\n")

for CellStr in csv.cells(Str) do

io.write(CellStr, "\n")

end

9. Save this file as csvtest.lua.

10. From a command shell, run the test script as follows:

lua csvtest.lua

--- csv.parse ---

Natty Bumppo,"Natty Bumppo, Pathfinder","Natty "Hawkeye"" Bumppo"

Pos 14, field [Natty Bumppo], escaped [Natty Bumppo]

Pos 41, field [Natty Bumppo, Pathfinder], escaped ["Natty Bumppo, Pathfinder"]

Pos -1, field [Natty "Hawkeye" Bumppo], escaped ["Natty ""Hawkeye"" Bumppo"]

--- csv.cells ---

Natty Bumppo

Natty Bumppo, Pathfinder

Natty "Hawkeye" Bumppo

How It Works

This extension processes CSV rows and leaves the matter of reading and writing those lines to the io library. It comprises two layers: the low level parsing component in C, and the escaping and iterator routines in Lua.

The format of csvparse.c is similar to that of iomode.c. The loader function luaopen_csvparse is called by Lua's package system. It calls luaL_register to create (or reuse, if it already exists) the csv namespace table and to place into it the LclCsvParse extension function keyed with the string “parse”.

The extension function uses Lua's string buffering routines. This mechanism lets you construct a string in stages without having to worry about memory issues. You have to pay attention to the virtual stack, however. Here's how string buffering works:

1. The following line initializes Buf (of type luaL_Buffer) in preparation to building up a CSV field string:

luaL_buffinit(L, &Buf);

2. Characters are added to the string in the following line:

if (CnaCopy & Context.A) luaL_addchar(BufPtr, Ch);

The function luaL_addchar appends the specified character to the string buffer. When called, it and the other string buffer functions except luaL_addvalue expect to find the stack as it was left after the previous string buffer call. (In the case of luaL_addvalue, the stack should be the same except that an additional value should be pushed on the top.) In between string buffer calls, you are able to perform operations that manipulate the stack. You just need to make sure the stack is returned to the expected level before calling a string buffer function again.

3. The completed string is pushed on the stack with the following line:

luaL_pushresult(&Buf);

The csv.parse function is called with a string to process and, optionally, a one-based position that indicates where in the string to begin parsing. The arguments are obtained and validated with the following lines:

Str = luaL_checkstring(L, 1);

if (lua_isnil(L, 2)) Pos = 1;

else Pos = luaL_checkinteger(L, 2);

If the second argument is missing, Pos is assigned a default value of one indicating the beginning of the string. If it's an integer, its value is assigned to Pos. If it is neither of these, an error is generated. The value of Pos is checked to be sure it points to a valid position in Str.

The csv.parse function is intended to be called repeatedly, each time returning the next field. It does this by returning, in addition to the field string, the position of the next field. When the last field of the CSV row is returned, the value of this position is set to -1. A scheme like this is efficient because parsing takes place only as needed and, using Lua's string buffer mechanism, no overhead is spent on transient, partial strings during string construction. If the function succeeds, the call to luaL_pushresult pushes the field string on the stack, and the call to lua_pushintegerpushes the next position or -1. The return value of 2 indicates to Lua that these two values are to be returned, in the order they were pushed, to the caller of csv.parse.

The Lua layer of this extension library, csv.lua, includes routines to help with reading and writing CSV fields. The csv.cells function is an iterator generator that wraps csv.parse, allowing the fields of a CSV record to be read in a generic for loop. The csv.escape function applies the quoting rules to a single field, helping with the construction of a CSV record.

Summary

The C API is a substantial part of Lua, and this chapter has glossed or outright skipped over some of its features. But, having come this far, you have learned enough to create a basic C extension library for Lua. In particular, you've learned the following:

· The distinction between embedding Lua and extending it

· To use the virtual stack to exchange values between Lua and C

· To write and register C functions that can be called from Lua

· To call Lua functions from C

· Where you can store persistent values in C

· To program userdata to create versatile Lua handles that, behind the scenes, can manage C resources

· Different ways to layer your extensions to get the most out of C and the most out of Lua

Take some time to test what you've learned by tackling the following exercises. From there, the next chapters will lead you through the use of Lua for database, web, and network access.

Exercises

1. In the ud_example Try It Out, you created and used an example handle as follows:

local Hnd = ud_example.open(1)

Hnd:set(2)

Hnd:close()

What changes would you need to make to the C library so that it would additionally support the following usage?

local Hnd = ud_example.open(1)

ud_example.set(Hnd, 2)

ud_example.close(Hnd)

2. Add stack diagrams in the form of single line comments to the following C fragment:

lua_newtable(L);

lua_pushstring(L, "Rip Van Winkle");

lua_setfield(L, -2, "Name");

lua_pushvalue(L, -1);

lua_pushcclosure(L, FncA, 1);

lua_setfield(L, -3, "a");

lua_pushcclosure(L, FncB, 1);

lua_setfield(L, -2, "b");

lua_pushvalue(L, -1);

lua_setglobal(L, "test");

Now explain what this code accomplishes:

3. Write an extension library for bit operations. The bit._and function should return each argument (which must be an integer) linked with and. Similarly, the bit._or function should return each argument (which must be an integer) linked with or.

and and or are reserved keywords and cannot be used as names. Nevertheless, you can use them as table keys for your functions, but then you need to invoke your functions as bitl'and'] and bit['or'].