Beginning Lua Programming (2007)

Chapter 10. Looking Under the Hood

This chapter covers various ways of looking at the inner workings of Lua. To continue the automotive metaphor of the title, you can write many useful programs without knowing the things in this chapter, just as you can drive a car without knowing how to change the oil. However, the knowledge and techniques in this chapter will help you find bugs, and let you do things with Lua that otherwise wouldn't be possible. You'll learn about the following:

· Using bytecode

· Controlling Lua's memory management

· Understanding the implementation of tables and strings

· Looking directly inside functions to see and alter their local variables

· Tracing the execution of your code on a call-by-call, line-by-line, or instruction-by-instruction basis

Bytecode and luac

As mentioned in Chapter 3, before Lua can execute source code, it needs to compile it into bytecode, which is then interpreted by the bytecode interpreter. Another name for the bytecode interpreter is the virtual machine, because it's kind of like an imaginary computer (imaginary in the sense that no one has ever built one out of silicon — it's just simulated by your real computer). Lua comes with two programs: the command-line interpreter lua, and the bytecode compiler luac. The main purpose of luac is to compile Lua source code to bytecode and save that bytecode to a file (so it can be run later), but it can also output a human-readable listing of the bytecode.

Try It Out

Reading Bytecode with luac

1. Save the following as test.lua:

-- A short script (for use as a luac test).

function Greet(Name)

print("Hello, " .. Name .. ".")

end

Greet("whoever you are")

2. Run luac as follows:

luac -p -l test.lua

The output should look something like this:

main <test.lua:0,0> (6 instructions, 24 bytes at 0x471410)

0+ params, 2 slots, 0 upvalues, 0 locals, 2 constants, 1 function

1 [5] CLOSURE 0 0 ; 0x471560

2 [3] SETGLOBAL 0 -1 ; Greet

3 [7] GETGLOBAL 0 -1 ; Greet

4 [7] LOADK 1 -2 ; "whoever you are"

5 [7] CALL 0 2 1

6 [7] RETURN 0 1

function <test.lua:3,5> (7 instructions, 28 bytes at 0x471560)

1 param, 5 slots, 0 upvalues, 1 local, 3 constants, 0 functions

1 [4] GETGLOBAL 1 -1 ; print

2 [4] LOADK 2 -2 ; "Hello, "

3 [4] MOVE 3 0

4 [4] LOADK 4 -3 ; "."

5 [4] CONCAT 2 2 4

6 [4] CALL 1 2 1

7 [5] RETURN 0 1

How It Works

The -p option tells luac to only parse the file (check it for syntactical correctness) rather than actually outputting any bytecode. The -l option tells it to print a bytecode listing. The last argument, test.lua, names the file to be used as source code. (This filename has to be the last argument; no options can follow it.)

The bytecode listing itself has two sections: one for the main chunk (the whole file test.lua) and one for the Greet function definition. This chapter isn't big enough to give anywhere close to a comprehensive description of the virtual-machine instructions that bytecode is composed of, but a line-by-line breakdown will give you at least an impression of how bytecode works.

First, the following lines give information about the main chunk (more on this in a moment):

main <test.lua:0,0> (6 instructions, 24 bytes at 0x471410)

0+ params, 2 slots, 0 upvalues, 0 locals, 2 constants, 1 function

Then the CLOSURE instruction creates a function, using the definition at 0x471560 where the [5] means that this corresponds to line 5 of the source code (the end of Greet's function definition):

1 [5] CLOSURE 0 0 ; 0x471560

This instruction assigns the function to the global variable Greet:

2 [3] SETGLOBAL 0 -1 ; Greet

This instruction gets the value (a function) of the global variable Greet:

3 [7] GETGLOBAL 0 -1 ; Greet

This instruction loads one of the main chunk's constants ("whoever you are") into register 1:

4 [7] LOADK 1 -2 ; "whoever you are"

The Lua virtual machine has up to 250 registers, or temporary storage locations. Here, a register is being used to store "whoever you are" until it is passed to a function. Registers are also used to store arguments and other local variables. In the example's output, where the second line says 2 slots, that means that the main chunk uses 2 registers. Registers reside on the call stack, so each function call gets a fresh set.

Continuing with the line-by-line breakdown of the example, the following

instruction calls register 0 (the Greet function) with 1 argument and 0 return

values (subtract 1 from both the 2 and the 1 to get these values): 5

[7] CALL 0 2 1

Then this instruction returns from the main chunk, thereby ending the script:

6 [7] RETURN 0 1

These lines give information about the Greet function definition—it resides at 0x471560, has one argument (parameter) and three constants, and uses five registers:

function <test.lua:3,5> (7 instructions, 28 bytes at 0x471560)

1 param, 5 slots, 0 upvalues, 1 local, 3 constants, 0 functions

This instruction gets the value (a function) of the global variable print:

1 [4] GETGLOBAL 1 -1 ; print

These instructions put the constant "Hello, ", the function's argument, and the constant "." into registers 2, 3, and 4:

2 [4] LOADK 2 -2 ; "Hello, "

3 [4] MOVE 3 0

4 [4] LOADK 4 -3 ; "."

And finally, these instructions concatenate registers 2 through 4, call print, and return from Greet:

5 [4] CONCAT 2 2 4

6 [4] CALL 1 2 1

7 [5] RETURN 0 1

If you want to know more about the Lua virtual machine, check out Kein-Hong Man's “A No-Frills Introduction to Lua 5.1 VM Instructions." It's available at luaforge.net, along with his ChunkSpy program, which is a tool for analyzing bytecode.

You can use bytecode listings to find certain kinds of bugs, troubleshoot efficiency issues, and understand the meaning of particularly hairy pieces of source code. To create a bytecode file, get rid of the -p option (and the -l option) and supply the output filename with the -o option, or don't supply it and look for the output in the file luac.out. A common convention is to end bytecode files in .luac, but (as with .lua files), you can name them anything you want and Lua won't care. For example, this line creates a test.luac file:

luac -o test.luac test.lua

You can run this like a regular Lua file, by typing the following:

lua test.luac

The output is this:

Hello, whoever you are.

Lua can tell the difference between bytecode and source code by looking at the first byte of the file. If it's the character "\27", then the file is treated as bytecode; otherwise it's treated as normal source code. This applies not only to the command-line interpreter, but also to loadstring and its cousinsload, loadfile, and dofile, all of which can accept either source code or bytecode.

The string.dump function converts from a Lua function to bytecode as follows:

> function Greet(Name)

>> print("Hello, " .. Name .. ".")

>> end

-- Convert Greet to bytecode:

> Bytecode = string.dump(Greet)

> -- Convert it back to a function and call it:

> (loadstring(Bytecode))("Insert name here")

Hello, Insert name here.

According to the Lua reference manual, the function given to string.dump cannot have upvalues. An undocumented feature (which therefore may change without notice in a future version) is that a function with upvalues may be dumped, but in the dumped version, all the upvalues will be private to the function (even if the original shared them with another function) and they'll be nil until assigned to from within the function. (In Lua 5.0, string.dump triggered an error if given a function with upvalues.)

luac is necessary less often than you might think. For example, you can use luac to do the following:

· Save the time you would otherwise spend to compile a source code file before running it. Because Lua's bytecode compiler is so fast, this is very seldom a big enough time saver to be worth the hassle.

· Hide scripts from snooping eyes and fiddling fingers. Other than constant strings, a .luac file looks like gobbledygook in a text editor, and this is enough to protect it from casual exploration, but a determined person (armed with the aforementioned ChunkSpy, for example) can still read and modify it.

· Save space. Because .luac files are not necessarily much smaller (or any smaller) than the corresponding .lua files, this is also seldom worth the hassle. One case where it can be worthwhile is in embedded systems, where space is at a premium. (If source code is never used on the embedded device itself, then the bytecode-compiler component of Lua doesn't need to be present, which saves more space.)

· Check syntax. The -p option allows you to check files for syntax errors without actually running them.

· Generate bytecode listings (as discussed previously) for debugging purposes

These last two uses are the domain of lint-type program checkers. For example, a program to find unintentional global variables could create a list of all globals by looking at GETGLOBAL and SETGLOBAL instructions in a bytecode listing.

Bytecode is not portable. A bytecode file generated on one machine can only be used on another if the second machine has the same byte order (little-endian versus big-endian) and word size (32-bit versus 64-bit, for example). Byte order just refers to the order in which multiple-byte numbers are laid out in memory.

Also, because the virtual machine is nothing more than a means of implementing Lua, bytecode is not portable between different versions. Therefore, Lua 5.0 bytecode won't work on Lua 5.1 or vice versa.

The following table lists the luac options and what they do.

10-1

Garbage Collection

If a value is of a type that uses a variable amount of space (such as a string, table, or function) or if it's used as an upvalue and hence may outlive its stack frame, Lua automatically sets aside the necessary space for it in memory. If Lua subsequently notices that this value is no longer either pointed to by a variable or is used as a key or value in a table, it marks that value's memory as no longer occupied so that it can be reused. This process is called garbage collection.

In the following Try It Out, you print bar charts to show how memory usage varies as strings are created and garbage-collected.

Try It Out

Measuring Memory Usage

1. Type the following function into the interpreter:

function GarbageTest(Len)

-- Clean up any garbage:

collectgarbage("collect")

for I = 0, 9 do

-- Create a Len-character string and put it in a variable

-- that will go out of scope at the end of this iteration:

local Junk = string.rep(I, Len)

-- Print a line of a bar chart showing memory usage:

print(string.rep("#", collectgarbage("count")))

-- This use -- of string.rep increases memory usage a bit too, but

-- that can be ignored.

end

2. Call GarbageTest with an argument of 200. The output should look something like this:

####################

#####################

######################

3. Call GarbageTest with an argument of 3000. The output should look something like this:

###########################

################################

#####################################

##########################################

###############################################

##############################

###################################

########################################

#############################################

##################################################

How It Works

The collectgarbage function does various things related to garbage collection, depending on the option string given as its first argument. At the beginning of GarbageTest, it's called with the "collect" option, which makes Lua run a garbage-collection cycle (so that GarbageTest can run with a clean slate). On every iteration of the loop, a new string is created, then collectgarbage is used with the "count" option that makes it return the current memory usage (in kilobytes), and this number is printed in bar chart form. Because Junk is local, a string created on one iteration is not accessible from the next iteration, which means it's eligible for garbage collection. When the strings are only 200 characters long, you can see memory usage slowly ramp up, but when they are 3000 characters long, memory usage ramps up more quickly, until the garbage collector kicks in and frees up the memory used by strings from previous iterations. That's why the second bar chart has a sawtooth shape.

Any code that creates a lot of large strings (or other collectable objects such as tables) that immediately become inaccessible is said to churn the memory pool. Memory churn causes the garbage collector to kick in often and can hence slow down your program. The previous example causes memory churn on purpose, but here are two rules of thumb for avoiding churn caused by strings:

· Don't do in several string.gsub strings what can be done in one string.

· Instead of building up a string piece by piece by concatenating onto the end of it on every iteration of a loop, put the pieces into an array and concatenate it all at once with table.concat.

These are important mainly when the strings are long; otherwise the churn is not noticeable.

collectgarbage has other options that enable you to stop and restart the garbage collector and tweak how often it kicks in automatically and how fast it runs. These options are summarized in Chapter 11. For more details, see the Lua reference manual.

collectgarbage did not take the same arguments in Lua 5.0 as it does in Lua 5.1, and it didn't have as many uses. Some of the slack was picked up by the gcinfo function.

If a table has a metatable whose _mode field is a string containing the letter "k", and the garbage collector sees a key in that table that, if it weren't a key in that table, would be inaccessible, then the garbage collector will remove that key-value pair. (This doesn't happen if the key is a string, number, or Boolean.) For example:

> F1 = function() end

> F2 = function() end

> F3 = function() end

> WeakKeyedTbl = {[F1] = 1, [F2] = 2, [F3] = 3}

> setmetatable(WeakKeyedTbl, {__mode = "k"})

> -- Make function F2 inaccessible except via WeakKeyedTbl:

> F2 = nil

> collectgarbage("collect")

> -- The key-value pair [F2] = 2 is no longer in WeakKeyedTbl:

> for F, N in pairs(WeakKeyedTbl) do

>> print(F, N)

>> end

function: 0x493890 1

function: 0x493e10 3

Such a table is said to have weak keys.

Similarly, if the letter "v" is contained in the string in the __mode field of a table, and the garbage collector sees a value in that table that, if it weren't a value in that table, would be inaccessible, then the garbage collector will remove that key-value pair (again, only if the value is not a string, number, or Boolean). For example:

> F1 = function() end

> F2 = function() end

> F3 = function() end

> WeakValuedTbl = {F1, F2, F3}

> setmetatable(WeakValuedTbl, {__mode = "v"})

> -- Make function F2 inaccessible except via WeakValuedTbl:

> F2 = nil

> collectgarbage("collect")

> -- The key-value pair [2] = F2 is no longer in WeakValuedTbl:

> print(WeakValuedTbl[1], WeakValuedTbl[2], WeakValuedTbl[3])

function: 0x493890 nil function: 0x493e10

Such a table is said to have weak values.

The reason this doesn't apply to strings, numbers, and Booleans is that they are never truly inaccessible. For example, if the string "ab" doesn't happen to already be in a variable or a table, it can still be obtained by concatenating "a" and "b".

Weak keys and values are useful when you want to make an association between a function, table, thread (coroutine), or userdata and some second value, and it's impossible or inconvenient to make that second value part of the first value, but you don't want the association to stick around when you're done with the first value. For instance, a weak-keyed table could have functions as keys and descriptions of those functions as values. When no other variable or table contained a particular function, it and its description would automatically disappear from the table.

The memory leak in orderedtbl.lua that was pointed out in Chapter 8 can be fixed by giving the three tables responsible for the leak weak keys this:

local RealTbls = {} -- Keys: proxy tables; values: the real

-- tables.

local NumToKeys = {} -- Keys: proxy tables; values: arrays

-- of real tables' keys, in order.

local KeyToNums = {} -- Keys: proxy tables; values: tables

-- mapping real tables' keys to their (order) numbers.

-- Prevent leakage by giving all of the above tables weak keys:

local ModeMeta = {__mode = "k"}

setmetatable(RealTbls, ModeMeta)

setmetatable(NumToKeys, ModeMeta)

setmetatable(KeyToNums, ModeMeta)

-- Does the bookkeeping necessary to add a key and its order

-- to the real table (or update/delete a key):

local function __newindex(Proxy, Key, Val)

The __mode string can contain both "k" and "v" (as in "kv"), in which case both the keys and the values of that table are weak. (Any other characters that might also be in the string are ignored.)

Don't change a metatable's __mode field after it has been used as a metatable; otherwise you'll get

unpredictable results (because Lua caches this field).

One other metamethod that has to do with garbage collection is specific to userdata. The userdata type will be covered in Chapter 13, but it's basically a way of exposing a C datatype (or a datatype of some other language) to Lua. When userdata that has a metatable with a __gc field is up for garbage collection, then the value of that field is called with the userdata as an argument. The __gc function should free any memory or other resources (such as open files or network connections) that were allocated outside of Lua when the userdata was created. In this way, languages that are not garbage collected (such as C) can reap the benefits of garbage collection when interfaced with Lua. (This rationale explains why __gc has no effect on tables, which already reap said benefits.)

The Implementation of Tables and Strings

Tables and strings are both implemented using a technique called hashing. Imagine that you had a formula for converting a person's name into a number between one and a thousand. If this formula gives a result of 719 for the name John Smith, then 719 is said to be the hash of John Smith. Now imagine a thousand-page phone book organized by hash. Assume that there aren't too many people in the phone book, and assume that the hash formula is such that they are distributed more or less randomly throughout the book (so that all the people whose last names start with S aren't clumped together, for example). Given these assumptions, if you turn to page 719, you'll have only one or at most a handful of names to look through to find John Smith.

This would be pretty impractical for a phone book meant for humans, but it turns out to be an efficient way for a computer to store things and retrieve them. A table index like Tbl.Xyz tells Lua to hash "Xyz" and look for its value in the same place as the other keys (if any) with the same hash.

Hashing is a good way to store key-value pairs if the keys can be any type at all. But for keys that are consecutive integers, nothing beats storing only the values in memory one right after the other, in order by key. Finding a particular key's value is then just a matter of adding that key to the memory address right before where the values start, which avoids the overhead of computing the hash and possibly searching through other keys with the same hash. If Lua notices that an array has consecutive integer keys starting at 1, then it lays their values out this way, in what is called thearray part of the table. The rest of the table is kept in the hash part.

This makes Lua tables more efficient for common uses, but it's just an implementation detail —it doesn't alter a table's behavior as far as what key-value pairs that table contains. For example, ipairs goes through all consecutive integer keys whether they're in the array part or the hash part of the table. (The distinction between a table's array part and hash part explains some aspects of Lua's behavior that are explicitly undefined, such as the unpredictable length of an array with gaps.)

Lua strings are said to be interned. This means that a new string is only created if a string does not already exist with the same sequence of characters. On the second line of the following example, when "X" and "YZ" are concatenated to form "XYZ", Lua asks itself, “do I already have a string that has this sequence of characters?” (This is the part of strings' implementation that uses hashing, so that answering this question doesn't require a search through every single currently existing string.)

A = "XYZ"

B = "X" .. "YZ"

In this case, the answer is yes, so rather than setting aside a new hunk of memory to store the string, it just assigns the string that's already in A to B. This is why Lua strings are immutable — otherwise A would change if B were changed. The main advantages of interning are that determining whether two strings are equal is very quick, and it takes the same amount of time even if the strings are very long. Another advantage is that storing lots of identical strings takes much less memory.

A disadvantage of interning is that even strings that are used only briefly need to go through the overhead of being hashed and then being interned (if they're not already equal to a previously interned string). This is why the GarbageTest example caused memory churn: On every iteration, a new string was interned, but at the end of every iteration, the just-interned string became eligible for garbage collection. The rules of thumb given for avoiding memory churn work because they avoid unnecessarily interning long but short-lived strings.

The Debug Library

The debug library includes various functions for looking under the hood of Lua code. These functions are available in the table debug.

Use the debug library with caution. Its functions violate the normal rules of what values and variables are accessible from where. These functions can also be slow. Unless you are writing a debugging tool or something else that needs to poke around in Lua's guts, you probably don't want to use the debug library.

All of the functionality described in this section is also available from C. In cases where speed is important, you may want to work from C rather than Lua.

Inspecting and Manipulating Running Code

Some of the debug library's functions enable you to look at and alter running functions and their variables. In the following Try It Out, you use these functions to access local variables that are not in scope.

Try It Out

Accessing Out-of-Scope Locals

1. Type the following into the interpreter:

I = "global"

function Inne

local I = "inside Inner, outside loop"

for I = 1, 10 do

if I == 5 then

debug.debug()

end

print("I (inside loop): " .. I)

end

function Outer()

local I = "inside Outer"

Inner()

end

2. Call Outer. You'll see the following:

> Outer()

I (inside loop): 1

I (inside loop): 2

I (inside loop): 3

I (inside loop): 4

lua_debug>

3. The last line ("lua_debug>") is a debugging prompt. It's similar to the prompt you're already familiar with, but less full-featured. In particular, it doesn't accept input spread between multiple lines, and you might also find that line-editing keys such as backspace don't work with it, so type the following carefully:

lua_debug> print(I)

global

lua_debug> local I = "this chunk"; print(debug.getlocal(1, 1))

I this chunk

lua_debug> print(debug.getlocal(2,1))

nil

lua_debug> print(debug.getlocal(3, 1))

I inside Inner, outside loop

lua_debug> print(debug.getlocal(3, 2))

(for index) 5

lua_debug> print(debug.getlocal(3, 3))

(for limit) 10

lua_debug> print(debug.getlocal(3, 4))

(for step) 1

lua_debug> print(debug.getlocal(3, 5))

I 5

lua_debug> print(debug.getlocal(3, 6))

nil

lua_debug> print(debug.getlocal(4, 1))

I inside Outer

lua_debug> print(debug.setlocal(3, 5 "Kilroy was here!"))

lua_debug> cont

I (inside loop): Kilroy was here!

I (inside loop): 6

I (inside loop): 7

I (inside loop): 8

I (inside loop): 9

I (inside loop): 10

How It Works

The debug.debug function opens an “interpreter within the interpreter” —an interpreter that runs while something else (in this case, the for loop inside of Inner) is still running. There is nothing magical about this—you could write it yourself using io.read, loadstring, and io.stderr:write. Because each line you type is a separate chunk, you can't access variables local to Inner and Outer in the normal way shown here:

lua_debug> print(I)

global

Instead, you can use the debug.getlocal function, which can access variables even if they're out of scope. This function takes two arguments, both numbers. You use the first argument to specify which stack frame you want to look in. In the second argument, you specify which local within that stack frame you want. debug.getlocal itself is considered to be at stack frame 0, and the function calling debug .getlocal is considered to be at stack frame 1, so the following means “get the first local variable in the current function” (the current function being the chunk typed at the prompt):

lua_debug> local I = "this chunk"; print(debug.getlocal(1, 1))

I this chunk

debug.getlocal returns both the variable's name and its value.

Local variables are numbered consecutively, starting at 1. In the following snippet, debug.getlocal is asked for the first local variable in stack frame 2, which is the debug.debug function itself. debug.debug has no local variables, so debug.getlocal returns nil:

lua_debug> print(debug.getlocal(2, 1))

nil

Because debug.debug is at stack frame 2, the function that called it (Inner) must be at stack frame 3. Not only can debug.getlocal see both of Inner's local variables named I, but it can also see the hidden variables that Lua uses to keep track of the loop. These are the ones whose names start with an open parenthesis:

lua_debug> print(debug.getlocal(3, 1))

I inside Inner, outside loop

lua_debug> print(debug.getlocal(3, 2))

(for index) 5

lua_debug> print(debug.getlocal(3, 3))

(for limit) 10

lua_debug> print(debug.getlocal(3, 4))

(for step) 1

lua_debug> print(debug.getlocal(3, 5))

I 5

lua_debug> print(debug.getlocal(3, 6))

nil

debug.getlocal can also see inside Outer as follows:

lua_debug> print(debug.getlocal(4, 1))

I inside Outer

The debug.setlocal function is similar to debug.getlocal, except it assigns a value to the specified local (and returns the local's name). If the debugging prompt sees a line with nothing but the magic word cont on it, then debug.debug returns and the code that called it continues. In this case, the loop continues where it left off, except I has received a new value:

lua_debug> print(debug.setlocal(3, 5, "Kilroy was here!"))

lua_debug> cont

I (inside loop): Kilroy was here!

I (inside loop): 6

I (inside loop): 7

I (inside loop): 8

I (inside loop): 9

I (inside loop): 10

Rather than figuring out in your head what functions are at what stack frames, you can use the debug .traceback function (introduced in Chapter 6) as shown in this example:

> Outer()

I (inside loop): 1

I (inside loop): 2

I (inside loop): 3

I (inside loop): 4

lua_debug> print(debug.traceback())

stack traceback:

(debug command):1: in main chunk

[C]: in function 'debug'

stdin:5: in function 'Inner'

stdin:3: in function 'Outer'

stdin:1: in main chunk

[C]: ?

For more detail about a given stack frame and the function running at that stack frame, use the function debug.getinfo, which returns either an associative table or nil if there is no frame at the requested stack level. The table's fields will be explained in a moment, but you can see the correspondence between each stack frame's short_src, currentline, and name fields in the following example and that stack frame's line in the previous traceback:

lua_debug> for K, V in pairs(debug.getinfo(1)) do print(K, V) end

nups 0

what main

func function: 0x495e40

lastlinedefined 0

source =(debug command)

currentline 1

namewhat

linedefined 0

short_src (debug command)

lua_debug> for K, V in pairs(debug.getinfo(2)) do print(K, V) end

source =[C]

what C

func function: 0x485730

nups 0

short_src [C]

name debug

currentline -1

namewhat field

linedefined -1

lastlinedefined -1

lua_debug> for K, V in pairs(debug.getinfo(3)) do print(K, V) end

source =stdin

what Lua

func function: 0x495918

nups 0

short_src stdin

name Inner

currentline 5

namewhat global

linedefined 1

lastlinedefined 9

lua_debug> for K, V in pairs(debug.getinfo(4)) do print(K, V) end

source =stdin

what Lua

func function: 0x496cc8

nups 0

short_src stdin

name Outer

currentline> 3

namewhat global

linedefined 1

lastlinedefined 4

lua_debug> for K, V in pairs(debug.getinfo(5)) do print(K, V) end

nups 0

what main

func function: 0x495240

lastlinedefined 0

source =stdin

currentline 1

namewhat

linedefined 0

short_src stdin

lua_debug> for K, V in pairs(debug.getinfo(6)) do print(K, V) end

nups 0

what C

func function: 0x480d78

lastlinedefined -1

source =[C]

currentline -1

namewhat

linedefined -1

short_src [C]

lua_debug> print(debug.getinfo(7))

nil

If no second argument is given, the contents of the table default to the fields shown in the previous output. If the second argument is a string, the characters in it are treated as code letters that specify which fields to include. The following table gives the meanings of the fields. All line numbers are line numbers of the chunk the function was defined in or -1 if no line number is available. Some of the code letters select multiple fields.

10-2_1

10-2_2

When you're figuring out the number to give functions like debug.getlocal and debug.getinfo to specify a stack frame, you should include tail calls in your reckoning, even though they don't really get their own stack frames. Because tail calls reuse their callers' stack frames, no other information is available about a function that did a tail call. For example:

> return (function() -- This whole chunk is a function that

>> -- does a tail call to an anonymous function. This

>> -- anonymous function can get its caller's info, but

>> -- there's not much there:

>> for K, V in pairs(debug.getinfo(2)) do

>> print(K, V)

>> end

>> end)()

source =(tail call)

what tail

nups 0

short_src (tail call)

name

currentline -1

namewhat

linedefined -1

lastlinedefined -1

The first argument to debug.getinfo can be a function rather than a stack level. For example:

> function Greet(Name)

>> print("Hello, " .. Name .. ".")

>> end

> for K, V in pairs(debug.getinfo(Greet)) do

>> print(K, V)

>> end

nups 0

what Lua

func function: 0x494078

lastlinedefined 3

source =stdin

currentline -1

namewhat

linedefined 1

short_src stdin

Notice that currentline is -1 because debug.getinfo doesn't know whether the function passed to it is even running, or, if it is, what stack levels it's running on. Also notice that name is not included. That's because as far as debug.getinfo is concerned, the function passed to it is just a value (it doesn't know that you got it from the global variable Greet).

The debug.getupvalue and debug.setupvalue functions are similar to debug.getlocal and debug.setlocal, except that they are for upvalues, whereas the latter functions are only for locals whose scope begins (and ends) within the function at the given stack frame. Also, instead of taking a stack frame as their first argument, debug.getupvalue and debug.setupvalue can only take a function, as specified here:

> do

>> local N = 0

>> -- Returns a number one higher than the last every time

>> -- it's called:

>> function Counter()

>> N = N + 1

>> return N

>> end

> -- Print Counter's first upvalue's name and value:

> print(debug.getupvalue(Counter, 1))

N 0

> -- Counter only has one upvalue; trying to get the second

> -- returns nothing:

> print(debug.getupvalue(Counter, 2))

> print(Counter())

> print(debug.getupvalue(Counter, 1))

N 2

> -- Set the upvalue to -100; Counter will add one to this

> -- before returning it:

> debug.setupvalue(Counter, 1, -100)

> print(Counter())

-99

Because of Lua's scoping rules, two locals in the same stack frame can have the same name, but two upvalues in the same function cannot.

Separate threads of execution have separate stacks. debug.getinfo, debug.getlocal, debug .setlocal, and debug.traceback work by default on the currently running thread, but take an optional first argument of the thread to be used (shifting any other arguments over to compensate). For example:

> function FncA()

>> FncB()

>> end

> function FncB()

>> coroutine.yield()

>> end

> Thread = coroutine.create(FncA)

> coroutine.resume(Thread)

> print(debug.traceback(Thread, "Inside another thread"))

Inside another thread

stack traceback:

[C]: in function 'yield'

stdin:2: in function 'FncB'

stdin:2: in function <stdin:1>

None of Lua 5.0's debug functions took this optional first argument.

Hooks

The debug library enables you to use hooks, which are functions that Lua calls automatically as other code is running. These are useful for debugging, and you can also use them to profile code (to find out how much time is spent in which parts of a program, for example).

In the following Try It Out, you define a hook that prints information about function calls and returns as they happen.

Try It Out

Tracing Function Calls with a Hook

1. Save the following as calltrace.lua:

-- Creates two global functions: a function that traces function calls

-- and returns, and another function to turn it off.

-- Usage:

-- require("calltrace")

-- Trace() -- Turns on tracing.

-- Untrace() -- Turns off tracing.

-- The current depth of the stack (nil when not tracing):

local Depth

-- Returns a string naming the function at StackLvl; this string will

-- include the function's current line number if WithLineNum is true. A

-- sensible string will be returned even if a name or line numbers

-- can't be found.

local function GetInfo(StackLvl, WithLineNum)

-- StackLvl is reckoned from the caller's level:

StackLvl = StackLvl + 1

local Ret

local Info = debug.getinfo(StackLvl, "nlS")

if Info then

local Name, What, LineNum, ShortSrc =

Info.name, Info.what, Info.currentline, Info.short_src

if What == "tail" then

Ret = "overwritten stack frame"

else

if not Name then

if What == "main" then

Name = "chunk"

else

Name = What .. " function"

end

if Name == "C function" then

Ret = Name

else

-- Only use real line numbers:

LineNum = LineNum >= 1 and LineNum

if WithLineNum and LineNum then

Ret = Name .. " (" .. ShortSrc .. ", line " .. LineNum

.. ")"

else

Ret = Name .. " (" .. ShortSrc .. ")"

end

else

-- Below the bottom of the stack:

Ret = "nowhere"

end

return Ret

end

-- Returns a string of N spaces:

local function Indent(N)

return string.rep(" ", N)

end

-- The hook function set by Trace:

local function Hook(Event)

-- Info for the running function being called or returned from:

local Running = GetInfo(2)

-- Info for the function that called that function:

local Caller = GetInfo(3, true)

if Event == "call" then

Depth = Depth + 1

io.stderr:write(Indent(Depth), "calling ", Running, " from ",

Caller, "\n")

else

local RetType

if Event == "return" then

RetType = "returning from "

elseif Event == "tail return" then

RetType = "tail-returning from "

end

io.stderr:write(Indent(Depth), RetType, Running, " to ", Caller,

"\n")

Depth = Depth - 1

end

-- Sets a hook function that prints (to stderr) a trace of function

-- calls and returns:

function Trace()

if not Depth then

-- Before setting the hook, make an iterator that calls

-- debug.getinfo repeatedly in search of the bottom of the stack:

Depth = 1

for Info in

function()

return debug.getinfo(Depth, "n")

end

Depth = Depth + 1

end

-- Don't count the iterator itself or the empty frame counted at

-- the end of the loop:

Depth = Depth - 2

debug.sethook(Hook, "cr")

else

-- Do nothing if Trace() is called twice.

end

-- Unsets the hook function set by Trace:

function Untrace()

debug.sethook()

Depth = nil

end

2. Type the following into the interpreter:

require("calltrace")

function FncA()

FncB()

end

function FncB()

FncC()

end

function FncC()

print("Inside FncC")

end

Trace()

FncA()

Untrace()

3. Starting from where you typed Trace() , the output should look like this:

> Trace()

returning from sethook ([C]) to Trace (./calltrace.lua, line 99)

returning from Trace (./calltrace.lua) to chunk (stdin, line 1)

returning from chunk (stdin) to C function

> FncA()

calling chunk (stdin) from C function

calling FncA (stdin) from chunk (stdin, line 1)

calling FncB (stdin) from FncA (stdin, line 2)

calling FncC (stdin) from FncB (stdin, line 2)

calling print ([C]) from FncC (stdin, line 2)

calling C function from print ([C])

returning from C function to print ([C])

Inside FncC

returning from print ([C]) to FncC (stdin, line 2)

returning from FncC (stdin) to FncB (stdin, line 2)

returning from FncB (stdin) to FncA (stdin, line 2)

returning from FncA (stdin) to chunk (stdin, line 1)

returning from chunk (stdin) to C function

> Untrace()

calling chunk (stdin) from C function

calling Untrace (./calltrace.lua) from chunk (stdin, line 1)

calling sethook ([C]) from Untrace (./calltrace.lua, line 107)

How It Works

When Trace is called, it measures the current depth of the stack (by calling debug.getinfo with deeper and deeper stack levels until it returns nil), and then it uses debug.sethook(Hook, "cr") to set Hook as a hook to be called whenever a function is called (the c in "cr") and whenever a function returns (the r). The hook itself is called with an argument of "call", "return", or "tail return", depending on why it was called. It runs on top of the stack, which means it can get information about the function that was running when the hook was called by using 2 as an argument todebug.getinfo, and information about the function that called that one by using 3. The retrieval of this information is hidden in the GetInfo function (which therefore needs to add 1 to its argument before calling debug .getinfo). The hook prints this information, indenting it by one column per stack frame.

As soon as the hook is set, it starts tracing. These lines show control returning from debug.sethook, through Trace and the interpreted chunk, back to the interpreter itself:

returning from sethook ([C]) to Trace (./calltrace.lua, line 99)

returning from Trace (./calltrace.lua) to chunk (stdin, line 1)

returning from chunk (stdin) to C function

Then when FncA is called, it calls FncB, which calls FncC, which calls print, which calls a C function (tostring, if you're curious). This C function returns, and then print prints the message given to it by FncC:

calling chunk (stdin) from C function calling FncA (stdin) from chunk (stdin, line 1)

calling FncB (stdin) from FncA (stdin, line 2)

calling FncC (stdin) from FncB (stdin, line 2)

calling print ([C]) from FncC (stdin, line 2)

calling C function from print ([C])

returning from C function to print ([C])

Inside FncC

Control then returns back to the main loop of the interpreter through FncC, FncB, FncA, and the chunk where FncA was called:

returning from print ([C]) to FncC (stdin, line 2)

returning from FncC (stdin) to FncB (stdin, line 2)

returning from FncB (stdin) to FncA (stdin, line 2)

returning from FncA (stdin) to chunk (stdin, line 1)

returning from chunk (stdin) to C function

In this example, the Inside FncC message was right where you'd expect it to be, just before print returned. Because that message was printed to standard output and the call trace was printed to standard error, your system may not output them in the right order, particularly if you use your shell's redirection operators to send the program's output to files or other programs.

A hook is not called again until after it has returned. That's why Hook's call to GetInfo is not traced. (If it were, there'd be an infinite regress and the rest of the program would never be reached.)

Untrace unsets the hook by calling debug.sethook with no arguments. (It also nils out Depth so that future calls to Trace from different stack levels will work.)

Tail calls are indented as though they grew the stack, like this:

do -- local scope for Count.

local Count = 5

-- Keeps tail-calling itself until Count is 0:

function Tail()

if Count == 0 then

return

else

Count = Count - 1

return Tail()

end

The function's call trace looks like this:

calling chunk (stdin) from C function

calling Tail (stdin) from chunk (stdin, line 1)

calling Tail (stdin) from Tail (stdin, line 10)

calling Tail (stdin) from Lua function (stdin, line 10)

returning from Lua function (stdin) to overwritten stack frame

tail-returning from Lua function (stdin) to overwritten stack frame

tail-returning from Tail (stdin) to chunk (stdin, line 1)

returning from chunk (stdin) to C function

Notice that a tail return is not a return from a tail call. It's a return from a function that did a tail call.

It's possible to write a version of Trace that takes a list of functions and only traces those functions. The hook would still be called on every call and return, but it would check the list before printing anything. Depending on what information you wanted to print about returns from tail calls, you might need to manage your own stack that would grow even on tail calls.

If you use an end-of-file to exit the interpreter while calls are still being traced, you'll see the trace hit the left margin like this:

> ^D

returning from C function to nowhere

Because the second argument to debug.sethook was "cr", the hook was called on calls and returns. If that argument includes the letter "l", the hook will be called on every line of code, not counting lines where nothing happens (lines not counted by the activelines field of debug.getinfo's return value). The hook will be called with two arguments: the string "line", and the line number.

If debug.sethook has a nonzero third argument (called the hook count), the hook will be called every time that many bytecode instructions have been executed. It will receive the "count" string as an argument.

The debug.gethook function returns three values: the current hook function, the current hook mask (the string containing zero or more of "c", "l", and "r"), and the current hook count. For example, if there's no hook set, then the following:

HookFnc, HookMask, HookCount = debug.gethook()

print("HookFnc: " .. tostring(HookFnc) .. "; HookMask: '"

.. HookMask .. "'; HookCount: " .. HookCount)

will print this:

HookFnc: nil; HookMask: ''; HookCount: 0

If the calltrace.lua hook is set, then it will print this:

HookFnc: function: 0x495e58; HookMask: 'cr'; HookCount: 0

Hooks are a per-thread setting. When you set a hook, it only gets set (and then called) in one thread. Like debug.getinfo, debug.getlocal, debug.setlocal, and debug.traceback, debug.sethook and debug.gethook work by default on the currently running thread, but take the thread to be used as an optional first argument. In debug.sethook, its other arguments are shifted over to compensate.

The Lua 5.0 versions didn't take this optional argument.

Other Functions in the Debug Library

debug.getfenv and debug.setfenv are just like getfenv and setfenv, except that they work with both Lua and C functions, as well as userdata and threads. (getfenv and setfenv only work with Lua functions.)

Similarly, debug.getmetatable and debug.setmetatable are like getmetatable and setmetatable except that they work with all values, not just tables.

debug.getregistry returns the registry table, a table that C code can use as a storage area. (See Chapter 13.)

Lua 5.0 did not include any of these five functions.

Summary

In this chapter you learned the following:

· You can use the luac command-line program to compile source code to bytecode, or to print a human-readable bytecode listing.

· Memory that is no longer used is reclaimed by the garbage collector. You can control the garbage collector's behavior with the collectgarbage function.

· You can use a technique called hashing to implement tables. If Lua thinks that part of a table is being used as an array, it lays that part out sequentially in memory, which is more efficient than hashing for array use.

· You can also implement strings with hashing. These strings are interned: two strings with the same contents are the same string, and their contents are at the same memory location.

· The debug library enables you to examine local variables and altered them even if they're out of scope.

· The debug library also enables you set a hook that will be called on every function call, function return, or line of source code, or every so many bytecode instructions.

The next chapter will complete this book's treatment of the Lua language itself, by summarizing Lua's built-in library functions. The chapter after that discusses libraries written by members of the Lua com-munity, then there's a chapter on the C side of Lua, and then the rest of the book is about doing various things in Lua (such as using databases, web programming, and more).

But first, here are a couple of exercises (with answers in the appendix) to enhance what you've learned in this chapter.

Exercises

1. Imagine you have a file of Lua source code (not bytecode) that the Lua interpreter refuses to run, giving instead the error message “bad header in precompiled chunk.” What must be wrong with the file to cause this message? (The hint here is the word “precompiled.”)

2. Write a GetVar function that takes a variable name and returns the value of the variable in the scope where it was called. It should follow the same scoping rules as Lua itself (locals, then upvalues, then globals) except that it doesn't need to find external locals that aren't actually upvalues (because of not being referred to as variables anywhere in the function being searched).

The variable D in the following test code is an example of this. According to Luas scoping rules, the value of D in InnerFnc should be "upvalue" and not "global", but finding would-be upvalues like D is actually impossible to do except in special cases.

To test your function, run the following code:

Gl = "global"

Up = "upvalue"

L = "local"

OL = "outer local"

IL = "inner local"

A, B, C, D, E = Gl, Gl, Gl, Gl, Gl

function OuterFnc()

local A, B, C, D = Up, Up, Up, Up

local function InnerFnc(A)

local A = IL

local B = L

local _ = C -- Without this, C, like D, would not be an

-- upvalue of InnerFnc.

for _, Name in ipairs({"A", "B", "C", "D", "E"}) do

print(Name, GetVar(Name))

end

InnerFnc(OL)

end

OuterFnc()

It should print the following:

A inner local

B local

C upvalue

D global

E global