Beginning Lua Programming (2007)
Chapter 15. Programming for the Web
Although the Internet is the infrastructure for a myriad of protocols, e-mail and the Web are the two services most people associate with it. In this chapter, you'll learn how nicely Lua fits into the Web part of the equation. In this chapter, you learn about the following:
· The basic operation of web servers
· Generating web content dynamically with Lua
· Issues with serving dynamic content
· Handling input from users by means of forms
· The basics of the Lua-based Kepler web server environment
Along the way, you'll see how Lua can make use of community libraries to dynamically generate attractive, informative web pages.
A Web Server Primer
To generate web pages using Lua, you should have a comfortable grasp on how web servers operate. This understanding will let you develop Lua programs that focus on the Web's strengths and deal with the Web's limitations in as graceful a manner as possible. As you'll see in this chapter, a successful web application draws on many different tools, with the web server itself playing the central role.
Conceptually, a web server is pretty simple. It's an application that runs on a host and listens for inbound connections. It has access to resources, generally web pages, that are handed out in response to requests made by other applications, usually web browsers. These requests and responses conform to a standard known as hypertext transfer protocol, or HTTP. Unlike many other protocols that define how two applications can have an ongoing conversation, HTTP defines a self-contained transaction. It describes how one application, the client, can properly initiate a request and how another application, the server, can properly respond to that request. The lack of dialog is the reason HTTP is often referred to as a connectionless protocol.
The notion of the World Wide Web derives from the presence of many web servers and many web browsers throughout the Internet. The one-shot nature of HTTP transactions simplifies the construction of web servers and allows them to scale efficiently to support frequent requests from many clients. The original intent of the Web as a collection of static, browsable, hyperlinked documents was served quite well by this simple model. The web pages themselves are usually prepared in HTML (HyperText Markup Language), a standard that defines how tags are used to define a page's structure, appearance, and links to other pages.
The request/response model has simplicity going for it, but it's got two drawbacks: poor performance and poor memory. Performance suffers when a client needs to connect repeatedly to the same server to retrieve related resources such as the images, scripts, and style sheets associated with a given page. Version 1.1 of the HTTP standard addresses this by specifying that connections be kept alive by default. The second drawback deals with the server's inability to retain the context of a series of transactions. Each transaction, even one that occurs over a keepalive connection, appears to the server as a brand new request from an unknown client. This is not a problem for static documents, but it becomes an issue for dynamically generated content in which a server response depends on the transactions that precede it. One solution to this deficiency is for the server to embed a limited amount of contextual information in the hyperlinks or HTML forms that are part of a dynamically created page. Requests that are based on these links or forms return this information to the server where it can be used to reconstruct the context of the transaction. Another method is for the server to use a cookie, which is a small packet of information that is optionally stored by a web browser at the request of a server and is returned by the browser when making a request of that server. Often, the information passed between the client and web server is simply a small identifier that indexes more information stored on the server.
Dynamic Web Content
Much of what you see in your web browser is generated on the server after you submit an HTTP request. This is where Lua fits into the picture, because it will be called on to generate web pages dynamically. There are two basic server configurations that are responsible: the embedded web server and the extended web server.
Embedded Web Server
A limited web server is simple enough to embed into even small applications. A growing number of devices, such as network routers, are implemented this way. One advantage of this approach is that managing the user interface can be made trivially easy. Another advantage is that resources such as file handles or database connections can be kept open for the duration of the application. For the purposes of most applications, an embedded web server needs only a small number of features that a mainstream web server would require. Even a file system is not required.
Embedding a web server is suitable for applications of all types, not just hardware devices. For example, using the LuaSocket library, you can write a short Lua script that opens a database connection and begins accepting connections on some specific port. Interaction between the user and the database can occur without the overhead of reestablishing the database connection for each transaction.
Extended Web Server
A general-purpose web server can be extended to run an application that it knows nothing about, as long as it and the application both agree on how to communicate. Mainstream web servers all implement a standard named the Common Gateway Interface (CGI). In this model, when a client requests a resource that specifies a program, the server invokes that program, passing it information about the client, server, and the request itself. The output of the program is then returned to the client. CGI is often associated with HTML forms and user interaction, but at its root, it simply specifies how a web server interfaces with external programs. Much of the dynamically created content on the web is generated this way.
FastCGI is an optimized variation of CGI in which the application and its resources remain active between HTTP transactions.
Creating Content at Run Time with Lua
For many applications, Lua is the ideal language with which to write CGI programs. One complaint that is leveled against CGI is that each request for dynamic content requires an external program to be executed. Large or otherwise slow programs can degrade performance noticeably. Lua, however, is known for its small footprint and its fast loading and execution of scripts, so it fits in nicely with the CGI model. Lua's flexible text handling and ability to connect with low-level libraries are features that definitely enhance the creation of web applications.
Executing CGI Scripts
A key to configuring CGI is to make sure the web server can distinguish programs from static deliverable resources. When a web server identifies a CGI program, it hands the task of actually executing it to the operating system. In order to function properly, the CGI program generally requires some information about the request, network connection, client, and server.
When you tell a web client to request a resource from a server, whether by clicking a hyperlink or using a browser address bar, you are submitting a uniform resource locator (URL, which is pronounced you-are-ell) to the client. Here is an example of a URL:
http://localhost/cgi-bin/date.lua?num=3
In this example, http:// is the scheme, localhost is the server, /cgi-bin/date.lua is the path, and num=3 is the query. In a hyperlink, the scheme and server are often left out, in which case, the browser uses the values that apply to the page that contains the link. The question mark and query apply only when a dynamic resource is requested and even then only sometimes.
When a web client like a browser contacts a web server, it submits a packet of information that starts with a command that looks like this:
POST /cgi-bin/postform.lua HTTP/1.1
or this:
GET /cgi-bin/submit.lua?name=Natty&number=11 HTTP/1.1
Other types of web clients, such as curl, can request that the web server accept a file for upload using a command like this:
PUT /address.html HTTP/1.1
In all cases, the command line is followed by headers that provide more details about the transaction. The web server conveys the command and headers to the CGI program by means of the shell environment, a per-process collection of key/value pairs.
You can recognize the portion of text immediately following the command as the URL with the scheme and server removed. In particular, notice in the GET example that the query is intact. This is one way of transferring user-provided information to the web server.
The other way of conveying information to the server is to include it in the information packet. This is how content is uploaded with the POST and PUT commands. As you can see from the examples of those commands, the information doesn't appear in the HTTP command. In this case, the headers are followed by the submitted data with one blank line in between. This submitted data will be delivered to the CGI program through a standard pipeline. This conduit connects the standard output of the web server to the standard input of the CGI program, permitting a large amount of data to be streamed efficiently.
All of the content that the CGI program generates, regardless of the initial command, is returned to the web server through a similar pipeline in the reverse direction. This model simplifies the creation of CGI programs because it keeps network communication issues with the web server and application details with the external program.
CGI Scripts on Unix-Type Systems
On Unix-type systems, scripts may begin with a line (known as a shebang) that specifies the path of the interpreter that is to process the program. For example, if a Lua script is marked as executable and begins with the following line, the system will invoke /usr/local/bin/lua with the name of the script as an argument:
#!/usr/local/bin/lua
The Lua interpreter will skip the first line of a script if it begins with the # character. This can be a convenience for general scripting because it allows you to launch a Lua script directly from a shell prompt without explicitly invoking lua itself. With CGI, however, it is imperative, because without the special first line, the system wouldn't know what to do with the script. If you installed Lua in another directory, make the appropriate change.
CGI Scripts on Windows
In general, Windows examines the extension of a script to determine how to launch it. You need to add some entries into the registry to tell Windows how to properly invoke Lua scripts. The following registry script will set things up for you. Make the appropriate changes if you installed Lua in a different location from the one shown here. Create a new file called lua.reg and put the following lines into it:
REGEDIT4
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.lua]
@="luafile""
Content Type"="application/lua"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.luac]
@="luafile""
Content Type"="application/lua"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\luafile]
@="Lua script"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\luafile\DefaultIcon]
@="C:\\Program Files\\utility\\lua.ico,0"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\luafile\Shell]
HKEY_LOCAL_MACHINE\SOFTWARE\Classes\luafile\Shell\open]
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\luafile\Shell\open\command]
@="\"c:\\program files\\utility\\lua.exe\"
To add the entries to the registry from the Windows Explorer, double-click the file. Alternatively, from the command shell, you can issue this command:
start lua.reg
In either case, choose Yes when you are asked if you want to add the entries.
Additionally, for recent versions of Windows, an environment variable named PATHEXT can be augmented with the .lua extension so that the system recognizes Lua scripts as executable. The value of this variable is a semicolon-delimited series of extensions including the period. To edit it, go to the environment variables dialog box in the control panel systems applet. After this is done, you can type the name of a script directly without specifying its extension.
Installing a Web Server
You'll need access to a working web server with CGI capability to proceed with the exercises in this chapter. Even if your Internet service provider has given you everything you need to create and run CGI scripts, you'll likely find it beneficial to develop and test your Lua CGI scripts with a local web server. Doing so will obviate the need to frequently upload scripts or edit them remotely.
The following sections focus on Apache and a small Win32 server named TinyWeb. Another excellent multi-platform web server is Abyss from Aprelium Technologies. Abyss is highly configurable using a web browser. Kepler is a Lua-based web server environment that will be covered briefly in a later section.
Apache
Apache is an excellent, free choice for your local web server. It operates smoothly and dependably on a wide variety of platforms. Although it is highly configurable, its default settings should need little or no adjustment after you install it. Should you decide to learn its advanced features, your knowledge can be applied to Apache installations on any platform. Source and precompiled binary distributions are available from httpd.apache.org. If you are a Windows user, you'll find the Apache setup program to be a straightforward way of installing this web server on your system.
On Windows, Apache will install an entry in the programs menu. When started, this server will place an icon in the task bar tray. You can right-click this tray icon to restart or stop the server. Apache on Unix-type systems will typically have a script in the /etc/rc.d directory to manage starting, restarting, and stopping the server.
After installing Apache, you'll need to make an adjustment to its configuration file, httpd.conf, to make it work with Lua.
For security reasons, Apache will not make its own environment fully available to the CGI programs it launches. This means that the usual LUA_PATH and LUA_CPATH environment variables will not be available to CGI scripts by default. One way or another, you'll need to let Lua scripts know where to look for the modules they need. One way would be to explicitly assign values to the package.path and package.cpath variables at the beginning of scripts that require modules. Alternatively, Apache lets you set environment variables for all or parts of its web directory tree. Here you will make theLUA_PATH and LUA_CPATH variables known throughout the entire tree. First, retrieve the current values of the variables. From a command shell, execute the following to scroll through the environment listing:
set | more
If your system has grep, you can use it as follows to focus the output:
set | grep LUA
Back up Apache's configuration file, httpd.conf, and then open it for editing. You'll find this file in the conf subdirectory of Apache's installation directory. Locate the following line:
<Directory />
Beneath it, add two lines such as the following, making the appropriate substitutions for your system's particular values and platform:
SetEnv LUA_CPATH "/usr/local/lib/lua/5.1/?.so"
SetEnv LUA_PATH "?.lua;/usr/local/lib/lua/5.1/?.lua"
Restart Apache to make the changes take effect.
TinyWeb
For Windows users looking for a very lightweight web server with CGI capabilities, TinyWeb from Ritlabs may be a suitable choice. It is free, easy to install, and consumes few resources. Obtain the tinyweb.zip package from www.ritlabs.com/en/products/tinyweb.
In a command shell, set up your server environment as follows:
The instructions shown here assume this will be beneath c:\program files.
c:
cd "\program files"
md tinyweb
cd tinyweb
md package
md htdocs
md log
cd htdocs
echo Hello from TinyWeb > index.html
md cgi-bin
md images
md scripts
md css
To start up properly, TinyWeb needs to find an index file. The one created here is not valid HTML, but it will satisfy the startup requirement.
Unzip the tinyweb package in the package subdirectory as follows:
The 7-zip utility is used here, but any unzipping utility should work.
cd ..\package
7z x tinyweb.zip
Start TinyWeb in its log directory as follows:
c:
cd "\program files\tinyweb\log"
start ..\package\tiny "c:\programfiles\tinyweb\htdocs"
The argument to tiny specifies the document root. It must be fully specified. You can create a batch file with these lines to facilitate starting the web server.
TinyWeb buffers its log files, so in general, you won't be able to read log entries right after submitting a request to the server or encountering a problem. If you need to do this, obtain sync, which is a free command-line program from www.sysinternals.com. Place sync in a directory that is in the Windows search path, for example the utility directory recommended in Chapter 1. Invoking the following command commits all pending file buffers on the C drive:
sync c
To stop TinyWeb, terminate it with the task manager.
Every file in TinyWeb's cgi-bin directory (and in every subdirectory you may make beneath it) needs to be a program that can be run from the command shell. As long as Lua has been registered properly (as described in the “CGI Scripts on Windows” section earlier in this chapter), you should have no problems running the Lua scripts you place in this directory. TinyWeb does not filter its own environment when executing a CGI program, so it will pass the LUA_PATH and LUA_CPATH variables intact to the Lua scripts it runs. Consequently, nothing special needs to be done to make sure that your Lua CGI scripts will be able to successfully load the modules they need.
Testing Your Web Server with Static Content
In the following sections, you will be accessing your web server by means of a web browser. Make sure your web server has been started before proceeding. The following examples assume you have installed the web server on your own machine. By default, a web server will listen on all network interfaces of the machine on which it is running. For example, if the machine you start the web server on is part of a network and is reachable with the name schubert, then the web server can be reached from a browser with both this:
http://localhost/
and this:
http://schubert/
The localhost address works only on the same machine as the web server. The following examples assume this to be the case. Make the appropriate changes if your setup is different.
Make sure that your web server is functioning properly by asking it for a static page. Point your browser to the following URL to verify that it is serving static pages correctly:
http://localhost/
In the case shown here, the trailing slash following the server name means that you are asking for the default page in the web server's document root, a directory that the web server is configured to treat as the topmost folder. Usually this refers to a page named index.html, although this can typically be configured to something else. If your browser reports that it cannot establish a connection with localhost, your web server has not been started or is somehow not listening on the right interface or port. If it reports a permissions problem, then the server doesn't have authority to read the page. In most cases, the error log of the web server will contain more details about the problem than will be reported to the client. Other problems here will almost certainly involve a visit to the web server's documentation.
Serving Dynamic Web Content
When your server is delivering static pages properly, test it to see how it does with CGI scripts. This is where you'll start using Lua in a web server environment.
Try It Out
Creating a Simple Time Server
In this Try It Out, you arrange to have your web server call Lua to fulfill a dynamic resource request. Lua will generate a plain-text page with the appropriate header and a line containing the server time.
1. In your web server's cgi-bin directory, make a file called date.lua with the following line:
io.write('Content-Type: text/plain\n\n”, os.date(), ”\n”)
2. On Unix-type systems that require scripts to have an interpreter path, insert the following line at the top:
If lua is installed somewhere else on your system, modify this line accordingly.
#!/usr/local/bin/lua
3. On Unix-type systems, mark the script as readable and executable by the web server, as follows:
chmod a+rx date.lua
4. Request the following resource with your web browser:
http://localhost/cgi-bin/date.lua
The response should be a simple, unformatted display of the system date.
How It Works
If you receive the system date in an unadorned web page, the web server correctly identified date.lua as a script, executed it with Lua, and returned the content that the script generated. In this case, the groundwork is in place to develop more advanced CGI programs in Lua. You may want to examine your web server's log to see how it records a successful CGI transaction.
If you don't receive the expected page, note the response that you get instead. Your browser will typically report a status code, but you should consult your web server's error log for more details. The following section covers the problems you are most likely to encounter.
Problems with CGI Scripts
A response from a web server is always accompanied by a three-digit response code. Values in the 200s indicate success; values in the 400s and 500s indicate problems.
When a CGI request goes bad, the error that is reported is generally one of these:
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found 500 Internal Server Error
Problems with CGI scripts usually involve one of the following usual suspects:
· The server mistakes the script for a static page. In this case, the server happily returns the file to your browser, which in turn may display the source code or ask you where you want to save it. This is a web server configuration issue. Remedy this by taking the steps needed to convince your web server that it's dealing with a CGI program, which generally entails placing the script in the designated cgi-bin directory or configuring the server to allow CGI execution in the script's directory.
· The server doesn't have authority to run the script. This stems from a permissions problem with the operating system and results in a 403 Forbidden response. When it comes to CGI programs, getting permissions right is an important task. You should run the web server as a user with very limited authority, because CGI scripts can be a source of security breaches. On Unix-type systems, this user (often nobody) should never be a member of groups that would widen its authority. However, this can make it tricky for a CGI developer to assign correct privileges to a script. A reasonable policy is to require all CGI scripts to be cleared by an administrator who has the necessary privileges to place the script in the cgi-bin directory and to change the script's permissions to make it executable by the web server.
· The script itself has an error. Lua's error message may be reported as a web page, or you may get a rather vague message like Premature end of script headers. In the latter case, the web server's error log should contain the full error message. Running a CGI script from the command shell can be a good way to make sure it doesn't contain errors. In general, it's easier to diagnose problems this way than in a web server environment. With a little effort, more sophisticated CGI scripts that depend on environment variables and possibly standard input can be made to work as well. If you are simply interested in catching syntax errors, you can use luac to compile it.
· Your CGI script may have executed properly, but it sent back content that your browser doesn't know how to render. For example, content of type text/plain cannot be displayed by older versions of Internet Explorer.
· On Unix-like systems, or on Windows when you have configured Apache to emulate Unix-type behavior, your Lua script may lack or have an improperly specified shebang. This is the first line of the script that looks like #!/usr/local/bin/lua.
Asynchronous Calls to the Server
The previous example uses a standard synchronous call to the web server to request the output of the date.lua CGI script. Each time you refresh the page, the contents of the page (which is not a whole lot in this example) are entirely replaced. Modern browsers support an asynchronous method of calling the server as well. In this case, your browser issues a standard request of the server, but rather than waiting for the server's response, it returns control immediately. Then, when the server does respond, a handler is called that can programmatically deal with the returned content, often by making localized changes to the displayed page.
In the following examples, a JavaScript page named AjaxRequest.js will be used to simplify the process of making asynchronous calls. It can be obtained freely from here:
http://www.ajaxtoolbox.com/request/full/AjaxRequest.js
If it doesn't already exist, create a directory named scripts beneath the document root of your web server. Place AjaxRequest.js in this directory. You can test whether it is properly accessible by requesting it explicitly. Point your browser to the following URL and verify that the script is returned:
http://localhost/scripts/AjaxRequest.js
Try It Out
Implementing the Time Server using AJAX
Enhance the previous server time example with an asynchronous call to the server. To do this, you'll use HTML which links in the AJAX toolkit script.
1. With your text editor, create a new file and copy the following lines to it:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>Server time</title>
<script type="text/javascript" src="/scripts/AjaxRequest.js"></script>
<script type="text/javascript">
function NodeReplace(NodeStr, ValStr) {
var El = document.getElementByld(NodeStr);
var TxtNode = document.createTextNode(ValStr);
El.replaceChild(TxtNode, El.firstChild);
} // NodeReplace
function Get(UrlStr, NodeStr) {
function LclPrint(Req) {
NodeReplace(NodeStr, Req.responseText);
} // LclPrint
var Req = {'url':UrlStr, 'onSuccess':LclPrint }
AjaxRequest.get(Req);
return false;
} // Get
</script>
</head>
<body>
<p><a href="#" onclick="return Get('/cgi-bin/date.lua', 'time');">Server
time</a> <i><span id="time"> </span></i></p>
</body>
</html>
2. Save this file as date.html in your web server's document root directory. Request the following page with your browser:
http://localhost/date.html
3. Click the Server Time hyperlink at intervals to update the time display.
How It Works
Clicking the hyperlink initiates a request for the output of the date.lua script. Rather than waiting for this call to return, the JavaScript call to the server returns immediately. When the updated system time does arrive, only its designated area (specifically, the span block with the id value"time") on the page is updated. Each time you click the server time link, another call is made, and when the response is received, the contents of the time compartment are updated. The browser page history is not modified, which means that clicking your browser's back button will not take you back to a page with the previous time.
AJAX is strictly a client-side mechanism. From the web server's point of view, the synchronous and asynchronous calls to date.lua are indistinguishable.
If, as you study this example, you begin to imagine a browser that supports a <script type="text/lua"> script tag and the use of Lua coroutines as an elegant replacement for event handlers, you are not alone. But for now, the province of browser scripting belongs to JavaScript.
For security reasons, an asynchronous call cannot be made to a web server other than the one that served the current page. If it's crucial for you to contact multiple servers from your application, you may need to add a level of indirection by sending all of your asynchronous requests to only one server. This server can then make calls to other web servers and return the content to the original requester.
Producing a Calendar Dynamically
A CGI script is essentially a noninteractive program and can call any library function that applies in that context. The following example demonstrates this by presenting a script that, except for producing HTML rather than plain text, functions just as well in a command shell as a web server environment.
Try It Out
Displaying a Calendar of the Current Month
Prepare a script that displays the current month with the current day highlighted. The script will make use of the os.date function to determine attributes of the current month, such as the day of week that the first day of the month falls on.
1. Save the following file as calendar.lua to your web server's cgi-bin directory:
#!/usr/local/bin/lua
local Cn = {}
Cn.HdrStr = [[Content-Type: text/html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>Calendar</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="/css/general.css" type="text/css" />
<style type="text/css">
table.data td {border-right : 1px solid; border-bottom : 1px solid;
border-color:inherit; color:#404040; background-color:#FFFFFF;
vertical-align:top; width:14%;}
table.data td.today {background-color:#f0f0ff;}
table.data td.name {font-weight:bold; text-align:center; color:#505060;
background-color:#f0f0ff;}
table.data td.edge {width:15%;}
</style>
</head>
<body>
]]
Cn.FtrStr = "</body>\n\n</html>"
local DaysInMonth = { 31, 0, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 }
-- For the specified month and year, this function returns the day of week of
-- the first day in the month, and the number of days in the month.
local function Calendar(Year, Month)
assert(Year > 1900 and Year < 2100,"Year out of range")
assert(Month >= 1 and Month <= 12, "Month out of range")
local Rec = os.date("*t", os.time{year = Year, month = Month, day = 1})
local DayMax = DaysInMonth[Rec.month]
if DayMax == 0 then
DayMax = math.mod(Rec.year, 4) == 0 and 29 or 28
end
return Rec.wday, DayMax
end
local Today = os.date("*t")
local Dow, DayMax = Calendar(Today.year, Today.month)
io.write(Cn.HdrStr)
io.write('<table class="data shade" border="0" cellspacing="0" ',
'cellpadding="5" summary="" width="100%">\n', '
<tbody>\n',
'<tr><th colspan="7">', os.date('%B'), '</th></tr>\n',
'<tr>\n')
local Base = 8 - Dow -- Map day to column
for Col = 1, 7 do
local Wd = (Col == 1 or Col == 7) and ' edge' or ''
io.write('<td class="name', Wd, '">', os.date("%a",
os.time{year = Today.year, month = Today.month, day = Base + Col}),
'</td>\n')
end
io.write('</tr>\n')
local Day = 2 - Dow -- Map day to column
while Day <= DayMax do
io.write('<tr>')
for Col = 1, 7 do
io.write(Today.day == Day and '<td class="today">' or '<td>',
(Day >= 1 and Day <= DayMax) and Day or ' ', '</td>\n')
Day = Day + 1
end
io.write("</tr>\n")
end
io.write('</tbody></table>\n')
io.write(Cn.FtrStr)
2. This script and others in this chapter generate web pages that refer to the following style sheet. Create a subdirectory named css in your web server's document. Save the following contents to a file named general.css in the css directory:
body {background-color : #FFFFFF; color : #404040; font: 9pt/13pt arial;
margin-left:4em; margin-top:2em; margin-bottom:1em; margin-right:4em;}
.hdr {padding:3pt; padding-left:1em; font-weight:bold; border-style: solid;
border-width:1px;}
.shade {color:#505060; background-color:#D0D0F0; border-color:#B0B0D0;}
.white {background-color:#FFFFFF;}
.top {vertical-align:top;}
.note {font: 8pt/12pt arial; color: #606060; margin-top: 0px;}
h1 {font: 11pt/15pt arial; color: #606060; font-weight:bold;}
.right {text-align: right;}
span.super {vertical-align: super; font-size: smaller;}
table.data {border-style:solid; border-top-width:1px; border-left-width:1px;
border-right-width:0px; border-bottom-width:0px;}
table.data tbody {border-color:inherit; font: 9pt/13pt arial;}
table.data tr {padding-top:0px; padding-bottom:0px; border-color:inherit;}
table.data th {border-right : 1px solid; border-bottom : 1px solid;
border-color:inherit;}
table.data td {border-right : 1px solid; border-bottom : 1px solid;
border-color:inherit; color:#404040; background-color:#f6f6ff;}
table.data td.in {padding-left:16px;}
form table {border: 1px solid; padding: 6px;}
form table td {border-right: 0px; border-bottom: 0px;}
form table td.label {text-align: right; font-weight:bold;}
form table td.input {text-align: left;}
3. Verify that your web server can read this page by requesting the following page:
http://localhost/css/general.css
Figure 15-1 shows the kind of page you should see when you request the following URL with your browser:
http://localhost/cgi-bin/calendar.lua
Figure 15-1
How It Works
The calendar script is invoked by the web server as a CGI program. It uses date arithmetics to construct the calendar's attributes, and writes out this calendar using a styled table. Notice that the function Calendar is independent of the script's web context. This makes it a prime candidate for inclusion in a general date module.
Producing Charts Dynamically
Charts and other graphs in your web pages are a great way to present at-a-glance data aggregation and to offer insights into informational patterns. The gd graphics library that you became familiar with in Chapter 12 is ideally suited for this because with it you can create images programmatically. In this section, you'll learn how to use the gd library to generate a bar chart that can be presented in a dynamically built web page.
Usually when you place a graph based on some live data in a web page, you want supporting textual details to accompany it. For example, in a business environment, you may want to produce a web page that details recent customer or vendor transactions. In this page, you may want to place a bar chart that summarizes this activity. Embedding the image right into the dynamically created page would be nice, but this isn't the way HTML works. An image in a web page must be referenced as a separate resource using the src attribute with the <img> tag. This presents a problem, because you've got two dynamically generated resources—the web page and the chart—that must somehow remain synchronized.
The solution presented here works as follows. When a request is made for a dynamic page, the appropriate CGI script is called. It produces a web page with the appropriate details, usually gleaned from a database. In this page, there is a reference to the image. The src attribute of the <img> tag identifies a CGI script that will actually produce the graph. The details of what the graph should look like are submitted as part of the query. This way, the web page details and the query that specifies the appearance of the graph are bound together and any synchronization issues are avoided.
In the cgi-bin directory of your web server, create a file with the following contents and name it bar.lua, marking the file as executable and readable by the web server where necessary:
#!/usr/local/bin/lua
require "gd"
require "iomode"
io.modeset(io.stdin, "binary")
io.modeset(io.stdout, "binary")
local function LclPng(Str)
local ValMax = 1 -- Prevent subsequent division by zero
local ValList = {}
local ValCount = 0
local ClrList = {}
local Factor, Ht, Wd, BarWd, Gap, Hnd, PngStr, ErrStr
Str = "_" .. string.gsub(string.lower(Str), "_") .. "_"
local function LclPrm(Prefix, Default)
local PatStr = "_" .. Prefix .. "(%d+)_"
local Pos, Pos, NumStr = string.find(Str, PatStr)
return Pos and tonumber(NumStr) or Default
end
Ht = LclPrm("h", 200)
BarWd = LclPrm("b", 12)
Gap = LclPrm("g", 4)
for ValStr in string.gmatch(Str, "_([%.%d%:]+)_") do
local Pos, NumStr, ClrStr, Val
Pos, Pos, NumStr, ClrStr = string.find(ValStr, "([%d%.]+)%:(%d+)")
Val = tonumber(Pos and NumStr or ValStr)
if Val > ValMax then
ValMax = Val
end
table.insert(ValList, { Val, tonumber(ClrStr) })
ValCount = ValCount + 1
end
if ValCount > 0 then
Wd = ValCount * (Gap + BarWd) + Gap + 1
Factor = (Ht - Gap) / ValMax
Hnd = gd.createPalette(Wd, Ht)
if Hnd then
-- Background is first color allocated
ClrList.White = Hnd:colorAllocate(255, 255, 255)
ClrList.Gray = Hnd:colorAllocate(150, 150, 150)
ClrList.Black = Hnd:colorAllocate(0, 0, 0)
-- Allocate colors specified in query string
for Mode, RdStr, GrStr, BlStr in string.gmatch(Str,
"_([cfl])(%x%x)(%x%x)(%x%x)_") do
local Clr = Hnd:colorAllocate(tonumber(RdStr, 16),
tonumber(GrStr, 16), tonumber(BlStr, 16))
if Mode == 'c' then
table.insert(ClrList, Clr)
elseif Mode == 'l' then
ClrList.Line = Clr
else
ClrList.Fill = Clr
end
end
local ClrDef = ClrList[1] or ClrList.Gray
ClrList.Line = ClrList.Line or ClrList.Gray
-- By default, background color is the first color allocated. If a fill
-- color was specified, fill in the image now.
if ClrList.Fill then
Hnd:filledRectangle(1, 1, Wd, Ht, ClrList.Fill)
end
local X = Gap
for J = 1, ValCount do
local Y = Factor * ValList[J][1]
local ClrPos = ValList[J][2] or 0
local Clr = ClrList[ClrPos] or ClrDef
Hnd:filledRectangle(X, Ht - Y, X + BarWd,
Ht, ClrList.Line)
Hnd:filledRectangle(X + 1, Ht + 1 - Y, X + BarWd - 1,
Ht, Clr)
X = X + Gap + BarWd
end
-- Create border around image
Hnd:line(0, 0, 0, Ht, ClrList.Line)
Hnd:line(0, 0, Wd, 0, ClrList.Line)
Hnd:line(Wd - 1, 0, Wd - 1, Ht - 1, ClrList.Line)
Hnd:line(0, Ht - 1, Wd - 1, Ht - 1, ClrList.Line)
-- Load image into Lua string
PngStr = gd.pngStr(Hnd)
if not PngStr then
ErrStr = "Error generating PNG image."
end
Hnd = nil
else
ErrStr = "Error creating palette-based PNG image."
end
else
ErrStr = "No values retrieved from query string."
end
return PngStr, ErrStr
end
local Http, PngStr, ErrStr, QueryStr
Http = os.getenv("SERVER_SOFTWARE")
QueryStr = arg[1] or os.getenv("QUERY_STRING")
if QueryStr then
PngStr, ErrStr = LclPng(QueryStr)
else
ErrStr = "Error retrieving query string."
end
if PngStr then
if Http then
io.write('Date: ', os.date(), '\r\n')
io.write('X-Scripting-Engine: ', _VERSION, '\r\n')
io.write('X-Graphics-Library: ', gd.VERSION, '\r\n')
io.write("Content-Transfer-Encoding: binary\r\n")
io.write('Content-Length: ', string.len(PngStr), '\r\n')
io.write("Content-Type: image/png\r\n\r\n");
end
io.write(PngStr)
else
io.write(Http and 'Content-Type: text/plain\r\n\r\n' or '', ErrStr, "\r\n")
end
This script uses the gd library to create a simple bar chart in PNG format based only on the arguments that are passed to it. This makes it easy for other dynamic scripts to include charts since no images need to be cached on the server.
Notice the following lines in bar.lua:
Http = os.getenv("SERVER_SOFTWARE")
QueryStr = arg[1] or os.getenv("QUERY_STRING")
These lines let the script know with a moderate degree of certainty whether it is being run in a web server environment or in a command shell. If Http is not nil, an HTTP header will be written before the PNG file contents. This is the response a web browser expects from its request. This technique is useful when developing a CGI script. It is generally easier to develop a script while working in a command shell than while working with a web server, and this practice permits the script to be run in either envi-ronment without modification.
To test bar.lua from a command shell, invoke it as in the following example:
lua bar.lua 12_23_34 > test.png
The string of numbers delimited with underscores is used by bar.lua to construct a simple bar chart. You can view the resulting test image with a web browser by opening it as a file.
To generate the same chart in a web server environment, request the following:
http://localhost/cgi-bin/bar.lua?12_23_34
The question mark after bar.lua in this reference separates the resource name from the arguments to be submitted by the web server to the CGI script.
To include a chart in a web page, reference it as follows:
<img src="/cgi-bin/bar.lua?12_23_34" alt="Chart" />
The arguments to a CGI script are conventionally divided into key/value pairs. For example:
height=200&width=500
To save some space in the URL, the bar chart script deviates from this practice and uses its own format. The argument string comprises a series of tokens that are separated from one another with an underscore character. Basically, you specify the height of each vertical bar in order. The following sequence creates a chart with bars of height 12, 23, and 34, scaled so that the tallest bar just fits into the height of the chart:
12_23_34
To specify the same chart with a non-default height, use a sequence like this:
H150_12_23_34
The token H150 indicates a chart height of 150 pixels.
The following table shows all of the supported tokens. Prefixes and hexadecimal numbers can be specified in either uppercase or lowercase.
The color prefixes (C, F, and L) are followed by a six-digit hexadecimal number in which the first pair of digits is red, the middle pair is green, and the last pair is blue. For example, the following specifies bright red for the bar color:
CFF0000
The first bar color value that you specify is used for each of the bars, unless the bar height token is followed by a colon and a bar color position. For example, this sequence results in a chart with three bars, with the first and last bars in red and the middle bar in blue:
CFF0000_C0000FF_12_23:2_34
Try It Out
Creating a Simulated Report with Bar Chart
1. Using the bar chart script, generate a live report. Create a new file and copy in these lines:
#!/usr/local/bin/lua
local Cn = {}
Cn.HdrStr = [[Content-Type: text/html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>Bogosity Index</title>
<link rel="stylesheet" href="/css/general.css" type= "text/css" />
</head>
<body>
]]
Cn.FtrStr = "</body>\n\n</html>"
-- This function returns an indexed table and bar chart specifier. The table
-- contains DayCount (default 31) simulated data records. Each record is an
-- indexed table of the form {date, value}. The chart specifier is a string
-- with each value separated by an underscore.
local function LclDataRetrieve(DayCount)
DayCount = DayCount or 31
local Lo, Hi, SecPerDay = -0.3, 2.3, 60 * 60 * 24
local Tm = os.time() - DayCount * SecPerDay
local ValList, RecList = {}, {}
for J = Lo, Hi, (Hi - Lo) / DayCount do
local DtStr = string.gsub(os.date("%A %d %B", Tm), " 0", " ")
Tm = Tm + SecPerDay
local Val = 100 + math.floor(300 * math.sin(J) + math.random(0, 60))
table.insert(RecList, {DtStr, Val})
table.insert(ValList, Val)
end
return RecList, table.concat(ValList, "_")
end
local Out = io.write
local CnColCount = 3
local List, UrlStr = LclDataRetrieve()
local RowCount = #List
local Split = math.ceil(RowCount / CnColCount)
Out(Cn.HdrStr, '<h1>Recent Bogon Flux Averages</h1>\n',
'<div><img src="/cgi-bin/bar.lua?h50_cD0D0F0_l9090B0_ff6f6ff_', UrlStr,
'" alt="Bogosity chart" /></div>\n',
'<p class="note">Average bogon flux values (10<span class="super">9</span> ',
'bogons / m<span class="super">2</span>) from ', List[1][1],
' to ', List[RowCount][1], '</p>\n',
'<table summary=""><tbody><tr>\n')
local Row = 1
for Col = 1, CnColCount do
Out('<td><table class="data shade" border="0" cellspacing="0" ',
'cellpadding="5" summary="">\n',
'<tbody>\n',
'<tr><th>Date</th><th>Bogon Flux</th></tr>\n')
for J = 1, Split do
local Dt, Val
local Rec = List[Row]
if Rec then
Dt, Val = Rec[1], Rec[2]
Row = Row + 1
else
Dt, Val = " ", " "
end
Out('<tr><td>', Dt, '</td><td class="right">', Val, '</td></tr>\n')
end
Out('</tbody>\n</table></td>\n')
end
Out('</tr></tbody></table>\n', Cn.FtrStr)
2. Save this file as bogosity.lua in your web server's cgi-bin directory. As always, make sure the permissions are set so that the web server can read and execute the script.
3. Request http://localh.ost/cgi-bin/bogosity.lua in your web browser. Figure 15-2 shows the kind of page you can expect.
Figure 15-2
How It Works
When you request the bogosity.lua resource, your web server recognizes it as a CGI script. During execution, it obtains a table of information to display. In this case, the data is generated internally, but it would be typical for this information to come from a database, system logs, or some other external source. It then writes out the text of what will become a valid HTML page. Note that the bar chart graphic is not created in this step; it is only specified in the <img> tag. When the browser displays the page, it makes another call to the server to obtain the chart. All of the information needed to produce the graph is passed along with the URL.
Even though the contents of this page are nonsensical, the script provides a good platform on which to base dynamic, informative pages.
A graph generated this way is limited by the amount of descriptive information you can convey to the script that creates it. Browsers and servers may each impose limits on the number of characters that can be transferred as part of URL. If you must submit more information than can fit within several hundred characters, you may need to devise an alternate method of describing the graph. For example, you could encode the bar heights in base 64 (a common method of expressing binary data using a set of 64 characters). Each bar could then be specified with two such characters (allowing a range from 0 to 4095), making the bar height delimiters unnecessary. Additionally, you could predefine a color table in bar.lua and specify colors by table offset rather than 24-bit values.
Interactive CGI Applications
The dynamic pages you've generated so far have not been interactive. You'll now learn about HTML forms that allow you to write scripts that process information provided by the user. HTML forms are rather primitive when compared with most desktop applications, and the simple HTTP request/response model that works so well with static content is decidedly clumsy with interactive content. However, even with its limitations, the interactive web is an integral part of the Internet experience.
CGI Helper Routines
The following file, cgi.lua, includes several routines that simplify writing CGI scripts in Lua. Place it in your server's module directory (as discussed in Chapter 7).
local Cgi = {}
-- Return a urlencoded copy of Str. For example,
-- "Here & there + 97% of other places"
-- is encoded as
-- "Here%20%26%20there%20%2B%2097%25%20of%20other%20places"
function Cgi.Encode(Str)
return string.gsub(Str, '%W, function(Str)
return string.format('%%%02X', string.byte(Str)) end )
end
-- Returns a urldecoded copy of Str. This function reverses the effects of
-- Cgi.Encode.
function Cgi.Decode(Str)
Str = string.gsub(Str, '%+', ' ')
Str = string.gsub(Str, '%%(%x%x)', function(Str)
return string.char(tonumber(Str, 16)) end )
return Str
end
-- Returns an escaped copy of Str in which the characters &<>" are escaped for
-- display in HTML documents.
function Cgi.Escape(Str)
Str = string.gsub(Str or "", "%&", "&")
Str = string.gsub(Str, '%"', """)
Str = string.gsub(Str, "%<", "<")
return string.gsub(Str, "%>", ">")
end
-- This function returns an associative array with the parsed contents of the
-- urlencoded string Str. Multiple values with the same name are placed into an
-- indexed table with the name.
local function LclParse(Str)
local Decode, Tbl = Cgi.Decode, {}
for KeyStr, ValStr in string.gmatch(Str .. '(.-)%=(.-)%&') do
local Key = Decode(KeyStr)
local Val = Decode(ValStr)
local Sub = Tbl[Key]
local SubCat = type(Sub)
-- If there are multiple values with the same name, place them in an
-- indexed table.
if SubCat == "string" then -- replace string with table
Tbl[Key] = { Sub, Val }
elseif SubCat == "table" then -- insert into existing table
table.insert(Sub, Val)
else -- add as string field
Tbl[Key] = Val
end
end
return Tbl
end
-- Returns an associative array with both the GET and POST contents which
-- accompany an HTTP request. Multi-part form data, usually used with uploaded
-- files, is not currently supported.
function Cgi.Params()
local PostLen, GetStr, PostStr, KeyList, Obj
KeyList = {
'PATH_INFO',
'PATH_TRANSLATED',
'REMOTE_HOST',
'REMOTE_ADDR',
'GATEWAY_INTERFACE',
'SCRIPT_NAME',
'REQUEST_METHOD',
'HTTP_ACCEPT',
'HTTP_ACCEPT_CHARSET',
'HTTP_ACCEPT_ENCODING',
'HTTP_ACCEPT_LANGUAGE',
'HTTP_FROM',
'HTTP_HOST',
'HTTP_REFERER',
'HTTP_USER_AGENT',
'QUERY_STRING',
'SERVER_SOFTWARE',
'SERVER_NAME',
'SERVER_PROTOCOL',
'SERVER_PORT',
'CONTENT_TYPE',
'CONTENT_LENGTH',
'AUTH_TYPE' }
Obj = {}
for J, KeyStr in ipairs(KeyList) do
Obj[KeyStr] = os.getenv(KeyStr)
end
if not Obj.SERVER_SOFTWARE then -- Command line invocation
Obj.QUERY_STRING = arg[1]
PostStr = arg[2]
elseif "application/x-www-form-urlencoded" == Obj.CONTENT_TYPE then
-- Web server invocation with posted urlencoded content
PostLen = tonumber(Obj.CONTENT_LENGTH)
if PostLen and PostLen > 0 then
PostStr = io.read(PostLen)
end
end
PostStr = PostStr or ""
Obj.Post = LclParse(PostStr)
Obj.POST_STRING = PostStr
GetStr = Obj.QUERY_STRING or ""
Obj.Get = LclParse(GetStr)
return Obj
end
return Cgi
Your CGI scripts can then make use of this module as follows:
local Cgi = require("cgi")
The module provides some routines that facilitate the transfer of information between the script and the web server. In particular, the Cgi.Params function makes it easy to collect information that a user has submitted in a form. Cgi.Escape prepares text for display in an HTML document by replacing the ampersand, less than, greater than and double quote characters with symbolic replacements. Similarly, Cgi.Encode makes character substitutions to prepare text for inclusion in a URL. This process is known as urlencoding. Cgi.Decode reverses this process to extract information.
The following Try It Out demonstrates how user input is processed by a CGI script.
Try It Out
Specifying a Simple Form
The advantages of inspecting Lua values apply as much to web applications as to other kinds of programs. Here, you will specify a form in a static web page. When you submit the form, a request for the cgishow.lua resource is made. This is a CGI program that simply displays the values that are part of this request.
1. Place the following form.html file in the document root of your web server, and then make sure it is readable by the web server:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>Simple HTML Form</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="/css/general.css" type="text/css" />
</head>
<body>
<h1>Simple form demonstration</h1>
<form action="/cgi-bin/cgishow.lua?Number=42" method="get">
<div><input type="hidden" name="Number" value="42" /></div>
<table class="shade" summary=""><tbody>
<tr>
<td class="label">User</td>
<td><input type="text" size="20" name="User" /></td>
<td class="label">Password</td>
<td><input type="password" size="20" name="Pass" /></td>
</tr>
<tr>
<td class="label top">Site(s)</td><td>
<select name="Site" size="4" multiple="multiple">
<option value="1">Abbey of Pomposa, near Ferrara</option>
<option value="2">Castle del Monte, Apulia</option>
<option value="3">Cathedral of Pisa, Pisa</option>
<option value="4">Chancellery Palace, Rome</option>
<option value="5">Church of San Spirito, Florence</option>
<option value="6">Doge's Palace, Venice</option>
<option value="7">Ducal Palace, Urbino</option>
<option value="8">Florence Cathedral, Florence</option>
<option value="9">Leaning Tower, Pisa</option>
<option value="10">Orvieto Cathedral, Orvieto</option>
<option value="11">Ospedale Degli Innocenti, Florence</option>
<option value="12">Palazzo Strozzi, Florence</option>
<option value="13">Pazzi Chapel, Florence</option>
<option value="14">Piazza del Campo, Siena</option>
<option value="15">Ponte Vecchio, Florence</option>
<option value="16">S. Andrea, Mantua</option>
<option value="17">S. Maria Della Pace, Rome</option>
<option value="18">S. Maria Novella, Florence</option>
<option value="19">S. Maria degli Angeli, Florence</option>
<option value="20">San Lorenzo, Florence, Florence</option>
<option value="21">San Sebastiano, at Mantua</option>
<option value="22">San Zeno Maggiore, Verona</option>
<option value="23">St. Mark's, Venice</option>
</select>
</td>
<td class="label top">Remarks</td>
<td><textarea name="Text" rows="3" cols="32"></textarea></td>
</tr>
<tr>
<td class="label">Artist</td>
<td colspan="3">
<input type="radio" name="Artist" value="1" checked="checked" />
Filippo Brunelleschi
<input type="radio" name="Artist" value="2" />Leon Battista Alberti —
<input type="radio" name="Artist" value="3" />Donato Bramante vjy
</td>
</tr>
<tr>
<td class="label">Art</td>
<td colspan="3">
<input type="checkbox" name="Art" value="Arch" checked="checked" />Architecture
<input type="checkbox" name="Art" value="Paint" />Painting
<input type="checkbox" name="Art" value="Sculpt" />Sculpture
</td>
</tr>
<tr>
<td class="right" colspan="4"><input type="submit" value="Submit" /></td>
</tr>
</tbody></table>
</form>
</body>
</html>
2. Place the following cgishow.lua file in the web server's cgi-bin directory, and then make sure it is readable and executable by the web server:
#!/usr/local/bin/lua
local Cgi = require("cgi")
require("show")
local Cn = {}
Cn.HdrStr = [[Content-Type: text/html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>CGI Environment</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="/css/general.css" type="text/css" />
</head>
<body>
]]
Cn.FtrStr = "</body>\n\n</html >"
local Prm = Cgi.Params()
local List = {}
ObjectDescribe(List, Prm, "CGI parameters")
io.write(Cn.HdrStr, '<p class="hdr shade">CGI Environment</p>\n')
for J, Rec in ipairs(List) do
local Key, Val = string.match(Rec[2], ,A%[%"?(.-)%"?%]%s+(.+)$')
if Key then
io.write('<div style="margin-left:', Rec[1] * 0.5, 'cm;">', '
<b>', Cgi.Escape(Key), '</b> ', Cgi.Escape(Val), '</div>\n')
else
io.write(Rec[2], '<br />\n')
end
end
io.write(Cn.FtrStr)
Two files, show.lua (introduced in Chapter 7) and cgi.lua, need to be located in Lua's module directory.
The general.css style sheet (which was used in a previous example) is used by both form.html and cgishow.lua. It must be located in css subdirectory beneath the document root.
3. With your web browser, request the following resource:
http://localhost/form.html
The resulting page should look like the one shown in Figure 15-3.
Figure 15-3
4. Fill in the various fields and click the Submit button. Figure 15-4 shows the kind of page you will get.
Figure 15-4
How It Works
In form.html, the following line specifies where the form contents should be sent (in this case, /cgi-bin/ cgishow.lua) and in what manner (using the GET command in this example):
<form action="/cgi-bin/cgishow.lua" method="get">
Because the scheme and server segments are missing from the action URL, the values of the current page (http://localhost) are used.
The GET command instructs the browser to append a question mark, followed by the name/value fields from the form, to the end of the URL. The principal advantage of this method is that the full URL can be saved and later used to request the resulting page without needing to revisit the initial form page. This is valuable in the case of a search engine. From another perspective, however, the GET method can be undesirable because it places the form's name/value pairs out in the open where they can show up in server logs and bookmarks. Examine the URL of thecgishow.lua page, and you'll see the unmasked password value. Also, nothing prevents you from pasting a very large amount of text into a form's text field, but limitations with the web browser and web server may prevent your CGI script from actually receiving the complete text.
The POST method places the form's name/value pairs after the HTTP headers, away from the URL. Try changing the form method in form.html from get to post and examine the differences in the CGI parameters table as shown in the resulting page.
A remark should be made regarding the way multiline text appears in the cgishow.lua page. If you type the following into a <textarea> control:
Line 1
Line 2
Line 3
it will appear as this in the display page:
Line 1\013\010Line 2\013\010Line 3
The actual data contains carriage returns (\013) and newlines (\010); the backslash notation is used by the show module to present control characters and conforms with Lua's method of embedding binary data in a string.
When you develop a CGI program in Lua, it's helpful to have a glimpse into the data that is transferred to your script. You can temporarily set the action attribute of a form to cgishow.lua in order to examine what your CGI script can expect.
The CGI module presented here supports only urlencoded data uploads. This method is always used with a GET command and is used by default with a POST command. Control characters such a carriage returns are always coded into a printable form in this type of encoding.
If you want to extend the module to accept uploaded files, you'll need to write a routine to handle data that has been encoded with the multipart/form-data method. This type of encoding uses a special form of delimiter that separates the various uploaded field values. These values can contain binary data and, on the Windows platform, special attention must be paid to avoid problems while reading from the standard input stream. Your script, or the revised CGI script, will need to call the following after requiring the iomode module:
io.modeset(io.stdin, "binary")
Without this, the content read from the standard input stream will have had carriage returns removed. In addition to corrupting binary uploads, this will cause the content to be smaller than the CONTENT_LENGTH value indicates. This discrepancy will cause io.read to block indefinitely as it attempts to read data that isn't there.
Developing CGI Scripts
As helpful as cgishow.lua is in developing CGI applications, the cycle of editing a Lua script and then testing it with a web browser can be tedious. Here are some hints to expedite the development process:
· Modularize your application by factoring out parts that don't involve the user interface. Place these in modules if appropriate.
· Test scripts in a command shell. Any error messages will be displayed in the same console window from which you launch the script, making it unnecessary to examine web server logs.
· Make a development replacement for Cgi.Params that fills the fields you require with test data.
· Make sure your script generates standard HTML. The Html Validator extension for Firefox is a convenient and valuable tool in this regard.
· Consider writing a Lua script that automates the testing of your dynamic web pages. Tools such as wget or curl can be used from the command shell to submit HTTP requests. Both allow you to specify posted content.
Security Issues
The CGI mechanism allows web users to run designated scripts from your web server. Your primary concern when developing CGI scripts is the validity of submitted form data. Put simply, expect the worst. HTML forms support enumerated selections and JavaScript can validate data at the browser, but don't for a moment think that these features do anything except make it easier for a well-intentioned user to enter data.
Lua's tonumber function can be used to validate numeric input. Regular expressions, covered in Chapter 5, are very useful in validating and cleaning data in your CGI script. Apply these techniques prior to including any user-provided data into a command to be executed by Lua, a database server, or some other inter-preter. For example, if you expect numeric data from a particular field, use a command like this:
local Quantity = tonumber(Prm.Post.Quantity)
if Quantity and Quantity >= Cn.QuantityLow and
Quantity <= Cn.QuantityHigh then
-- database code goes here
end
A very common practice is to construct a database or system command dynamically. When you do this, be especially wary of any quotes that are submitted in form data. Devious quoting can allow a malicious user to alter the intent of your command.
The Kepler Project
Kepler is a versatile Lua-based web server environment that supports a number of extensions that facilitate web applications. It is named after the German astronomer Johannes Kepler (1571-1630), who mathematically described the motion of orbiting bodies. Follow the link to the Kepler project on LuaForge (luaforge.net/) to read about Kepler and download the package. A mailing list for Kepler is hosted by LuaForge at lists.luaforge.net/pipermail/kepler-project.
Kepler's objective is to provide an integrated web development environment that functions with all mainstream web servers as well as Xavante, its own web server. The project comprises many interrelated components. One of them, CGILua, manages user interface issues, while the others provide support for services such as database connectivity, session management, FastCGI on Apache and platform-independent filesystem access. Kepler is compatible with the LuaBinaries distribution, and a Windows installer for it is available. Lua enthusiasts are encouraged to explore its many features.
CGI the Kepler Way
The following examples assume that you have Kepler installed and are using the Xavante web server with its default configuration. Examine the Lua configuration files for cgilua and xavante to see more details about how Kepler is set up.
CGILua is the layer between the web server and your CGI scripts. With it, you can register handlers to process scripts based on their extension.
As a first step in using Kepler, this Try It Out re-implements the time server covered earlier in this chapter.
Try It Out
Creating a Time Server with Kepler
This Try It Out generates a simple page with the current server time. It demonstrates how headers and ordinary content are output in the Kepler web environment.
1. In the Kepler installation directory, locate the subdirectory named web. In this directory, create two new subdirectories named cgi-bin and scripts, as follows:
cd web
mkdir cgi-bin
mkdir scripts
2. Using your text editor, prepare a CGI script that shows the system date. Save the following file as date.lua in the cgi-bin directory created in the previous step:
cgilua.contentheader("text", "plain")
cgilua.put(os.date())
3. Copy AjaxRequest.js to the scripts directory created in step 1.
4. With your web browser, request the following:
http://localh.ost/cgi-bin/date.lua
You should see the system date displayed in plain text.
5. Copy date.html from the AJAX-based server date example to the web directory and verify that it works as expected with the new date.lua.
How It Works
With Kepler, writing to standard output does not work as it does with the CGI scripts you've seen so far. Instead, Lua scripts use functions in the cgilua namespace. There are many functions and variables in this namespace that help with tasks such as encoding issues and error handling. This module is included automatically with Kepler CGI scripts.
Lua Pages
A Lua page is a hybrid between a static HTML page and a Lua script. In the examples you've worked with so far, you generated HTML output completely within Lua CGI scripts. With Lua pages, that approach is turned around so that you embed Lua code into an HTML page. The Lua page handler in CGILua processes the embedded Lua and replaces each Lua snippet with the content it generates.
Try It Out
Displaying the Server Time Using a Lua Page
1. Generate an HTML page with the system time using a Lua page. Save the following file as date.lp in Xavante's web directory:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>Server time</title>
</head>
<body>
<p>The system time is <i><% cgilua.put(os.date()) %></i>.</p>
</body>
</html>
2. With your web browser, request this:
http://localhost/date.lp
The system date is displayed in an HTML page.
How It Works
The Lua page handler in CGILua copies the content of an .lp file verbatim, except for the text beginning with <% and ending with %>. It considers the text between these markers to be Lua code. It executes this code and replaces the entire sequence with whatever content is generated. The Lua code doesn't actually have to generate any content; it can contain variables, even local variables, that are referenced in Lua code elsewhere in the Lua page. For example, near the top of date.lp you could have this line:
<% local function TimePut() cgilua.put(os.date()) end %>
And, in the body of the page, you could have this line:
<p>The system time is <i><% TimePut() %></i>.</p>
The function declaration doesn't generate any content, so the sequence is replaced with nothing.
CGILua will detect the <% and %> markers even within Lua strings. If you want to display them in HTML or a Lua string, represent them as <% and %> respectively.
Summary
In this chapter, you had a chance to use Lua to dynamically generate web content. In particular, you learned the following:
· Some details about how the Common Gateway Interface (CGI) works
· How to implement CGI scripts in Lua
· A method for presenting dynamically generated graphics in your web pages
· How to retrieve and examine user-provided data from HTML forms
· The rudiments of using Lua pages with Kepler
Before you head to the next chapter, where you'll learn how to use Lua in a wider variety of network applications, try out your new skills with the following exercises.
Exercises
1. Write a CGI script that displays a message like “Good Afternoon” based on the system time. Follow this with the value of the Name parameter on the URL. For example:
http://localhost/cgi-bin/ex1.lua?Name=Kit
should result in a message like this:
Good afternoon, Kit!
2. Write an HTML form that accepts a name from the user. When the form is submitted, call the CGI script from the previous exercise appropriately.
3. Modify the calendar script to accept a month and year as follows:
http://localhost/cgi-bin/calendar.lua?Month=3&Year=2007
If you're feeling ambitious, obtain these values from an HTML form. And if you really want to go the extra mile, set the default values in this form to the current month and year.