Beginning Lua Programming (2007)
Chapter 16. Connecting to a Larger World
In the previous chapter, you used Lua in conjunction with web servers and web browsers—applications that take care of networking details and let you focus on dynamic content and presentation. In this chapter, you'll burrow in a little deeper and learn how to manage those communication details using Lua. The principal tool you'll use in doing this is the LuaSocket library. This package lets you connect your Lua scripts to other programs, whether those programs are running on your own machine, on another machine in your home or office network, or on an Internet server located on the other side of the globe. The facets of this library and networking in general that you'll learn about are as follows:
· Building and installing LuaSocket
· The rudiments of the Berkeley sockets interface
· The basics of programming for the Internet
· Implementing simple client and server scripts
· Retaining a server-side state with coroutines
· Sending and receiving e-mail
· Serving and retrieving web pages
· Processing content with filters
· Using standard streams in a networked environment
You'll find the LuaSocket library to be a natural extension of the language that adds an entirely new dimension to your applications. At its simplest, it makes communicating with another program as easy as reading from and writing to a file.
Installing LuaSocket
LuaSocket is the work of Diego Nehab, an active member of the Lua community, and is made available with the same terms as Lua itself. It is widely used on the Linux, Mac OS X, and Windows platforms, and should work fine on all Unix-like systems. In addition to the source package, apackage with precompiled dynamic link libraries is available for Windows. Obtain LuaSocket from luaforge.net. The instructions that follow assume version 2.0.1, but you'll want to download a later version if one is available. In the event that a more recent version is available, the instructions regarding the compatibility module may no longer apply.
Compiling LuaSocket
Compiling the LuaSocket library is straightforward. The installable product comprises two core dynamically linked libraries and a number of Lua scripts.
Compiling on Linux and Other Unix-Like Systems
To compile the LuaSocket library on Linux and other Unix-like systems, follow these steps:
1. Extract the downloaded package.
2. Replace /path/to in the following command with the location of the source package:
tar xzvf /path/to/luasocket-2.0.1.tar.gz
3. Drop into the source package's directory:
cd luasocket-2.0.1
4. Using your text editor, open the file named config and make the following changes, using values that pertain to your particular system:
LUAINC=-I/usr/local/include
INSTALL_TOP_SHARE=/usr/local/lib/lua/5.1
INSTALL_TOP_LIB=/usr/local/lib/lua/5.1
5. After saving these changes, open the file named makefile and remove all lines that refer to compat, such as this:
$(COMPAT)/compat-5.1.o
6. Save your changes and do the same with the src/makefile.
7. Build and install the library:
make
make install
Compiling on Windows
You can run the following instructions from a Windows command shell. The makefiles that come with LuaSocket are Unix-oriented, so you won't use them here. These instructions assume that you've got your development environment set up as shown in Chapter 1.
Follow these steps:
1. Extract the contents of the luasocket-2.0.1.tar.gz package to the directory in which you will compile the library. To do this with 7-zip, use the following commands, replacing \path\to with the location of the downloaded package:
7z x \path\to\luasocket-2.0.1.tar.gz
7z x luasocket-2.0.1.tar
del luasocket-2.0.1.tar
2. Drop into the src subdirectory of the newly created tree, like this:
cd luasocket-2.0.1\src
3. The unix.c and usocket.c files are not used in the Windows version. Effectively remove them from the set by renaming their extension as follows:
ren unix.c unix.c00
ren usocket.c usocket.c00
4. Compile the C files as follows:
cl /c /nologo /Zl /Zd /Yd /MD /W3 /DWIN32 /DWIN32_LEAN_AND_MEAN *.c
5. Separate the MIME library module from the others like this:
ren mime.obj mime.o
6. Link the socket library as follows:
link /DLL /out:socket.dll /base:0x67800000 /export:luaopen_socket_core 
*.obj msvcrt.lib lua5.1.lib wsock32.lib
7. Link the MIME library as follows:
link /DLL /out:mime.dll /base:0x67700000 /export:luaopen_mime_core
mime.o msvcrt.lib lua5.1.lib
8. Copy the dynamic link libraries and support scripts to the installation directory as follows:
xcopy socket.dll "%LUA_DIR%\socket\core.*"
xcopy mime.dll "%LUA_DIR%\mime\core.*"
xcopy socket.lua "%LUA_DIR%\*.*"
xcopy mime.lua "%LUA_DIR%\*.*"
xcopy ltn12.lua "%LUA_DIR%\*.*"
xcopy ftp.lua "%LUA_DIR%\socket\*.*"
xcopy http.lua "%LUA_DIR%\socket\*.*"
xcopy smtp.lua "%LUA_DIR%\socket\*.*"
xcopy tp.lua "%LUA_DIR%\socket\*.*"
xcopy url.lua "%LUA_DIR%\socket\*.*"
Installing Windows Binaries
A binary package of LuaSocket that is compatible with the Window binary package of Lua is available on luaforge.net. Download luasocket-2-0.1-lua5.1-win32.zip or a higher version if one is available. The zip file's contents are as follows, with directory names shown in italic:
lib
mime
core.dll
socket
core.dll
lua
socket.lua
mime.lua
ltn12.lua
socket
url.lua
tp.lua
smtp.lua
http.lua
ftp.lua
You can use an interactive tool like WinZip to extract the files, or you can use the Windows command shell as follows:
1. Replace \path\to with the location of the downloaded zip file:
7z x \path\to\luasocket-2.0.1-lua5.1-win32.zip
xcopy lua\*.lua "%LUA_DIR%\*.*"
xcopy lua\socket\*.lua "%LUA_DIR%\socket\*.*"
xcopy lib\mime\*.dll "%LUA_DIR%\mime\*.*"
xcopy lib\socket\*.dll "%LUA_DIR%\socket\*.*"
2. Remove the lua and lib directories by executing the following:
del /s /q lib lua
The del command on older versions of Windows doesn't support these switches. In this case, try the following command instead:
deltree lib lua
Network Overview
To effectively use the LuaSocket library, you need at least a rudimentary understanding of networks, such as how computers on a network identify one another and how they exchange information.
Virtually all mainstream networks nowadays support the Internet Protocol (IP) as a means of routing data packets from one network device to another. The wide acceptance of this open standard ushered in the Internet boom. IP is just one of the well-engineered layers that allow remote computers to communicate with each other. Beneath it are layers involving standards for low-level protocols—such as communication by means of Ethernet and wireless devices—and above it are layers that coordinate packets and present them in a format suitable for applications like email programs and web browsers. Each of these layers encapsulates its tasks and presents a well-defined interface to adjacent layers, allowing for varied implementations that work well and reliably together. The LuaSocket library doesn't give you access to the lower layers of this model but, as you'll see in the sections that follow, it gives you a lot of control over the upper layers.
Routed Packets
From its inception, the Internet was designed to be fault tolerant. As long as a path exists between two computers on the Internet, they should be able to communicate even if shorter and more direct paths are inaccessible due to equipment failure. The Internet supports this objective by breaking all data transfer into individual packets that make the journey from the originating machine to the destination machine independently. At each step along the way, routers guide a packet closer to its destination based on routing tables and current performance metrics.
Addresses
Each machine with direct access to the Internet is identified with a unique address, either a 32-bit value (the format specified in version 4 of the IP protocol, or IPv4) or a 128-bit value (the format specified by the backwards compatible version 6 of the IP protocol, or IPv6). IPv4 addresses are conventionally expressed in dotted-decimal or dotted-quad notation in which the individual byte values, or octets, are delimited with a dot, for example 192.168.0.1. The most significant portion of the address identifies the network in which a machine resides and the least significant portion identifies the device within the network.
Domain Names
An important level of indirection allows you to use symbolic names like www.lua.org rather than IP addresses. If you were to move the Lua website to another network, its nameserver record would be updated to point to the new address relieving users throughout the world from the need to modify their records. The mechanism that supports this mapping is hierarchical and distributed and is known as DNS (Domain Name System). The hierarchy is apparent in the ordering of the dot-delimited names; for example, www is just one of possibly many hosts or subdomains at the Lua site, and lua is just one of many second level domains in the org top level domain. The mappings between host name and IP address are managed by name servers distributed throughout the Internet, each having authority for its own particular domain.
LuaSocket gives you access to DNS, allowing you to obtain host information either by IP address or by host name. There are three functions in the dns namespace (which in turn belongs to the socket namespace):
· socket.dns.gethostname(): Returns the host name of the machine on which the function is called.
· socket.dns.tohostname(IP address or host name): Returns, on success, the canonical name corresponding to the specified address followed by a table containing summary information including alias host names. On error, it returns nil followed by an error message. The canonical name is the principal host name registered with an IP address; aliases are alternative host names that resolve to this same IP address.
· socket.dns.toip(host name or IP address): Returns, on success, the IP address corresponding to the specified address followed by a table containing summary information including alias host names. On error, it returns nil followed by an error message.
The last two functions accept either an IP address or a host name. This is a convenient feature shared by most functions in the LuaSocket library. You should note that the convenience of using host names comes at a cost; a potentially long DNS lookup will need to be made to resolve the name to an IP address. However, subsequent lookups for a given name are usually fast because the name and address association will be cached locally.
Try It Out
Name and Number, Please
Usually when you access the principal Lua website, you specify www.lua.org and let your browser handle the details of obtaining the site's IP address. Here, you dig a little beneath the surface and use DNS to find out more about this site.
1. With your text editor, create a new Lua file with the following contents:
local socket = require("socket")
require("show")
local Ip = {socket.dns.toip("Error! Hyperlink reference not valid.
ObjectShow(Ip, "toip")
The ObjectShow function comes from the show module introduced in Chapter 7.
2. Save this file as dns_test.lua.
3. While connected to the Internet, run this script from a command shell:
lua dns_test.lua
["toip"] table: 00414F28 (n = 2)
[1] "62.197.40.9"
[2] table: 00414FD8 (n = 0)
["alias"] table: 00410C68 (n = 2)
[1] "www.lua.org"
[2] "zeus.pepperfish.net"
["ip"] table: 00410C00 (n = 1)
[1] "62.197.40.9"
["name"] "babel.pepperfish.net"
How It Works
The socket.dns.toip function calls on the services of a resolver library to obtain information about the specified host. The first value that is returned is the current IP address of www.lua.org; this is followed by a table containing summary information including alias host names, canonical host name and IP address. In this case, you can see that www.lua.org is an alias for babel.pepperfish.net. Note that this behind-the-scenes information is subject to change; when you run this script, the site hosting www.lua.org may have changed.
Identifying Internet Resources
Most resources on the Internet can be named with a URL, which is a standard way of identifying web pages, images, and other resources and the means by which you access them. URLs are described in more detail in Chapter 15. LuaSocket provides support for, according to Diego, “anything you could possibly want to do with” a URL.
If you write an application that works with Internet resources, you can use url.parse to get the details you need to connect with a remote server and make a request. This function receives a URL string and breaks it into its constituent parts.
url.parse(URL string, optional result table) returns a table with the specified URL's string components keyed with the names url, scheme, authority, path, params, query, fragment, userinfo, host, port, user, and password. If the result table is provided, the components are placed into it rather than a new table. Fields in this table that are not overwritten are left alone.
Try It Out
Unfurling a URL
As one example pertaining to the socket.url module, use url.parse to break a rather contrived URL into its component parts.
1. Create a Lua file with the following contents:
local url = require("socket.url")
require("show")
local UrlStr = "http://natty:pathfinder@www.example.net:8888" ..
"/susquehanna;loc?date=1793#title"
ObjectShow(url.parse(UrlStr), "URL")
2. Save this script as url_01.lua.
3. Run the script as follows:
lua url_01.lua
["URL"] table: 00413760 (n = 0)
["authority"] "natty:pathfinder@www.example.net:8888"
["fragment"] "title"
["host"] "www.example.net" —
["params"] "loc"
["password"] "pathfinder"
["path"] "/susquehanna"
["port"] "8888"
["query"] "date=1793"
["scheme"] "http"
["user"] "natty"
["userinfo"] "natty:pathfinder"
How It Works
The url.parse function extracts components within the specified URL as substrings. For example, notice that the value of port is saved as a string rather than a number. Also, note that certain intermediate components are stored in the result table as well. For example, in this case theauthority field includes the intermediate userinfo field which in turn includes the user and password fields.
Transport Protocols
On top of IP are two protocols to which LuaSocket gives you access: UDP (User Datagram Protocol) and TCP (Transmission Control Protocol). These are so-called transport protocols, which means that they're in charge of getting information from one application to another. In a characteristic layered approach, they use IP to handle packet routing details, and IP in turn uses the underlying hardware to handle the physical signal transmission details needed to transfer information.
Here are more detailed descriptions of UDP and TCP:
· UDP: A short step away from IP is UDP, a protocol that specifies how datagrams are exchanged between applications. This protocol has low overhead and is consequently very fast. However, UDP doesn't attempt to straighten out problems with datagrams that are lost in transit, arrive at their destination out-of-sequence, or contain errors introduced in transmission. The high reliability of modern networks has made problems like this less likely to occur than in earlier years. The LuaSocket documentation includes an example client script that uses UDP to connect with a remote daytime server.
· TCP: The predominant transport protocol of the Internet is TCP. A veritable roster of application protocols with initialized names depend on it: the web (HTTP and HTTPS), mail (SMTP, IMAP, POP), file transfer (FTP), and secure shell (SSH) among many others. This protocol allows applications to send and receive a stream of data without worrying about the problems that may beset the data in transit. When data is transmitted, TCP splits it into packets suitable for conveyance by IP. On the receiving end, TCP requests that missing or corrupted packets be resent, puts arriving packets in their original order, and recombines the packets into a stream of bytes.
Sockets: Streams and Datagrams
A library of routines and data structures that provide applications with abstracted access to the UDP and TCP protocols was developed at Berkeley in the early 80s. The model this library implements is known as the Berkeley sockets interface and has become the standard way for applications to communicate with one another on the Internet. In the Berkeley model, a socket represents one end of a connection. It has associated with it an IP address and a port and must be of type TCP or UDP. The IP address must be associated with one of the machine's interfaces; these can be physical such as Ethernet interfaces, or virtual such as the loopback interfaces or virtual private network interfaces. The port number is a two-byte value that identifies a particular service accessible on an interface.
· UDP: This is sometimes referred to as a datagram socket. You generally want to read the entire datagram when receiving data with this type of socket; anything not read is discarded.
· TCP: This is often referred to as a stream or connection-oriented socket because, when you connect it to a peer, you can use it for a two way dialog in which streams of bytes are exchanged. Reading data from a remotely connected socket is done with a client socket's receive method. On success, ClientHnd:receive(read pattern, optional prefix) returns a Lua string containing the received data matching the specified pattern. If the method fails, nil is returned followed by an error message and the partial contents of the received data. The first argument to this method indicates how much data is to be read: the next line (the default), the next fixed block of bytes, or everything that is sent by the remote socket until the connection is closed. All patterns have comparable performance, so select the one that is easiest to work with for your particular application. The following table provides a summary of the LuaSocket receive patterns.
If a prefix is specified, it will be prepended to the returned data string.
Client TCP sockets send data to one another by means of the send method. On success, ClientHnd:send (data string, optional start, optional end) returns the position of the last byte sent. On error, the method returns nil followed by an error message and the position of the last byte successfully sent. You can use the start and end arguments to send a substring rather than the entire string.
By default, the functions used to send and receive data will block; that is, they will retain control until the operation has completed. A blocking function simplifies programming somewhat, but it can be a trapdoor for the unwary because it can deprive your program of control needed for other connections. The Berkeley sockets interface provides an effective way to deal with this by including a select function that lets you know which sockets in a pool are ready for reading and which are ready for writing. Additionally, LuaSocket lets you set timeout values to bail out of a function if it doesn't respond in the specified time. As you'll see, a combination of these two features makes it possible to write responsive application servers in Lua.
TCP Socket Sociology
ATCP socket is created by calling socket.tcp. If this function succeeds, a new socket handle is returned, otherwise nil followed by an error message is returned. The returned socket is referred to in the LuaSocket documentation as a master object, but in some respects it isn't yet even an apprentice; it has a limited skill set and is unable to communicate with peers. There are two career paths open to such a socket: through calls to bind and listen it can become a server object, or through a call to connect it can become a client object. One thing it cannot do in this phase is transfer information with another socket.
When a TCP socket graduates to client or server status, its associated methods change. LuaSocket lets you know if you attempt to call a method on a socket that isn't of the right type. For example, calling accept on a master socket will result in an error with the following message:
calling 'accept' on bad self (tcp{server} expected, got userdata)
The tcp{server} notation is how LuaSocket renders a server socket with tostring. In general, a server socket's job is to listen on a well-known port, which is a port that is associated with the service that the application renders. For example, a web server listens on port 80 and an FTP server listens on port 21. When the accept method of a server socket detects an inbound connection request, it creates and returns a new socket that is connected with the remote socket. The returned value is a client socket. The nomenclature can be a little misleading because such a socket is part of a server application, but it's arguably appropriate, because the socket has the same capabilities as a client socket returned from a call to connect.
A client socket created by accept and a client socket created by connect are peers. Their job is to transfer information with each other, and the principal means used to do this are the send and receive methods. When peers are connected, there is an IP address and a port at each end. The address and port of the server are usually well-known, for example port 80 at www.example.com, but in general the address and port of the connecting client are decided by the sockets library rather than the application itself. The port that the library selects for you is called ephemeral because it is recycled after the connection is closed. When using the socket.connect shortcut to create a client socket, you can optionally specify an address and port. You might want to do this if you have more than one interface that could be used to connect with the remote server and have some reason to favor one. If you want to specify an address but don't care about the port, you can use 0 as the port value.
Using LuaSocket for Network Communication
The following Try It Outs demonstrate sockets in action. Keep in mind that these examples use LuaSocket at a rather low level; many applications using the package never deal directly with sockets. The first exercise will use Lua scripts for both ends—the server and the client—of a communication link. In the second exercise, you'll implement a very simplistic web server with a Lua script. In this case, you'll use a web browser for the client side of the connection.
Try It Out
Using Two Servers and a Client
LuaSocket gives your Lua scripts access to Berkeley sockets. Here, you'll open up three command shells and explore the basic practice of listening and connecting sockets.
1. With your text editor, create a new Lua file with the following contents:
local socket = require("socket")
local Addr = arg[1] or "127.0.0.1"
local Port = tonumber(arg[2] or 11250)
local Str, Len, SckHnd, ClientHnd, ErrStr, BindAddr, BindPort, ClAddr, ClPort
SckHnd, ErrStr = socket.bind(Addr, Port)
if SckHnd then
BindAddr, BindPort = SckHnd:getsockname()
io.write("Listening on ", BindAddr, ", port ", BindPort, "\n")
ClientHnd, ErrStr = SckHnd:accept()
if ClientHnd then
ClAddr, ClPort = ClientHnd:getpeername()
io.write("Connection from ", ClAddr, ", port ", ClPort, "\n")
Str = string.format("Greetings from %s:%d to %s:%d\r\n",
BindAddr, BindPort, ClAddr, ClPort)
Len, ErrStr = ClientHnd:send(Str)
if Len then
Str, ErrStr = ClientHnd:receive()
if Str then
io.write("Received from client: [", Str, "]\n")
else
io.write("Receive error: ", ErrStr, "\r\n")
end
ClientHnd:shutdown("both")
else
io.write("Send error: ", ErrStr, "\n")
end
ClientHnd:close()
else
io.write("Client connection.", ErrStr, "\n")
end
SckHnd:close()
else
io.write("Listening socket. ",ErrStr, "\n")
end
2. Save this file as server_01.lua.
3. With your text editor, create another new Lua file with the following contents:
local socket = require("socket")
local Addr = arg[1] or "127.0.0.1"
local Port = tonumber(arg[2] or 11250)
local SckHnd, ErrStr, Str, ClAddr, ClPort, SrvAddr, SrvPort
SckHnd, ErrStr = socket.connect(Addr, Port, "127.0.0.1", 0)
if SckHnd then
ClAddr, ClPort = SckHnd:getsockname()
SrvAddr, SrvPort = SckHnd:getpeername()
io.write("Connected with ", SrvAddr, " on port ", SrvPort, "\n")
Str, ErrStr = SckHnd:receive()
if Str then
SckHnd:send(string.format("Greetings from %s:%d to %s:%d\r\n",
ClAddr, ClPort, SrvAddr, SrvPort))
io.write("Got [", Str, "] from server\n")
else
io.error("Error. ", ErrStr, "\n")
end
SckHnd:close()
else
io.write("Connecting socket. ", ErrStr, "\n")
end
4. Save this file as client_01.lua.
5. Open three command shells and, in each, use cd to change to the directory in which the two scripts are saved. If possible, size and arrange each shell display so they are all visible on the screen.
6. In one command shell, invoke the server as follows:
lua server_01.lua 127.1.2.3
Listening on 127.1.2.3, port 11250
7. In another command shell, invoke another server as follows:
lua server_01.lua 127.101.102.103
Listening on 127.101.102.103, port 11250
8. In the third command shell, invoke the client script as follows:
lua client_01.lua 127.1.2.3
Connected with 127.1.2.3 on port 11250
Got [Greetings from 127.1.2.3:11250 to 127.0.0.1:32822] from server
The script in the first shell will terminate after displaying the following line:
Connection from 127.0.0.1, port 32822
Received from client: [Greetings from 127.0.0.1:32822 to 127.1.2.3:11250]
In general, you'll see an ephemeral port value different than 32822 in the server's response.
9. In the third command shell, reinvoke the client script as follows:
lua client_01.lua 127.101.102.103
Connected with 127.101.102.103 on port 11250
Got [Greetings from 127.101.102.103:11250 to 127.0.0.1:32821] from server
The script in the second shell will terminate after displaying the following line:
Connection from 127.0.0.1, port 32821
Received from client: [Greetings from 127.0.0.1:32821 to 127.101.102.103:11250]
As before, you'll likely see a value different than 32821 when you run the script.
How It Works
This example illustrates network connections that take place over your machine's virtual loopback device, effectively connecting your machine to itself. A number of observations can be made from the output of these scripts:
· Your machine can listen for incoming connections on the same port as long as the listening sockets are bound to different interfaces. Virtual web servers are set up this way.
· A socket-based network connection involves client sockets at each end. The server application creates a client socket when accept responds to a connection request, and the client application creates a client socket by calling connect.
· IP address and port information about the remote socket is available through the getpeername method, but this information is not required to conduct a dialog.
A design objective of the World Wide Web was to keep web servers simple. While today's mainstream servers are a good deal more complex than their early predecessors, they are still considerably simpler than web browsers because they don't have to deal with content rendering and the user interface.
Try It Out
Creating a Simple Web Server
LuaSocket has everything you need to script a functional web server. The server you'll build here is light-duty by any definition, but it provides you with a basic framework on which you can add features, including the generation of dynamic content.
1. Create a new Lua file with your text editor and add the following contents:
local socket = require("socket")
require "show"
local Cn = {}
Cn.Host = "localhost"
Cn.Port = 80
Cn.MimeList = {
css = "text/css",
gif = "image/gif",
htm = "text/html",
html = "text/html",
png = "image/png",
txt = "text/plain" }
local function LclHdrRead(ClSck)
local Hdr = {}
local LineStr, ErrStr
LineStr, ErrStr = ClSck:receive()
if LineStr then
-- "GET /page.html HTTP/1.1" -> "GET", "page.html"
Hdr.Cmd, Hdr.Path = string.match(LineStr, "^(%S+)%s+%/(%S*)")
while LineStr do
LineStr, ErrStr = ClSck:receive()
if LineStr then
if LineStr ~= "" then
local Key, Val = string.match(LineStr, "^(.-)%:%s*(.*)$")
Hdr[string.lower(Key)] = Val
else
LineStr = nil -- End loop at first blank line
end
end
end
end
if (not Hdr.Path) or (Hdr.Path == "") then
Hdr.Path = "index.html"
end
return Hdr
end
local function LclSend(Client, BodyStr, MimeStr, CodeStr)
local SendStr =
'HTTP/1.1 ' .. (CodeStr or '200 OK') .. '\r\n' ..
'Date: ' .. os.date() .. '\r\n' ..
'Server: webserver.lua/0.1\r\n' ..
'Content-Length: ' .. string.len(BodyStr) .. '\r\n' ..
'Content-Type: ' .. (MimeStr or 'text/html') .. '\r\n\r\n' .. BodyStr
Client:send(SendStr)
Client:shutdown() -- We're finished with this transaction
end
local function LclSendFile(Client, FileStr)
local Hnd = io.open(FileStr, "rb")
if Hnd then
local Str = Hnd:read("*all")
if Str then
local ExtStr = string.lower(string.match(FileStr, "%P+$"))
local MimeStr = Cn.MimeList[ExtStr] or "application/octet-stream"
LclSend(Client, Str, MimeStr)
else
LclSend(Client, 'Error reading file.', 'text/plain',
'500 Internal Server Error')
end
Hnd:close()
else
LclSend(Client, 'Error opening file.', 'text/plain', '404 Not Found')
end
end
local Addr, Port, Server, Client, Hdr, Loop
Server = socket.bind(Cn.Host, Cn.Port)
if Server then
Addr, Port = Server:getsockname()
if Addr and Port then
io.write("Waiting for connection from client on ", Addr, Port, "\n")
local PortStr = Port == 80 and "" or (":" .. Port)
io.write('To end server, request "http://11, Cn.Host, PortStr,
'/quit" from browser\n')
Loop = true
while Loop do
Client = Server:accept()
if Client then
io.write("Got client request\n")
Addr, Port = Client:getpeername()
if Addr and Port then
io.write("Connected to ", Addr, Port, "\n")
Hdr = LclHdrRead(Client)
ObjectShow(Hdr, "Hdr")
if not string.find(Hdr.Path, 1, true) then
if Hdr.Path == "quit" then
LclSend(Client, "Shutdown", "text/plain")
Loop = false
else
LclSendFile(Client, Hdr.Path)
end
else
LclSend(Client, 'Unauthorized', 'text/plain', '401 Unauthorized')
end
else
io.write("Could not retrieve client address\n")
end
Client:close()
else
io.write("Error connecting to client\n")
end
end
io.write("Ending server loop\n")
else
io.write("Could not retrieve server address\n")
end
Server:close()
else
io.write("Error creating server socket\n")
end
2. Save the file as webserver.lua.
3. Before starting the server, create a web page and name it index.html. If you want to generate a sample page with a Lua script, you can use the following:
local function FncList(Tbl, Name)
local List = {}
for Key, Val in pairs(Tbl) do
if type(Val) == "function" then
List[#List + 1] = Key
end
end
table.sort(List)
io.write('<h1>', Name, ' library</h1>\n\n<p>')
for J, Str in ipairs(List) do
io.write(Str, " ")
end
io.write("</p>\n\n")
end
io.write('<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"\n',
'"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n\n',
'<html>\n\n<head>\n\n<title>Lua Environment</title>',
'\n\n</head>\n\n<body>\n\n')
local Namespace = {"coroutine", "debug", "io", "math", "os", "package",
"string", "table"}
FncList(_G, "base")
for J, Tbl in ipairs(Namespace) do
FncList(_G[Tbl], Tbl)
end
io.write('</body>\n\n</html>\n')
4. Save this file as env_page.lua and use it to create an index page as follows:
lua env_page.lua > index.html
5. Running this script requires that you don't currently have a TCP service listening on port 80 of your machine. If you do, change the value of Cn.Port at the top of the script to an unused port number. Fire up the web server as follows:
lua webserver.lua
6. With your web browser, access http://localhost/ to test the server. You can add links and references to images and style sheets in your index page to verify that the server returns everything as it should.
How It Works
This script has a very simple structure. First it creates a socket and binds it to the local machine (localhost is associated with the loopback address 127.0.0.1), as follows:
Server = socket.bind(Cn.Host, Cn.Port)
It then enters a loop in which it blocks on the following line, waiting for a connection request:
Client = Server:accept()
Because the socket is bound to the loopback device, you can only access the server from the local machine, but if your machine is part of a network, you can modify the value of Cn.Host and access the script from a different machine.
One limitation of this server is that only one request is handled at a time. A fair amount of work is required to address this deficiency in a robust and efficient way. An elegant way to approach this problem with Lua is to use coroutines, but even with these it can be a trick to get a central event dispatcher to conform to the Berkeley sockets model. Another limitation is that it expects each inbound header to occupy only one line. Finally, this implementation does not handle nested directories in the document root robustly.
Handling Multiple Persistent Connections
This section examines an effective way for a server to maintain ongoing connections with clients. When you interact with an FTP or telnet server, your connection is persistent: the server maintains the context of the interaction between transactions until you log off. The emerging SSE (Server-Sent Events) standard will carry this behavior over to web browsers as well.
Using Lua Coroutines with the select Function
Lua coroutines and the LuaSocket select function make it easy for a server application to handle simultaneous connections from multiple clients. In this example, a coroutine is dedicated to each client connection. Each coroutine is uncluttered with connection details or even an awareness of other concurrent connections, allowing it to focus on the ongoing dialog with a particular client.
Follow these steps:
1. Create a new file with your text editor and copy in the following Lua script:
local socket = require("socket")
local Cn = {SrvPort = 3072, SrvAddr = "localhost"}
local SckHnd = socket.connect(Cn.SrvAddr, Cn.SrvPort)
if SckHnd then
local Loop = true
local CnnSrvStr, CnnSrvPort = SckHnd:getpeername()
local CnnNameStr = socket.dns.tohostname(CnnSrvStr)
io.write(string.format("Connected to %s (%s) on port %d.\n",
CnnSrvStr, CnnNameStr, CnnSrvPort))
io.write('Issue .quit to end connection, .shutdown to terminate server.\n')
while Loop do
local Str, ErrStr
io.write("Send: ")
Str = io.read() or ".quit"
SckHnd:send(Str .. "\r\n")
Str, ErrStr = SckHnd:receive()
if Str then
io.write("Received: ", Str, "\n")
else
Loop = false
if ErrStr == "closed" then
io.write("Closing connection to server\n")
else
io.write("Error: ", ErrStr, "\n")
end
end
end
SckHnd:close()
else
io.write("Error creating client socket\n")
end
2. Save this file as client_02.lua.
3. Create another new file with your editor and copy in the following Lua script:
local socket = require("socket") —
local Cn = {HostPort = 3 072, HostAddr = "*"}
local function ClientSession(SckHnd)
local Loop, Str, ErrStr = true
local Rcv = {}
while Loop do
coroutine.yield(Str)
Str, ErrStr = SckHnd:receive()
if Str then
Loop = Str ~= ".quit" and Str ~= ".shutdown"
if Loop then
for J = 1, #Str do
Rcv[string.byte(Str, J, J)] = true
end
local SendStr = ""
for J = 33, 255 do
if Rcv[J] then
SendStr = SendStr .. string.char(J)
end
end
SckHnd:send(SendStr .. "\r\n")
end
else
io.write("Error: ", ErrStr, "\n")
end
end
return Str
end
local SckHnd = socket.bind(Cn.HostAddr, Cn.HostPort)
if SckHnd then
local SckList = {} -- Array of sockets, beginning with accepting socket
local CoList = {} -- Table of coroutines keyed by socket
local Loop = true
-- Prevent this socket from blocking for too long in call to accept
SckHnd:settimeout(250)
SckList[1] = SckHnd
while Loop do
io.write('Waiting for connection or data from\n')
for J, Hnd in ipairs(SckList) do
io.write(' ', J, ': ', tostring(Hnd), ', ', tostring(CoList[Hnd]), '\n')
end
local ReadTbl, WriteTbl, ErrStr = socket.select(SckList)
for K, SckHnd in ipairs(ReadTbl) do
if SckHnd == SckList[1] then -- Server socket
local ClientHnd, ErrStr = SckHnd:accept()
if ClientHnd then
local NewPos = #SckList + 1
SckList[NewPos] = ClientHnd
CoList[ClientHnd] = coroutine.wrap(ClientSession)
CoList[ClientHnd](ClientHnd)
elseif ErrStr ~= "timeout" then
io.write(ErrStr, "\n")
Loop = false
end
else -- Client connection
local Cmd = CoList[SckHnd]()
if ".quit" == Cmd then
CoList[SckHnd] = nil
SckHnd:close()
local L, Pos = #SckList
while L > 1 do
if SckHnd == SckList[L] then
table.remove(SckList, L)
L = 1 -- Terminate search
else
L = L - 1
end
end
elseif ".shutdown" == Cmd then
io.write("Shutting down server\n")
Loop = false
end
end
end
end
for J, SckHnd in ipairs(SckList) do
SckHnd:close()
end
else
io.write("Error creating server socket\n")
end
4. Save this file as server_02.lua.
5. Open a command shell, use cd to change to the directory where you saved the scripts, and execute the following server script:
The hexadecimal socket identifier will almost certainly be different in your case.
lua server_02.lua
Waiting for connection or data from
1: tcp{server}: 00658CA0, nil
6. Open another command shell, use cd to change to the directory where you saved the scripts, and execute the following client script:
lua client_02.lua
Connected to 127.0.0.1 (localhost) on port 3072.
Issue .quit to end connection, .shutdown to terminate server.
Send:
At this point, the server script prints new information indicating that the client has connected with it:
Waiting for connection or data from
1: tcp{server}: 00658CA0, nil
2: tcp{client}: 0065BD18, function: 007684A0
7. In response to the Send prompt, transmit some character sequences to the server, such as the following:
Send: ajx
Received: ajx
Send: bky
Received:abjkxy
Send: clz
Received:abcjklxyz
Send:
8. Open another command shell, use cd to change to the directory where you saved the scripts, and execute the following instance of the client script:
lua client_02.lua
Connected to 127.0.0.1 (localhost) on port 3072.
Issue .quit to end connection, .shutdown to terminate server.
Send:
The server script will now indicate multiple connected clients:
Waiting for connection or data from
1: tcp{server}: 00658CA0, nil
2: tcp{client}: 0065BD18, function: 007684A0
3: tcp{client}: 0065DD8C, function: 0076A1D0
9. Alternate between the two client scripts, inputting characters and verifying that the server is returning the set of accumulated characters for the particular connection.
10. End a client session as follows:
Send: .quit
Closing connection to server
Note that the server prints a summary with one fewer client connection.
11. Terminate the server from the remaining active client as follows:
Send: .shutdown
Closing connection to server
The server script ends after printing the following:
Shutting down server
The client script creates a TCP socket and connects it to a server with a well-known port (here, an arbitrarily selected value). This is done in one command:
local SckHnd = socket.connect(Cn.SrvAddr, Cn.SrvPort)
As presented, the server and clients all run on the same machine, but with the appropriate adjustment to Cn.SrvAddr, you can run the client script on a remotely connected machine. The following lines obtain information about the server for display purposes:
local CnnSrvStr, CnnSrvPort = SckHnd:getpeername()
local CnnNameStr = socket.dns.tohostname(CnnSrvStr)
A loop is entered in which the following things are done repeatedly:
1. Text is obtained from the user (io.read).
2. The acquired text is sent to the server (SckHnd:send).
3. The server's response is obtained (SckHnd:receive).
4. The response is displayed to the user (io.write).
This loop continues until one of the following occurs:
· The user issues the .quit command to end the session.
· The user issues the .shutdown command to terminate the server.
· Another client issues the .shutdown command. In this case, the client loop won't end until it attempts to send some data to the now defunct server.
Multiple Connections on the Server Side
Things are a little more involved on the server side, but the actual business logic (the code that actually interacts with the client) is refreshingly simple due to coroutines. The actual connection logic is something that you can write once and tuck away in a module.
The key to handling concurrent client connections is the socket.select function. As you saw in Chapter 9, a properly implemented dispatcher forms the basis for properly managing asynchronous events, which are events that originate outside of the application. The feature that you want is the ability to call a function that blocks until an event occurs. In this case, the event can be a connection request by a new client or the arrival of information from an already connected client. The socket.select fills this role by blocking until one or more of the sockets you indicate are ready for reading or writing. socket.select(receive array, send array, optional timeout in seconds) returns three values: a table of sockets from the receive array that have data waiting, a table of sockets from the send array that are ready to be written to, and an error message. On success the error message is nil.
Follow these steps to use socket.select:
1. Place the client and server sockets you expect input from into an array (a table indexed with contiguous integers beginning with 1), and pass this table as the first argument.
2. Place all the sockets that you are waiting to write to into another array. If either of these tables is empty, you can use nil instead. If you need the function to return after some designated number of seconds in the event that none of the specified sockets become ready for reading or writing, indicate a timeout value as the third argument.
The returned tables are structured as follows:
{
[1] = tcp{client}: 0065E214,
[2] = tcp{client}: 0065E394,
[tcp{client}: 0065E214] = 1,
[tcp{client}: 0065E394] = 2
}
You can see that each socket is represented two ways in the table: once as an array element and once as an associative key. The script presented here uses an ipairs loop to handle only the arrayed sockets.
All incoming connection requests are handled by the server socket. Recall that a server socket calls accept to establish a connection and create a new client socket, but does not exchange data with the remote socket. The server socket is kept in the first position of an array that holds all active sockets. Subsequent positions hold client sockets. This array is in the form required by socket.select so it is passed as the function's first argument (the receive array) to wait until inbound data or a connection request arrives.
This example is line oriented: the server expects to receive a single line from the client and in return sends back a line. Because of this, there are no special writing considerations and the second argument to socket.select (the send array) is passed as nil.
Setting Timeout Values for the Server Socket
One quirk in the Windows implementation of sockets is worked around in the multiple-connection example. Generally, a server socket can be included along with client sockets in the receive array, and an inbound connection request for the server socket is treated like inbound data for a client socket. However, Windows is known to occasionally report that a server socket is ready to accept a new connection when in fact it is not. To deal with this, you can do the following:
1. Set a timeout value for the server socket:
SckHnd:settimeout(250)
This forces a call to accept to return in a quarter second if in fact no connection requests are pending.
2. Check the return values of this function to determine whether a viable connection was made. In this example, a timeout condition terminates the main loop. In a real application, you would ignore it.
3. For each new client connection, create a coroutine and associate it with the client socket. When a client connection is terminated, this coroutine is set to nil and the client socket is removed from the read array.
The actual dialog between client and server is quite simple. Essentially it is a loop that begins with a yield. This transfers control back to the main script loop where it spends most of its time blocking in a call to socket.select. When this mechanism indicates that data has arrived from a client, the associated coroutine is resumed. The client data is read and examined. If it constitutes a termination request (either to end the connection or shut down the server) the dialog loop is terminated and the coroutine returns rather than yields. Otherwise, it processes the data and responds with its own data. In this example, the data returned is simply a string comprising the unique characters received from the client so far. This illustrates the ease with which the dialog state can be retained between transmissions.
The Application Protocols
LuaSocket's C layer interfaces with platform-specific sockets libraries to handle networking details at the TCP and UDP level. LuaSocket also provides a rich interface to the Internet's application protocols such as SMTP, FTP, HTTP and a number of modules to support them.
The application protocols in wide use throughout the Internet are presented in RFC (Request for Comments) documents that are informative and generally easy to read. If you need Lua to interact with an Internet server, such as a server that uses NNTP (Network News Transfer Protocol), the relevant RFCs will help you implement a working client. Visit www.rfc-editor.org for indexes to all RFCs.
Many application protocols involve transforming data prior to sending it or after receiving it. The following section describes an elegant solution that the LuaSocket libraries bring to this task.
Filtering the Flow of Data
As data is moved from one point to another, it often has to be manipulated to do the following:
· Conform with protocol constraints
· Enhance transmission efficiency
· Accommodate application formats and platform-dependent issues such as end-of-line conventions
LuaSocket includes a framework for flexibly massaging inbound and outbound data. It treats data as a fluid that is pumped from a source through a series of filters to a sink. While Diego Nehab created this framework in conjunction with the development of LuaSocket version 2, it is quite general and has been made into a standalone module named ltn12. An article Diego wrote detailing the principles of this framework (available on the Lua wiki at http://lua-users.org/wiki/FiltersSourcesAndSinks) was evidently slated to become the 12th in a series of Lua technical notes. The first eleven of these articles are maintained on the main Lua website; newer contributions by members of the Lua community are placed on the lua-users wiki.
The sources, filters and sinks framework implemented in ltn12 is based on the modification of data chunks rather than transacted data as a whole. This reduces the memory requirements of applications, especially server applications that handle concurrent connections involving large transactions. The basic parts of an ltn12 circuit are shown in the following table.
Many of the application protocols in use today were created when only ASCII text was being moved around on networks. To support the transmission of text that includes characters outside of the ASCII range as well as binary data such as images and compressed data, various encoding techniques have evolved to work within the original protocol limitations. MIME (Multipurpose Internet Mail Extensions) standardizes the ways non-ASCII data can be exchanged. Two common encoding methods are Quoted-Printable and Base64, which do the following:
· The Quoted-Printable encoding expands non-ASCII characters to a three-character sequence: an equal sign followed by a two-character hexadecimal representation of the out-of-range character. The equal sign itself is given the same treatment, as are tabs and spaces at the end of a line. For text messages that include occasional non-ASCII characters, this method of encoding has the advantage of being readable and compact.
· Base64 encoding is suited for transmitted data that includes many non-ASCII characters. In this method of encoding, all data is expanded so that each sequence of three bytes becomes four ASCII characters. In a sense, data is transformed from a 256 character alphabet to a 64 character alphabet.
The mime module included in LuaSocket implements encodings from the MIME standard. It requires the ltn12 module but is independent of the socket routines.
Try It Out
Text Hydraulics
The ltn12 module gives you everything you need to construct a source-to-sink data transformation routine. The mime module provides filters for encodings, text wrapping and end-of-line manipulation. Here you'll use both modules to demonstrate the ways to combine a source, filters, sink, and pump to encode and decode a string.
1. Create a Lua file with the following contents:
local mime = require("mime")
local ltn12 = require("ltn12")
-- This function receives one string and one or more high level filters. On
-- success it returns the filtered string, otherwise nil followed by an error
-- message.
local function Transform(Str, ... )
-- Source is specified string
local Src = ltn12.source.string(Str)
-- Chain all specified filters into one
local Filter = ltn12.filter.chain(... )
-- Send all data chunks to table
local Snk, Tbl = ltn12.sink.table()
-- Filter chunks before delivering to
sink Snk = ltn12.sink.chain(Filter, Snk)
-- Open the valve
local Code, ErrStr = ltn12.pump.all(Src, Snk)
return Code and table.concat(Tbl) or nil, ErrStr
end
local function Test(Str, EncodingStr)
local CodeStr, StatStr
CodeStr = Transform(Str, mime.encode(EncodingStr)) or ""
StatStr = Str == Transform(CodeStr, mime.decode(EncodingStr)) and
"OK" or "Not OK"
io.write(string.format("%-18s [%s] %s\n", EncodingStr, CodeStr, StatStr))
end
local Str = "Gabriel Garc\237a M\225rquez"
io.write(Str, "\n")
Test(Str, "base64")
Test(Str, "quoted-printable")
2. Run the script as follows:
lua mime_01.lua
Gabriel Garcia Marquez
base64 [R2FicmllbCBHYXJj7WEgTeFycXVleg==] OK
quoted-printable [Gabriel Garc=EDa M=E1rquez] OK
How It Works
The Transform function is a general-purpose routine for applying one or more filters to a string. Receiving and returning whole strings in some respects defeats the purpose of a filtered circuit, but even if the input and output strings are enormous, the actual filtering operations take place on smaller portions of data. The routine illustrates how you can place filters between a source and a sink. In this case, the amalgamated filter is chained to the sink (in ltn12.sink.chain), but you could just as easily have chained to the source instead (by using ltn12.source.chain).
The Test function receives a string to transform and a MIME encoding identifier (either base64 or quoted-printable). It uses this identifier to obtain an encoding filter with which it calls Transform to generate an encoded string. To test this, it compares the original string with a decoded version to make sure the process is functioning properly.
Examine the encoded versions of the text. For this example, the Quoted-Printable encoding makes the most sense, as most of the data is already in ASCII characters. The two equal signs at the end of the Base64-encoded string are padding added to make the length of the encoded string a multiple of four.
Accessing Web Pages
The http module of LuaSocket lets you retrieve a web resource with one function call. http.request (URL string, optional post body) requests a resource from a web server. If there is no second argument, the request is made using the GET method; otherwise the second argument is sent to the server as post data and the POST method is used. On success, the function returns four response values from the server. In order, these are the web resource body, the status code, the headers and the status line. On error, the function returns nil followed by an error message. If the function succeeds, remember to also check the status code.
When data is posted to the server, it is assumed to be urlencoded. See the section on CGI programming in Chapter 15 for more details.
The following global variables are consulted by http.request and can be used to modify its 'margin-top:1.5pt;margin-right:0cm;margin-bottom:1.5pt; margin-left:5.0pt;text-align:justify;text-indent:0cm;line-height:normal'>· PORT specifies the port used to contact the web server if a value isn't included in the URL.
· PROXY is used as a proxy server. If it is used, it should be in the form http://proxy.example.com:8080.
· TIMEOUT specifies the timeout value in seconds of the request.
· USERAGENT specifies the user-agent header that will be sent to the server.
An alternate version of the same function gives you finer-grained control over the HTTP request. Instead of submitting a URL to the function, you pass an associative table instead. http.request(request attribute table) requests a resource from a web server. Its return values are like the URL string version of this function, except that on success, the first value returned is 1 rather than the resource body.
The request attribute table can have values for the following keys (only the url field is mandatory):
· url specifies the URL of the requested resource.
· sink indicates where you want the retrieved resource to be stored. Common values are ltn12.sink.file to save the resource as a file, and ltn12.sink.table to store the retrieved chunks in a table where they can be easily converted to a string using table.concat.
· method should be "GET", "HEAD", or "POST".
· headers is an associative table that includes headers that will be sent to the server in addition to the standard headers. For example:
headers = {["content-length"] = 1094}
· You use source if you are posting data to the server. When using this, you should specify content-length in the headers table. For example:
source = ltn12.source.string(PostStr)
headers = {["content-length"] = string.len(PostStr)}
· step enables you to specify a step pump other than the default ltn12.pump.step.
· proxy allows you to request a resource through a proxy server. You use this like the PROXY global variable.
· You can set redirect to false to prevent server redirection.
· create specifies an alternate function to create the client socket.
Additionally, you can include any or all of the keys user, password, host, port, and path in the table with appropriate values to override the fields embedded in url.
When http.request is called with a table, it saves the retrieved resource to the specified sink rather than returning it.
Try It Out
Grabbing a File
You can conveniently retrieve binary files with http.request. Here the table form of the function is used with a file sink.
1. With your text editor, create a new Lua file with the following contents:
local http = require("socket.http")
require("show")
local ResFileStr = "lascii85.tar.gz"
local PathStr = "http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.0/"
local ResHnd, ErrStr = io.open(ResFileStr, "wb")
if ResHnd then
local Req = {
url = PathStr .. ResFileStr,
sink = ltn12.sink.file(ResHnd),
-- proxy = "http://proxy.example.com:8888"
}
local Response = {http.request(Req)}
ObjectShow(Response, "Response")
else
io.write("Error opening ", ResFileStr, " for writing\n")
end
2. Save this file as http_01.lua.
3. Run the script as follows:
lua http_01.lua
["Response"] table: 0041C6B0 (n = 4)
[1] 1
[2] 200
[3] table: 00429F28 (n = 0)
["accept-ranges"] "bytes"
["content-encoding"] "x-gzip"
["content-length"] "2641"
["content-type"] "application/x-gzip"
["date"] "Thu, 07 Sep 19:49:29 GMT"
["etag"] ""3f60e-a51-3fbb6735""
["last-modified"] "Wed, 19 Nov 2003 12:51:01 GMT"
["server"] "Apache"
["via"] "1.1 tinyproxy (tinyproxy/1.7.0)"
[4] "HTTP/1.1 200 OK"
Some headers may be different when you run the script. When the request completes, the lascii85.tar.gz file is saved in the current directory.
How It Works
The table form of http.request is used here for extra control over the file retrieval request. To save the downloaded file directly to disk, the ltn12.sink.file factory is called with the handle to a file opened for writing. The function this factory returns is called for each chunk of data received. When there is no more data to process, it closes the file handle.
If there are no special request attributes, you can retrieve the file directly into a string with the following call:
UrlStr = "http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.0/lascii85.tar.gz"
BodyStr, Code, Hdr, StatStr = http.request(UrlStr)
Sending and Receiving E-mail Messages
LuaSocket provides high-level support for sending messages with SMTP (Simple Mail Transfer Protocol), which is the standard means of sending Internet e-mail. LuaSocket currently doesn't support email retrieval directly, but its lower-level TCP socket routines let you do this if you follow the protocol—usually POP (Post Office Protocol) or IMAP (Internet Message Access Protocol)—that your mail server uses.
Try It Out
Sending E-mail
The smtp module comprises two functions: smtp.message and smtp.send. The first function prepares a message for sending by the second function. Both are used in the following example. This exercise works only with SMTP servers that accept plain-text password authentication.
1. Create a new file with the following Lua script:
local smtp = require("socket.smtp")
local Recipient = {"<recipient_1@example.net>", "<recipient_2@example.net>"}
-- Run the specified configuration file in an empty environment. On success,
-- returns a table that contains all global variables that are assigned in
-- configuration file, otherwise (nil, error string).
local function Configure(CfgFileStr)
local Gl, Cfg = getfenv(0), {}
setfenv(0, Cfg)
local Code, ErrStr = pcall(dofile, CfgFileStr)
setfenv(0, Gl)
return Code and Cfg or nil, ErrStr
end
local BodyStr = [[
Now the hungry lion roars,
And the wolf behowls the moon;
Whilst the heavy ploughman snores,
All with weary task foredone.
]]
local Msg = {
headers = {
to = "Oberon <oberon@example.com> Titania <titania@example.com>",
cc = "William Shakespeare <bard@globe.example.org>",
subject = "Moonlight",
from = "Puck <puck@example.com>",
},
body = BodyStr
}
local Packet = {}
local Code
local CfgFileStr = "smtp.cfg"
local Cfg, ErrStr = Configure(CfgFileStr)
if Cfg then
if type(Cfg.ServerStr) == "string" and type(Cfg.ServerPort) == "number" and
type(Cfg.UserStr) == "string" and type(Cfg.PassStr) == "string" then
io.write("Connecting to ", Cfg.ServerStr, "\n")
Packet.from = "<puck@example.com>"
Packet.rcpt = Recipient
Packet.source = smtp.message(Msg)
Packet.server = Cfg.ServerStr
Packet.port = Cfg.ServerPort
Packet.password = Cfg.PassStr
Packet.user = Cfg.UserStr
Code, ErrStr = smtp.send(Packet)
if Code then
io.write("Successfully sent messages")
end
else
ErrStr = "Improper configuration: '.. CfgFileStr
end
end
if ErrStr then
io.write(ErrStr, "\n")
end
2. Modify the contents of the Recipient table to include one or more of your e-mail addresses.
3. Save this file as smtp_01.lua.
4. Create a configuration file with your editor and add the following fields, using appropriate values for your SMTP account:
ServerStr = "mail.example.com"
ServerPort = 25
UserStr = "user"
PassStr = "password"
5. Save this file as smtp.cfg
6. Send the message as follows:
lua smtp_01.lua
Connecting to mail.example.com
Successfully sent messages
7. Use your regular e-mail retrieval software to verify that the message was sent or, if you have POP access to your e-mail account, you can retrieve it programmatically in the next exercise.
How It Works
The e-mail sending process begins by obtaining account particulars (such as the server name, user name, and password) as follows:
Cfg, ErrStr = Configure(CfgFileStr)
It's a good policy to keep this information out of general-purpose scripts. The technique used in the Configure function is to execute the configuration file as a Lua script that does not have access to library functions. Environment manipulation places the global variables it creates into a table that is returned by Configure.
After the presence of the fields expected in the configuration table have been verified, a table is prepared that will be used by smtp.send. This includes fields such as from, rcpt, source, server, port, password, and user.
The rcpt table specifies the only addresses to which the message is sent. As shown in this example, the to and cc fields of the message headers table are completely incidental to this process and are included for the purpose of display by the email client. Don't make the mistake of including a bcc field. To send a blind copy, simply include the recipient in the rcpt table and do not include it in either the to or cc header fields. This is done in the example.
The source field is a simple ltn12 source. Here, the smtp.message factory is used to condition the specified message and generate a suitable source.
Now that you've sent a message, you can retrieve it programmatically if you have POP access to your e-mail account.
Try It Out
Getting E-mail
The POP specification lets you list, retrieve, and delete e-mail that has arrived at your mail server. To do this with LuaSocket, you create a client socket and connect it with a POP server on which you have an account. The following example assumes that your POP server does not require an SSL (Secure Sockets Layer) or the APOP (Authenticated Post Office Protocol) command.
1. Create a new Lua file with the following contents:
local socket = require("socket")
-- Run the specified configuration file in an empty environment. On success,
-- returns a table that contains all global variables that are assigned in
-- configuration file, otherwise (nil, error string).
local function Configure(CfgFileStr)
local Gl, Cfg = getfenv(0), {}
setfenv(0, Cfg)
local Code, ErrStr = pcall(dofile, CfgFileStr)
setfenv(0, Gl)
return Code and Cfg or nil, ErrStr
end
-- Obtain POP server response. On success, return (string), otherwise (nil,
-- error string).
local function LclGet(PopHnd)
local Str, ErrStr = PopHnd:receive()
if Str then
-- All POP responses begin with "+OK " or "-ERR
if string.sub(Str, 1, 1) == "+" then
Str = string.sub(Str, 5)
else
ErrStr = string.sub(Str, 6)
Str = nil
end
end
-- return Ok, string.sub(Str, Ok and 5 or 6)
return Str, ErrStr
end
-- Send a command to the POP server and obtain its response. Returns (string)
-- on success and (nil, error string) otherwise. The Multi argument is true if
-- the caller expects a multiline response from the POP server. If Multi is
-- true and command succeeds, returns (string, table) where table contains
-- returned data lines, otherwise (nil, error string).
local function LclTransact(PopHnd, SendStr, Multi)
local Ok, Str, ErrStr, Tbl
Ok, ErrStr = PopHnd:send(SendStr .. "\r\n")
if Ok then
Str, ErrStr = LclGet(PopHnd)
if Str and Multi then
Tbl = {}
local DataStr
while Multi and not ErrStr do
DataStr, ErrStr = PopHnd:receive()
if DataStr then
if string.sub(DataStr, 1, 1) == "." then
DataStr = string.sub(DataStr, 2)
if DataStr == "" then
Multi = false
end
end
if Multi then
Tbl[#Tbl + 1] = DataStr
end
end
end
if ErrStr then
Str = nil
else
ErrStr = Tbl -- 2nd return value for Multi
end
end
end
return Str, ErrStr
end
-- Close the connection to the POP server. It is assumed that the server is in
-- a state to receive the "quit" command.
local function LclClose(PopHnd)
LclTransact(PopHnd, "quit")
PopHnd:close()
end
-- Connect to the POP server and authenticate user. If this function succeeds,
-- (client socket) is returned, otherwise (nil, error string) is returned.
local function LclOpen(Cfg)
local Ok, Str, RetHnd, PopHnd, ErrStr
PopHnd, ErrStr = socket.connect(Cfg.ServerStr, Cfg.ServerPort)
if PopHnd then
Str, ErrStr = LclGet(PopHnd)
if Str then
if LclTransact(PopHnd, "user " .. Cfg.UserStr) then
if LclTransact(PopHnd, "pass " .. Cfg.PassStr) then
RetHnd = PopHnd
end
end
if not RetHnd then
ErrStr = "Invalid username or password submitted"
end
end
if not RetHnd then
LclClose(PopHnd)
end
end
return RetHnd, ErrStr
end
-- This function requests header information for each stored email message. On
-- success, it returns (header table), otherwise it returns (nil, error
-- string). Only the first line of each multiline value is saved.
local function LclHeaders(PopHnd)
local Ok, Str, ErrStr, RetTbl, ListTbl
Str, ListTbl = LclTransact(PopHnd, "list", true)
if Str then
RetTbl = {}
for J, Str in ipairs(ListTbl) do
local HdrTbl, MsgTbl = {}
HdrTbl.size = tonumber(string.match(Str, "(%d+)$") or 0)
Str, MsgTbl = LclTransact(PopHnd, "top " .. J .. " 1", true)
if Str then
for J, Str in ipairs(MsgTbl) do
local KeyStr, ValStr = string.match(Str, nA([%a%-]+)%:%s*(.*)")
if KeyStr then
HdrTbl[string.lower(KeyStr)] = ValStr
end
end
RetTbl[#RetTbl + 1] = HdrTbl
else
ErrStr = ErrStr and (ErrStr .. ', ' .. MsgTbl) or MsgTbl
end
end
if ErrStr then
RetTbl = nil
end
else
ErrStr = ListTbl
end
return RetTbl, ErrStr
end
local Cfg, PopHnd, ErrStr, Ok, HdrTbl, CfgFileStr
CfgFileStr = "cfg"
Cfg, ErrStr = Configure(CfgFileStr)
if Cfg then
if type(Cfg.ServerStr) == "string" and type(Cfg.ServerPort) == "number" and
type(Cfg.UserStr) == "string" and type(Cfg.PassStr) == "string" then
io.write("Connecting to ", Cfg.ServerStr, "\n")
PopHnd, ErrStr = LclOpen(Cfg)
if PopHnd then
io.write("Retrieving headers\n")
HdrTbl, ErrStr = LclHeaders(PopHnd)
if HdrTbl then
for J, Rec in ipairs(HdrTbl) do
io.write("---Msg ", J, "---\n",
" Subject: ", Rec.subject or "---", "\n",
" From: ", Rec.from or "---", "\n".
" Date: ", Rec.date or "---", "\n".
" Size: ", Rec.size or "---", "\n")
end
end
io.write("Disconnecting from ", Cfg.ServerStr, "\n")
LclClose(PopHnd)
end
else
io.write("Improper configuration: ", CfgFileStr, "\n")
end
end
if ErrStr then
io.write(ErrStr, "\n")
end
2. Save this file as pop_01.lua.
3. Using your editor, create a new configuration file following this template:
PassStr = "pass"
ServerPort = 110
ServerStr = "mail.example.com"
UserStr = "user"
4. Save this file as cfg.
5. Substitute values that apply to your POP account.
6. Run the script as follows:
The particular headers that are displayed may vary according to your POP server.
lua pop_01.lua
Connecting to localhost
Retrieving headers
---Msg 1---
Subject: Moonlight
To: Oberon <oberon@example.com> Titania <titania@example.com>
Cc: William Shakespeare <bard@globe.example.org>
From: Puck <puck@example.com>
Date: Thu, 07 Sep 19:52:19 GMT
Size: 970
Disconnecting from localhost
How It Works
Like the previous example, the e-mail header retrieval process begins by obtaining account particulars (such as server name, user name, and password) as follows:
Cfg, ErrStr = Configure(CfgFileStr)
After the presence of the fields expected in the configuration table has been verified, a connection with the POP server is established. The authentication process shown here involves sending the following lines (where username and secret are replaced with values from the configuration file):
user username
pass secret
If the POP server is okay with each of these, a list of the waiting e-mail sizes is requested by sending the list command. If this command succeeds, the response will contain multiple lines. In this case, lines are read from the server until a line made up of only a single period is encountered. To address the possibility of such a line being transmitted in an email message, servers conventionally prepend a period to every line beginning with a period. Clients, including the one shown here, remove these initial periods. Based on the retrieved message list, the headers of each waiting message are requested using the top command. This is done so that a list of the message details can be presented to the user prior to downloading message bodies. Use the RETR command to retrieve the full body of the message. You can delete a message with theDELE command. Both of these commands are followed by a space and the one-based position of the message.
For more information about retrieving message bodies and dealing with possible encoding issues, refer to documentation concerning MIME techniques.
Networking with Lua and Streams
In this chapter, you've surveyed some (but by no means all) of the uses of the LuaSocket library. Other viable means of communicating over the Internet exist in the form of programs like inetd and ssh that handle the networking details for you.
On the Server Side: inetd and Friends
Unix-type systems and the Cygwin system for Windows come with a super-server application named inetd, xinetd or launchd that you can configure to listen on various ports. When a connection request arrives on one of these ports, it executes the program associated with the port, mapping the program's standard input, output and error streams to the new socket connection. In this way, your program can receive data from a remote client simply by reading its standard input stream, and send data back to the client by writing to its standard output stream. Many of the issues that arise when writing a full-fledged server application are taken care of by the super-server. These include managing execution privileges and restricting connections by address and frequency. Any program that can read from and write to its standard streams can be registered with a super-server, so what makes Lua special in this regard? In a nutshell, speed. Unlike a dedicated application server that runs continuously as a process, a server application registered with a super-server is executed whenever a new connection for it is made. Lua's small size and rapid loading make it a perfect choice for use in this arrangement.
Try It Out
Creating a Stream-Based Server
If you work on a Unix-type platform, including Cygwin for Windows, you can arrange to have a Lua program act as a network server by registering it with your system's super-server. The example shown here assumes you have xinetd available.
If you use a system with inetd, consult its man page for configuration details. In particular, you need to add an entry in /etc/services to give the listening port a service name. It is this service name rather than the port number itself that is put in the inetd.conf file.
The super-server usually runs as root and executes server programs with minimal permissions to reduce the possibility of a security breach. In the following example, the user nobody (often associated with web servers) is used in this capacity. You may want or need to replace references tonobody in the following files with another suitable low-authority user. In general, you should restrict the writers of server scripts as much as practical. In this example, only root can write to the script.
1. With your text editor, create a new Lua script with the following contents:
-- Sample echo server for testing with inetd and cousin xinetd
local Str = "Lua echo server opening / " .. os.date("%T")
while Str and Str ~= "" and Str ~= "\r" do
io.write(Str, "\r\n")
io.flush()
Str = io.read()
end
io.write("Lua echo server closing / ", os.date("%T"), "\r\n")
2. Save this file as echo.lua in a suitable directory. (/var/local/lua is used here, but the choice is yours.)
3. As root, modify the permissions of echo.lua so that it is readable by nobody and writeable only by suitable users, like this:
chown root:nobody echo.lua
chmod u=rw,g=r,o= echo.lua
4. In the /etc/xinetd.d directory, create a new file named lua-echo with the following contents, changing the directory and user if necessary:
# description: A sample Lua server
service lua-echo
{
socket_type = stream
type = UNLISTED
protocol = tcp
user = nobody
wait = no
server = /usr/local/bin/lua
server_args = /var/local/lua/echo.lua
disable = no
port = 23032
log_type = FILE /var/local/lua/log
log_on_success = PID HOST DURATION
log_on_failure = HOST
}
5. Restart the super-server. On some systems this is done with a command like this:
service xinetd restart
or this:
/etc/rc.d/rc.inetd restart
Consult the documentation for your particular super-server and platform.
6. Use telnet to connect with the server (localhost if you are accessing the echo server from the machine on which it will run or the name of the host if you are accessing the script remotely), and submit a blank line when you are ready to terminate the session, as follows:
You can also invoke concurrent connections.
telnet localhost 23032
Trying 127.0.0.1...
Connected to localhost.
Escape character is ,^]'.
Lua echo server opening / 09:40:56
abc
abc
xyz
xyz
Lua echo server closing / 09:41:08
Connection closed by foreign host.
How It Works
When the super-server receives a TCP connection request on port 23032, it makes sure that the remote client is eligible to be connected. If so, it executes the following, with the stdin, stdout, and stderr channels hooked to the socket connection:
/usr/local/bin/lua /var/local/lua/echo.lua
The echo script begins by writing a welcome message with the time. It then waits for input from the client and, when it receives a non-empty line, writes it back. The loop is broken when an empty line arrives from the client. The script then writes a closing line with the time and terminates. This causes the Lua interpreter to terminate as well, a condition that prompts the super-server to close the network connection.
On the Client Side: ssh and Friends
You can use the Lua io.popen function in conjunction with programs like ssh and nc (sometimes named netcat) to do for clients one-half of what super-servers provide for servers. Why one-half? Unfortunately, you can only open the stock popen in the C runtime library, on which Lua's io.popen is based, in either read mode or write mode but not both together. Here's the way it works: io.popen accepts the name of a program and program arguments as its first argument, and either "w" (or "wb") for writing or "r" (or "rb") for reading as its second argument. The "b" qualifiers indicate binary mode, an important consideration on the Windows platform. Here's an example use of io.popen:
Hnd, ErrStr = io.popen("nc server.example.net 4583", "r")
In this example, the netcat program (nc) will be invoked with the name of a server followed by the port on which the server is listening. Whatever this program writes to its standard output will be readable from the handle that io.popen returns. In this case, because the pipe is read-only, the launched program has to run without input.
The io.popen function isn't supported on all platforms, including older versions of Windows.
A versatile program named plink can be used with io.popen on Windows and Unix-type systems. This stream-based program is part of Simon Tatham's outstanding PuTTY suite of secure shell client applications. With it, you can connect to a server using the SSH or telnet protocol or use it to transfer data in “raw” mode. The SSH option makes it an excellent tool for accessing servers protected behind a firewall.
Try It Out
Using a Stream-Based Client
In this example, you programmatically connect to the server from the previous example. You need to have either netcat (www.vulnwatch.org/netcat) or plink (www.chiark.greenend.org.uk/~sgtatham/putty) to run the client script.
Because of the one-way nature of io.popen, the stdin channel of netcat or plink will be taken from a file. The stdout channel is attached to the handle returned from io.popen.
1. Create a new Lua script with the following contents. If you run the client script from a machine other than the one the echo server is running on, change the value of ServerStr accordingly. Also, there are two variations of CmdStr—comment or delete the one that doesn't apply depending on which network tool you will use.
local ServerStr = "localhost"
local Port = 23032
local CmdStr = "nc " .. ServerStr .. " " .. Port
-- local CmdStr = "plink -raw -P " .. Port .. " " .. ServerStr
local SendFileStr = "send.txt"
local Hnd = io.open(SendFileStr, "w")
if Hnd then
Hnd:write("One\nTwo\nThree\n\n")
Hnd:close()
Hnd = io.popen(CmdStr .. " > " .. SendFileStr)
if Hnd then
for Str in Hnd:lines() do
io.write(Str, "\n")
end
Hnd:close()
else
io.write("Error executing '", CmdStr,"'\n")
end
else
io.write("Error opening ", SendFileStr," for writing\n")
end
2. Save this file as client_03.lua.
3. Run the script as follows:
lua client_03.lua
Lua echo server opening / 10:39:31
One
Two
Three
Lua echo server closing / 10:39:31
How It Works
The first order of business for the client script is to prepare some lines to transmit to the echo server. It writes these lines to a file, taking care to include a blank line at the end to signal to the echo server a request to end the session. Putting these lines in a file is necessary because afterio.popen is called the script will have no way to send data to the echo server. The network utility, either netcat or plink, is invoked with a call to io.popen. The command uses the < redirector to attach the utility's standard input stream to the file just prepared. After making a network connection with the server, the lines are sent by the utility to the server. Server output is obtained by the client through the io.popen handle. The following line is used to loop through the lines from the server until the session has been ended:
for Str in Hnd:lines() do
Note that the Lua scripts on both the client and the server have external requirements for network connectivity, but don't require any Lua modules.
The technique of using external tools for network communication can facilitate many tasks. For example, if you administer a number of network hosts, you can write short scripts that collect and print critical information about each script's host. You can register each script with the super-server on its host or, more securely, invoke it through SSH. You can then run a client script to collect this information with one invocation. You can use the plink tool whether you access the scripts in raw mode or through SSH.
When implementing client and server scripts for network operations, moving basic data back and forth in the form of text lines often suffices. This can be limiting, however, for exchanging richly structured data. Currently, the prevailing standard is to exchange data in XML format, and libraries for Lua are available on luaforge.net to read and write in this format. If the scripts exchanging data are both written in Lua, however, another solution may be the easiest. As a data description language, Lua has a very clean and compact notation. Translating the Booleans, numbers, and strings of nested tables to Lua code is a straightforward variation of the ObjectDescribe you worked with in Chapter 7. In addition to being easy to write, data that is received in this format couldn't be easier to read—you let Lua do it for you with the loadstring function. For example, suppose you have a Lua data structure that would be created with the following code:
Classes = {
Laura = {"BIO 102", "SPA 221", "MTH 150", "CHM 100", Id = 120},
Jasmine = {"ENG 154", "LIT 204", "HUM 120", "PHY 100", Id = 390}
}
You can use a routine with the following characteristics to convert this structure back to Lua code:
· Special characters like quotes and newlines in strings are properly escaped.
· Boolean values are converted to “true” and “false.”
· Tables are handled by calling the routine recursively, with attention paid to already-defined tables to avoid unbound recursion.
The string that is returned by such a routine might look something like this:
['Classes'] = {Laura={[1] = "BIO 102",[2]="SPA 221",[3]="MTH 150", 
[4]="CHM 100",['Id']=120},Jasmine={[1]="ENG 154",[2]="LIT 204",
[3]="HUM 120",[4]="PHY 100",['Id']=390}}
With a little finesse, you could generate a rendition that is more readable. This readability doesn't matter to the receiving machine, but it might be desirable if some type of monitoring by people is required. When the receiving script receives the string, it can reconstruct the data structure as follows:
Fnc = loadstring("return {" .. Str .. "}")
if Fnc then
Tbl = Fnc()
end
To avoid the concatenation on the receiving side, wrap the string with return { and } on the sending side instead.
Summary
Network communication with Lua is well-supported by the excellent LuaSocket library and other tools such as the plink utility. In this chapter, you learned about the following:
· Berkeley sockets and their use in Internet communication
· LuaSocket's support for domain name queries, URL parsing, and socket communication
· Sending e-mail with SMTP and receiving it with POP
· Requesting web pages and other Internet resources via the Web
· The system of filters, sources, sinks, and pumps that can condition data in transit
· Using external tools for networking with standard streams
In the next chapter, you'll explore how Lua is used in computer games. But before you leave this chapter, try the following exercises to enhance your Lua networking skills.
Exercises
1. A simple cipher known as rot13 is often used to casually obscure text. This method is often used in newsgroup postings to veil potentially offensive jokes or the answers to riddles. This forces the reader to intentionally apply the cipher to make the text understandable. Each ASCII character in the text is effectively rotated 13 positions, so that characters in the range from A through M are shifted up to N through Z, and characters in the range N through Z are shifted down to A through M. A similar rotation takes place with lowercase letters. All characters outside of these ranges remain unaffected. The cipher rotates from the midpoint of each character range (that is A-Z and a-z), so it is its own inverse in the sense that applying it to converted text restores the original text.
Write a simple filter (a function that accepts a string and returns a converted string) that implements rot13. Use the Transform function from the Text Hydraulics example to check your filter. The decimal values of the characters A and a are 65 and 97, respectively.
Tbbq yhpx!
2. Extend the e-mail retrieval example to include the message body when displaying the headers if the size of the message is less than or equal to 4096 bytes. Do this by writing a function named LclBodyPrint that is called in the existing header loop as follows:
for J, Rec in ipairs(HdrTbl) do
io.write("--- Msg ", J, " ---\n",
"Subject: ",Rec.subject or "---", "\n".
"To: ",Recto or "---", "\n".
"Cc: ",Rec.cc or "---", "\n",
"From: ",Rec.from or "---", "\n".
"Date: ",Rec.date or "---", "\n".
"Size: ",Rec.size or "---", "\n")
if Rec.size <= 4096 then
LclBodyPrint(PopHnd, J)
end
end
The POP command to retrieve a message body is RETR position, where position is the value J in the header loop shown above. If the RETR command is successful, it returns the entire message including headers in multiple lines. Use the blank line between the headers and the message body to print only the body.
3. In the Stream-Based Client exercise, you used netcat or plink to pass three lines to an echo server. Do the same using the LuaSocket package rather than an external tool.