Creating an Application - Programming Google App Engine (2012)

Programming Google App Engine

Chapter 2. Creating an Application

The App Engine development model is as simple as it gets:

1. Create the application.

2. Test the application on your own computer by using the web server software included with the App Engine development kit.

3. Upload the finished application to App Engine.

In this chapter, we walk through the process of creating a new application, testing it with the development server, registering a new application ID and setting up a domain name, and uploading the app to App Engine. We look at some of the features of the Python and Java software development kits (SDKs) and the App Engine Administration Console. We also discuss the workflow for developing and deploying an app.

We will take this opportunity to demonstrate a common pattern in web applications: managing user preferences data. This pattern uses several App Engine services and features.

Setting Up the SDK

All the tools and libraries you need to develop an application are included in the App Engine SDK. There are separate SDKs for Python and Java, each with features useful for developing with each language. The SDKs work on any platform, including Windows, Mac OS X, and Linux.

The Python and Java SDKs each include a web server that runs your app in a simulated runtime environment on your computer. The development server enforces the sandbox restrictions of the full runtime environment and simulates each of the App Engine services. You can start the development server and leave it running while you build your app, reloading pages in your browser to see your changes in effect.

Both SDKs include a multifunction tool for interacting with the app running on App Engine. You use this tool to upload your app’s code, static files, and configuration. The tool can also manage datastore indexes, task queues, scheduled tasks, and service configuration, and can download messages logged by the live application so you can analyze your app’s traffic and behavior.

The Python SDK has a few tools not available in the Java SDK, mostly because the tools are written in Python (and so require that Python be installed). Notably, the Python SDK includes tools for uploading and downloading data to and from the datastore. This is useful for making backups, changing the structure of existing data, and for processing data offline. This tool and others work fine with Java applications, and if you’re using Java, you should consider installing Python and the App Engine Python SDK.

The Python SDKs for Windows and Mac OS X include a “launcher” application that makes it especially easy to create, edit, test, and upload an app, using a simple graphical interface. Paired with a good programming text editor (such as Notepad++ for Windows, or Sublime Text for Mac OS X), the launcher provides a fast and intuitive Python development experience.

For Java developers, Google provides a plug-in for the Eclipse integrated development environment that implements a complete App Engine development workflow. The plug-in includes a template for creating new App Engine Java apps, as well as a debugging profile for running the app and the development web server in the Eclipse debugger. To deploy a project to App Engine, you just click a button on the Eclipse toolbar.

Both SDKs also have cross-platform command-line tools that provide these features. You can use these tools from a command prompt, or otherwise integrate them into your development environment as you see fit. The Java SDK also includes an Apache Ant plug-in that makes it easy to integrate these tasks into an Ant-based workflow.

We discuss the Python SDK first, then the Java SDK in Installing the Java SDK. Feel free to skip the section that does not apply to your chosen language.

Installing the Python SDK

App Engine includes two Python runtime environments: a legacy environment based on Python 2.5, and a newer environment running Python 2.7. The newer environment has more than just a slightly newer version of the Python interpreter. In particular, the newer environment can serve multiple requests simultaneously from a single application instance, a performance-related feature that’ll prove useful when you start to get large amounts of traffic. If you’re creating a new app, there is no reason not to use Python 2.7, and if you have an existing app using the 2.5 runtime environment, you should consider upgrading. We assume the 2.7 environment for this tutorial and the rest of this book.

The App Engine SDK for the Python runtime environment runs on any computer that runs Python 2.7. If you are using Mac OS X or Linux, or if you have used Python previously, you may already have Python on your system. You can test whether Python is installed on your system and check which version is installed by running the following command at a command prompt (in Windows, Command Prompt; in Mac OS X, Terminal):

python -V

(That’s a capital “V.”) If Python is installed, it prints its version number, like so:

Python 2.7.1

You can download and install Python 2.7 for your platform from the Python website:

http://www.python.org/

Be sure to get Python version 2.7 (such as 2.7.2) from the “Download” section of the site. As of this writing, the latest major version of Python is 3.2, and the latest 2.x-compatible release is 2.7.

TIP

App Engine Python does not yet support Python 3. Python 3 includes several new language and library features that are not backward compatible with earlier versions. When App Engine adds support for Python 3, it will likely be in the form of a new runtime environment, in addition to the Python 2.5 and 2.7 environments. You control which runtime environment your application uses with a setting in the app’s configuration file, so your application will continue to run as intended when new runtime environments are released.

You can download the App Engine Python SDK bundle for your operating system from the Google App Engine website:

http://developers.google.com/appengine/downloads

Download and install the file appropriate for your operating system:

§ For Windows, the Python SDK is an .msi (Microsoft Installer) file. Click on the appropriate link to download it, then double-click on the file to start the installation process. This installs the Google App Engine Launcher application, adds an icon to your Start menu, and adds the command-line tools to the command path.

§ For Mac OS X, the Python SDK is a Mac application in a .dmg (disk image) file. Click on the link to download it, then double-click on the file to mount the disk image. Drag the GoogleAppEngineLauncher icon to your Applications folder. To install the command-line tools, double-click the icon to start the Launcher, then allow the Launcher to create the “symlinks” when prompted.

§ If you are using Linux or another platform, the Python SDK is available as a .zip archive. Download and unpack it (typically with the unzip command) to create a directory named google_appengine. The command-line tools all reside in this directory. Adjust your command path as needed.

To test that the App Engine Python SDK is installed, run the following command at a command prompt:

dev_appserver.py --help

The command prints a helpful message and exits. If instead you see a message about the command not being found, check that the installer completed successfully, and that the location of the dev_appserver.py command is on your command path.

Windows users, if when you run this command, a dialog box opens with the message “Windows cannot open this file... To open this file, Windows needs to know what program created it,” you must tell Windows to use Python to open the file. In the dialog box, choose “Select the program from a list,” and click OK. Click Browse, then locate your Python installation (such as C:\Python27). Select python from this folder, then click Open. Select “Always use the selected program to open this kind of file.” Click OK. A window will open and attempt to run the command, then immediately close. You can now run the command from the Command Prompt.

NOTE

Before proceeding, you will want to make sure that the Launcher is using your Python 2.7 installation, and not another version of Python that may be on your system. In particular, the Mac version of the Launcher will use /usr/bin/python2.6 by default, even if /usr/bin/python is Python 2.7.

To change the version of Python used by the Launcher, select Preferences from the appropriate menu, then specify a “Python Path” value of /usr/bin/python. Close the window to store this preference. If you already have a development server running, stop it, then start it again for the change to take effect.

You can confirm that the Launcher is using the correct version of Python by starting the server, then clicking the Logs button. Scroll up to the top, and look for the line that says: Python command: /usr/bin/python. If the path setting did not take effect, close the Launcher application, then start it and try again.

A brief tour of the Launcher

The Windows and Mac OS X versions of the Python SDK include an application called the Google App Engine Launcher (hereafter, just “Launcher”). With the Launcher, you can create and manage multiple App Engine Python projects using a graphical interface. Figure 2-1 shows an example of the Launcher window in Mac OS X.

The Google App Engine Launcher for Mac OS X main window, with a project selected

Figure 2-1. The Google App Engine Launcher for Mac OS X main window, with a project selected

To create a new project, select New Application from the File menu (or click the plus-sign button at the bottom of the window). Browse to where you want to keep your application files, then enter a name for the application. The Launcher creates a new directory at that location, named after the application, to hold the app’s files, and creates several starter files. The app appears in the application list in the main launcher window.

To start the development web server, make sure the application is selected, then click the Run button. You can stop the server with the Stop button. To open the home page of the running app in a browser, click the Browse button. The Logs button displays messages logged by the app in the development server.

The SDK Console button opens a web interface for the development server with several features for inspecting the running application, including tools to inspect the contents of the (simulated) datastore and memory cache, and an interactive console that executes Python statements and displays the results.

The Edit button opens the app’s files in your default text editor. In the Mac OS X version, this is especially useful with text editors that can open a directory’s worth of files, such as TextMate or Emacs. In the Windows version, this just opens app.yaml for editing.

The Deploy button uploads the application to App Engine. Before you can deploy an application, you must register an application ID with App Engine and edit the application’s configuration file with the registered ID. The Dashboard button opens a browser window with the App Engine Administration Console for the deployed app. We’ll look at the configuration file, the registration process, and the Administration Console later in this chapter.

The complete App Engine Python SDK, including the command-line tools, resides in the Launcher’s application directory. In the Windows version, the installer adds the appropriate directory to the command path, so you can run these tools from a Command Prompt.

In Mac OS X, when you start the Launcher for the first time it asks for permission to create “symlinks.” This creates symbolic links in the directory /usr/local/bin/ that refer to the command-line tools in the application bundle. With the links in this directory, you can type just the name of a command at a Terminal prompt to run it. If you didn’t create the symlinks, you can do so later by selecting the Make Symlinks item from the GoogleAppEngineLauncher menu.

You can set command-line flags for the development server within the Launcher. To do so, select the application, then go to the Edit menu and select Application Settings. Add the desired command-line options to the Extra Flags field, then click Update.

NOTE

The Mac OS X version of the Launcher installs Google’s software update facility to check for new versions of the App Engine SDK. When a new version is released, this feature notifies you and offers to upgrade.

Immediately after you upgrade, you’ll notice the symlinks stop working. To fix the symlinks, reopen the Launcher app and follow the prompts. The upgrade can’t do this automatically because it needs your permission to create new symlinks.

Installing the Java SDK

The App Engine SDK for the Java runtime environment runs on any computer that runs the Java SE Development Kit (JDK). The App Engine for Java SDK supports JDK 6, and when running on App Engine, the Java runtime environment uses the Java 6 JVM and JRE. (JDK 5 support is limited and deprecated.)

If you don’t already have it, you can download and install the Java 6 JDK for most platforms from Oracle’s website (Mac users, see the next section):

http://www.oracle.com/technetwork/java/javase/downloads/index.html

You can test whether the Java development kit is installed on your system and check which version it is by running the following command at a command prompt (in Windows, Command Prompt; in Mac OS X, Terminal):

javac -version

If you have the Java 6 JDK installed, the command will print a version number similar to javac 1.6.0. The actual output varies depending on which specific version you have.

App Engine Java apps use interfaces and features from Java Enterprise Edition (Java EE). The App Engine SDK includes implementations for the relevant Java EE features. You do not need to install a separate Java EE implementation.

The steps for installing the App Engine SDK for Java depend on whether you wish to use the Google Plugin for the Eclipse IDE. We’ll cover these situations separately.

Java on Mac OS X

Mac OS X versions 10.8 (Mountain Lion) and 10.7 (Lion) do not include the Java 6 runtime environment by default. If you are running Mac OS X 10.8, you may be prompted to download and install Java 6 when you first run Eclipse. If you are running Mac OS X 10.7, and you did not upgrade from a previous major version of the operating system, you may have to download and install Java for OS X Lion from Apple’s website:

http://support.apple.com/kb/DL1421

If you are running Mac OS X 10.6 (Snow Leopard), or have upgraded to 10.7 from 10.6, you should already have Java 6 installed. You may want to run Apple’s Software Update (from the Apple menu) to ensure you have the latest minor version.

If you are using Mac OS X 10.5 (Leopard) and a 64-bit processor, Java 6 is installed, but you must explicitly change the default version of Java to be Java 6 by using the Java Preferences Utility. You can find this under /Applications/Utilities/. In the Java Applications list, drag the desired version (such as “Java SE 6, 64-bit”) to the top of the list. OS X uses the topmost version in the list that is compatible with your system. Leopard’s version of Java 6 only works with 64-bit processors.

If you have a 32-bit Mac running Leopard, you’re stuck using Java 5. Java 5 support in the App Engine SDK is deprecated. Consider upgrading to Mac OS X 10.6 Snow Leopard, which includes a 32-bit version of Java 6. (Mac OS X 10.7 does not work with 32-bit processors.)

If you are using Eclipse, make sure you get the version that corresponds with your processor. Separate versions of the “Eclipse IDE for Java EE Developers” bundle are available for 32-bit and 64-bit processors.

For more information about Java and Mac OS X, see Apple’s developer website:

http://developer.apple.com/java/

Installing the Java SDK with the Google Plugin for Eclipse

One of the easiest ways to develop App Engine applications in Java is to use the Eclipse IDE and the Google Plugin for Eclipse. The plug-in works with all versions of Eclipse from Eclipse 3.3 (Europa) to the Eclipse 4.2 (Juno). You can get Eclipse for your platform for free at the Eclipse website:

http://www.eclipse.org/

If you’re getting Eclipse specifically for App Engine development, get the “Eclipse IDE for Java EE Developers” bundle. This bundle includes several useful components for developing web applications, including the Eclipse Web Tools Platform (WTP) package.

You can tell Eclipse to use the JDK you have installed in the Preferences window. In Eclipse 4.2, select Preferences (Windows and Linux, in the Window menu; Mac OS X, in the Eclipse menu). In the Java category, select “Installed JREs.” If necessary, add the location of the SDK to the list, and make sure the checkbox is checked.

To install the App Engine Java SDK and the Google Plugin, use the software installation feature of Eclipse. In Eclipse 4.2, select Install New Software from the Help menu, then type the following URL in the “Work with” field and click the Add button:

http://dl.google.com/eclipse/plugin/4.2

(This URL does not work in a browser; it only works with the Eclipse software installer.)

In the dialog box that opens, enter “Google” for the name, then click OK. Several items are added to the list. For a minimal App Engine development environment, select Google Plugin for Eclipse, then expand the SDKs category and select Google App Engine Java SDK. Figure 2-2 shows the Install Software window with these items selected.

The Eclipse 4.2 (Juno) Install Software window, with the Google Plugin selected

Figure 2-2. The Eclipse 4.2 (Juno) Install Software window, with the Google Plugin selected

There’s other good stuff in here, all free of charge. Google Web Toolkit (GWT) is a development suite for making rich web user interfaces using Java, without having to write a single line of JavaScript. The Eclipse plug-in makes it easy to create GWT apps that run on App Engine. There’s also a set of tools for making apps for Android devices that use App Engine as a networked backend. If that’s of interest, you’ll also want to get the Android Development Toolkit from developer.android.com.

Check the boxes for the desired items, then click the Next button and follow the prompts.

For more information on installing the Google Plugin for Eclipse, including instructions for Eclipse 3.3 through 3.7, see the website for the plug-in:

http://developers.google.com/eclipse/

After installation, the Eclipse toolbar has a new drop-down menu button. The notifications bar at the bottom may also include a prompt to sign in with your Google account. These additions are shown in Figure 2-3.

The Eclipse 4.2 window with the Google Plugin installed, with the drop-down menu button open

Figure 2-3. The Eclipse 4.2 window with the Google Plugin installed, with the drop-down menu button open

The plug-in adds several features to the Eclipse interface:

§ The drop-down menu button, with shortcuts for creating a new web application project, deploying to App Engine, and other features

§ A Web Application Project item under New in the File menu

§ A Web Application debug profile, for running an app in the development web server under the Eclipse debugger

You can use Eclipse to develop your application, and to deploy it to App Engine. To use other features of the SDK, like downloading log data, you must use the command-line tools from the App Engine SDK. Eclipse installs the SDK in your Eclipse application directory, undereclipse/plugins/. The actual directory name depends on the specific version of the SDK installed, but it looks something like this:

com.google.appengine.eclipse.sdkbundle_1.7.1/appengine-java-sdk-1.7.1/

This directory contains command-line tools in a subdirectory named bin/. In Mac OS X or Linux, you may need to change the permissions of these files to be executable in order to use the tools from the command line:

chmod 755 bin/*

You can add the bin/ directory to your command path, but keep in mind that the path will change each time you update the SDK.

Installing the Java SDK without Eclipse

If you are not using the Eclipse IDE or otherwise don’t wish to use the Google Plugin, you can download the App Engine Java SDK as a .zip archive from the App Engine website:

http://developers.google.com/appengine/downloads

The archive unpacks to a directory with a name like appengine-java-sdk-1.7.1.

The SDK contains command-line launch scripts in the bin/ subdirectory. You can add this directory to your command path to make the commands easier to run.

TIP

Both the AppCfg tool and the development web server execute Java classes to perform their functions. You can integrate these tools into your IDE or build scripts by calling the launch scripts, or by calling the Java classes directly. Look at the contents of the launch scripts to see the syntax.

The App Engine SDK includes a plug-in for Apache Ant that lets you perform functions of the SDK from an Ant build script. See the App Engine documentation for more information about using Ant with App Engine.

Test that the App Engine Java SDK is installed properly by running the following command at a command prompt:

dev_appserver --help

Mac OS X and Linux users, use dev_appserver.sh as the command name.

The command prints a helpful message and exits. If instead you see a message about the command not being found, check that the archive unpacked successfully, and that the SDK’s bin/ directory is on your command path.

Developing the Application

An App Engine application responds to web requests. It does so by calling request handlers, routines that accept request parameters and return responses. App Engine determines which request handler to use for a given request from the request’s URL, using a configuration file included with the app that maps URLs to handlers.

An app can also include static files, such as images, CSS stylesheets, and browser JavaScript. App Engine serves these files directly to clients in response to requests for corresponding URLs without invoking any code. The app’s configuration specifies which of its files are static, and which URLs to use for those files.

The application configuration includes metadata about the app, such as its application ID and version number. When you deploy the app to App Engine, all the app’s files, including the code, configuration files, and static files, are uploaded and associated with the application ID and version number mentioned in the configuration. An app can also have configuration files specific to the services, such as for datastore indexes, task queues, and scheduled tasks. These files are associated with the app in general, not a specific version of the app.

The structure and format of the code and configuration files differ for Python apps and for Java apps, but the concepts are similar. In the next few sections, we create the files needed for a simple application in Python and Java, and look at how to use the tools and libraries included with each SDK.

The User Preferences Pattern

The application we create in this section is a simple clock. When a user visits the site, the app displays the current time of day according to the server’s system clock. By default, the app shows the current time in the Coordinated Universal Time (UTC) time zone. The user can customize the time zone by signing in using Google Accounts and setting a preference.

This app demonstrates three App Engine features:

§ The datastore, primary storage for data that is persistent, reliable, and scalable

§ The memory cache (or memcache), secondary storage that is faster than the datastore, but is not necessarily persistent in the long term

§ Google Accounts, the ability to use Google’s user account system for authenticating and identifying users

Google Accounts works similarly to most user account systems. If the user is not signed in to the clock application, she sees a generic view with default settings (the UTC time zone) and a link to sign in or create a new account. If the user chooses to sign in or register, the application directs her to a sign-in form managed by Google Accounts. Signing in or creating an account redirects the user back to the application.

Of course, you can implement your own account mechanism instead of using Google Accounts. You can also use an OpenID provider (or a provider of the user’s choosing) with App Engine’s built-in OpenID support. Using Google Accounts or OpenID has advantages and disadvantages—the chief advantage being that you don’t have to implement your own account mechanism. If a user of your app already has a Google account, the user can sign in with that account without creating a new account for your app.

If the user accesses the application while signed in, the app loads the user’s preferences data and uses it to render the page. The app retrieves the preferences data in two steps. First, it attempts to get the data from the fast secondary storage, the memory cache. If the data is not present in the memory cache, the app attempts to retrieve it from the primary storage (the datastore), and if successful, it puts it into the memory cache to be found by future requests.

This means that for most requests, the application can get the user’s preferences from the memcache without accessing the datastore. While reading from the datastore is reasonably fast, reading from the memcache is much faster and avoids the cost of a datastore call. The difference is substantial when the same data must be accessed every time the user visits a page.

Our clock application has two request handlers. One handler displays the current time of day, along with links for signing in and out. It also displays a web form for adjusting the time zone when the user is signed in. The second request handler processes the time zone form when it is submitted. When the user submits the preferences form, the app saves the changes and redirects the browser back to the main page.

The application gets the current time from the application server’s system clock. It’s worth noting that App Engine makes no guarantees that the system clocks of all its web servers are synchronized. Since two requests for this app may be handled by different servers, different requests may see different clocks. The server clock is not consistent enough as a source of time data for a real-world application, but it’s good enough for this example.

In the next section, we implement this app using Python. We do the same thing with Java in the section Developing a Java App. As before, feel free to skip the section that doesn’t apply to you.

Developing a Python App

The simplest Python application for App Engine is a single directory with two files: a configuration file named app.yaml, and a file of Python code for a request handler. The directory containing the app.yaml file is the application root directory. You’ll refer to this directory often when using the tools.

TIP

If you are using the Launcher, you can start a new project by selecting the File menu, New Application. The Launcher creates a new project with several files, which you may wish to edit to follow along with the example. Alternatively, you can create the project directory and files by hand, then add the project to the Launcher by clicking the File menu, and then Add Existing Application.

Create a directory named clock to contain the project. Using your favorite text editor, create a file inside this directory named app.yaml similar to Example 2-1.

Example 2-1. The app.yaml configuration file for a simple application, using the Python 2.7 runtime environment

application: clock

version: 1

runtime: python27

api_version: 1

threadsafe: true

handlers:

- url: .*

script: main.application

libraries:

- name: webapp2

version: "2.5.1"

This configuration file is in a format called YAML, an open format for configuration files and network messages. You don’t need to know much about the format beyond what you see here.

In this example, the configuration file tells App Engine that this is version 1 of an application called clock, which uses version 1 (api_version) of the Python 2.7 runtime environment. Every request for this application (every URL that matches the regular expression .*, which is all of them) is to be handled by an application object defined in the application variable of a Python module named main.

TIP

In the Python 2.5 runtime environment, URLs are mapped to the names of source files, which are executed as CGI scripts. While this is still supported with Python 2.7, concurrent requests (multithreading) require this new way of referring to the WSGI instance global variable in the app.yaml configuration.

Create a file named main.py similar to Example 2-2, in the same directory as app.yaml.

Example 2-2. A simple Python web application, using the webapp2 framework

import datetime

import webapp2

class MainPage(webapp2.RequestHandler):

def get(self):

message = '<p>The time is: %s</p>' % datetime.datetime.now()

self.response.out.write(message)

application = webapp2.WSGIApplication([('/', MainPage)],

debug=True)

This simple Python web application uses a web application framework called “webapp2,” which is included with App Engine. This framework conforms to a common standard for Python web application frameworks known as the Web Server Gateway Interface (WSGI). You don’t need to know much about WSGI, except that it’s a Python standard, there are many useful frameworks to choose from, and it’s easy to port a WSGI application to other application hosting environments using various adapters (such as a WSGI-to-CGI adapter). webapp2 is a simple example of a WSGI framework. Django, a popular open source web app framework for Python that’s also included with App Engine, is another.

We’ll walk through this example in a moment, but first, let’s get it running. If you are using the Launcher app, start the development web server by clicking the Run button. The icon next to the project turns green when the server starts successfully.

Open a browser to view the project by clicking the Browse button. The browser displays a page similar to Figure 2-4.

The first version of the clock application viewed in a browser

Figure 2-4. The first version of the clock application viewed in a browser

You can leave the web server running while you develop your application. The web server notices when you make changes to your files, and reloads them automatically as needed.

If you are not using the Launcher, you can start the development server from a command prompt by running the dev_appserver.py command, specifying the path to the project directory (clock) as an argument:

dev_appserver.py clock

TIP

If your current working directory is the clock directory you just created, you can run the command using a dot (.) as the path to the project:

dev_appserver.py .

The server starts up and prints several messages to the console. If this is the first time you’re running the server from the command line, it may ask whether you want it to check for updates; type your answer, then hit Enter. You can safely ignore warnings that say “Could not read datastore data” and “Could not initialize images API.” These are expected if you have followed the installation steps so far. The last message should look something like this:

INFO ... Running application dev-clock on port 8080: http://localhost:8080

INFO ... Admin console is available at: http://localhost:8080/_ah/admin

This message indicates the server started successfully. If you do not see this message, check the other messages for hints, and double-check that the syntax of your app.yaml file is correct.

Test your application by visiting the server’s URL in a web browser:

http://localhost:8080/

Introducing the webapp framework

App Engine’s Python 2.7 runtime environment uses WSGI as the interface between your application and the server instance running the application. Typically, you would not write the code that implements this interface. Instead, you would use a framework, a suite of libraries and tools that form an easy way to think about building web applications and perform common web tasks.

There are dozens of web frameworks written in Python, and several are mature, well documented, and have active developer communities. Django, web2py, and Pylons are examples of well-established Python web frameworks. But not every Python web application framework works completely with the App Engine Python runtime environment. Constraints imposed by App Engine’s sandboxing logic limit which frameworks work out of the box. In addition to App Engine’s own webapp2 framework, Django (http://www.djangoproject.com/) is known to work well and is included with App Engine. web2py (http://web2py.com/) includes special support for App Engine. Others have been adapted for App Engine with additional software. We’ll discuss how to use Django with App Engine in Chapter 18.

The webapp2 framework (the successor to “webapp,” the framework included with the legacy Python 2.5 environment) is intended to be small and easy to use. It doesn’t have the features of more established frameworks, but it’s good enough for small projects. For simplicity, most of the Python examples in this book use the webapp2 framework. We’ll introduce some of its features here.

Let’s take a closer look at our simple web application, line by line:

import datetime

import webapp2

This loads the libraries we intend to use in the main module. We use datetime to get the system time for our clock. The webapp2 framework is in the webapp2 module:

class MainPage(webapp2.RequestHandler):

def get(self):

message = '<p>The time is: %s</p>' % datetime.datetime.now()

self.response.out.write(message)

webapp2 applications consist of one or more request handlers, units of code mapped to URLs or URL patterns that are executed when a client (or other process) requests a URL. As we saw earlier, the first URL mapping takes place in app.yaml, which associates the request URL with its WSGI application object in a Python module. The webapp2 application maps this to a RequestHandler class.

To produce the response, webapp2 instantiates the class and then calls a method of the class that corresponds to the HTTP method of the request. When you type a URL into your browser’s address bar, the browser uses the HTTP GET method with the request, so webapp2 calls the get()method of the request handler. Similarly, when you submit a web form, the browser uses the HTTP POST method, which would attempt to call a post() method.

The code can access the request data and produce the response data, using attributes of the instance. In this case, we prepared a response string (message), then used the output stream of the response attribute to write the message. You can also use the response attribute to set response headers, such as to change the content type. (Here, we leave the content type at its default of text/html.)

application = webapp2.WSGIApplication([('/', MainPage)],

debug=True)

The application module global variable contains the object that represents the WSGI application. This value is created when the main module is imported for the first time, and stays in memory for the lifetime of the application instance. (App Engine creates and destroys application instances as needed to serve your app’s traffic. More on that later.) App Engine knows which module and variable to use from the mapping in the app.yaml file.

The application object is an instance of the WSGIApplication class provided by the webapp2 module. The constructor is called with two values. The first is a list of URL pattern and RequestHandler class pairs. When the application is called to handle a request, the URL is tested against each pattern in the order it appears in the list. The first to match wins. The URL pattern is a regular expression.

In this case, our application simply maps the root URL path (/) to MainPage. If the application is asked to handle any other URL path (any path that doesn’t match), webapp2 serves an HTTP 404 error page. Notice that the app.yaml file maps all URL paths to this application, effectively putting webapp2 in charge of serving 404 errors. (If a URL does not match any pattern in app.yaml, App Engine serves its own 404 error.)

The WSGIApplication constructor is also given a debug=True parameter. This tells webapp2 to print detailed error messages to the browser when things go wrong. webapp2 knows to only use this in the development server, and disable this feature when it is running on App Engine, so you can just leave it turned on.

A single WSGIApplication instance can handle multiple URLs, routing the request to different RequestHandler classes based on the URL pattern. But we’ve already seen that the app.yaml file maps URL patterns to handler scripts. So which URL patterns should appear in app.yaml, and which should appear in the WSGIApplication? Many web frameworks include their own URL dispatcher logic, and it’s common to route all dynamic URLs to the framework’s dispatcher in app.yaml. With webapp2, the answer mostly depends on how you’d like to organize your code. For the clock application, we will create a second request handler as a separate script to take advantage of a feature of app.yaml for user authentication, but we could also put this logic in main.py and route the URL with the WSGIApplication object.

Users and Google Accounts

So far, our clock shows the same display for every user. To allow each user to customize the display and save her preferences for future sessions, we need a way to identify the user making a request. An easy way to do this is with Google Accounts, a.k.a. the Users service.

Before we make the user interface of our app more elaborate, let’s introduce a templating system to manage our HTML. User-facing web applications have a browser-based user interface, consisting of HTML, CSS, and sometimes JavaScript. Mixing markup and code for the browser in your server-side code gets messy fast. It’s nearly always better to use a library that can represent the user interface code separately from your app code, using templates. The app code calls the templating system to fill in the blanks with dynamic data and render the result.

For this example, we’ll use the Jinja2 templating system. Jinja2 is an open source templating system written in Python, based on the templating system included with the Django web application framework. App Engine will provide this library to your app if you request it in the app.yaml file.

Edit app.yaml, and add these lines to the libraries: section near the bottom:

libraries:

# ...

- name: jinja2

version: latest

- name: markupsafe

version: latest

While libraries such as Jinja2 are available to your app when it is running on App Engine, you must install the library yourself on your own computer. (App Engine does not include every library in the SDK because it would have to include every supported version of every library, and that could get large.) To install Jinja2, use the easy_install command included with Python. For example, on Mac OS X:

sudo easy_install jinja2 markupsafe

(Enter your administrator password when prompted.)

TIP

Make sure to use the same version of Python to install the library as the version of Python you’re using to run the development server! See the tip back in Installing the Python SDK on setting up the Launcher.

Let’s add something to our app’s home page that indicates whether the user is signed in, and provides links for signing in and signing out of the application. Edit main.py to resemble Example 2-3.

Example 2-3. A version of main.py that invites the user to sign in with Google Accounts, using a Jinja2 template

import datetime

import jinja2

import os

import webapp2

from google.appengine.api import users

template_env = jinja2.Environment(

loader=jinja2.FileSystemLoader(os.getcwd())

class MainPage(webapp2.RequestHandler):

def get(self):

current_time = datetime.datetime.now()

user = users.get_current_user()

login_url = users.create_login_url(self.request.path)

logout_url = users.create_logout_url(self.request.path)

template = template_env.get_template('home.html')

context = {

'current_time': current_time,

'user': user,

'login_url': login_url,

'logout_url': logout_url,

}

self.response.out.write(template.render(context)

application = webapp2.WSGIApplication([('/', MainPage)],

debug=True)

Next, create a new file in the same directory named home.html, and edit it to resemble Example 2-4. This is the Jinja2 template.

Example 2-4. The HTML template for the home page, using the Jinja2 template system

<html>

<head>

<title>The Time Is...</title>

</head>

<body>

{% if user %}

<p>

Welcome, {{ user.email() }}!

You can <a href="{{ logout_url }}">sign out</a>.

</p>

{% else %}

<p>

Welcome!

<a href="{{ login_url }}">Sign in or register</a> to customize.

</p>

{% endif %}

<p>The time is: {{ current_time }}</p>

</body>

</html>

Reload the page in your browser. The new page resembles Figure 2-5.

The clock app with a link to Google Accounts when the user is not signed in

Figure 2-5. The clock app with a link to Google Accounts when the user is not signed in

We’ve added a few new things to main.py:

import jinja2

import os

# ...

from google.appengine.api import users

You import the Jinja2 library the same way you would with a typical installation on your computer. The libraries: section of app.yaml puts Jinja2 on the library search path when running on App Engine.

We also import the os module to use in the next part, as well as the API for the Users service:

template_env = jinja2.Environment(

loader=jinja2.FileSystemLoader(os.getcwd())

One way to configure Jinja2 is with an Environment object. This object maintains aspects of the template system that are common across your app. In this case, we use the Environment to declare that our template files are loaded from the file system, using the FileSystemLoader.

We store the Jinja2 Environment object in a module global variable because we only need to create this object once in the lifetime of the application instance. As with the WSGIApplication object, the constructor is called when the module is imported, and the object stays resident in memory.

TIP

Remember that we’ve turned on concurrent requests using the threadsafe: true line in app.yaml. This tells App Engine to use one instance to process multiple requests simultaneously. These requests will share global module variables, and may interleave instructions. This is fine for most common read-only uses of global variables, such as configuration data and compiled regular expressions.

The os.getcwd() value passed to the FileSystemLoader constructor tells it to find templates in the current working directory. When the request handler is called, the current working directory is the application root directory. If you move your templates into a subdirectory (and that’s probably a good idea), this value needs to be modified accordingly:

current_time = datetime.datetime.now()

user = users.get_current_user()

login_url = users.create_login_url(self.request.path)

logout_url = users.create_logout_url(self.request.path)

The request handler code calls the Users service API by using functions in the module users, from the package google.appengine.api. users.get_current_user() returns an object of class users.User that represents the user making the request if the user is signed in, or None(Python’s null value) if the user is not signed in. You can use this value to access the user’s email address, which our application does from within the template.

To allow a user to sign in or sign out, you direct the user’s browser to the Google Accounts system “login” or “logout” URLs. The app gets these URLs using the users.create_login_url() and users.create_logout_url(), respectively. These functions take a URL path for your application as an argument. Once the user has signed in or signed out successfully, Google Accounts redirects the user back to your app using that URL path. For this app, we direct the user to sign in or sign out by presenting her with links to click. (In other situations, redirecting the user might be more appropriate.)

template = template_env.get_template('home.html')

context = {

'current_time': current_time,

'user': user,

'login_url': login_url,

'logout_url': logout_url

}

self.response.out.write(template.render(context)

Here, we load the home.html template, set the dynamic data in the template’s “context,” render the template with the context values into the text of the page, and finally write it to the response.

Within the template, we use a conditional section to display a different welcome message depending on whether the user is signed in or not. {% if user %} is true if the context value we set to 'user' is considered true in Python, which it would be if users.get_current_user()returned a User object. The {% else %} and {% endif %} directives delimit the sections of the template to render, based on the condition. {{ user.email() }} calls the email() method of the object, and interpolates its return value into the template as a string. Similarly, {{ logout_url }}, {{ login_url }}, and {{ current_time }} interpolate the generated URLs we set in the context.

TIP

For more information about Jinja2 template syntax and features, see the Jinja2 website:

http://jinja.pocoo.org/

If you click on the “Sign in or register” link with the app running in the development server, the link goes to the development server’s simulated version of the Google Accounts sign-in screen, as shown in Figure 2-6. At this screen, you can enter any email address, and the development server will proceed as if you are signed in with an account that has that address.

The development server’s simulated Google Accounts sign-in screen

Figure 2-6. The development server’s simulated Google Accounts sign-in screen

If this app were running on App Engine, the login and logout URLs would go to the actual Google Accounts locations. Once signed in or out, Google Accounts redirects back to the given URL path for the live application.

Click on “Sign in or register,” then click on the Login button on the simulated Google Accounts screen, using the default test email address (test@example.com). The clock app now looks like Figure 2-7. To sign out again, click the “sign out” link.

The clock app, with the user signed in

Figure 2-7. The clock app, with the user signed in

Web forms and the datastore

Now that we know who the user is, we can ask her for her preferred time zone, remember her preference, and use it on future visits.

First, we need a way to remember the user’s preferences so future requests can access them. The App Engine datastore provides reliable, scalable storage for this purpose. The Python API includes a data modeling interface that maps Python objects to datastore entities. We can use it to write a UserPrefs class.

Create a new file named models.py, as shown in Example 2-5.

Example 2-5. The file models.py, with a class for storing user preferences in the datastore

from google.appengine.api import users

from google.appengine.ext import db

class UserPrefs(db.Model):

tz_offset = db.IntegerProperty(default=0)

user = db.UserProperty(auto_current_user_add=True)

def get_userprefs(user_id=None):

if not user_id:

user = users.get_current_user()

if not user:

return None

user_id = user.user_id()

key = db.Key.from_path('UserPrefs', user_id)

userprefs = db.get(key)

if not userprefs:

userprefs = UserPrefs(key_name=user_id)

return userprefs

The Python data modeling interface is provided by the module db in the package google.appengine.ext. A data model is a class whose base class is db.Model. The model subclass defines the structure of the data in each object by using class properties. This structure is enforced bydb.Model when values are assigned to instance properties. For our UserPrefs class, we define two properties: tz_offset, an integer, and user, a User object returned by the Google Accounts API.

Every datastore entity has a primary key. Unlike a primary key in a relational database table, an entity key is permanent and can only be set when the entity is created. A key is unique across all entities in the system, and consists of several parts, including the entity’s kind (in this case'UserPrefs'). An app can set one component of the key to an arbitrary value, known in the API as the key name.

The clock application uses the user’s unique ID, provided by the user_id() method of the User object, as the key name of a UserPrefs entity. This allows the app to fetch the entity by key, since it knows the user’s ID from the Google Accounts API. Fetching the entity by key is faster than performing a datastore query.

In models.py, we define a function named get_userprefs() that gets the UserPrefs object for the user. After determining the user ID, the function constructs a datastore key for an entity of the kind 'UserPrefs' with a key name equivalent to the user ID. If the entity exists in the datastore, the function returns the UserPrefs object.

If the entity does not exist in the datastore, the function creates a new UserPrefs object with default settings and a key name that corresponds to the user. The new object is not saved to the datastore automatically. The caller must invoke the put() method on the UserPrefs instance to save it.

Now that we have a mechanism for getting a UserPrefs object, we can make two upgrades to the main page. If the user is signed in, we can get the user’s preferences (if any) and adjust the clock’s time zone.

Edit main.py. With the other import statements, import the models module we just created:

import models

In the request handler code, call the models.get_userprefs() function, and use the return value to adjust the current_time value. Also, add the userprefs value to the template context:

class MainPage(webapp2.RequestHandler):

def get(self):

# ...

userprefs = models.get_userprefs()

if userprefs:

current_time += datetime.timedelta(

0, 0, 0, 0, 0, userprefs.tz_offset)

template = template_env.get_template('home.html')

context = {

# ...

'userprefs': userprefs,

}

self.response.out.write(template.render(context)

Let’s also add a web form to the template so the user can set a time zone preference. Edit home.html, and add the following near the bottom of the template, above the </body>:

{% if user %}

<form action="/prefs" method="post">

<label for="tz_offset">

Timezone offset from UTC (can be negative):

</label>

<input name="tz_offset" id="tz_offset" type="text"

size="4" value="{{ userprefs.tz_offset }}" />

<input type="submit" value="Set" />

</form>

{% endif %}

To enable the preferences form, we need a new request handler to parse the form data and update the datastore. Let’s implement this as a new request handler module. (We’ll see why in a moment.)

Create a file named prefs.py with the contents shown in Example 2-6.

Example 2-6. A new handler module, prefs.py, for the preferences form

import webapp2

import models

class PrefsPage(webapp2.RequestHandler):

def post(self):

userprefs = models.get_userprefs()

try:

tz_offset = int(self.request.get('tz_offset')

userprefs.tz_offset = tz_offset

userprefs.put()

except ValueError:

# User entered a value that wasn't an integer. Ignore for now.

pass

self.redirect('/')

application = webapp2.WSGIApplication([('/prefs', PrefsPage)],

debug=True)

This request handler handles HTTP POST requests to the URL /prefs, which is the URL (“action”) and HTTP method used by the form. Because it’s an HTTP POST action, the code goes in the post() method (instead of the get() method used in main.py). The handler code calls theget_userprefs() function from models.py to get the UserPrefs object for the current user, which is either a new unsaved object with default values, or the object for an existing entity. The handler parses the tz_offset parameter from the form data as an integer, sets the property of the UserPrefs object, then saves the object to the datastore by calling its put() method. The put() method creates the object if it doesn’t exist, or updates the existing object.

If the user enters something other than an integer in the form field, we don’t do anything. It’d be appropriate to return an error message, but we’ll leave this as is to keep the example simple.

The form handler redirects the user’s browser to the / URL. In webapp2, the self.redirect() method takes care of setting the appropriate response headers for redirecting the browser.

Finally, edit app.yaml to map the handler module to the URL /prefs in the handlers: section, as shown in Example 2-7.

Example 2-7. A new version of app.yaml mapping the URL /prefs, with login required

application: clock

version: 1

runtime: python27

api_version: 1

threadsafe: true

handlers:

- url: /prefs

script: prefs.application

login: required

- url: .*

script: main.application

libraries:

- name: webapp2

version: "2.5.1"

- name: jinja2

version: latest

- name: markupsafe

version: latest

The login: required line says that the user must be signed in to Google Accounts to access the /prefs URL. If the user accesses the URL while not signed in, App Engine automatically directs the user to the Google Accounts sign-in page, then redirects her back to this URL afterward. This makes it easy to require sign-in for sections of your site, and to ensure that the user is signed in before the request handler is called.

Be sure to put the /prefs URL mapping before the /.* mapping. URL patterns are tried in order, and the first pattern to match determines the handler used for the request. Since the pattern /.* matches all URLs, /prefs must come first or it will be ignored.

Reload the page to see the customizable clock in action. Try changing the time zone by submitting the form. Also try signing out, then signing in again using the same email address, and again with a different email address. The app remembers the time zone preference for each user.

Caching with memcache

The code that gets user preferences data in Example 2-5 fetches an entity from the datastore every time a signed-in user visits the site. User preferences are often read and seldom changed, so getting a UserPrefs object from the datastore with every request is more expensive than it needs to be. We can mitigate the cost of reading from primary storage by using a caching layer.

We can use the memcache service as secondary storage for user preferences data. Because of the way we wrote models.py, adding caching requires just a few minor changes. Edit this file as shown in Example 2-8.

Example 2-8. A new version of models.py that caches UserPrefs objects in memcache

from google.appengine.api import memcache

from google.appengine.api import users

from google.appengine.ext import db

class UserPrefs(db.Model):

tz_offset = db.IntegerProperty(default=0)

user = db.UserProperty(auto_current_user_add=True)

def cache_set(self):

memcache.set('UserPrefs:' + self.key().name(), self)

def put(self):

super(UserPrefs, self).put()

self.cache_set()

def get_userprefs(user_id=None):

if not user_id:

user = users.get_current_user()

if not user:

return None

user_id = user.user_id()

userprefs = memcache.get('UserPrefs:' + user_id)

if not userprefs:

key = db.Key.from_path('UserPrefs', user_id)

userprefs = db.get(key)

if userprefs:

userprefs.cache_set()

else:

userprefs = UserPrefs(key_name=user_id)

return userprefs

The Python API for the memcache service is provided by the module memcache in the package google.appengine.api. The memcache stores key-value pairs. The value can be of any type that can be converted to and from a flat data representation (serialized), using the Python picklemodule, including most data objects.

The new version of the UserPrefs class overrides the put() method. When the put() method is called on an instance, the instance is saved to the datastore by using the superclass’s put() method, then it is saved to the memcache.

A new UserPrefs method called cache_set() makes the call to memcache.set(). memcache.set() takes a key and a value. Here, we use the string 'UserPrefs:' followed by the entity’s key name as the memcache key, and the full object (self) as the value. The API takes care of serializing the UserPrefs object, so we can put in and take out fully formed objects.

The new version of get_userprefs() checks the memcache for the UserPrefs object before going to the datastore. If it finds it in the cache, it uses it. If it doesn’t, it checks the datastore, and if it finds it there, it stores it in the cache and uses it. If the object is in neither the memcache nor the datastore, get_userprefs() returns a fresh UserPrefs object with default values.

Reload the page to see the new version work. To make the caching behavior more visible, you can add logging statements in the appropriate places in models.py, like so:

import logging

class UserPrefs(db.Model):

# ...

def cache_set(self):

logging.info('cache set')

# ...

The development server prints logging output to the console. If you are using the Launcher, you can open a window of development server output by clicking the Logs button.

That’s it for our Python app. Next, we’ll take a look at the same example using the Java runtime environment. If you’re not interested in Java, you can skip ahead to Registering the Application.

Developing a Java App

Java web applications for App Engine use the Java Servlet standard interface for interacting with the application server. An application consists of one or more servlet classes, each extending a servlet base class. Servlets are mapped to URLs using a standard configuration file called a “deployment descriptor,” also known as web.xml. When App Engine receives a request for a Java application, it determines which servlet class to use based on the URL and the deployment descriptor, instantiates the class, and then calls an appropriate method on the servlet object.

All the files for a Java application, including the compiled Java classes, configuration files, and static files, are organized in a standard directory structure called a Web Application Archive, or “WAR.” Everything in the WAR directory gets deployed to App Engine. It’s common to have your development workflow build the contents of the WAR from a set of source files, either using an automated build process or WAR-aware development tools.

If you are using the Eclipse IDE with the Google Plugin, you can create a new project by using the Web Application wizard. Click the Google drop-down menu button, then select New Web Application Project. (Alternately, from the File menu, select New, then Web Application Project.) In the window that opens, enter a project name (such as Clock) and package name (such as clock).

Uncheck the “Use Google Web Toolkit” checkbox, and make sure the “Use Google App Engine” checkbox is checked. (If you leave the GWT checkbox checked, the new project will be created with GWT starter files. This is cool, but it’s outside the scope of this chapter.) Figure 2-8 shows the completed dialog box for the Clock application. Click Finish to create the project.

The Google Plugin for Eclipse New Web Application Project dialog, with values for the Clock application

Figure 2-8. The Google Plugin for Eclipse New Web Application Project dialog, with values for the Clock application

If you are not using the Google Plugin for Eclipse, you will need to create the directories and files another way. If you are already familiar with Java web development, you can use your existing tools and processes to produce the final WAR. For the rest of this section, we assume you are using the directory structure that is created by the Eclipse plug-in.

Figure 2-9 shows the project file structure, as depicted in the Eclipse Package Explorer.

A new Java project structure, as shown in the Eclipse Package Explorer

Figure 2-9. A new Java project structure, as shown in the Eclipse Package Explorer

The project root directory (Clock) contains two major subdirectories: src and war. The src/ directory contains all the project’s class files in the usual Java package structure. With a package path of clock, Eclipse created source code for a servlet class named ClockServlet in the fileclock/ClockServlet.java.

The war/ directory contains the complete final contents of the application. Eclipse compiles source code from src/ automatically and puts the compiled class files in war/WEB-INF/classes/, which is hidden from Eclipse’s Package Explorer by default. Eclipse copies the contents of src/META-INF/ to war/WEB-INF/classes/META-INF/ automatically, as well. Everything else, such as CSS or browser JavaScript files, must be created in the war/ directory in its intended location.

Let’s start our clock application with a simple servlet that displays the current time. Open the file src/clock/ClockServlet.java for editing (creating it if necessary), and give it contents similar to Example 2-9.

Example 2-9. A simple Java servlet

package clock;

import java.io.IOException;

import java.io.PrintWriter;

import java.text.SimpleDateFormat;

import java.util.Date;

import java.util.SimpleTimeZone;

import javax.servlet.http.*;

@SuppressWarnings("serial")

public class ClockServlet extends HttpServlet {

public void doGet(HttpServletRequest req,

HttpServletResponse resp)

throws IOException {

SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss.SSSSSS");

fmt.setTimeZone(new SimpleTimeZone(0, "");

resp.setContentType("text/html");

PrintWriter out = resp.getWriter();

out.println("<p>The time is: " + fmt.format(new Date() + "</p>");

}

}

The servlet class extends javax.servlet.http.HttpServlet, and overrides methods for each of the HTTP methods it intends to support. This servlet overrides the doGet() method to handle HTTP GET requests. The server calls the method with an HttpServletRequest object and an HttpServletResponse object as parameters. The HttpServletRequest contains information about the request, such as the URL, form parameters, and cookies. The method prepares the response, using methods on the HttpServletResponse, such as setContentType() andgetWriter(). App Engine sends the response when the servlet method exits.

To tell App Engine to invoke this servlet for requests, we need a deployment descriptor: an XML configuration file that describes which URLs invoke which servlet classes, among other things. The deployment descriptor is part of the servlet standard. Open or create the file war/WEB-INF/web.xml, and give it contents similar to Example 2-10.

Example 2-10. The web.xml file, also known as the deployment descriptor, mapping all URLs to ClockServlet

<?xml version="1.0" encoding="utf-8"?>

<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns="http://java.sun.com/xml/ns/javaee"

xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"

xsi:schemaLocation="http://java.sun.com/xml/ns/javaee

http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">

<servlet>

<servlet-name>clock</servlet-name>

<servlet-class>clock.ClockServlet</servlet-class>

</servlet>

<servlet-mapping>

<servlet-name>clock</servlet-name>

<url-pattern>/</url-pattern>

</servlet-mapping>

</web-app>

Eclipse may open this file in its XML Design view, a table-like view of the elements and values. Select the Source tab at the bottom of the editor pane to edit the XML source.

web.xml is an XML file with a root element of <web-app>. To map URL patterns to servlets, you declare each servlet with a <servlet> element, then declare the mapping with a <servlet-mapping> element. The <url-pattern> of a servlet mapping can be a full URL path, or a URL path with a * at the beginning or end to represent a part of a path. In this case, the URL pattern / matches just the root URL path.

WARNING

Be sure that each of your <url-pattern> values starts with a forward slash (/). Omitting the starting slash may have the intended behavior on the development web server but unintended behavior on App Engine.

App Engine needs one additional configuration file that isn’t part of the servlet standard. Open or create the file war/WEB-INF/appengine-web.xml, and give it contents similar to Example 2-11.

Example 2-11. The appengine-web.xml file, with App Engine-specific configuration for the Java app

<?xml version="1.0" encoding="utf-8"?>

<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">

<application>clock</application>

<version>1</version>

<threadsafe>true</threadsafe>

</appengine-web-app>

In this example, the configuration file tells App Engine that this is version 1 of an application called clock. We also declare the app to be thread-safe, authorizing App Engine to reuse an application instance to serve multiple requests simultaneously. (Of course, we must also make sure our code is thread-safe when we do this.) You can also use this configuration file to control other behaviors, such as static files and sessions. For more information, see Chapter 3.

The WAR for the application must include several JARs from the App Engine SDK: the Java EE implementation JARs, and the App Engine API JAR. The Eclipse plug-in installs these JARs in the WAR automatically. If you are not using the Eclipse plug-in, you must copy these JARs manually. Look in the SDK directory in the lib/user/ and lib/shared/ subdirectories. Copy every .jar file from these directories to the war/WEB-INF/lib/ directory in your project.

Finally, the servlet class must be compiled. Eclipse compiles all your classes automatically, as needed. If you are not using Eclipse, you probably want to use a build tool such as Apache Ant to compile source code and perform other build tasks. See the official App Engine documentation for information on using Apache Ant to build App Engine projects.

I suppose it’s traditional to explain how to compile a Java project from the command line using the javac command. You can do so by putting each of the JARs from war/WEB-INF/lib/ and the war/WEB-INF/classes/ directory in the classpath, and making sure the compiled classes end up in the classes/ directory. But in the real world, you want your IDE or an Ant script to take care of this for you.

One more thing for Eclipse users: the Eclipse new-project wizard created a static file named war/index.html. Delete it by right-clicking on it in the Project Explorer, selecting Delete, then clicking OK. (If you don’t delete it, this static file will take precedence over the servlet mapping we just created.)

It’s time to test this application with the development web server. The Eclipse plug-in can run the application and the development server inside the Eclipse debugger. To start it, select the Run menu, Debug As, and Web Application. The server starts, and prints the following message to the Console panel:

The server is running at http://localhost:8888/

If you are not using Eclipse, you can start the development server, using the dev_appserver command (dev_appserver.sh for Mac OS X or Linux). The command takes the path to the WAR directory as an argument, like so:

dev_appserver war

The command-line tool uses a different default port than the Eclipse plug-in uses (8080 instead of 8888). You can change the port used by the command-line tool with the --port argument, such as --port=8888.

Test your application by visiting the server’s URL in a web browser:

http://localhost:8888

The browser displays a page similar to the Python example, shown earlier in Figure 2-4.

Introducing JSPs, JSTL, and EL

Right now, our clock displays the time in the UTC time zone. We’d like for our application to let the user customize the time zone, and to remember the user’s preference for future visits. To do that, we use Google Accounts to identify which user is using the application.

Before we go any further, we should introduce a way to keep our HTML separate from our servlet code. This allows us to maintain the “business logic”—the code that implements the main purpose of our app—separately from the appearance of the app, making our logic easier to test and our appearance easier to change. Typically, you would use a templating system to define the appearance of the app in files that contain the HTML, CSS, and JavaScript, and leave blanks where the dynamic data should go. There are many fine templating systems to choose from in Java, such as Apache Velocity.

For this example, we use Java Servlet Pages, or JSPs. JSPs are a standard part of J2EE, which means you do not have to install anything else to use them. A JSP contains a mix of text (HTML) and Java code that defines the logic of the page. The JSP compiles to a servlet, just like theClockServlet we already defined, that’s equivalent to writing out the HTML portions, and evaluating the Java portions. In a sense, JSPs are just another way of writing servlet code.

JSPs are often criticized for being too powerful. Since the full Java language is available from within a JSP, there is a risk that business logic may creep into the templates, and you no longer have a useful separation. To mitigate this, later versions of the JSP specification included new ways of describing template logic that are intentionally less powerful than full Java code: the Java Servlet Templating Language (JSTL) and the JSP Expression Language (EL). We use these features for this example, and other places in the book where templated output is required.

Edit ClockServlet.java to resemble Example 2-12.

Example 2-12. Code for ClockServlet.java that displays Google Accounts information and links

package clock;

import java.io.IOException;

import java.text.SimpleDateFormat;

import java.util.Date;

import java.util.SimpleTimeZone;

import javax.servlet.RequestDispatcher;

import javax.servlet.ServletException;

import javax.servlet.http.*;

import com.google.appengine.api.users.User;

import com.google.appengine.api.users.UserService;

import com.google.appengine.api.users.UserServiceFactory;

@SuppressWarnings("serial")

public class ClockServlet extends HttpServlet {

public void doGet(HttpServletRequest req,

HttpServletResponse resp)

throws IOException, ServletException {

SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss.SSSSSS");

fmt.setTimeZone(new SimpleTimeZone(0, "");

UserService userService = UserServiceFactory.getUserService();

User user = userService.getCurrentUser();

String loginUrl = userService.createLoginURL("/");

String logoutUrl = userService.createLogoutURL("/");

req.setAttribute("user", user);

req.setAttribute("loginUrl", loginUrl);

req.setAttribute("logoutUrl", logoutUrl);

req.setAttribute("currentTime", fmt.format(new Date());

resp.setContentType("text/html");

RequestDispatcher jsp = req.getRequestDispatcher("/WEB-INF/home.jsp");

jsp.forward(req, resp);

}

}

Next, create a new file named home.jsp in the war/WEB-INF/ directory of your project, and give it contents similar to Example 2-13.

Example 2-13. Code for ClockServlet.java that displays Google Accounts information and links

<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>

<head>

<title>The Time Is...</title>

</head>

<body>

<c:choose>

<c:when test="${user != null}">

<p>

Welcome, ${user.email}!

You can <a href="${logoutUrl}">sign out</a>.

</p>

</c:when>

<c:otherwise>

<p>

Welcome!

<a href="${loginUrl}">Sign in or register</a> to customize.

</p>

</c:otherwise>

</c:choose>

<p>The time is: ${currentTime}</p>

</body>

</html>

Using Eclipse, you can leave the development web server running while you edit code. When you save changes to code, Eclipse compiles the class, and if it compiles successfully, Eclipse injects the new class into the already-running server. In most cases, you can simply reload the page in your browser, and it will use the new code.

If you are not using Eclipse, shut down the development server by hitting Ctrl-C. Recompile your project, then start the server again.

Reload the new version of the clock app in your browser. The new page resembles the Python example, shown previously in Figure 2-5.

Here’s everything we’re going to say about JSPs, JSTL, and EL in this book:

§ Remember that home.jsp represents a servlet. In this case, it’s one that expects certain attributes to be set in its context. ClockServlet invokes home.jsp by setting attributes on the HttpServletRequest object via its setAttribute() method, then forwarding the request and response to the home.jsp servlet.

§ The forwarding takes place via a RequestDispatcher set up for the home.jsp servlet. In this case, we keep home.jsp inside the /WEB-INF/ directory, so that the servlet container doesn’t map it to a URL. If the JSP resided outside of /WEB-INF/, a URL to that path (from the WAR root) would map to the JSP servlet, and the servlet container would invoke it directly (assuming no explicit URL pattern matched the URL).

§ The getRequestDispatcher() method of the request instance takes a path to a JSP and returns its RequestDispatcher. Make sure the path starts with a forward slash (/).

§ To invoke the JSP, ClockServlet calls the forward() method of the RequestDispatcher, passing the HttpServletRequest and HttpServletResponse objects as arguments. The forward() method may throw the ServletException; in this example, we just add athrows clause to doGet().

§ A JSP contains text (HTML) and specially formatted directives. This example contains one directive, <%@ taglib ... %>, which loads a JSTL tag library. You might also see <% ... %>, which contains Java code that becomes part of the servlet, and <%= ... %>, which contains a Java expression whose string form is printed to the page.

§ <c:choose>...</c:choose>, <c:when>...</c:when>, and <c:otherwise>...</c:otherwise> are examples of JSTL tags. These come from the /jsp/jstl/core tag library imported by taglib import directive in the first line. The c: is the prefix associated with the library in the import directive. Here, the <c:choose> structure renders the <c:when> block when the user is signed in, and the <c:otherwise> block otherwise.

§ ${user != null} is an example of an EL expression. An EL expression can appear in the text of the document, where its value is rendered into the text, or in a JSTL tag attribute, where its value is used by the tag. The expression ${logoutUrl} renders the String value of thelogoutUrl attribute set by ClockServlet. ${user.email} is an example of accessing a JavaBean property of a value: the result is equivalent to calling the getEmail() method of the User object value. ${user != null} shows how an EL expression can use simple operators, in this case producing a boolean value used by the <c:when test="...">.

For this book, we’ll stick to simple features of JSPs, JSTL, and EL, and not provide additional explanation. For more information about these J2EE features, see Head First Servlets and JSP by Brian Basham et al. (O’Reilly).

WARNING

Be careful when mapping the URL pattern /* to a servlet in your deployment descriptor when using request dispatchers in this way. Explicit URL mappings override the default JSP path mapping, and the request dispatcher will honor it when determining the servlet for the path. If you have a /* URL mapping that might match a JSP path, you must have an explicit JSP URL mapping in the deployment descriptor that overrides it:

<servlet>

<servlet-name>home-jsp</servlet-name>

<jsp-file>/WEB-INF/home.jsp</jsp-file>

</servlet>

<servlet-mapping>

<servlet-name>home-jsp</servlet-name>

<url-pattern>/WEB-INF/home.jsp</url-pattern>

</servlet-mapping>

<servlet>

<servlet-name>clock</servlet-name>

<servlet-class>clock.ClockServlet</servlet-class>

</servlet>

<servlet-mapping>

<servlet-name>clock</servlet-name>

<url-pattern>/*</url-pattern>

</servlet-mapping>

Note that a mapping in /WEB-INF/ is still hidden from clients. A request for /WEB-INF/home.jsp will return a 404 Not Found error, and will not invoke the JSP servlet.

Users and Google Accounts

The ClockServlet in Example 2-12 calls the Users API to get information about the user who may or may not be signed in with a Google Account. This interface is provided by the com.google.appengine.api.users package. The app gets a UserService instance by calling thegetUserService() method of the UserServiceFactory class. Then it calls the getCurrentUser() method of the UserService, which returns a User object, or null if the current user is not signed in. The getEmail() method of the User object returns the email address for the user.

The createLoginURL() and createLogoutURL() methods of the UserService generate URLs that go to Google Accounts. Each of these methods takes a URL path for the app where the user should be redirected after performing the desired task. The login URL goes to the Google Accounts page where the user can sign in or register for a new account. The logout URL visits Google Accounts to sign out the current user, and then immediately redirects back to the given application URL without displaying anything.

If you click on the “Sign in or register” link with the app running in the development server, the link goes to the development server’s simulated version of the Google Accounts sign-in screen, similar to the Python version shown earlier in Figure 2-6. At this screen, you can enter any email address, and the development server will proceed as if you are signed in with an account that has that address.

If this app were running on App Engine, the login and logout URLs would go to the actual Google Accounts locations. Once signed in or out, Google Accounts redirects back to the given URL path for the live application.

Click on “Sign in or register,” then enter an email address (such as test@example.com) and click on the Login button on the simulated Google Accounts screen. The clock app now looks like Figure 2-7 (shown earlier). To sign out again, click the “sign out” link.

In addition to the UserService API, an app can also get information about the current user with the servlet “user principal” interface. The app can call the getUserPrincipal() method on the HttpServletRequest object to get a java.security.Principal object, or null if the user is not signed in. This object has a getName() method, which in App Engine is equivalent to calling the getEmail() method of a User object.

The main advantage to getting user information from the servlet interface is that the servlet interface is a standard. Coding an app to use standard interfaces makes the app easier to port to alternate implementations, such as other servlet-based web application environments or private servers. As much as possible, App Engine implements standard interfaces for its services and features.

The disadvantage to the standard interfaces is that not all standard interfaces represent all of App Engine’s features, and in some cases the App Engine services don’t implement every feature of an interface. All services include a nonstandard “low-level” API, which you can use directly or use to implement adapters to other interfaces.

Web forms and the datastore

Now that we can identify the user, we can prompt for the user’s preferences and remember them for future requests. We can store preferences data in the App Engine datastore.

There are several ways to use the datastore from Java. The simplest way is to call the datastore API directly. This API lets you create and manipulate entities (records) in the datastore by using instances of an Entity class. Entities have named properties, which you can get and set usinggetProperty() and setProperty() methods. The API is easy to understand, and has a direct correspondence with the concepts of the datastore. We’ll use the datastore API for this tutorial and the next few chapters.

TIP

The datastore API is sufficient for many uses, but it’s not particularly Java-like to represent to data objects as instances of a generic Entity class. It’d be better if data objects could be represented by real Java objects, with classes, fields, and accessor methods that describe the role of the data in your code.

The App Engine SDK includes support for two major standard interfaces for manipulating data in this way: Java Data Objects (JDO) and the Java Persistence API (JPA). With JDO and JPA, you use regular Java classes to describe the structure of data, and include annotations that tell the interface how to save the data to the datastore and re-create the objects when the data is fetched. We discuss JPA in detail in Chapter 10.

You may also want to consider Objectify, a third-party open source library with many of these benefits. Objectify is easier to use than JDO and JPA, although it is specific to App Engine.

First, let’s set up a way for a signed-in user to set a time zone preference. Edit home.jsp and add the following, just above the closing </body> tag:

<c:if test="${user != null}">

<form action="/prefs" method="post">

<label for="tz_offset">

Timezone offset from UTC (can be negative):

</label>

<input name="tz_offset" id="tz_offset" type="text"

size="4" value="${tzOffset}" />

<input type="submit" value="Set" />

</form>

</c:if>

(We will populate the tzOffset attribute in a moment. If you’d like to test the form now, remove the ${tzOffset} reference temporarily.)

This web form includes a text field for the user’s time zone preference, and a button that submits the form. When the user submits the form, the browser issues an HTTP POST request (specified by the method attribute) to the URL /prefs (the action attribute), with a request body containing the form field data.

Naturally, we need a new servlet to handle these requests. Create a new servlet class PrefsServlet (PrefsServlet.java) with the code shown in Example 2-14.

Example 2-14. The PrefsServlet class, a servlet that handles the user preferences form

package clock;

import java.io.IOException;

import javax.servlet.http.HttpServlet;

import javax.servlet.http.HttpServletRequest;

import javax.servlet.http.HttpServletResponse;

import com.google.appengine.api.datastore.DatastoreService;

import com.google.appengine.api.datastore.DatastoreServiceFactory;

import com.google.appengine.api.datastore.Entity;

import com.google.appengine.api.datastore.Key;

import com.google.appengine.api.datastore.KeyFactory;

import com.google.appengine.api.users.User;

import com.google.appengine.api.users.UserService;

import com.google.appengine.api.users.UserServiceFactory;

@SuppressWarnings("serial")

public class PrefsServlet extends HttpServlet {

public void doPost(HttpServletRequest req,

HttpServletResponse resp)

throws IOException {

UserService userService = UserServiceFactory.getUserService();

User user = userService.getCurrentUser();

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Key userKey = KeyFactory.createKey("UserPrefs", user.getUserId();

Entity userPrefs = new Entity(userKey);

try {

int tzOffset = new Integer(req.getParameter("tz_offset").intValue();

userPrefs.setProperty("tz_offset", tzOffset);

userPrefs.setProperty("user", user);

ds.put(userPrefs);

} catch (NumberFormatException nfe) {

// User entered a value that wasn't an integer. Ignore for now.

}

resp.sendRedirect("/");

}

}

A datastore entity has a kind, which groups related entities together for the purpose of queries. Here, the user’s time zone preference is stored in an entity of the kind "UserPrefs". This entity has two properties. The first is named "tzOffset", and its value is the user’s time zone offset, an integer. The second is "user", which contains a representation of the User value that represents the currently signed-in user.

Each datastore entity has a key that is unique across all entities. A simple key contains the kind, and either an app-assigned string (the key name) or a system-assigned number (referred to in the API as an ID). In this case, we provide a key name equal to the user ID from the User value.

The entity is saved to the datastore by the call to the ds.put() method. If an entity with the given key does not exist, the put() method creates a new one. If the entity does exist, put() replaces it. In a more typical case involving more properties, you would fetch the entity by key or with a query, update properties, then save it back to the datastore. But for PrefsServlet, it is sufficient to replace the entity without reading the old data.

When we are done updating the datastore, we respond with a redirect back to the main page (/). It is a best practice to reply to the posting of a web form with a redirect, to prevent the user from accidentally resubmitting the form by using the “back” button of the browser. It’d be better to offer more visual feedback for this action, but this will do for now.

Edit web.xml to map the new servlet to the /prefs URL used by the form. Add these lines just before the closing </web-app> tag:

<servlet>

<servlet-name>Prefs</servlet-name>

<servlet-class>clock.PrefsServlet</servlet-class>

</servlet>

<servlet-mapping>

<servlet-name>Prefs</servlet-name>

<url-pattern>/prefs</url-pattern>

</servlet-mapping>

<security-constraint>

<web-resource-collection>

<web-resource-name>prefs</web-resource-name>

<url-pattern>/prefs</url-pattern>

</web-resource-collection>

<auth-constraint>

<role-name>*</role-name>

</auth-constraint>

</security-constraint>

The order in which the URL mappings appear in the file does not matter. Longer patterns (not counting wildcards) match before shorter ones.

The <security-constraint> block tells App Engine that only users signed in with a Google Account can access the URL /prefs. If a user who is not signed in attempts to access this URL, App Engine redirects the user to Google Accounts to sign in. When the user signs in, she is directed back to the URL she attempted to access. A security constraint is a convenient way to implement Google Accounts authentication for a set of URLs. In this case, it means that PrefsServlet does not need to handle the case where someone tries to submit data to the URL without being signed in.

Finally, edit the doGet() method in the ClockServlet class to fetch the UserPrefs entity and use its value, if one exists. Add this code prior to where the "currentTime" attribute is set, with the imports in the appropriate place:

import com.google.appengine.api.datastore.DatastoreService;

import com.google.appengine.api.datastore.DatastoreServiceFactory;

import com.google.appengine.api.datastore.Entity;

import com.google.appengine.api.datastore.EntityNotFoundException;

import com.google.appengine.api.datastore.Key;

import com.google.appengine.api.datastore.KeyFactory;

// ...

Entity userPrefs = null;

if (user != null) {

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

Key userKey = KeyFactory.createKey("UserPrefs", user.getUserId();

try {

userPrefs = ds.get(userKey);

} catch (EntityNotFoundException e) {

// No user preferences stored.

}

}

if (userPrefs != null) {

int tzOffset = (Long) userPrefs.getProperty("tz_offset").intValue();

fmt.setTimeZone(new SimpleTimeZone(tzOffset * 60 * 60 * 1000, "");

req.setAttribute("tzOffset", tzOffset);

} else {

req.setAttribute("tzOffset", 0);

}

This code retrieves the UserPrefs entity by reconstructing its key, then calling the get() method of the DatastoreService. This either returns the Entity, or throws EntityNotFoundException. In this case, if it’s not found, we fall back on the default setting of 0 for the time zone offset. If it is found, we update the SimpleDateFormat to use the setting. In either case, we populate the "tzOffset" attribute with a value, which is displayed in the form.

TIP

Notice the casting of the property value in this example. When we set the property in the PrefsServlet class, we used an int value. The datastore only has a long integer value type internally, so it upgraded the value before storing it. When we retrieve the property with getProperty(), it comes out as a Long. (This value is null if the entity has no property of that name.) We must cast the value to a Long, then call its intValue() method to get our int.

This is where a data framework like JPA or Objectify comes in handy. Such a framework handles the marshaling of values and casting of types automatically, and can provide a degree of protection for type consistency. The datastore itself is “schemaless,” which is important to understand as you modify your data model over the lifetime of your application.

Restart your development server, then reload the page to see the customizable clock in action. Try changing the time zone by submitting the form. Also try signing out, then signing in again using the same email address, and again with a different email address. The app remembers the time zone preference for each user.

Caching with memcache

So far, our application fetches the object from the datastore every time a signed-in user visits the site. Since user preferences data doesn’t change very often, we can speed up the per-request data access using the memory cache (memcache) as secondary storage.

We need to make two changes to our app to achieve this. The first is to update ClockServlet to check the cache. If the value is found in the cache, ClockServlet uses it. If it is not found, it falls back on reading from the datastore as before. If the value is found in the datastore but not in the cache, it stores the value in the cache for use by future requests.

The second change we need is to PrefsServlet. When the user updates her preference, we must invalidate (delete) the value in the cache. The next attempt to read the value—likely the redirect to ClockServlet that immediately follows the form submission—will see the cache does not have the value, get it from the datastore, and update the cache.

TIP

We could have PrefsServlet update the cache itself, but we must have ClockServlet populate the cache as needed. Memcache is not durable storage, and could delete any value at any time. That’s what makes it fast. If ClockServlet did not update the cache, the value could go missing, and performance would degrade until the next time the user updates her preference. To keep things simple, we make ClockServlet responsible for all cache updates, and simply delete the value from the cache when it changes in the datastore in PrefsServlet.

Edit the lines we just added to ClockServlet to use the cache, putting the new imports in the appropriate place:

import com.google.appengine.api.memcache.MemcacheService;

import com.google.appengine.api.memcache.MemcacheServiceFactory;

// ...

Entity userPrefs = null;

if (user != null) {

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();

MemcacheService memcache = MemcacheServiceFactory.getMemcacheService();

String cacheKey = "UserPrefs:" + user.getUserId();

userPrefs = (Entity) memcache.get(cacheKey);

if (userPrefs == null) {

Key userKey = KeyFactory.createKey("UserPrefs", user.getUserId();

try {

userPrefs = ds.get(userKey);

memcache.put(cacheKey, userPrefs);

} catch (EntityNotFoundException e) {

// No user preferences stored.

}

}

}

if (userPrefs != null) {

int tzOffset = (Long) userPrefs.getProperty("tz_offset").intValue();

fmt.setTimeZone(new SimpleTimeZone(tzOffset * 60 * 60 * 1000, "");

req.setAttribute("tzOffset", tzOffset);

} else {

req.setAttribute("tzOffset", 0);

}

Similarly, edit the doPost() method of PrefsServlet to delete the cached value (if any) when performing an update:

import com.google.appengine.api.memcache.MemcacheService;

import com.google.appengine.api.memcache.MemcacheServiceFactory;

// ...

MemcacheService memcache = MemcacheServiceFactory.getMemcacheService();

String cacheKey = "UserPrefs:" + user.getUserId();

try {

int tzOffset = new Integer(req.getParameter("tz_offset").intValue();

userPrefs.setProperty("tz_offset", tzOffset);

userPrefs.setProperty("user", user);

ds.put(userPrefs);

memcache.delete(cacheKey);

} catch (NumberFormatException nfe) {

// User entered a value that wasn't an integer. Ignore for now.

}

Any object you store in the memcache must be serializable. That is, it must implement the Serializable interface from the java.io package. The Entity class is serializable. You can also use any serializable object as the key for a cache value; in this case we just use a String.

Reload the page to see the new version work. To make the caching behavior more visible, you can add logging statements to ClockServlet, like so:

import java.util.logging.*;

// ...

public class ClockServlet extends HttpServlet {

private static final Logger log = Logger.getLogger(ClockServlet.class.getName();

public void doGet(HttpServletRequest req,

HttpServletResponse resp)

throws IOException, ServletException {

// ...

userPrefs = (Entity) memcache.get(cacheKey);

if (userPrefs == null) {

log.warning("CACHE MISS");

} else {

log.warning("CACHE HIT");

}

// ...

}

}

The development server prints logging output to the console. If you are using Eclipse, these messages appear in the Console pane.

The Development Console

Both the Python and Java development web servers include a handy feature for inspecting and debugging your application while testing on your local machine: a web-based development console. With your development server running, visit the following URL in a browser to access the console:

http://localhost:8080/_ah/admin

(Java Eclipse users, remember that your default port number is 8888.)

In the Python Launcher, you can also click the SDK Console button to open the console in a browser window.

Figure 2-10 shows the Datastore Viewer in the Python console. The Java Development Console has most of the same features as the Python version, but not all.

The Development Console’s Datastore Viewer, Python version

Figure 2-10. The Development Console’s Datastore Viewer, Python version

The Development Console’s Datastore Viewer lets you list and inspect entities by kind, edit entities, and create new ones. You can edit the values for existing properties, but you cannot delete properties or add new ones, nor can you change the type of the value. For new entities, the console makes a guess as to which properties belong on the entity based on existing entities of that kind, and displays a form to fill in those properties. Similarly, you can only create new entities of existing kinds, and cannot create new kinds from the console.

You can use the console to inspect the application’s task queues, to see tasks currently on the queue (in the local instance of the app), run them ahead of schedule, and flush them. By default, the development server drives task queues in the background. You can disable this behavior for debugging purposes. See Chapter 16.

You can test how your app receives email and XMPP messages by sending it mock messages through the console. Simply select the Inbound Mail or XMPP section of the console, then fill out a form with the message data.

Some features are exclusive to the Python development server. An “interactive console” lets you type Python code directly in the browser, execute it in the (local) server environment, and see the result. In addition to the datastore viewer, the Python console has a memcache viewer for inspecting and manipulating cache values. The Python console also makes it easy to inspect task queue and scheduled task (“cron job”) configuration.

The Java development server includes a way to simulate conditions of the Capabilities API. This API lets Java apps interrogate service status programmatically, so the app can respond automatically to scheduled or unplanned maintenance outages. You can tell the development server to simulate various conditions to test your app.

The Python Interactive Console

An especially powerful feature of the Python console is the “Interactive Console.” This feature lets you type arbitrary Python code directly into a web form and see the results displayed in the browser. You can use this to write ad hoc Python code to test and manipulate the datastore, memcache, and global data within the local development server.

Here’s an example: run your clock application, sign in with an email address, and then set a time zone preference, such as -8. Now open the Python development console, then select Interactive Console. In the lefthand text box, enter the following, where -8 is the time zone preference you used:

from google.appengine.ext import db

import models

q = models.UserPrefs.gql("WHERE tz_offset = -8")

for prefs in q:

print prefs.user

Click the Run Program button. The code runs, and the email address you used appears in the righthand box.

Code run in the development console behaves just like application code. If you perform a datastore query that needs a custom index, the development server adds configuration for that index to the application’s index.yaml configuration file. Datastore index configuration is discussed inChapter 6.

Registering the Application

Before you can upload your application to App Engine and share it with the world, you must first create a developer account, then register an application ID. If you intend to use a custom domain name (instead of the free appspot.com domain name included with every app), you must also set up the Google Apps service for the domain. You can do all of this from the App Engine Administration Console.

To access the Administration Console, visit the following URL in your browser:

https://appengine.google.com/

Sign in using the Google account you intend to use as your developer account. If you don’t already have a Google account (such as a Gmail account), you can create one using any email address.

Once you have signed in, the Console displays a list of applications you have created, if any, and a button to “Create an Application,” similar to Figure 2-11. From this screen, you can create and manage multiple applications, each with its own URL, configuration, and resource limits.

The Administration Console application list, with one app

Figure 2-11. The Administration Console application list, with one app

When you register your first application ID, the Administration Console prompts you to verify your developer account, using an SMS message sent to your mobile phone. After you enter your mobile phone number, Google sends an SMS to your phone with a confirmation code. Enter this code to continue the registration process. You can verify only one account per phone number, so if you have only one mobile number (like most people), be sure to use it with the account you intend to use with App Engine.

If you don’t have a mobile phone number, you can apply to Google for manual verification by filling out a web form. This process takes about a week. For information on applying for manual verification, see the official App Engine website.

You can have up to 10 active applications created by a given developer account. You can disable an application to reclaim a slot.

You have four decisions to make when creating a new application:

§ The application ID

§ The application title

§ Authentication options

§ Storage options

Of these, only the application title can be changed later.

The Application ID and Title

When you click the “Create an Application” button, the Console prompts for an application identifier. The application ID must be unique across all App Engine applications, just like an account username.

The application ID identifies your application when you interact with App Engine using the developer tools. The tools get the application ID from the application configuration file. For Python applications, you specify the app ID in the app.yaml file, on the application: line. For Java applications, you enter it in the <application> element of the appengine-web.xml file.

TIP

In the example earlier in this chapter, we chose the application ID “clock” arbitrarily. If you’d like to try uploading this application to App Engine, remember to edit the app’s configuration file (app.yaml for Python, appengine-web.xml for Java) after you register the application to change the application ID to the one you chose.

The application ID is part of the domain name you can use to test the application running on App Engine. Every application gets a free domain name that looks like this:

app-id.appspot.com

The application ID is also part of email and XMPP addresses the app can use to receive incoming messages. See Chapter 14 and Chapter 15.

Because the application ID is used in the domain name, an ID can contain only lowercase letters, numbers, or hyphens, and must be shorter than 32 characters. Additionally, Google reserves every Gmail username as an application ID that only the corresponding Gmail user can register. As with usernames on most popular websites, a user-friendly application ID may be hard to come by.

When you register a new application, the Console also prompts for an “application title.” This title is used to represent your application throughout the Console and the rest of the system. In particular, it is displayed to a user when the application directs the user to sign in with a Google account. Make sure the title is what you want your users to see.

Once you have registered an application, its ID cannot be changed, although you can delete the application and create a new one. You can change the title for an app at any time from the Administration Console.

Registering an application ID makes it permanently unavailable for others to register, or for you to reregister later, even if you disable or delete the app.

Setting Up a Domain Name

If you are developing a professional or commercial application, you probably want to use your own domain name instead of the appspot.com domain as the official location of your application. You can set up a custom domain name for your App Engine app by using Google’s “software as a service,” Google Apps.

Google Apps provides hosted applications for your business or organization, including email (with Gmail and POP/IMAP interfaces); calendaring (Google Calendar); chat (Google Talk); hosted word processing, spreadsheets, and presentations (Google Docs); easy-to-edit websites (Google Sites); video hosting; and so forth. You can also purchase access to third-party applications for your domain at the Google Apps Marketplace. You associate these services with your organization’s domain name by mapping the domain to Google’s servers in its DNS record, either by letting Google manage the DNS for the domain or by pointing subdomains to Google in your own DNS configuration. Your organization’s members access the hosted services by using your domain name.

With App Engine, you can add your own applications to subdomains of your domain. Even if you do not intend to use the other Google Apps services, you can use Google Apps to associate your own domain with your App Engine application.

TIP

The website for Google Apps indicates that Standard Edition accounts are “ad-supported.” This refers to ads that appear on Google products such as Gmail. It does not refer to App Engine: Google does not place ads on the pages of App Engine applications, even those using free accounts. Of course, you can put ads on your own sites, but that’s your choice—and your ad revenue.

If you have not set up Google Apps for your domain already, you can do so during the application ID registration process. You can also set up Google Apps from the Administration Console after you have registered the app ID. If you haven’t yet purchased a domain name, you can do so while setting up Google Apps, and you can host the domain on Google’s name servers for free. To use a domain you purchased previously, follow the instructions on the website to point the domain to Google’s servers.

Once you have set up Google Apps for a domain, you can access the Google Apps dashboard at a URL similar to the following:

http://www.google.com/a/example.com

To add an App Engine application as a service, click the “Add more services” link, then find Google App Engine in the list. Enter the application ID for your app, then click “Add it now.” On the following settings screen, you can configure a subdomain of your domain name for the application. All web traffic to this subdomain will go to the application.

NOTE

Google Apps does not support routing web traffic for the top-level domain (such as http://example.com/) directly to an App Engine app. If you set up Google Apps to manage the DNS for the full domain name, an HTTP request to the top-level domain will redirect to http://www.example.com, and you can assign the www subdomain to your App Engine app. If Google does not maintain the DNS record for your domain, you will need to set up the redirect yourself using a web server associated with the top-level domain.

By default, the subdomain www is assigned to Google Sites, even if you do not have the Sites app activated. To release this subdomain for use with App Engine, first enable the Sites service, then edit the settings for Sites and remove the www subdomain.

Google Apps and Authentication

The authentication options section of the new application form controls the behavior of the app’s use of Google Accounts. By default, any Google account is allowed to sign in, and it’s up to the app to decide which accounts can proceed. The alternative is to automatically restrict sign-in to accounts on the Google Apps domain. You would pick the latter option if you are using Google Apps to manage accounts on your domain, and wish to restrict access to the app to just those accounts.

Google Apps allows your organization’s members (employees, contractors, volunteers) to create user accounts with email addresses that use your domain name (such as juliet@example.com). Members can sign in with these accounts to access services that are private to your organization, such as email or word processing. Using Apps accounts, you can restrict access to certain documents and services to members of the organization. It’s like a hosted intranet that members can access from anywhere.

Similarly, you can limit access to your App Engine applications to just those users with accounts on the domain. This lets you use App Engine for internal applications such as project management or sales reporting. When an App Engine application is restricted to an organization’s domain, only members of the organization can sign in to the application’s Google Accounts prompt. Other Google accounts are denied access.

This authentication restriction must be set when the application is registered, in the Authentication Options section of the registration form. The default setting allows any user with a Google account to sign in to the application, leaving it up to the application to decide how to respond to each user. When the app is restricted to a Google Apps domain, only users with Google accounts on the domain can sign in.

After the application ID has been registered, the authentication options cannot be changed. If you want different authentication options for your application, you must register a new application ID.

The restriction applies only to the application’s use of Google Accounts. If the application has any URLs that can be accessed without signing in to Google Accounts (such as a welcome page), those URLs will still be accessible by everyone. One of the simplest ways to restrict access to a URL is with application configuration. For example, a Python application can require sign-in for all URLs with the following in the app.yaml file:

handlers:

- url: /.*

script: main.application

login: required

A Java app can do something similar in the application’s deployment descriptor (web.xml). See Chapter 3.

The sign-in restriction applies even when the user accesses the app, using the appspot.com domain. The user does not need to be accessing the app with the Apps domain for the authentication restriction to be enforced.

If you or other members of your organization want to use Google Apps accounts as developer accounts, you must access the Administration Console by using a special URL. For example, if your Apps domain is example.com, you would use the following URL to access the Administration Console:

https://appengine.google.com/a/example.com

You sign in to the domain’s Console with your Apps account (for instance, juliet@example.com).

If you create an app by using a non-Apps account and restrict its authentication to the domain, you will still be able to access the Administration Console by using the non-Apps account. However, you will not be able to sign in to the app itself with that account, including when accessing URLs restricted to administrators.

Uploading the Application

In a traditional web application environment, releasing an application to the world can be a laborious process. Getting the latest software and configuration to multiple web servers and backend services in the right order and at the right time to minimize downtime and prevent breakage is often difficult and delicate. With App Engine, deployment is as simple as uploading the files with a single click or command. You can upload and test multiple versions of your application, and set any uploaded version to be the current public version.

For Python apps, you can upload an app from the Launcher, or from a command prompt. From the Launcher, select the app to deploy, then click the Deploy button. From a command prompt, run the appcfg.py command as follows, substituting the path to your application directory forclock:

appcfg.py update clock

As with dev_appserver.py, clock is just the path to the directory. If the current working directory is the clock/ directory, you can use the relative path, a dot (.).

For Java apps, you can upload from Eclipse using the Google plug-in, or from a command prompt. In Eclipse, click the “Deploy to App Engine” button (the little App Engine logo) in the Eclipse toolbar. Or from a command prompt, run the appcfg (or appcfg.sh) command from the SDK’s bin/ directory as follows, using the path to your application’s WAR directory for war:

appcfg update war

When prompted by these tools, enter your developer account’s email address and password. The tools remember your credentials for subsequent runs so you don’t have to enter them every time. (If your developer account uses two-step authentication, see the next section.)

The upload process determines the application ID and version number from the app configuration file—app.yaml for Python apps, appengine-web.xml for Java apps—then uploads and installs the files and configuration as the given version of the app. After you upload an application for the first time, you can access the application immediately using either the .appspot.com subdomain or the custom Google Apps domain you set up earlier. For example, if the application ID is clock, you can access the application with the following URL:

http://clock.appspot.com/

TIP

If your app is written in Python, a Python developer can download all files for an app she uploaded, including Python code. Only the developer that uploaded the app can download the files. This feature can be disabled, and once it is disabled, it cannot be reenabled. You can only do this with Python apps. See the official documentation for the appcfg.py download_app command.

This feature is not a substitute for proper file management practices. Make sure you are retaining copies of your application files, such as with a revision control system and regular backups.

Using Two-Step Verification

All Google accounts have an optional security feature called two-step verification. With this feature enabled, when you sign in to a Google web application, you are prompted for two things: your account password, and a temporary security code that Google sends to your mobile phone number via text message (or a recorded voice message) when you try to sign in. This protects your account by requiring something you know (your password) and something you have (your phone).

Two-step verification is an all-around good idea, and if you have a mobile phone, you should enable it for your Google account. To enable two-step verification, visit your Google account settings:

https://www.google.com/settings/

In the Security section, find “2-step verification.” If this is “off,” click Edit, then follow the prompts. The set-up process involves registering your phone number and receiving a test code.

For web applications and applications that can use browser-based sign-in (OAuth), two-step verification works as expected. When you sign in to the Google Plugin for Eclipse, for example, the plug-in opens a browser-like window to display the sign-in prompts on Google’s website. The process grants Eclipse a token that it can use to interact with App Engine, without needing to know or store your Google account password. The token stays valid indefinitely: Eclipse will not prompt you again until the next time you sign out then sign back in again.

The Python command-line tool appcfg.py/appcfg can also use browser-based sign-in. If you provide the --oauth2 flag, the command will attempt to open a web browser for you to sign in with your Google account and grant access to the account for the tool:

appcfg.py update --oauth2 clock

Some applications can’t use, or don’t yet know how to use, browser-based authentication. This includes the Python launcher, the Java appcfg.sh tool, and the Python appcfg.py tool when run without the --oauth2 flag. With two-step authentication enabled, your regular Google account password will not work with these applications. Instead, you generate an application-specific password, and use it in that application where you previously used your account password.

To generate an application-specific password, go to your Google account settings, Security, find “Authorizing applications and sites,” and click Edit. Enter your account password when prompted. Scroll down to “Application-specific passwords,” then type a name you can use to remember what this password is for. (This name can be anything, such as “App Engine.” It is only used in the list of generated passwords for when you wish to revoke it later.) Click “Generate password.” Google generates the password and displays it.

You do not need to memorize this password. But you shouldn’t write it down, either: it otherwise acts like an old-fashioned (one-step) password. The Python Launcher on the Mac can store this password in your OS X keychain. The command-line tools will attempt to remember this password for your current session. If any of these tools needs a password again, open your account settings in a browser, revoke the old password, then generate a new one. There is no way to retrieve a previously generated password.

You can revoke access for both kinds of applications from the “Authorizing applications and sites” screen. It’s always safe to revoke access for apps you aren’t currently using. You can reauthorize an application later by signing in again, either with two-step verification or with a new application-specific password.

Introducing the Administration Console

You manage your live application from your browser, using the App Engine Administration Console. You saw the Console when you registered the application, but as a reminder, you can access the Console at the following URL:

https://appengine.google.com/

If your app uses a Google Apps domain name and you are using an Apps account on the domain as your developer account, you must use the Apps address of the Administration Console:

https://appengine.google.com/a/example.com

Select your application (click its ID) to go to the Console for the app.

The first screen you see is the Dashboard, shown in Figure 2-12. The Dashboard summarizes the current and past status of your application, including traffic and load, resource usage, and error rates. You can view charts for the request rate, the amount of time spent on each request, error rates, bandwidth and CPU usage, and whether your application is hitting its resource limits.

The Administration Console dashboard for a new app

Figure 2-12. The Administration Console dashboard for a new app

If you’ve already tested your new application, you should see a spike in the requests-per-second chart. The scale of the chart goes up to the highest point in the chart, so the spike reaches the top of the graph even though you have only accessed the application a few times.

The Administration Console is your home base for managing your live application. From here, you can examine how the app is using resources, browse the application’s request and message logs, and query the datastore and check the status of its indexes. You can also manage multiple versions of your app, so you can test a newly uploaded version before making it the live “default” version. You can invite other people to be developers of the app, allowing them to access the Administration Console and upload new files. And when you’re ready to take on large amounts of traffic, you can establish a billing account, set a daily budget, and monitor expenses.

Take a moment to browse the Console, especially the Dashboard, Logs, and Data sections. Throughout this book, we discuss how an application consumes system resources, and how you can optimize an app for speed and cost effectiveness. You will use the Administration Console to track resource consumption and diagnose problems.