Deploying and Managing Applications - Programming Google App Engine (2012)

Programming Google App Engine

Chapter 20. Deploying and Managing Applications

Uploading your application to App Engine is easy: just click a button or run a command, then enter your developer account email address and password. All your application’s code, configuration, and static files are sent to App Engine, and seconds later your new app is running.

Easy deployment is one of App Engine’s most useful features. You don’t have to worry about which servers have which software, how servers connect to services, or whether machines need to be rebooted or processes restarted. Other than your developer account, there are no database passwords to remember, no secret keys to generate, and no need to administer and troubleshoot individual machines. Your application exists as a single logical entity, and running it on large-scale infrastructure is as easy as running it on your local computer.

App Engine includes features for testing a new version of an app before making it public, reverting to a previous version quickly in case of a problem, and migrating the datastore and service configuration for the app from one version to another. These features let you control how quickly changes are applied to your application: you can make a new version public immediately to fix small bugs, or you can test a new version for a while before making it the version everyone sees.

Service configuration is shared across all versions of an app, including datastore indexes, task queues, and scheduled tasks. When you upload the app, the service configuration files on your computer are uploaded as well, and take effect immediately for all app versions. You can also upload each configuration file separately. This is especially useful for datastore indexes, since new indexes based on existing entities take time to build before they are ready to serve datastore queries.

Notice that service configuration is separate from the application configuration, which includes URL mappings, runtime environment selection, and inbound service activation. Application configuration is bound to a specific app version.

App Engine provides rich facilities for inspecting the performance of an app while it is serving live traffic. Most of these features are available in the Administration Console, including analytic graphs of traffic and resource usage, and browsable request and message logs. You can also use the Console to inspect and make one-off changes to datastore entities.

You also use the Administration Console to perform maintenance tasks, such as giving other developers access to the Console, changing settings, and setting up a billing account.

In this chapter, we discuss how to upload a new app, how to update an existing app, and how to use App Engine’s versioning feature to test a new version of an app on App Engine while your users continue to use the previous version. We look at how to migrate the datastore and service configuration from one version to another. We also look at features of the SDK and the Administration Console for inspecting, troubleshooting, and analyzing how your live application is running. And finally, we discuss other application maintenance tasks, billing, and where to get more information.

Uploading an Application

We introduced uploading an application way back in Chapter 2, but let’s begin with a brief review.

If you’re developing a Python app with the Launcher for Windows or Mac OS X, you can deploy your app by selecting it in the app list and then clicking the Deploy button. The Launcher prompts for your developer account email address and password. The Mac version can remember your account credentials on your Keychain, but it always prompts in case you want to specify a different account. The upload begins, and the Launcher opens a window to display status messages emitted by the uploader tool.

If you’re developing a Java app with Eclipse and the Google Plugin, you can deploy your app by clicking on the Deploy App Engine Project button in the Eclipse toolbar (the one that looks like the App Engine logo, a gray airplane engine with blue wings). It prompts for your developer account email address and password. The upload begins, and an Eclipse progress window reports on the status.

You can also upload a Python or Java app with the AppCfg command-line tool from the SDK. For Python, the tool takes the update action and a path to your application root directory (the directory containing the app.yaml file):

appcfg.py update clock

For Java, the tool takes the update action and a path to the WAR directory:

appcfg update clock/war

(For Mac OS X or Linux and a Java app, use appcfg.sh.)

WARNING

We said this earlier, but it’s worth repeating: there is no way to download the app’s files once they have been uploaded. We strongly advise that you keep backups and use a revision control system.

Using Versions

The upload tool determines the application ID from the appropriate configuration file. For Python, this is the application element in the app.yaml file. For Java, this is the <application> element in the appengine-web.xml file.

The tool also uses this file to determine the version ID to use for this upload, from the version element. If App Engine does not yet have a version of this app with this ID, then it creates a new version with the uploaded files. If App Engine does have such a version, then it replaces the existing version. The replacement is total: no remnants of the previous version’s code or static files remain. The new app has only the files present in the project directory on your computer at the time of the upload. Of course, data stored by the services for the app remain, including the datastore, memcache, log data, and enqueued tasks.

The version of the app that is visible on your app’s primary domain name—either app-id.appspot.com or your Google Apps domain—is known as the default version. When you upload your app for the first time, the initial version becomes the default version automatically. If you subsequently upload a version with a different version ID, the original version remains the default until you change the default using the Administration Console.

Recall from Chapter 3 that each version has its own appspot.com URL that includes the version ID as well as the application ID:

version-id.app-id.appspot.com

TIP

Remember that there are no special protections on the version URLs. If the app does not restrict access by using code or configuration, then anyone who knows an unrestricted URL can access it. If you don’t want a user to be able to access a version other than the default, you can check the Host header in the app and respond accordingly. You can also upload the nondefault version with configuration that restricts all URLs to administrators. Be sure to upload it again with the real configuration before making it the default version.

When you replace an existing version by uploading the app with that version ID, App Engine starts using the uploaded app for requests for that version within seconds of the upload. It is not guaranteed that every request after a particular time will use the new code and static files, but it usually doesn’t take more than a few seconds for the App Master to update all the frontend servers. The App Master ensures that all the files are in place on a frontend server before using the new files to handle requests.

Internally, App Engine maintains a “minor” version number for each version ID. When you upload the app, App Engine creates a new minor version for the app ID and version ID mentioned in the app’s configuration file, and associates all code, static files, and frontend configuration with that minor version. When it is done creating the new minor version, it declares the new minor version the latest for the version ID, and the frontend starts to use it. Minor version numbers are an internal implementation detail, and are not exposed to the app or the Administration Console. You cannot access previous minor versions or the latest minor version number, and you cannot revert to previous minor versions. (This explains the word latest in the version URL, but you can’t actually replace the word with a minor version number to access other minor versions.)

If you upload the app with the same version ID as that of the version that’s currently the default, your users will start seeing the updated app within a few seconds of uploading. This is fine for small, low-risk changes that don’t depend on changes to your data schemas or datastore indexes.

For larger changes, it’s better to upload the app with a new version ID (in the application configuration file), test the app with the version URL, then switch the default version to direct traffic to the new version. To switch the default version, sign in to the Administration Console, and then select Versions from the sidebar. Select the radio button next to the desired version, and then click the Make Default button. This is shown in Figure 20-1.

The Administration Console Versions panel

Figure 20-1. The Administration Console Versions panel

App Engine can host up to 10 different version IDs per app at one time. You can delete unused versions from the Administration Console by clicking the Delete button on the appropriate row.

Many actions in the Administration Console refer to a specific version of the app, including usage statistics and the log viewer. You can control which version you’re looking at by selecting it from the drop-down menu in the top-left corner of the screen, next to the application ID. The version ID only appears as a drop-down menu if you have more than one version of the app. Similarly, the application ID appears as a drop-down menu if you have more than one app associated with your developer account.

Managing Service Configuration

All versions of an application use the same services. Service configuration and application data are shared across all versions of the app.

An app can have several service-related configuration files:

index.yaml or datastore-indexes.xml

A description of all the required datastore indexes.

queue.yaml or queue.xml

Configuration for task queues.

cron.yaml or cron.xml

The schedule for scheduled tasks (cron jobs).

Whenever you upload the app, these configuration files are uploaded from your computer to the services and take effect for the entire app, replacing any configuration that was once there. This is true regardless of whether the app version ID is new or already exists, or whether the version ID is the default.

You can update the configuration for each service without uploading the entire app by using the AppCfg tool. To update just the index configuration, use the update_indexes action, with the project directory (e.g., app-dir):

appcfg.py update_indexes app-dir

To update just the task queue configuration, use update_queues:

appcfg.py update_queues app-dir

And to update just the pending task schedule, use update_cron:

appcfg.py update_cron app-dir

Application Settings

The Application Settings panel of the Administration Console lets you change or view several miscellaneous aspects of your application.

This panel lets you view the application ID and the authentication method (Google accounts or Google Apps) that you set when you registered the application ID. These cannot be changed. If you need different values, you must register a new application ID.

The application title is the user-visible name of your application. It appears on the Google Accounts sign-in screen. You set this when you created the application, and you can change it at any time from this panel.

The “cookie expiration” time is the amount of time a user signed in with a Google account will remain signed in. If the user signs in to your app, he will not have to sign in again from the computer he is using until the expiration time elapses.

Also on this screen, you can view which optional services are enabled for this application. Optional services are those that have passive behavior that affect the app, specifically the inbound services of incoming email and incoming XMPP. You can enable these for an app in the app’s configuration file, as discussed in previous chapters.

This screen offers an alternate path for setting up a domain name for an app using Google Apps. Instead of adding the app as a service within Google Apps, you can use this setup procedure to associate the app with an Apps domain with default settings. Either setup procedure has the same effect, and you must still register the domain and create the Google Apps account separately. (There is a link to set up Google Apps on this screen.)

Managing Developers

When you register an application ID, you become a developer for the application automatically. You can invite other people to be developers for the application from the Developers section of the Administration Console.

To invite a developer, you enter the person’s email address in the appropriate field and click Invite. App Engine sends an email to the developer inviting her to set up an account. If the email address you invited is for a Google account, the developer can use the existing account to access App Engine, although she must still accept the invitation by clicking on a link in the invitation email message. If the email address does not have a corresponding Google account, the developer can create a Google account for that address by following the instructions in the message. The developer cannot accept the invitation from a Google account with a different address; you must invite the alternate address explicitly.

An invited developer who has not yet accepted the invitation appears in the list with a status of “Pending.” After the developer accepts the invitation, she appears with a status of “Active.”

You can remove any developer from the list by clicking the Remove button for the developer. The developer loses all access immediately.

All developers of an application have the same rights and access. They can access the Administration Console, upload new application files, create and delete versions, change the default version, access logs, and inspect and tweak the datastore. They can even remove your administrator access, take over billing, and disable or delete the app. When a developer signs in to the application itself via Google Accounts, the app recognizes her as an administrator, allowing her to access URLs configured as administrator-only and telling the app about the administrator status when the app inquires via the API.

While you can only register up to 10 application IDs yourself, you can be a developer on as many applications as you like, assuming you can get invited. Of course, it’s against the Terms of Service to create fake accounts just to be able to create more apps and invite your main account to them, so don’t do that.

Every administrative action performed by a developer adds a corresponding entry in a special log. Any developer can view this log in the Admin Logs section of the Console. This log cannot be edited or erased. Activities that get logged include app updates, version changes, developer invites and removals, index changes, settings changes, and changes to datastore entities made from the Console.

Quotas and Billing

The first thing you see when you visit the Administration Console and select an app is the “dashboard.” This handy screen provides a visual overview of your app’s traffic, resource usage, and errors. For an example of the dashboard for a new app, refer back to Figure 2-12 in Chapter 2.

The topmost chart displays time-based data over the past 24 hours. You can select from several data sets to view via the drop-down menu, including requests per second, clock time or CPU time per request, bandwidth, errors, and quota denials. You can adjust the period for this chart by clicking on the buttons (such as “6 hr”).

Below the chart is a graph showing how much of the billable quotas have been consumed for the calendar day, and how much of your daily budget has been spent for each quota. A message at the upper-right of the chart indicates how much of the calendar day is remaining. If any of the bars look like they might fill up before the next reset, you may need to increase your budget for that quota to avoid quota denials.

Near the bottom of the dashboard are lists of popular URL paths and URLs that are returning errors. You can click on a URL to view the detailed request logs for that URL path.

The dashboard’s time-based chart and URL traffic lists show data for the version of the app selected by the drop-down menu in the upper-left corner of the screen. When you first sign in to the Console, the default version is selected. To view data for a different version of the app, select it from the drop-down menu.

You can view a more comprehensive chart of how the app is consuming resources with quotas from the Quota Details section of the Administration Console. This chart shows billable quotas as well as several fixed quotas, such as API calls and service bandwidth. If your app is having quota-denial errors, check this screen for information on how the app is consuming resources.

The resource usage chart on the dashboard and the quota details screen show the total of all resource usage for all versions of the app. All versions of an app share the same budget and quotas.

When your app is ready to outgrow the free quotas, you can set up billing and set a budget for additional resources. App Engine allocates more resources as needed according to the budget you establish, and you are only billed for the resources actually consumed. To set up billing for an application, you select the Billing Settings panel in the Administration Console and click the Enable Billing button. The developer account used to enable billing becomes the “billing account,” and the owner of that account is solely responsible for setting the budget and paying for resources consumed.

When you enable billing, the Console prompts you for billing information and authorization to charge via Google Checkout, Google’s payment service. The process looks as though you’re making a purchase via Google Checkout, but no money is charged right away. You set the budget for the app, a maximum amount of money that can be charged to your account, then authorize Google Checkout to charge up to that amount. If you increase the budget at a later date, you must reauthorize the amount with Google Checkout. Figure 20-2 is an example of the budget settings screen.

Setting the budget in the Administration Console

Figure 20-2. Setting the budget in the Administration Console

You can view a history of charges made to the billing account, including a breakdown of the billed resource usage for each calendar day, in the Billing History panel.

Getting Help

If you have questions not answered by this book, you may find your answers in the official documentation on Google’s website:

http://developers.google.com/appengine/

The documentation includes complete references for the APIs and tools for both the Python and Java runtime environments; a list of frequently asked questions and answers (the FAQ); and a large collection of articles describing best practices, interesting features, and complete working examples.

You may also want to browse the contents of the SDK, as installed by either the Eclipse plug-in or the Python Launcher, and also available as a ZIP archive from the website. The source code for the Python SDK serves as supplementary documentation, and includes several undocumented (and unsupported) features and extensions. Both SDKs also include a set of functional example applications.

All App Engine developers should subscribe to Google’s App Engine downtime mailing list. This low-traffic, announcement-only list is used by the App Engine team to announce scheduled times when services are taken down for maintenance, and also to report the status of unexpected problems:

http://groups.google.com/group/google-appengine-downtime-notify

You can check the current and past health of App Engine and its services by consulting the system status site:

http://code.google.com/status/appengine

By far the best place to ask questions about Google App Engine is Stack Overflow. Post your question with the google-app-engine tag, and it’ll be seen by the App Engine developer community, as well as the App Engine team at Google. As you learn more, you can answer other people’s questions and build your reputation as an App Engine expert. Use Google to search past answers, which may have what you’re looking for:

http://stackoverflow.com/questions/tagged/google-app-engine

If you believe you have found a bug in App Engine, the SDK, or the Administration Console, or if you have a feature you’d like to request, you can post to the App Engine issue tracker. You can also see features others have requested, and vote for your favorites. (The App Engine team does indeed consider the highly requested features in this list when determining the product road map.)

http://code.google.com/p/googleappengine/issues/list

Google has a general discussion group for App Engine developers and an IRC channel, and also does regular live-streamed video question-and-answer sessions with the team. This page has more information on community resources:

http://developers.google.com/appengine/community