The Django Web Application Framework - Programming Google App Engine with Python (2015)

Programming Google App Engine with Python (2015)

Chapter 18. The Django Web Application Framework

As with all major categories of software, web applications have a common set of problems that need to be solved in code. Most web apps need software to interface with the server’s networking layer, communicate using the HTTP protocol, define the resources and actions of the application, describe and implement the persistent data objects, enforce site-wide policies such as access control, and describe the browser interface in a way that makes it easily built and modified by designers. Many of these components involve complex and detailed best practices for interoperating with remote clients and protecting against a variety of security vulnerabilities.

A web application framework is a collection of solutions and best practices that you assemble and extend to make an app. A framework provides the structure for an app, and most frameworks can be run without changes to demonstrate that the initial skeleton is functional. You use the toolkit provided by the framework to build the data model, business logic, and user interface for your app, and the framework takes care of the details. Frameworks are so useful that selecting one is often the first step when starting a new web app project.

Notice that App Engine isn’t a web application framework, exactly. App Engine provides scaling infrastructure, services, and interfaces that solve many common problems, but these operate at a level of abstraction just below most web app frameworks. A better example of a framework is webapp2, a framework for Python included with the App Engine Python SDK that we’ve been using in examples throughout the book so far. webapp2 lets you implement request handlers as Python classes, and it takes care of the details of interfacing with the Python runtime environment and routing requests to handler classes.

Several major frameworks for Python work well with App Engine. Django, Pyramid, web2py, and Flask work well, and some frameworks have explicit support for App Engine. These frameworks are mature, robust, and widely used, and have large thriving support communities and substantial online documentation. You can buy books about some of these frameworks.

Not every feature of every framework works with App Engine. Most notably, many frameworks include a mechanism for defining data models, but these are usually implemented for relational databases, and don’t work with the App Engine datastore. In some cases, you can just replace the framework’s data modeling library with App Engine’s ndb library. Some features of frameworks also have issues running within App Engine’s sandbox restrictions, such as by depending upon unsupported libraries. Developers have written adapter components that work around many of these issues.

In general, to use a framework, you add the framework’s libraries to your application directory and then map all dynamic URLs (all URLs except those for static files) to a script that invokes the framework. Because the interface between the runtime environment and the app is WSGI, you can associate the framework’s WSGI adapter with the URL pattern in app.yaml, just as we did with webapp2. Most frameworks have their own mechanism for associating URL paths with request handlers, and it’s often easiest to send all dynamic requests to the framework and let it route them. You may still want to use app.yaml to institute Google Accounts–based access control for some URLs.

Django is a popular web application framework for Python, with a rich stack of features and pluggable components. It’s also large, consisting of thousands of files. To make it easier to use Django on App Engine, the Python runtime environment includes the Django libraries, so you do not have to upload all of Django with your application files. The Python SDK bundles several versions of Django as well. App Engine includes Django 1.5 as a third-party library that you can request from app.yaml. You can use a newer version by downloading it and adding it to your application directory.

Django has its own data modeling library that is typically used with relational database as its backing store. Some Django components rely on this modeling library and don’t work without it. On App Engine, an easy option is to use Google Cloud SQL (introduced in Chapter 11) as the backing store for the Django library, though this doesn’t have the automatic scaling features of Cloud Datastore. Alternatively, you can use an adapter layer, such as djangae (currently only for Django 1.6 and 1.7), which uses Cloud Datastore as the backing store for Django and works with most components. Many features of Django don’t need its native data modeling facility at all, and you can always use App Engine’s ndb library directly in your own app code to access Cloud Datastore.

The official documentation for Django is famously good, although it relies heavily on Django’s own data modeling features for examples. For more information about Django, see the Django project website.

In this chapter, we discuss how to use Django 1.5 via the provided libraries, and discuss which of Django’s features work and which ones don’t when using Django this way. We’ll also take a quick look at how to use Django with Cloud SQL.

Using the Bundled Django Library

The App Engine Python SDK provides Django 1.5 in its lib/django-1.5/ subdirectory. With Django, you use a command-line tool to set up a new web application project. This tool expects lib/django-1.5/ to be in the Python library load path, so it can load modules from the django package.

One way to set this up is to add it to the PYTHONPATH environment variable on your platform. For example, on the Mac OS X or Linux command line, using a bash-compatible shell, run this command to change the environment variable for the current session to load Django 1.5 from the Python SDK located at ~/google-cloud-sdk/platform/google_appengine/:

export APPENGINE_PATH=~/google-cloud-sdk/platform/google_appengine

export PYTHONPATH=$PYTHONPATH:$APPENGINE_PATH/lib/django-1.5

The commands that follow will assume the SDK is in ~/google-cloud-sdk/platform/google_appengine/, and this PYTHONPATH is set.

The Django library is available in the runtime environment by using a libraries: directive in app.yaml, just like other libraries. We’ll see an example of this in a moment.

TIP

Django 1.5 is the most recent version included with the Python runtime environment as of App Engine version 1.9.18. Later versions of Django are likely to be added to the runtime environment in future releases.

Instructions should be similar for later versions, although it isn’t clear whether all future versions of Django will be added to the Python SDK. All previously included versions must remain in the SDK for compatibility, and the SDK might get a little large if it bundles every version. You may need to install Django on your local computer separately from the Python SDK in future versions. Django will likely be included in the runtime environment itself, similar to other third-party Python libraries. If you install Django yourself, you do not need to adjust the PYTHONPATH, and can run the Django commands without the library path.

You can install Django in your Python environment (or virtual environment) using pip:

pip install Django==1.5

Check the App Engine website for updates on the inclusion of future versions of Django in the runtime environment.

Creating a Django Project

For this tutorial, we will create an App Engine application that contains a Django project. In Django’s terminology, a project is a collection of code, configuration, and static files. A project consists of one or more subcomponents called apps. The Django model encourages designing apps to be reusable, with behavior controlled by the project’s configuration. The appearance of the overall website is also typically kept separate from apps by using a project-wide template directory.

You create a new Django project by running a command called django-admin.py startproject. Run this command to create a project named myproject in a subdirectory called myproject/:

export DJANGO_DIR=~/google-cloud-sdk/platform/google_appengine/lib/django-1.5

python $DJANGO_DIR/django/bin/django-admin.py startproject myproject

This command creates a project root directory in the current directory named myproject/ with several starter files. This root directory contains a subdirectory, also named myproject/. The starter files are as follows:

manage.py

A command-line utility you will use to build and manage this project, with many features

myproject/__init__.py

A file that tells Python that code files in this directory can be imported as modules (this directory is a Python package)

myproject/settings.py

Configuration for this project, in the form of a Python source file

myproject/urls.py

Mappings of URL paths to Python code, as a Python source file

myproject/wsgi.py

Code that sets up the WSGI middleware

The django-admin.py tool has many features, but most of them are specific to managing SQL databases. This is the last time we’ll use it here.

If you’re following along with a Django tutorial or book, the next step is usually to start the Django development server by using the manage.py command. If you did so now, you would be running the Django server, but it would know nothing of App Engine. We want to run this application in the App Engine development server. To do that, we need a couple of additional pieces.

Hooking It Up to App Engine

To connect our Django project to App Engine, we need a short script that instantiates the Django WSGI adapter, and an app.yaml configuration file that maps all (nonstatic) URLs to the Django project.

Create a file named main.py in the application root directory (the outer myproject/ directory) with the following contents:

import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

The first two lines tell Django where to find the project’s settings module, which in this case is at the module path myproject.settings (the myproject/settings.py file). This must be set before importing any Django modules. The remaining two lines import the WSGI adapter, instantiate it, and store it in a global variable.

Next, create app.yaml in the application root directory, like so:

application: myapp

version: 1

runtime: python27

api_version: 1

threadsafe: yes

handlers:

- url: .*

script: main.application

libraries:

- name: django

version: "1.5"

This should be familiar by now, but to review, this tells App Engine this is an application with ID myapp and version ID 1 running in the Python 2.7 runtime environment, with multithreading enabled. All URLs are routed to the Django project we just created, via the WSGI adapter instantiated in main.py. The libraries: declaration selects Django 1.5 as the version to use when importing django modules.

Our directory structure so far looks like this:

myproject/

app.yaml

main.py

manage.py

myproject/

__init__.py

settings.py

urls.py

wsgi.py

We can now start this application in a development server. Start the development server from the command line, using the current working directory as the application root directory:

dev_appserver.py .

Load the development server URL (http://localhost:8080/) in a browser, and enjoy the welcome screen (Figure 18-1).

pgap 1801

Figure 18-1. The Django welcome screen

Creating a Django App

The next step is to add a Django “app” to the Django project. You create new Django apps by using the project’s manage.py script, which was generated when you created the project. (You can also use django-admin.py to create apps.)

With the current working directory still set to the application root, create a new app named bookstore for this project:

python manage.py startapp bookstore

This creates the subdirectory bookstore, with four new files:

__init__.py

A file that tells Python that code files in this directory can be imported as modules (this directory is a Python package)

models.py

A source file for data models common to this app

tests.py

A starter file illustrating how to set up automated tests in Django

views.py

A source file for Django views (request handlers)

This layout is Django’s way of encouraging a design philosophy that separates data models and request handlers into separate, testable, reusable units of code. This philosophy comes naturally when using the datastore and ndb. Django does not depend on this file layout directly, and you can change it as needed.

In addition to creating the app, it’s useful to tell Django explicitly that this app will be used by this project. Edit myproject/settings.py, and find the INSTALLED_APPS value. Set it to a tuple containing the Python path to the app’s package:

INSTALLED_APPS = (

'bookstore',

)

Be sure to include the trailing comma, which tells Python this is a one-element tuple.

The settings.py file is a Python source file. It contains settings for the Django framework and related components, in the form of variables. The INSTALLED_APPS setting lists all apps in active use by the project, which enables some automatic features such as template loading. (It’s primarily used by Django’s data modeling tools, which we won’t be using, in favor of App Engine’s ndb.)

Let’s define our first custom response. Edit bookstore/views.py, and replace its contents with the following:

from django import http

def home(request):

return http.HttpResponse('Welcome to the Book Store!')

In Django, a view is a function that is called with a django.http.HttpRequest object and returns a django.http.HttpResponse object. This view creates a simple response with some text. As you would expect, the HttpRequest provides access to request parameters and headers. You can set headers and other aspects of the HttpResponse. Django provides several useful ways to make HttpResponse objects, as well as specialized response classes for redirects and other return codes.

We still need to connect this view to a URL. URLs for views are set by the project, in the myproject/urls.py file. This allows the project to control the URLs for all of the apps it uses. You can organize URLs such that each app specifies its own set of mappings of URL subpaths to views, and the project provides the path prefix for each app. For now, we’ll keep it simple, and just refer to the app’s view directly in the project’s URL configuration.

Each Django app defines its own URL patterns. Create a new file in the bookstore/ directory named urls.py, and give it the following contents:

from django.conf.urls import patterns, url

from bookstore import views

urlpatterns = patterns('',

url(r'^$', views.home),

)

Add a reference to the bookstore/urls.py file from the project’s main myproject/urls.py, like so:

from django.conf.urls import include, patterns, url

urlpatterns = patterns('',

url(r'^books/', include('bookstore.urls')),

)

The entry in the project’s myproject/urls.py file maps all URL paths that begin with books/ to the bookstore Django app. The entry in bookstore/urls.py matches subpaths of this path. The pattern r'^books/$' matches books/, and this is followed by the pattern r'^$', which matches the empty string.

With the development server still running, load the /books/ URL in your browser. The app calls the view to display some text.

You may notice that, now that you have defined a URL pattern, the URL / no longer displays the Django welcome screen. Eventually, you will want to add an entry to myproject/urls.py to associate the root URL path with an appropriate view.

Using Django Templates

Django includes a templating system for building web pages and other displayable text. The Jinja2 templating library we’ve used throughout the book so far is based on the Django template system. Their syntaxes are mostly similar, but you’ll notice minor differences between the two systems. As just one example, Jinja2 lets you call methods on template values with arguments, and so requires that you use parentheses after the method even when not using arguments: {{ someval.method() }}. In Django, templates can call methods of values but cannot pass arguments, and so the parentheses are omitted: {{ someval.method }}.

Django templates are baked into the Django framework, so they’re easy to use. It’s possible to use Jinja2 templates with a Django application, but Jinja2 does not automatically support some of the organizational features of Django.

Let’s update the example to use a Django template. First, we need to set up a template directory for the bookstore Django app. In the bookstore/ directory, create a directory named templates/, and a subdirectory in there named bookstore:

mkdir -p bookstore/templates/bookstore

(The -p option to mkdir tells it to create the entire path of subdirectories, if any subdirectory does not exist.)

By default, Django knows how to look for templates in each app’s templates/ subdirectory. We create another subdirectory named after the app (templates/bookstore/) so that bookstore templates all have a common path prefix.

Inside the bookstore/templates/bookstore/ subdirectory, create the file index.html with the following template text:

<html>

<body>

<p>Welcome to The Book Store! {{ clock }}</p>

</body>

</html>

Finally, edit bookstore/views.py to look like this:

from django.shortcuts import render_to_response

import datetime

def home(request):

return render_to_response(

'bookstore/index.html',

{ 'clock': datetime.datetime.now() },

)

Reload the page to see the template displayed by the new view.

The render_to_response() shortcut function takes as its first argument the path to the template file. This path is relative to bookstore/templates/. The second argument is a Python mapping that defines variables to be used within the template. In this example, we set a template variable named clock to be the current datetime.datetime. Within the template, {{ code }} interpolates this value as a string.

The behavior of the template engine can be extended in many ways. You can define custom tags and filters to use within templates. You can also change how templates are loaded, with template loader classes and the TEMPLATE_LOADERS setting variable. The behavior we’re using here is provided by the django.template.loaders.app_directories.Loader class, which appears in the default settings file created by Django.

Using ndb with Django

In Chapter 9, we discussed several powerful automatic behaviors of the ndb library. To support these features, ndb needs an opportunity to clean up pending operations when a request handler is finished. Django needs to be told to give ndb this opportunity. ndb provides a class that Django knows how to call at the beginning and end of each request. Django calls this class middleware, because it lives in the middle of the call stack.

Edit myproject/settings.py, then locate MIDDLEWARE_CLASSES. At the top of this list, insert an entry for the ndb middleware, like so:

MIDDLEWARE_CLASSES = (

'google.appengine.ext.ndb.django_middleware.NdbDjangoMiddleware',

# ...

)

You don’t have to think too hard about how this works. Just remember to include it when using ndb and Django together. If your app doesn’t use the ndb library, you don’t need the middleware.

Using WTForms with ndb

One of the many arduous tasks that web frameworks can do well is forms, those pages of data entry fields that users fill out and submit, and applications store away for processing later. A good framework makes it easy to describe forms and their corresponding data models, and takes care of HTML rendering, data validation, error reporting, and security. Django includes a forms framework, but it only works with Django’s data modeling library, and doesn’t work with App Engine’s ndb.1 Instead, you can use a work-alike library called WTForms. You don’t need Django to use WTForms: it works just as well with webapp2, Flask, or any other framework.

We won’t go into the details of how WTForms works—see the WTForms documentation for a complete explanation—but let’s walk through a quick example to see how the pieces fit together. Our example will use the following behavior for creating and editing Book entities:

§ An HTTP GET request to /books/book/ displays an empty form for creating a new Book.

§ An HTTP POST request to /books/book/ processes the book creation form, and either creates the book and redirects to /books (the book listing page) or redisplays the form with errors, if any.

§ An HTTP GET request to /books/book/1234 displays the form to edit the Book entity, with the fields filled out with the current values.

§ An HTTP POST request to /books/book/1234 updates the book with that ID, with the same error-handling behavior as the book creation form.

WTForms is a third-party library that is not provided by App Engine, so it must be added directly to the application code. From the application root directory, run the following command:

pip install -t lib WTForms WTForms-Appengine

# Clean up unnecessary installation control files.

rm -rv lib/*-info

This creates a lib/ directory, and installs the WTForms library and its App Engine ndb plug-in. When you deploy your app, these files will be deployed with it and will be available to the app.

To make libraries in lib/ available for import, we must add it to the lookup path. Edit main.py, and add these lines near the top:

import sys

sys.path.append('lib')

Let’s set up the new form URLs. Edit bookstore/urls.py to use a new view function named book_form() to handle these URLs:

from django.conf.urls.defaults import patterns, url

urlpatterns = patterns('myproject',

url(r'^book/(\d*)', views.book_form),

url(r'^$', views.home),

)

The regular expression '^book/(\d*)' captures the book ID in the URL, if any, and passes it to the view function as an argument.

Edit bookstore/models.py and replace its contents with the following ndb model definitions:

from google.appengine.ext import ndb

class Book(ndb.Model):

title = ndb.StringProperty()

author = ndb.StringProperty()

copyright_year = ndb.IntegerProperty()

author_birthdate = ndb.DateProperty()

class BookReview(ndb.Model):

book = ndb.KeyProperty(kind='Book')

review_author = ndb.UserProperty()

review_text = ndb.TextProperty()

rating = ndb.StringProperty(choices=['Poor', 'OK', 'Good',

'Very Good', 'Great'],

default='Great')

create_date = ndb.DateTimeProperty(auto_now_add=True)

No surprises here. These are just ndb models for datastore entities of the kinds Book and BookReview.

Now for the views. Edit bookstore/views.py, and replace its contents with the following:

from django import template

from django.http import HttpResponseRedirect

from django.shortcuts import render_to_response

from google.appengine.ext import ndb

from wtforms_appengine.ndb import model_form

from bookstore import models

def home(request):

q = models.Book.query().order('title')

return render_to_response('bookstore/index.html',

{'books': q})

BookForm = model_form(models.Book)

def book_form(request, book_id=None):

if request.method == 'POST':

# The form was submitted.

if book_id:

# Fetch the existing Book and update it from the form.

book = models.Book.get_by_id(int(book_id))

form = BookForm(request.POST, obj=book)

else:

# Create a new Book based on the form.

book = models.Book()

form = BookForm(request.POST)

if form.validate():

form.populate_obj(book)

book.put()

return HttpResponseRedirect('/books/')

# else fall through to redisplay the form with error messages

else:

# The user wants to see the form.

if book_id:

# Show the form to edit an existing Book.

book = models.Book.get_by_id(int(book_id))

form = BookForm(obj=book)

else:

# Show the form to create a new Book.

form = BookForm()

return render_to_response('bookstore/bookform.html', {

'book_id': book_id,

'form': form,

}, template.RequestContext(request))

We’ve updated the home() view to set up a query for Book entities, and pass that query object to the template. Edit bookstore/templates/bookstore/index.html to display this information:

<html>

<body>

<p>Welcome to The Book Store!</p>

<p>Books in our catalog:</p>

<ul>

{% for book in books %}

<li>{{ book.title }}, by {{ book.author }} ({{ book.copyright_year }})

[<a href="/books/book/{{ book.key.id }}">edit</a>]</li>

{% endfor %}

</ul>

<p>[<a href="/books/book/">add a book</a>]</p>

</body>

</html>

Finally, create the template for the form used by the new book_form() view, named bookstore/templates/bookstore/bookform.html:

<html>

<body>

{% if book_id %}

<p>Edit book {{ book_id }}:</p>

<form action="/books/book/{{ book_id }}" method="POST">

{% else %}

<p>Create book:</p>

<form action="/books/book/" method="POST">

{% endif %}

{% csrf_token %}

<p>

{{ form.title.label|safe }}: {{ form.title|safe }}

{% if form.title.errors %}

<ul>

{% for error in form.title.errors %}

<li>{{ error }}</li>

{% endfor %}

</ul>

{% endif %}

</p>

<p>

{{ form.author.label|safe }}: {{ form.author|safe }}

{% if form.author.errors %}

<ul>

{% for error in form.author.errors %}

<li>{{ error }}</li>

{% endfor %}

</ul>

{% endif %}

</p>

<p>

{{ form.copyright_year.label|safe }}: {{ form.copyright_year|safe }}

{% if form.copyright_year.errors %}

<ul>

{% for error in form.copyright_year.errors %}

<li>{{ error }}</li>

{% endfor %}

</ul>

{% endif %}

</p>

<p>

{{ form.author_birthdate.label|safe }}: {{ form.author_birthdate|safe }}

{% if form.author_birthdate.errors %}

<ul>

{% for error in form.author_birthdate.errors %}

<li>{{ error }}</li>

{% endfor %}

</ul>

{% endif %}

</p>

<input type="submit" />

</form>

</body>

</html>

BookForm is a class generated by the model_form() function, based on the models.Book ndb model. You can also define form classes manually by subclassing wtforms.Form. A form model is very similar to an ndb data model, using class members to declare typed fields. Because form data is typically stored in a datastore once it is submitted, it is convenient to define the form model in terms of the underlying data model, so you don’t have to describe the same thing twice and keep both models in sync.

The form class has useful default rendering and processing behavior for each of the default property declaration types, and you can customize this extensively. For now, we’ll use the defaults produced by model_form(). An instance of the form class is responsible for storing the form data entered by the user, keeping track of validation errors, and populating the ndb.Model instance when we’re ready to store the data.

The book_form() view function takes the HTTP request object and the book_id captured by the regular expression in urls.py as arguments. If the request method is 'POST', then it processes the submitted form. Otherwise it assumes the method is 'GET' and just displays the form. In either case, the form is represented by an instance of the BookForm class.

If constructed without arguments, the BookForm represents an empty form for creating a new Book entity. If constructed with the obj argument set to a Book object, the form’s fields are prepopulated with the object’s property values.

To process a submitted form, you pass the dictionary of POST parameters (request.POST) to the BookForm constructor as its first positional argument. If you also provide the obj argument, the instance sets the initial values—including the entity key—and the form data overwrites everything else, as provided.

The BookForm object knows how to render the form based on the model class and the provided model instance (if any). It also knows how to validate data submitted by the user, and render the form with the user’s input and any appropriate error messages included. The validate() method tells you if the submitted data is acceptable for saving to the datastore. If it isn’t, you send the BookForm to the template just as you would when displaying the form for the first time.

If the data submitted by the user is valid, the BookForm knows how to produce the final entity object. The populate_obj() method takes an entity instance and populates its properties based on the submitted form data. In this example, a successful create or update redirects the user to/books/ (which we’ve hardcoded in the view for simplicity) instead of rendering a template.

To display the form, we pass the BookForm object to the bookstore/templates/bookstore/bookform.html template. Methods on the object generate HTML for the labels and form fields for each property. Our template also tests for validation errors, in case the form is being redisplayed because of a data entry error, and prints error messages as needed.

Restart the development server, then load the book list URL (/books/) in the browser. Click “add a book” to show the book creation form. Enter some data for the book, then submit it to create it. Because the development server may try to display the book list prior to updating the global datastore index, you may need to reload the book list after creating a book to see it appear.

NOTE

The default form widget for a date field is just a text field, and it’s finicky about the format. In this case, the “author birthdate” field expects input in the form YYYY-MM-DD, such as 1902-02-27.

Continue the test by clicking the “edit” link next to one of the books listed. The form displays with that book’s data. Edit some of the data and then submit the form to update the entity.

Also try entering invalid data for a field, such as nonnumeric data for the “copyright year” field, or a date that doesn’t use the expected format. Notice that the form redisplays with your original input, and with error messages.

The main thing to notice about this example is that the data model class itself (in this case Book) completely describes the default form, including the display of its form fields and the validation logic. The default field names are based on the names of the properties. You can change a field’s name by specifying a verbose_name argument to the property declaration on the model class:

class Book(ndb.Model):

title = ndb.StringProperty(verbose_name="Book title")

# ...

With WTForms, you can customize the display, the error messages, and the validation routines in many ways. See the WTForms documentation for more information.

CROSS-SITE REQUEST FORGERY

Cross-site request forgery (CSRF) is a class of security issues with web forms where the attacker lures a victim into submitting a web form whose action is your web application, but the form is under the control of the attacker. The malicious form may intercept the victim’s form values, or inject some of its own, and cause the form to be submitted to your app on behalf of the victim.

Django has a built-in feature for protecting against CSRF attacks, and it is enabled by default. The protection works by generating a token that is added to forms displayed by your app, and is submitted with the user’s form fields. If the form is submitted without a valid token, Django rejects the request before it reaches the view code. The token is a digital signature, and is difficult to forge.

This requires the cooperation of our example code in two places. The template/bookstore/bookform.html template includes the {% csrf_token %} template tag somewhere inside the <form> element. Also, the render_to_response() function needs to pass the request to the template when rendering the form, with a third argument, template.RequestContext(request).

The blocking magic happens in a component known as middleware, similar to the middleware we added for ndb support. This architectural feature of Django lets you compose behaviors that act on some or all requests and responses, independently of an app’s views. The MIDDLEWARE_CLASSES setting in settings.py activates middleware, anddjango.middleware.csrf.CsrfViewMiddleware is enabled by default. If you have a view that accepts POST requests and doesn’t need CSRF protection (such as a web service endpoint), you can give the view the @csrf_exempt decorator, from the django.views.decorators.csrf module.

This feature of Django illustrates the power of a full-stack web application framework. Not only is it possible to implement a security feature like CSRF protection across an entire site with a single component, but this feature can be provided by a library of such components. (You could argue that this is a poor example, because it imposes requirements on views and templates that render forms. But the feature is useful enough to be worth it.)

See Django’s CSRF documentation for more information.

Using a Newer Version of Django

The versions of the Django library that are available in the App Engine runtime environment tend to lag behind the latest releases. As of App Engine 1.9.18, App Engine provides up to Django 1.5, while the latest release of Django is 1.8. There are good reasons to use newer versions, and it’s usually not a problem to do so.2 Instead of using a built-in version, you add Django to your application files.

You can add Django 1.8 to your app just as we did earlier with the WTForms library. Run these commands from the application root directory:

pip install -t lib Django==1.8

rm -rv lib/*-info

If you haven’t already, add these lines near the top of main.py:

import sys

sys.path.append('lib')

Remove django from the libraries: section of app.yaml.

We must make one small change to main.py to upgrade to Django 1.8’s method of setting up the WSGI application instance. The complete file looks like this:

import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'

import sys

sys.path.append('lib')

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

Restart the development server, and confirm that the app is still working.

If you ever need to run the django-admin.py command, or if you just want to start a new project from scratch using the new version, be sure to use the new installation, like so:

export DJANGO_DIR=.../myproject/lib/

python $DJANGO_DIR/django/bin/django-admin.py startproject newproject

A complete installation of Django contains nearly 4,000 files. Be aware that App Engine imposes a file count limit of 10,000 application files per version per app. You may be able to remove components from lib/django/contrib that you are not using to reduce the number of files. For example, in Django 1.8, GeoDjango (lib/django/contrib/gis) contributes 608 files.

Using Django with Google Cloud SQL

While Django as we’ve seen it so far is already quite useful for App Engine apps, the framework really shines when paired with a relational database. You can configure Django to use Google Cloud SQL from App Engine to take advantage of a rich collection of database-backed features and plug-ins.

If you haven’t already, perform the setup steps described in Chapter 11 to install MySQL and the MySQLdb library on your local computer, set up a Cloud SQL instance, and create a database.

You configure Django to use a database with settings in the myproject/settings.py file. With App Engine and Cloud SQL, you can use the django.db.backends.mysql driver included with Django, setting the 'HOST' parameter to be the /cloudsql/ socket path for the Cloud SQL instance.

Continuing the example from the Cloud SQL chapter, with a project ID of saucy-boomerang-123, an instance name of mydb, a database named mmorpg, and a user named app, the configuration that would work when running on App Engine to connect to Cloud SQL looks like this:

DATABASES = {

'default': {

'ENGINE': 'django.db.backends.mysql',

'HOST': '/cloudsql/saucy-boomerang-123:mydb',

'NAME': 'mmorpg',

'USER': 'app'

}

}

When testing the app locally, we need different configuration to tell Django to use the local MySQL test database. The settings.py file is Python code, so we can use conditional logic to select database configuration based on the environment:

import os

if os.environ.get('SERVER_SOFTWARE', '').startswith('Google App Engine'):

# This is App Engine.

DATABASES = {

'default': {

'ENGINE': 'django.db.backends.mysql',

'HOST': '/cloudsql/saucy-boomerang-123:mydb',

'NAME': 'mmorpg',

'USER': 'app'

}

}

else:

# This is a development server.

DATABASES = {

'default': {

'ENGINE': 'django.db.backends.mysql',

'HOST': '127.0.0.1',

'NAME': 'mmorpg',

'USER': 'app',

'PASSWORD': 'p4$$w0rd'

}

}

In addition to the application code, Django’s command-line tools use this configuration to create and manage table schemas. These tools run locally, and during development you use these tools to update your local MySQL database. You also use these tools to update the live database prior to deploying new software.

For this to work, the configuration needs to support a third case: connecting to the live database from the local machine. The App Engine SDK includes a custom driver to support this case, called google.appengine.ext.django.backends.rdbms. The custom driver uses gcloudauthentication to access your app’s databases. Here is one way to set this up in settings.py:

import os

if os.environ.get('SERVER_SOFTWARE', '').startswith('Google App Engine'):

# This is App Engine.

# ...

elif os.environ.get('MANAGE_DATABASE_MODE') == 'live':

# The administrator is running a command-line tool in "live" mode.

DATABASES = {

'default': {

'ENGINE': 'google.appengine.ext.django.backends.rdbms',

'INSTANCE': 'saucy-boomerang-123:mydb',

'NAME': 'mmorpg',

'USER': 'app'

}

}

else:

# This is a development server.

# ...

The App Engine SDK must be on the PYTHONPATH library lookup path for the command-line tools to find and use the custom driver. If you haven’t already, add ~/google-cloud-sdk/platform/google_appengine (where ~/google-cloud-sdk is the path to your Cloud SDK installation) to this path in your environment. The yaml library is also needed, and is included with the SDK. If you’re using the version of Django distributed with the SDK, add the path to this library to PYTHONPATH as well.

For example, if using the bash shell, add this to your .bashrc file:

export APPENGINE_PATH=~/google-cloud-sdk/platform/google_appengine

export PYTHONPATH=$PYTHONPATH:$APPENGINE_PATH

export PYTHONPATH=$PYTHONPATH:$APPENGINE_PATH/lib/yaml/lib

export PYTHONPATH=$PYTHONPATH:$APPENGINE_PATH/lib/django-1.5

The MANAGE_DATABASE_MODE environment variable is just an environment variable you can set when you run a Django tool. In Mac OS, Linux, or Windows with Cygwin, you can set this environment variable when you run a command, like so:

MANAGE_DATABASE_MODE='live' ./manage.py syncdb

To use your local database, simply leave the environment variable unset:

./manage.py syncdb

The ./manage.py tool was created in your application root directory when you ran the django-admin.py startproject command. You use this to update and manage your database.

With all of this set up, you can now perform every step of the official Django tutorial. As before, use dev_appserver.py . instead of ./manage.py runserver to start the development server.

TIP

The manage.py tool needs enough database privileges to create, update, and drop tables. In Chapter 11, we created the app account and gave it specific privileges that did not include creating and dropping tables. To expand the privileges on this account, connect to the database using your root account:

mysql -h ... -u root -p

At the mysql> prompt, grant all privileges to app:

GRANT ALL ON mmorpg.* TO 'app';

You may want to create a separate admin database account that is used exclusively by manage.py, then either modify manage.py to modify settings.DATABASES, or use the environment variable technique to select this account in settings.py.

1 In the old ext.db library, this compatibility was provided by the google.appengine.ext.db.djangoforms module, which wrapped the Django forms interface. There is no equivalent adapter for Django and ndb, but WTForms is a capable substitute.

2 If you intend to use Django with Google Cloud SQL, stick with Django 1.5. As of App Engine 1.9.18, the google.appengine.ext.django.backends.rdbms custom driver is not compatible with Django 1.8.