Django Design Patterns and Best Practices (2015)

Chapter 7. Forms

In this chapter, we will discuss the following topics:

· Form workflow

· Untrusted input

· Form processing with class-based views

· Working with CRUD views

Let's set aside Django Forms and talk about web forms in general. Forms are not just long, boring pages with several items that you have to fill. Forms are everywhere. We use them every day. Forms power everything from Google's search box to Facebook's Likebutton.

Django abstracts most of the grunt work while working with forms such as validation or presentation. It also implements various security best practices. However, forms are also common sources of confusion due to one of several states they could be in. Let's examine them more closely.

How forms work

Forms can be tricky to understand because interacting with them takes more than one request-response cycle. In the simplest scenario, you need to present an empty form, and the user fills it correctly and submits it. In other cases, they enter some invalid data and the form needs to be resubmitted until the entire form is valid.

So, a form goes through several states:

· Empty form: This form is called an unbound form in Django

· Filled form: This form is called a bound form in Django

· Submitted form with errors: This form is called a bound form but not a valid form

· Submitted form without errors: This form is called a bound and valid form

Note that the users will never see the form in the last state. They don't have to. Submitting a valid form should take the users to a success page.

Forms in Django

Django's form class contains the state of each field and, by summarizing them up a level, of the form itself. The form has two important state attributes, which are as follows:

· is_bound: If this returns false, then it is an unbound form, that is, a fresh form with empty or default field values. If true, then the form is bound, that is, at least one field has been set with a user input.

· is_valid(): If this returns true, then every field in the bound form has valid data. If false, then there was some invalid data in at least one field or the form was not bound.

For example, imagine that you need a simple form that accepts a user's name and age. The form class can be defined as follows:

# forms.py

from django import forms

class PersonDetailsForm(forms.Form):

name = forms.CharField(max_length=100)

age = forms.IntegerField()

This class can be initiated in a bound or unbound manner, as shown in the following code:

>>> f = PersonDetailsForm()

>>> print(f.as_p())

<p><label for="id_name">Name:</label> <input id="id_name" maxlength="100" name="name" type="text" /></p>

<p><label for="id_age">Age:</label> <input id="id_age" name="age" type="number" /></p>

>>> f.is_bound

False

>>> g = PersonDetailsForm({"name": "Blitz", "age": "30"})

>>> print(g.as_p())

<p><label for="id_name">Name:</label> <input id="id_name" maxlength="100" name="name" type="text" value="Blitz" /></p>

<p><label for="id_age">Age:</label> <input id="id_age" name="age" type="number" value="30" /></p>

>>> g.is_bound

True

Notice how the HTML representation changes to include the value attributes with the bound data in them.

Forms can be bound only when you create the form object, that is, in the constructor. How does the user input end up in a dictionary-like object that contains values for each form field?

To find this out, you need to understand how a user interacts with a form. In the following diagram, a user opens the person's details form, fills it incorrectly first, submits it, and then resubmits it with the valid information:

As shown in the preceding diagram, when the user submits the form, the view callable gets all the form data inside request.POST (an instance of QueryDict). The form gets initialized with this dictionary-like object—referred to this way since it behaves like a dictionary and has a bit of extra functionality.

Forms can be defined to send the form data in two different ways: GET or POST. Forms defined with METHOD="GET" send the form data encoded in the URL itself, for example, when you submit a Google search, your URL will have your form input, that is, the search string visibly embedded, such as ?q=Cat+Pictures. The GET method is used for idempotent forms, which do not make any lasting changes to the state of the world (or to be more pedantic, processing the form multiple times has the same effect as processing it once). For most cases, this means that it is used only to retrieve data.

However, the vast majority of the forms are defined with METHOD="POST". In this case, the form data is sent along with the body of the HTTP request, and they are not seen by the user. They are used for anything that involves a side effect, such as storing or updating data.

Depending on the type of form you have defined, the view will receive the form data in request.GET or request.POST, when the user submits the form. As mentioned earlier, either of them will be like a dictionary. So, you can pass it to your form class constructor to get a bound form object.

Note

The Breach

Steve was curled up and snoring heavily in his large three-seater couch. For the last few weeks, he had been spending more than 12 hours at the office, and tonight was no exception. His phone lying on the carpet beeped. At first, he said something incoherently, still deep in sleep. Then, it beeped again and again, in increasing urgency.

By the fifth beep, Steve awoke with a start. He frantically searched all over his couch, and finally located his phone. The screen showed a brightly colored bar chart. Every bar seemed to touch the high line except one. He pulled out his laptop and logged into the SuperBook server. The site was up and none of the logs indicated any unusual activity. However, the external services didn't look that good.

The phone at the other end seemed to ring for eternity until a croaky voice answered, "Hello, Steve?" Half an hour later, Jacob was able to zero down the problem to an unresponsive superhero verification service. "Isn't that running on Sauron?" asked Steve. There was a brief hesitation. "I am afraid so," replied Jacob.

Steve had a sinking feeling at the pit of his stomach. Sauron, a mainframe application, was their first line of defense against cyber-attacks and other kinds of possible attack. It was three in the morning when he alerted the mission control team. Jacob kept chatting with him the whole time. He was running every available diagnostic tool. There was no sign of any security breach.

Steve tried to calm him down. He reassured him that perhaps it was a temporary overload and he should get some rest. However, he knew that Jacob wouldn't stop until he found what's wrong. He also knew that it was not typical of Sauron to have a temporary overload. Feeling extremely exhausted, he slipped back to sleep.

Next morning, as Steve hurried to his office building holding a bagel, he heard a deafening roar. He turned and looked up to see a massive spaceship looming towards him. Instinctively, he ducked behind a hedge. On the other side, he could hear several heavy metallic objects clanging onto the ground. Just then his cell phone rang. It was Jacob. Something had moved closer to him. As Steve looked up, he saw a nearly 10-foot-tall robot, colored orange and black, pointing what looked like a weapon directly down at him.

His phone was still ringing. He darted out into the open barely missing the sputtering shower of bullets around him. He took the call. "Hey Steve, guess what, I found out what actually happened." "I am dying to know," Steve quipped.

"Remember, we had used UserHoller's form widget to collect customer feedback? Apparently, their data was not that clean. I mean several serious exploits. Hey, there is a lot of background noise. Is that the TV?" Steve dived towards a large sign that said "Safe Assembly Point". "Just ignore that. Tell me what happened," he screamed.

"Okay. So, when our admin opened their feedback page, his laptop must have gotten infected. The worm could reach other systems he has access to, specifically, Sauron. I must say Jacob, this is a very targeted attack. Someone who knows our security system quite well has designed this. I have a feeling something scary is coming our way."

Across the lawn, a robot picked up an SUV and hurled it towards Steve. He raised his hands and shut his eyes. The spinning mass of metal froze a few feet above him. "Important call?" asked Hexa as she dropped the car. "Yeah, please get me out of here," Steve begged.

Why does data need cleaning?

Eventually, you need to get the "cleaned data" from the form. Does this mean that the values that the user had entered were not clean? Yes, for two reasons.

First, anything that comes from the outside world should not be trusted initially. Malicious users can enter all sorts of exploits through a form that can undermine the security of your site. So, any form data must be sanitized before you use them.

Tip

Best Practice

Never trust the user input.

Secondly, the field values in request.POST or request.GET are just strings. Even if your form field can be defined as an integer (say, age) or date (say, birthday), the browser would send them as strings to your view. Invariably, you would like to convert them to the appropriate Python types before use. The form class does this conversion automatically for you while cleaning.

Let's see this in action:

>>> fill = {"name": "Blitz", "age": "30"}

>>> g = PersonDetailsForm(fill)

>>> g.is_valid()

True

>>> g.cleaned_data

{'age': 30, 'name': 'Blitz'}

>>> type(g.cleaned_data["age"])

int

The age value was passed as a string (possibly, from request.POST) to the form class. After validation, the cleaned data contains the age in the integer form. This is exactly what you would expect. Forms try to abstract away the fact that strings are passed around and give you clean Python objects that you can use.

Displaying forms

Django forms also help you create an HTML representation of your form. They support three different representations: as_p (as paragraph tags), as_ul (as unordered list items), and as_table (as, unsurprisingly, a table).

The template code, generated HTML code, and browser rendering for each of these representations have been summarized in the following table:

Notice that the HTML representation gives only the form fields. This makes it easier to include multiple Django forms in a single HTML form. However, this also means that the template designer has a fair bit of boilerplate to write for each form, as shown in the following code:

<form method="post">

{% csrf_token %}

<table>{{ form.as_table }}</table>

<input type="submit" value="Submit" />

</form>

Note that to make the HTML representation complete, you need to add the surrounding form tags, a CSRF token, the table or ul tags, and the submit button.

Time to be crisp

It can get tiresome to write so much boilerplate for each form in your templates. The django-crispy-forms package makes writing the form template code more crisp (in the sense of short). It moves all the presentation and layout into the Django form itself. This way, you can write more Python code and less HTML.

The following table shows that the crispy form template tag generates a more complete form, and the appearance is much more native to the Bootstrap style:

So, how do you get crisper forms? You will need to install the django-crispy-forms package and add it to your INSTALLED_APPS. If you use Bootstrap 3, then you will need to mention this in your settings:

CRISPY_TEMPLATE_PACK = "bootstrap3"

The form initialization will need to mention a helper attribute of the type FormHelper. The following code is intended to be minimal and uses the default layout:

from crispy_forms.helper import FormHelper

from crispy_forms.layout import Submit

class PersonDetailsForm(forms.Form):

name = forms.CharField(max_length=100)

age = forms.IntegerField()

def __init__(self, *args, **kwargs):

super().__init__(*args, **kwargs)

self.helper = FormHelper(self)

self.helper.layout.append(Submit('submit', 'Submit'))

Understanding CSRF

So, you must have noticed something called a CSRF token in the form templates. What does it do? It is a security mechanism against Cross-Site Request Forgery (CSRF) attacks for your forms.

It works by injecting a server-generated random string called a CSRF token, unique to a user's session. Every time a form is submitted, it must have a hidden field that contains this token. This token ensures that the form was generated for the user by the original site, rather than a fake form created by an attacker with similar fields.

CSRF tokens are not recommended for forms using the GET method because the GET actions should not change the server state. Moreover, forms submitted via GET would expose the CSRF token in the URLs. Since URLs have a higher risk of being logged or shoulder-sniffed, it is better to use CSRF in forms using the POST method.

Form processing with Class-based views

We can essentially process a form by subclassing the Class-based view itself:

class ClassBasedFormView(generic.View):

template_name = 'form.html'

def get(self, request):

form = PersonDetailsForm()

return render(request, self.template_name, {'form': form})

def post(self, request):

form = PersonDetailsForm(request.POST)

if form.is_valid():

# Success! We can use form.cleaned_data now

return redirect('success')

else:

# Invalid form! Reshow the form with error highlighted

return render(request, self.template_name,

{'form': form})

Compare this code with the sequence diagram that we saw previously. The three scenarios have been separately handled.

Every form is expected to follow the Post/Redirect/Get (PRG) pattern. If the submitted form is found to be valid, then it must issue a redirect. This prevents duplicate form submissions.

However, this is not a very DRY code. The form class name and template name attributes have been repeated. Using a generic class-based view such as FormView can reduce the redundancy of form processing. The following code will give you the same functionality as the previous one in fewer lines of code:

from django.core.urlresolvers import reverse_lazy

class GenericFormView(generic.FormView):

template_name = 'form.html'

form_class = PersonDetailsForm

success_url = reverse_lazy("success")

We need to use reverse_lazy in this case because the URL patterns are not loaded when the view file is imported.

Form patterns

Let's take a look at some of the common patterns when working with forms.

Pattern – dynamic form generation

Problem: Adding form fields dynamically or changing form fields from what has been declared.

Solution: Add or change fields during initialization of the form.

Problem details

Forms are usually defined in a declarative style with form fields listed as class fields. However, sometimes we do not know the number or type of these fields in advance. This calls for the form to be dynamically generated. This pattern is sometimes called Dynamic Forms or Runtime form generation.

Imagine a flight passenger check-in system, which allows for the upgrade of economy class tickets to first class. If there are any first-class seats left, there needs to be an additional option to the user if they would like to fly first class. However, this optional field cannot be declared since it will not be shown to all users. Such dynamic forms can be handled by this pattern.

Solution details

Every form instance has an attribute called fields, which is a dictionary that holds all the form fields. This can be modified at runtime. Adding or changing the fields can be done during form initialization itself.

For example, if we need to add a checkbox to a user details form only if a keyword argument named "upgrade" is true at form initialization, then we can implement it as follows:

class PersonDetailsForm(forms.Form):

name = forms.CharField(max_length=100)

age = forms.IntegerField()

def __init__(self, *args, **kwargs):

upgrade = kwargs.pop("upgrade", False)

super().__init__(*args, **kwargs)

# Show first class option?

if upgrade:

self.fields["first_class"] = forms.BooleanField(

label="Fly First Class?")

Now, we just need to pass the, PersonDetailsForm(upgrade=True) keyword argument to make an additional Boolean input field ( a checkbox) appear.

Note

Note that a newly introduced keyword argument has to be removed or popped before we call super to avoid the unexpected keyword error.

If we use a FormView class for this example, then we need to pass the keyword argument by overriding the get_form_kwargs method of the view class, as shown in the following code:

class PersonDetailsEdit(generic.FormView):

...

def get_form_kwargs(self):

kwargs = super().get_form_kwargs()

kwargs["upgrade"] = True

return kwargs

This pattern can be used to change any attribute of a field at runtime, such as its widget or help text. It works for model forms as well.

In many cases, a seeming need for dynamic forms can be solved using Django formsets. They are used when a form needs to be repeated in a page. A typical use case for formsets is while designing a data grid-like view to add elements row by row. This way, you do not need to create a dynamic form with an arbitrary number of rows. You just need to create a form for the row and create multiple rows using a formset_factory function.

Pattern – user-based forms

Problem: Forms need to be customized based on the logged-in user.

Solution: Pass the logged-in user as a keyword argument to the form's initializer.

Problem details

A form can be presented in different ways based on the user. Certain users might not need to fill all the fields, while certain others might need to add additional information. In some cases, you might need to run some checks on the user's eligibility, such as verifying whether they are members of a group, to determine how the form should be constructed.

Solution details

As you must have noticed, you can solve this using the solution given in the Dynamic form generation pattern. You just need to pass request.user as a keyword argument to the form. However, we can also use mixins from the django-braces package for a shorter and more reusable solution.

As in the previous example, we need to show an additional checkbox to the user. However, this will be shown only if the user is a member of the VIP group. Let's take a look at how PersonDetailsForm gets simplified with the form mixin UserKwargModelFormMixin fromdjango-braces:

from braces.forms import UserKwargModelFormMixin

class PersonDetailsForm(UserKwargModelFormMixin, forms.Form):

...

def __init__(self, *args, **kwargs):

super().__init__(*args, **kwargs)

# Are you a member of the VIP group?

if self.user.groups.filter(name="VIP").exists():

self.fields["first_class"] = forms.BooleanField(

label="Fly First Class?")

Notice how self.user was automatically made available by the mixin by popping the user keyword argument.

Corresponding to the form mixin, there is a view mixin called UserFormKwargsMixin, which needs to be added to the view, along with LoginRequiredMixin to ensure that only logged-in users can access this view:

class VIPCheckFormView(LoginRequiredMixin, UserFormKwargsMixin, generic.FormView):

form_class = PersonDetailsForm

...

Now, the user argument will be passed to the PersonDetailsForm form automatically.

Do check out other form mixins in django-braces such as FormValidMessageMixin, which are readymade solutions to common form-usage patterns.

Pattern – multiple form actions per view

Problem: Handling multiple form actions in a single view or page.

Solution: Forms can use separate views to handle form submissions or a single view can identify the form based on the Submit button's name.

Problem details

Django makes it relatively straightforward to combine multiple forms with the same action, for example, a single submit button. However, most web pages need to show several actions on the same page. For example, you might want the user to subscribe or unsubscribe from a newsletter in two distinct forms on the same page.

However, Django's FormView is designed to handle only one form per view scenario. Many other generic class-based views also share this assumption.

Solution details

There are two ways to handle multiple forms: a separate view and single view. Let's take a look at the first approach.

Separate views for separate actions

This is a fairly straightforward approach with each form specifying different views as their actions. For example, take the subscribe and unsubscribe forms. There can be two separate view classes to handle just the POST method from their respective forms.

Same view for separate actions

Perhaps you find the splitting views to handle forms to be unnecessary, or you find handling logically related forms in a common view to be more elegant. Either way, we can work around the limitations of generic class-based views to handle more than one form.

While using the same view class for multiple forms, the challenge is to identify which form issued the POST action. Here, we take advantage of the fact that the name and value of the Submit button is also submitted. If the Submit button is named uniquely across forms, then the form can be identified while processing.

Here, we define a subscribe form using crispy forms so that we can name the submit button as well:

class SubscribeForm(forms.Form):

email = forms.EmailField()

def __init__(self, *args, **kwargs):

super().__init__(*args, **kwargs)

self.helper = FormHelper(self)

self.helper.layout.append(Submit('subscribe_butn', 'Subscribe'))

The UnSubscribeForm unsubscribe form class is defined in exactly the same way (and hence is, omitted), except that its Submit button is named unsubscribe_butn.

Since FormView is designed for a single form, we will use a simpler class-based view say, TemplateView, as the base for our view. Let's take a look at the view definition and the get method:

from .forms import SubscribeForm, UnSubscribeForm

class NewsletterView(generic.TemplateView):

subcribe_form_class = SubscribeForm

unsubcribe_form_class = UnSubscribeForm

template_name = "newsletter.html"

def get(self, request, *args, **kwargs):

kwargs.setdefault("subscribe_form", self.subcribe_form_class())

kwargs.setdefault("unsubscribe_form", self.unsubcribe_form_class())

return super().get(request, *args, **kwargs)

The keyword arguments to a TemplateView class get conveniently inserted into the template context. We create instances of either form only if they don't already exist, with the help of the setdefault dictionary method. We will soon see why.

Next, we will take a look at the POST method, which handles submissions from either form:

def post(self, request, *args, **kwargs):

form_args = {

'data': self.request.POST,

'files': self.request.FILES,

}

if "subscribe_butn" in request.POST:

form = self.subcribe_form_class(**form_args)

if not form.is_valid():

return self.get(request,

subscribe_form=form)

return redirect("success_form1")

elif "unsubscribe_butn" in request.POST:

form = self.unsubcribe_form_class(**form_args)

if not form.is_valid():

return self.get(request,

unsubscribe_form=form)

return redirect("success_form2")

return super().get(request)

First, the form keyword arguments, such as data and files, are populated in a form_args dictionary. Next, the presence of the first form's Submit button is checked in request.POST. If the button's name is found, then the first form is instantiated.

If the form fails validation, then the response created by the GET method with the first form's instance is returned. In the same way, we look for the second forms submit button to check whether the second form was submitted.

Instances of the same form in the same view can be implemented in the same way with form prefixes. You can instantiate a form with a prefix argument such as SubscribeForm(prefix="offers"). Such an instance will prefix all its form fields with the given argument, effectively working like a form namespace.

Pattern – CRUD views

Problem: Writing boilerplate for CRUD interfaces to a model is repetitive.

Solution: Use generic class-based editing views.

Problem details

In most web applications, about 80 percent of the time is spent writing, creating, reading, updating, and deleting (CRUD) interfaces to a database. For instance, Twitter essentially involves creating and reading each other's tweets. Here, a tweet would be the database object that is being manipulated and stored.

Writing such interfaces from scratch can get tedious. This pattern can be easily managed if CRUD interfaces can be automatically created from the model class itself.

Solution details

Django simplifies the process of creating CRUD views with a set of four generic class-based views. They can be mapped to their corresponding operations as follows:

· CreateView: This view displays a blank form to create a new object

· DetailView: This view shows an object's details by reading from the database

· UpdateView: This view allows to update an object's details through a pre-populated form

· DeleteView: This view displays a confirmation page and, on approval, deletes the object

Let's take a look at a simple example. We have a model that contains important dates, which are of interest to everyone using our site. We need to build simple CRUD interfaces so that anyone can view and modify these dates. Let's take a look at the ImportantDatemodel itself:

# models.py

class ImportantDate(models.Model):

date = models.DateField()

desc = models.CharField(max_length=100)

def get_absolute_url(self):

return reverse('impdate_detail', args=[str(self.pk)])

The get_absolute_url() method is used by the CreateView and UpdateView classes to redirect after a successful object creation or update. It has been routed to the object's DetailView.

The CRUD views themselves are simple enough to be self-explanatory, as shown in the following code:

# views.py

from django.core.urlresolvers import reverse_lazyfrom . import forms

class ImpDateDetail(generic.DetailView):

model = models.ImportantDate

class ImpDateCreate(generic.CreateView):

model = models.ImportantDate

form_class = forms.ImportantDateForm

class ImpDateUpdate(generic.UpdateView):

model = models.ImportantDate

form_class = forms.ImportantDateForm

class ImpDateDelete(generic.DeleteView):

model = models.ImportantDate

success_url = reverse_lazy("impdate_list")

In these generic views, the model class is the only mandatory member to be mentioned. However, in the case of DeleteView, the success_url function needs to be mentioned as well. This is because after deletion get_absolute_url cannot be used anymore to find out where to redirect users.

Defining the form_class attribute is not mandatory. If it is omitted, a ModelForm method corresponding to the specified model will be created. However, we would like to create our own model form to take advantage of crispy forms, as shown in the following code:

# forms.py

from django import forms

from . import models

from crispy_forms.helper import FormHelper

from crispy_forms.layout import Submit

class ImportantDateForm(forms.ModelForm):

class Meta:

model = models.ImportantDate

fields = ["date", "desc"]

def __init__(self, *args, **kwargs):

super().__init__(*args, **kwargs)

self.helper = FormHelper(self)

self.helper.layout.append(Submit('save', 'Save'))

Thanks to crispy forms, we need very little HTML markup in our templates to build these CRUD forms.

Note

Note that explicitly mentioning the fields of a ModelForm method is a best practice and will soon become mandatory in future releases.

The template paths, by default, are based on the view class and the model names. For brevity, we omitted the template source here. Note that we can use the same form for CreateView and UpdateView.

Finally, we take a look at urls.py, where everything is wired up together:

url(r'^impdates/create/$',

pviews.ImpDateCreate.as_view(), name="impdate_create"),

url(r'^impdates/(?P<pk>\d+)/$',

pviews.ImpDateDetail.as_view(), name="impdate_detail"),

url(r'^impdates/(?P<pk>\d+)/update/$',

pviews.ImpDateUpdate.as_view(), name="impdate_update"),

url(r'^impdates/(?P<pk>\d+)/delete/$',

pviews.ImpDateDelete.as_view(), name="impdate_delete"),

Django generic views are a great way to get started with creating CRUD views for your models. With a few lines of code, you get well-tested model forms and views created for you, rather than doing the boring task yourself.

Summary

In this chapter, we looked at how web forms work and how they are abstracted using form classes in Django. We also looked at the various techniques and patterns to save time while working with forms.

In the next chapter, we will take a look at a systematic approach to work with a legacy Django codebase, and how we can enhance it to meet evolving client needs.