Django Design Patterns and Best Practices (2015)

Chapter 2. Application Design

In this chapter, we will cover the following topics:

· Gathering requirements

· Creating a concept document

· HTML mockups

· How to divide a project into Apps

· Whether to write a new app or reuse an existing one

· Best practices before starting a project

· Why Python 3?

· Starting the SuperBook project

Many novice developers approach a new project by beginning to write code right away. More often than not it leads to incorrect assumptions, unused features and lost time. Spending some time with your client in understanding core requirements even in a project short on time can yield incredible results. Managing requirements is a key skill worth learning.

How to gather requirements

I saved several doomed projects by spending a few days with the client to carefully listen to their needs and set the right expectations. Armed with nothing but a pencil and paper (or their digital equivalents), the process is incredibly simple but effective. Here are some of the key points to remember while gathering requirements:

1. Talk directly to the application owners even if they are not technical savvy.

2. Make sure you listen to their needs fully and note them.

3. Don't use technical jargon such as "models". Keep it simple and use end-user friendly terms such as a "user profile".

4. Set the right expectations. If something is not technically feasible or difficult, make sure you tell them right away.

5. Sketch as much as possible. Humans are visual in nature. Websites more so. Use rough lines and stick figures. No need to be perfect.

6. Break down process flows such as user signup. Any multistep functionality needs to be drawn as boxes connected by arrows.

7. Finally, work through the features list in the form of user stories or in any easy way to understand the form.

8. Play an active role in prioritizing the features into high, medium, or low buckets.

9. Be very, very conservative in accepting new features.

10. Post-meeting, share your notes with everyone to avoid misinterpretations.

The first meeting will be long (perhaps a day-long workshop or couple of hour-long meetings). Later, when these meetings become frequent, you can trim them down to 30 minutes or one hour.

The output of all this would be a one page write-up and a couple of poorly drawn sketches.

In this book, we have taken upon ourselves the noble project of building a social network called SuperBook for superheroes. A simple sketch based off our discussions with a bunch of randomly selected superheroes is shown as follows:

A sketch of the SuperBook website in responsive design. Desktop (left) and smartphone (right) layouts are shown.

Are you a story teller?

So what is this one page write-up? It is a simple document that explains how it feels to use the site. In almost all the projects I have worked with, when someone new joins the team, they don't normally go through every bit of paperwork. He or she would be thrilled if they find a single-page document that quickly tells them what the site is meant to be.

You can call this document whatever you like—concept document, market requirements document, customer experience documentation, or even an Epic Fragile StoryLog™ (patent pending). It really doesn't matter.

The document should focus on the user experience rather than technical or implementation details. Make it short and interesting to read. In fact, Joel Spolsky's rule number one on documenting requirements is "Be Funny".

If possible, write about a typical user (persona in marketing speak), the problem they are facing, and how the web application solves it. Imagine how they would explain the experience to a friend. Try to capture this.

Here is a concept document for the SuperBook project:

Note

The SuperBook concept

The following interview was conducted after our website SuperBook was launched in the future. A 30 minute user test was conducted just prior to the interview.

Please introduce yourself.

My name is Aksel. I am a gray squirrel living in downtown New York. However, everyone calls me Acorn. My dad, T. Berry, a famous hip-hop star, used to call me that. I guess I was never good enough at singing to take up the family business.

Actually, in my early days, I was a bit of a kleptomaniac. I am allergic to nuts, you know. Other bros have it easy. They can just live off any park. I had to improvise—cafes, movie halls, amusement parks, and so on. I read labels very carefully too.

Ok, Acorn. Why do you think you were chosen for the user testing?

Probably, because I was featured in a NY Star special on lesser-known superheroes. I guess people find it amusing that a squirrel can use a MacBook (Interviewer: this interview was conducted over chat). Plus, I have the attention span of a squirrel.

Based on what you saw, what is your opinion about SuperBook?

I think it is a fantastic idea. I mean, people see superheroes all the time. However, nobody cares about them. Most are lonely and antisocial. SuperBook could change that.

What do you think is different about Superbook?

It is built from the ground up for people like us. I mean, there is no "Work and Education" nonsense when you want to use your secret identity. Though I don't have one, I can understand why one would.

Could you tell us briefly some of the features you noticed?

Sure, I think this is a pretty decent social network, where you can:

· Sign up with any user name (no more, "enter your real name", silliness)

· Fans can follow people without having to add them as "friends"

· Make posts, comment on them, and re-share them

· Send a private post to another user

Everything is easy. It doesn't take a superhuman effort to figure it out.

Thanks for your time, Acorn.

HTML mockups

In the early days of building web applications, tools such as Photoshop and Flash were used extensively to get pixel-perfect mockups. They are hardly recommended or used anymore.

Giving a native and consistent experience across smartphones, tablets, laptops, and other platforms is now considered more important than getting that pixel-perfect look. In fact, most web designers directly create layouts on HTML.

Creating an HTML mockup is a lot faster and easier than before. If your web designer is unavailable, developers can use a CSS framework such as Bootstrap or ZURB Foundation framework to create pretty decent mockups.

The goal of creating a mockup is to create a realistic preview of the website. It should not merely focus on details and polish to look closer to the final product compared to a sketch, but add interactivity as well. Make your static HTML come to life with working links and some simple JavaScript-driven interactivity.

A good mockup can give 80 percent of customer experience with less than 20 percent of the overall development effort.

Designing the application

When you have a fairly good idea of what you need to build, you can start to think about the implementation in Django. Once again, it is tempting to start coding away. However, when you spend a few minutes thinking about the design, you can find plenty of different ways to solve a design problem.

You can also start designing tests first, as advocated in Test-driven Design (TDD) methodology. We will see more of the TDD approach in the testing chapter.

Whichever approach you take, it is best to stop and think—"Which are the different ways in which I can implement this? What are the trade-offs? Which factors are more important in our context? Finally, which approach is the best?"

Experienced Django developers look at the overall project in different ways. Sticking to the DRY principle (or sometimes because they get lazy), they think—"Have I seen this functionality before? For instance, can this social login feature be implemented using a third-party package such as django-all-auth?"

If they have to write the app themselves, they start thinking of various design patterns in the hope of an elegant design. However, they first need to break down a project at the top level into apps.

Dividing a project into Apps

Django applications are called projects. A project is made up of several applications or apps. An app is a Python package that provides a set of features.

Ideally, each app must be reusable. You can create as many apps as you need. Never be afraid to add more apps or refactor the existing ones into multiple apps. A typical Django project contains 15-20 apps.

An important decision to make at this stage is whether to use a third-party Django app or build one from scratch. Third-party apps are ready-to-use apps, which are not built by you. Most packages are quick to install and set up. You can start using them in a few minutes.

On the other hand, writing your own app often means designing and implementing the models, views, test cases, and so on yourself. Django will make no distinction between apps of either kind.

Reuse or roll-your-own?

One of Django's biggest strengths is the huge ecosystem of third-party apps. At the time of writing, djangopackages.com lists more than 2,600 packages. You might find that your company or personal library has even more. Once your project is broken into apps and you know which kind of apps you need, you will need to take a call for each app—whether to write or reuse an existing one.

It might sound easier to install and use a readily available app. However, it not as simple as it sounds. Let's take a look at some third-party authentication apps for our project, and list the reasons why we didn't use them for SuperBook at the time of writing:

· Over-engineered for our needs: We felt that python-social-auth with support for any social login was unnecessary

· Too specific: Using django-facebook would mean tying our authentication to that provided by a specific website

· Python dependencies: One of the requirements of django-allauth is python-openid, which is not actively maintained or unapproved

· Non-Python dependencies: Some packages might have non-Python dependencies, such as Redis or Node.js, which have deployment overheads

· Not reusable: Many of our own apps were not used because they were not very easy to reuse or were not written to be reusable

None of these packages are bad. They just don't meet our needs for now. They might be useful for a different project. In our case, the built-in Django auth app was good enough.

On the other hand, you might prefer to use a third-party app for some of the following reasons:

· Too hard to get right: Do your model's instances need to form a tree? Use django-mptt for a database-efficient implementation

· Best or recommended app for the job: This changes over time but packages such as django-redis are the most recommended for their use case

· Missing batteries: Many feel that packages such as django-model-utils and django-extensions should have been part of the framework

· Minimal dependencies: This is always good in my book

So, should you reuse apps and save time or write a new custom app? I would recommend that you try a third-party app in a sandbox. If you are an intermediate Django developer, then the next section will tell you how to try packages in a sandbox.

My app sandbox

From time to time, you will come across several blog posts listing the "must-have Django packages". However, the best way to decide whether a package is appropriate for your project is Prototyping.

Even if you have created a Python virtual environment for development, trying all these packages and later discarding them can litter your environment. So, I usually end up creating a separate virtual environment named "sandbox" purely for trying such apps. Then, I build a small project to understand how easy it is to use.

Later, if I am happy with my test drive of the app, I create a branch in my project using a version control tool such as Git to integrate the app. Then, I continue with coding and running tests in the branch until the necessary features are added. Finally, this branch will be reviewed and merged back to the mainline (sometimes called master) branch.

Which packages made it?

To illustrate the process, our SuperBook project can be roughly broken down into the following apps (not the complete list):

· Authentication (built-in django.auth): This app handles user signups, login, and logout

· Accounts (custom): This app provides additional user profile information

· Posts (custom): This app provides posts and comments functionality

· Pows (custom): This app tracks how many "pows" (upvotes or likes) any item gets

· Bootstrap forms (crispy-forms): This app handles the form layout and styling

Here, an app has been marked to be built from scratch (tagged "custom") or the third-party Django app that we would be using. As the project progresses, these choices might change. However, this is good enough for a start.

Before starting the project

While preparing a development environment, make sure that you have the following in place:

· A fresh Python virtual environment: Python 3 includes the venv module or you can install virtualenv. Both of them prevent polluting your global Python library.

· Version control: Always use a version control tool such as Git or Mercurial. They are life savers. You can also make changes much more confidently and fearlessly.

· Choose a project template: Django's default project template is not the only option. Based on your needs try others such as twoscoops (https://github.com/twoscoops/django-twoscoops-project) or edge (https://github.com/arocks/edge).

· Deployment pipeline: I usually worry about this a bit later, but having an easy deployment process helps to show early progress. I prefer Fabric or Ansible.

SuperBook – your mission, should you choose to accept it

This book believes in a practical and pragmatic approach of demonstrating Django design patterns and the best practices through examples. For consistency, all our examples will be about building a social network project called SuperBook.

SuperBook focusses exclusively on the niche and often neglected market segment of people with exceptional super powers. You are one of the developers in a team comprised of other developers, web designers, a marketing manager, and a project manager.

The project will be built in the latest version of Python (Version 3.4) and Django (Version 1.7) at the time of writing. Since the choice of Python 3 can be a contentious topic, it deserves a fuller explanation.

Why Python 3?

While the development of Python 3 started in 2006, its first release, Python 3.0, was released on December 3, 2008. The main reasons for a backward incompatible version were—switching to Unicode for all strings, increased use of iterators, cleanup of deprecated features such as old-style classes, and some new syntactic additions such as the nonlocal statement.

The reaction to Python 3 in the Django community was rather mixed. Even though the language changes between Version 2 and 3 were small (and over time, reduced), porting the entire Django codebase was a significant migration effort.

On February 13, Django 1.5 became the first version to support Python 3. Developers have clarified that, in future, Django will be written for Python 3 with an aim to be backward compatible to Python 2.

For this book, Python 3 was ideal for the following reasons:

· Better syntax: This fixes a lot of ugly syntaxes, such as izip, xrange, and __unicode__, with the cleaner and more straightforward zip, range, and __str__.

· Sufficient third-party support: Of the top 200 third-party libraries, more than 80 percent have Python 3 support.

· No legacy code: We are creating a new project, rather than dealing with legacy code that needs to support an older version.

· Default in modern platforms: This is already the default Python interpreter in Arch Linux. Ubuntu and Fedora plan to complete the switch in a future release.

· It is easy: From a Django development point of view, there are very few changes, and they can all be learnt in a few minutes.

The last point is important. Even if you are using Python 2, this book will serve you fine. Read Appendix A to understand the changes. You will need to make only minimal adjustments to backport the example code.

Starting the project

This section has the installation instructions for the SuperBook project, which contains all the example code used in this book. Do check the project's README file for the latest installation notes. It is recommended that you create a fresh directory, superbook, first that will contain the virtual environment and the project source code.

Ideally, every Django project should be in its own separate virtual environment. This makes it easy to install, update, and delete packages without affecting other applications. In Python 3.4, using the built-in venv module is recommended since it also installs pip by default:

$ python3 -m venv sbenv

$ source sbenv/bin/activate

$ export PATH="`pwd`/sbenv/local/bin:$PATH"

These commands should work in most Unix-based operating systems. For installation instructions on other operating systems or detailed steps please refer to the README file at the Github repository: https://github.com/DjangoPatternsBook/superbook. In the first line, we are invoking the Python 3.4 executable as python3; do confirm if this is correct for your operating system and distribution.

The last export command might not be required in some cases. If running pip freeze lists your system packages rather than being empty, then you will need to enter this line.

Tip

Before starting your Django project, create a fresh virtual environment.

Next, clone the example project from GitHub and install the dependencies:

$ git clone https://github.com/DjangoPatternsBook/superbook.git

$ cd superbook

$ pip install -r requirements.txt

If you would like to take a look at the finished SuperBook website, just run migrate and start the test server:

$ cd final

$ python manage.py migrate

$ python manage.py createsuperuser

$ python manage.py runserver

In Django 1.7, the migrate command has superseded the syncdb command. We also need to explicitly invoke the createsuperuser command to create a super user so that we can access the admin.

You can navigate to http://127.0.0.1:8000 or the URL indicated in your terminal and feel free to play around with the site.

Summary

Beginners often underestimate the importance of a good requirements-gathering process. At the same time, it is important not to get bogged down with the details, because programming is inherently an exploratory process. The most successful projects spend the right amount of time preparing and planning before development so that it yields the maximum benefits.

We discussed many aspects of designing an application, such as creating interactive mockups or dividing it into reusable components called apps. We also discussed the steps to set up SuperBook, our example project.

In the next chapter, we will take a look at each component of Django in detail and learn the design patterns and best practices around them.