Helping with Ubuntu Development - Programming Linux - Ubuntu Unleashed 2017 Edition (2017)

Ubuntu Unleashed 2017 Edition (2017)

Part V: Programming Linux

Chapter 40. Helping with Ubuntu Development


In This Chapter

Image Introduction to Ubuntu Development

Image Setting Up Your Development System

Image Developing Apps and Scopes

Image Fixing Bugs and Packaging

Image Finding Bugs to Fix with Harvest

Image Masters of the Universe

Image References


Where the previous chapter talked about developing to scratch your own itch, this chapter focuses on how to become a part of a larger team, specifically the group of talented men and women who develop and package software for Ubuntu as a whole. Ubuntu is made up of thousands of different components. These are written in several different programming languages. Every component is available as source code, which is gathered into source packages. Source packages also include other content, specifically metadata describing things like copyright and licensing information, required dependencies, and build instructions for compiling or assembling the software for use. These source packages are used to build binary .deb packages, which are then included in the Ubuntu software repositories and made available for quick and easy installation by users. There are ways for anyone interested and willing to do a bit of study to use their skills to help.

Every time a bug is found, it only gets fixed if someone helps out and spends the time exploring the program code to find and fix it. Then, the fix only becomes available to the wider community if someone takes the time to build a new package that includes the fix and then uploads that package to an appropriate location in the Ubuntu software repositories.

This chapter has several parts. Each builds upon the previous, at least if you want to have a full understanding of the process. If only one part of the process interests you, say a later one like packaging, that is okay. You will benefit from reading the previous sections of the chapter, even if you only end up focusing your work on later steps in the process. You can participate in any part of the process, provided you are willing to learn how to do so. This chapter gets you started, but it is not the end of your required study. The process here is accurate, but there are more interesting and intricate details not included. Software development, especially on a big project, is not something to be done lightly or flippantly. For that reason, patches and bug fixes are only accepted from people willing to put in the effort necessary to produce detailed work with the highest quality. If a person hasn’t the time or isn’t interested in learning the steps properly, his software will probably also reflect this and is therefore not really wanted. The Ubuntu community is really nice, but they are also really busy. They will make time to mentor new developers and packagers who do their homework and make quality efforts, but it isn’t polite to waste their time with half-hearted attempts at getting the details correct.

Introduction to Ubuntu Development

Ubuntu development follows a six-month cycle. The process begins before and at a Ubuntu Developers’ Summit (UDS) with planning and discussion. Here, the goal is to unify the efforts of all those who will be involved to make the best use of limited resources. Everyone gets together via remote online participation, and in live discussions they make decisions about the priorities for the upcoming or the current release. The UDS happens once every three months, once at the start of a development cycle and once in the middle of the same cycle.

When a new release is created, is given a new code name. The release is referred to using this code name, such as “Wily Werewolf,” until its official release date, when it starts to be officially referred to using the release number, such as 16.04.

Some development begins by taking the newest versions of software from upstream software developers, either via Debian or directly from the specific application’s source. Some will have modifications, or patches, added to the code. All these are made into Ubuntu packages.

Other development begins internally, as there are some packages that have been written entirely by Ubuntu developers, including both those paid by Canonical and those who are volunteers in the wider Ubuntu community doing so for personal or community benefit or just for fun. These packages receive the new features slated for the release and are then made into new Ubuntu packages.

Testing occurs throughout the cycle, starting as soon as the first new packages begin to be uploaded. As a result of testing, bugs are often found. Bugs are problems with the software, such as errors that occur when the program is run. Bug reports are made using Launchpad, which is the issue tracker and code hosting solution created by Canonical for Ubuntu. Bugs are read by developers and community members and triaged into categories based on their importance. Critical bugs must be fixed before final release. Important bugs should be fixed as soon as possible. Other bugs might be irritating, but they are not vital to the operation of the program or the overall Ubuntu system, and, because of limited resources, they might be put off until someone has the time to deal with them. Every time bugs are fixed a new package must be made for the software so that it can be again uploaded into a repository for testing and review.

Alpha releases are made. Beta releases are made. Testing and bug fixing continue. Some software might be found to include new bugs that make the software unsuitable for use, so the previous version of the software is used. This is called “rolling back” to the previous version.

Eventually, the release day arrives. Everything works. The Ubuntu community rejoices and takes a week or two off. Then, it is time for the next UDS.

Setting Up Your Development System

Before you can begin, you must set up your system so that it works seamlessly with the rest of the Ubuntu community. There is a standard set of packages to install. There are some standard steps you must perform.

Install Basic Packages and Configure

The Ubuntu development community has created a convenient package that installs everything you need to get started. Install packaging-dev from the Ubuntu software repositories, which includes

Image gnupg, the GNU Privacy Guard, which includes the cryptographic tools you need to sign your packages when you upload them to Launchpad

Image pbuilder, which creates reproducible builds of a package in a clean, isolated environment

Image bzr-builddeb, which includes bzr; together these serve as your version control and package management systems

Image apt-file, which helps you find the binary package that includes a needed file

Set Up GPG

Generate your GPG key so that you can sign packages. Packages must be signed before they will be accepted for upload into the Ubuntu software repositories. This allows for tracking who is creating software and minimizes the risk of malicious software acts.

Click here to view code image

matthew@seymour:~$ gpg --gen-key

You will be asked what kind of key you want to generate. You can safely choose the default settings. Create a passphrase when asked. When this is done, a message like this one will be returned:

Click here to view code image

pub 2048R/38E0C789 2012-08-25

Key fingerprint = 6363 387F 7455 8929 E6E2 4619 4798 DFD9 38E0 C789

uid Matthew Helmke <matthew@matthewhelmke.com>

sub 2048R/BDE097FF 2012-08-2


Note

I created this GPG key specifically for this book and it is not used anywhere else. You won’t see it in use on Launchpad, for example, or for signing packages.


You need the key ID, which in this example is 38E0C789. Use this below. Upload your key ID to a keyserver, replacing keyID with your key ID.

matthew@seymour:~$ gpg --send-keys keyID

Create Your SSH Key

Next, create an SSH key that enables you to connect securely to another computer for file transfer. This is used when you’re uploading source files to Launchpad.

matthew@seymour:~$ ssh-keygen -t rsa

Choose the defaults and create a passphrase.

Set Up pbuilder

Finally, set up pbuilder so that you can build packages on your local machine. Replace release below with the name of the release for which you will develop; for example, the release being developed when this was written is quantal. You can do this for multiple releases at the same time, and not only for Ubuntu, but also for Debian releases like sid.

matthew@seymour:~$ pbuilder-dist release create

It takes some time for all the needed files to be downloaded and installed, but when they are, this step is complete.

Create a Launchpad Account

Launchpad was described in Chapter 39, “Opportunistic Development.” If you have already signed up for an account, you can skip this step. If not, open https://help.launchpad.net/YourAccount/NewAccount and sign up. You can develop for yourself without a Launchpad account, as in Chapter 39, “Opportunistic Development,” but to develop for Ubuntu, you must have an account.

Set Up Your Environment to Work with Launchpad

This involves several steps and builds upon the previous steps you just completed.

Upload your GPG Key to Launchpad

Find your GPG fingerprint. Replace youremail with the email address you used earlier to create your GPG key.

matthew@seymour:~$ gpg --fingerprint youremail

Something like this will be returned.

Click here to view code image

pub 2048R/38E0C789 2012-08-25

Key fingerprint = 6363 387F 7455 8929 E6E2 4619 4798 DFD9 38E0 C789

uid Matthew Helmke <matthew@matthewhelmke.com>

sub 2048R/BDE097FF 2013-08-25

Open https://launchpad.net/~/+editpgpkeys and copy the entire set of numbers and letter to the right of Key fingerprint = into the text box on the web page. Click Import Key on the web page. This fingerprint is used by Launchpad to check the Ubuntu key server for the key that you uploaded earlier.

You will be sent an encrypted email asking you to confirm the key import. You need to use an email client that supports OpenPGP encryption and enter the passphrase you created when you created the key to read this email. If you do not have an email reader that supports OpenPGP, you can also perform this step at the command line by typing gpg and copying/pasting the text of the email into the terminal window before hitting Enter.

Follow the directions in the email to complete this step.

Upload Your SSH Key to Launchpad

Open https://launchpad.net/~/+editsshkeys. Also open ~/.ssh/id_rsa.pub in a text editor. The contents of this file are the public part of your SSH key, so it is safe to share it with Launchpad. Copy the contents of the file and paste them into the Add an SSH Key text box on the web page that says. Now click Import Public Key.

Configure Bazaar

Bazaar is covered in Chapter 39, “Opportunistic Development.” Here, all you need to do is tell Bazaar you who are. This is a two-step process. The first step tells Bazaar which name and email address it should use when creating commit messages. Replace my name and email address with your information.

Click here to view code image

matthew@seymour:~$ bzr whoami "Matthew Helmke" <matthew@matthewhelmke.com>

This second step sets up Bazaar with your Launchpad ID so that the code you sign and upload is associated with your Launchpad account. Replace yourLaunchpadID with your Launchpad ID. If you can’t remember your ID, see where https://launchpad.net/~ redirects you to. The part that is automatically added after the ~ is your Launchpad ID.

Click here to view code image

matthew@seymour:~$ bzr launchpad-login yourLaunchpadID

Configure Your Local Bash Shell

The Ubuntu packaging tools that run at the command line on your development machine need to be configured with your information as well, in the same way that they are configured for Ubuntu’s parent, Debian. Open ~/.bashrc in your favorite text editor and add the following lines at the end, changing them to use your information instead of mine.

Click here to view code image

matthew@seymour:~$ gpg --fingerprint youremail

Reload the Bash shell configuration file.

Click here to view code image

matthew@seymour:~$ export DEBFULLNAME="Matthew Helmke"
matthew@seymour:~$ export DEBEMAIL="matthew@matthewhelmke.com"

If you are using a shell other than the default, Bash, then you need to configure your shell similarly.

Developing Apps and Scopes

If you want to develop apps specific to Ubuntu or scopes, which are an Ubuntu-specific way of interacting with content and services, that fit smoothly into the Ubuntu UI, this section is for you. In an attempt to draw developers in, Ubuntu has created a software development kit (SDK) called Ubuntu SDK. It is an integrated development environment (IDE) that contains a large number of tools and widgets to set up and create projects quickly and much more easily than doing so from scratch. You will find the details at https://developer.ubuntu.com/en/start/, from which you can download the Ubuntu SDK and find instructions and a quick start guide to help you begin. I haven’t seen this in wide use yet, as it is quite new, but it looks promising.

Fixing Bugs and Packaging

This section covers the process for fixing bugs and packaging your code. It does not cover the mechanics of reading program code and fixing it. Here, it is assumed that you know how to program in the language used in the software you are fixing.

From a high level, the process is easy to understand. Here are the steps:

1. Find a problem, a software bug.

2. Download the source code.

3. Fix the problem.

4. Test the fix on your local machine.

5. Commit the changes.

6. Request that your changes be merged into the main source.

Finding the problems to fix is something you learn with time and experience. The next section, “Finding Bugs to Fix with Harvest,” can help you. Other places to look, if you aren’t fixing an issue you discovered yourself, are in Ubuntu mailing lists, Launchpad bug reports, and in community gathering places like the Ubuntu Forums or AskUbuntu.

Before you do any work on a bug, do an extensive search to make sure it has not already been fixed and that someone else is not already working on the bug. You should obviously look in Launchpad in the section for the specific package. You can also check the upstream and/or Debian bug trackers for open and closed bugs and the upstream revision history or newer release(s). When you know of a bug you want to fix, download the source code from the Ubuntu software repositories. To start, find out which file contains the code you want to work on. The example uses an imaginary application, matthewsapp. In all the examples that follow, replace matthewsapp with the name of the program or application that you want to work on. We know that the matthewsapp binary is located at /usr/bin/matthewsapp. To find the Ubuntu package that contains matthewsapp, enter

Click here to view code image

matthew@seymour:~$ apt-file find /usr/bin/matthewsapp

This returns output something like this.

matthewsapp: /usr/bin/matthewsapp

You can also use apt-cache to learn the source package for a binary package. Sometimes, source packages have the same name as their resulting binary packages. Sometimes they do not. Here are two examples.

Click here to view code image

matthew@seymour:~$ apt-cache showsrc tomboy | grep ^Package:
Package: tomboy

matthew@seymour:~$ apt-cache showsrc python-vigra | grep ^Package:
Package: libvigraimpex

In the first example, the binary tomboy is in the tomboy package. In the second example, the binary python-vigra is in the librigraimpex package. When you know the package that contains the source code you need, get a copy of the source code itself. You do this in Ubuntu by branching the source package. Launchpad manages all the code for source packages and is the place from which you will download and create a local branch. You fix your bugs in the local branch and test them on your machine and then submit a merge proposal so that code from your branch can be examined and merged back into the main branch of the code, to then be used to build a new package of the code.

Create a local repository on your machine for the code. The following again uses matthewsapp, which you should replace with the name of the source package you need.

Click here to view code image

matthew@seymour:~$ bzr init-repo matthewsapp

Change to the newly created directory.

matthew@seymour:~$ cd matthewsapp

Create a new local branch. Name it something obvious. Many Ubuntu developers name the new target directory the same as the original, with .dev added at the end, like this.

Click here to view code image

matthew@seymour:~$ bzr branch ubuntu:matthewsapp matthewsapp.dev

After you have done this a few times with packages already in the Ubuntu repositories, you can read the official Ubuntu Packaging Guide to find out how to perform similar tasks with code from other Ubuntu releases, from Debian, or from an upstream tar file. For now, we concentrate on doing this the easy way to help you get started.

Create a patch that will include the fix for the bug.

Click here to view code image

matthew@seymour:~$ edit-patch 99-new-patch

This copies the packaging of the file to a new temporary directory. Edit the files with a text editor, or use a patch to do so in a temporary shell, like this.

Click here to view code image

matthew@seymour:~$ patch -pl < ../bugfix.patch

You can exit the temporary shell by entering exit or using Control+D.

Build a test package using your patch to test your changes. Replace release with the name of the release, such as quantal. Replace package and version with the name and number of the package.

Click here to view code image

matthew@seymour:~$ bzr builddeb -- -S -us -uc
matthew@seymour:~$ pbuilder-dist release build ../package_version.dsc

When the build completes, install the package from ~/pbuilder/release_result/ and test to see if the bug is fixed.

Click here to view code image

matthew@seymour:~$ sudo dpkg -i package_version.deb

Test it. Get it running. Try to break it. If you are convinced the bug is fixed in the software, only then is it time to move on. Feel free to repeat this cycle as often as necessary until it is really fixed.

When your changes are complete, create a new entry in the debian/changelog file.

matthew@seymour:~$ dch -i

Boilerplate text for the changelog entry that includes the first and last lines with placeholder text for the middle is provided for you. Use a text editor to edit the middle line(s) and include in your entry a specific bug fix tag that indicates which Launchpad bug you are fixing. Make sure you include where in the code you made the change(s), what you changed, and where the discussion of the change occurred.

The format for this is quite strict and looks like this:

Click here to view code image

matthewsapp (0.9.2-1ubuntu3) quantal; urgency=low

* debian/control: don't bacon the narwhals at midnight in line 35 as
discussed on launchpad (LP: #3263827)

-- Matthew Helmke <matthew@matthewhelmke.com> Sat, 25 Aug 2013 13:29:01 -0500

Commit the change locally.

matthew@seymour:~$ debcommit

Push the change to Launchpad. Replace items in italic with your information.

Click here to view code image

matthew@seymour:~$ bzr push lp:~yourLaunchpadID/ubuntu/release/package/branchname
matthew@seymour:~$ bzr lp-propose

The first command sends your code to Launchpad. The last command opens the Launchpad page of the merge proposal in your browser.

Finding Bugs to Fix with Harvest

For many, this is a great place to start because it helps you find ways to practice while also helping the community. Harvest is a to-do list for Ubuntu development. Some of the bugs listed in Harvest are things that have been fixed upstream but have not been incorporated yet into Ubuntu packaged versions. Others are small items that no one has yet had time to fix; these are called “bitesize” bugs in Ubuntu parlance.

The bitesize bugs are the perfect place for newcomers to help out. Some of these are as simple as fixing typos in displayed text that is somewhere in a program’s code. Fixing simple bugs at first lets you focus on learning the system. This frees you from wondering whether it is your programming skill or your packaging if you run into problems. The Resources section includes a link to the Harvest web page to help you get started.

Masters of the Universe

Fixing bugs is not the only reason people get involved with Ubuntu development and packaging. The Masters of the Universe—or, as it is better known, the MOTU—is a team of volunteers that finds useful software from upstream developers and packages it for easy installation from the Ubuntu repositories. This means that end users will find more software available without having to go and find source code packages and learn how to package and install themselves. You can find packages created by the MOTU in the Universe and Multiverse repositories. You have probably been using their work without even knowing it.

Everyone here starts as a contributor. To earn the MOTU title, you work with the MOTU team, who you can easily get in touch with and who will answer your questions to help you through the process. Some are very active, hands-on, and others are busier and might only respond when you ask questions. There are tons of opportunities. The References section includes a link to the MOTU web page.

References

Image https://uds.ubuntu.com—The main website for the Ubuntu Developers’ Summit.

Image http://developer.ubuntu.com/—The main website for Ubuntu development.

Image http://developer.ubuntu.com/apps—Useful resources for developing apps for Ubuntu to be run on multiple device platforms. Included here and available for download is the Ubuntu SDK, loaded with tools to assist in this task.

Image https://myapps.developer.ubuntu.com/dev/—An additional collection of resources for creating Ubuntu apps.

Image http://developer.ubuntu.com/resources/—A great list of useful resources for Ubuntu development.

Image http://packaging.ubuntu.com/—The main website for the Ubuntu Packaging Guide. This chapter would have been much more difficult to write without this guide, and the guide goes into far greater detail than this chapter. This packaging guide is also available directly from the Ubuntu software repositories in the ubuntu-packaging-guide package.

Image http://harvest.ubuntu.com/—The main website for Harvest.

Image https://wiki.ubuntu.com/UbuntuDevelopment/BugFixingInitiative—The main website for the Ubuntu community Bug Fixing Initiative.

Image https://wiki.ubuntu.com/MOTU—The main website for the Ubuntu community Masters of the Universe (MOTU) team.

Image http://daniel.holba.ch/blog/—Daniel Holbach works for Canonical. He organized the original MOTU team. His primary job today is making it easy and fun for people in the community to join in and participate with Ubuntu development. His blog is an excellent resource.