Maximizing Software Engineering Methods - C++ Software Engineering - Professional C++ (2014)

Professional C++ (2014)

Part VC++ Software Engineering

· CHAPTER 24: Maximizing Software Engineering Methods

· CHAPTER 25: Writing Efficient C++

· CHAPTER 26: Conquering Debugging

Chapter 24Maximizing Software Engineering Methods

WHAT’S IN THIS CHAPTER?

· What a software life cycle model is, with examples of the Stagewise Model, the Waterfall Model, the Spiral Model, and RUP

· What software engineering methodologies are, with examples of Agile, Scrum, XP, and Software Triage

· What Source Code Control means

Chapter 24 starts the last part of this book, which is about software engineering. This part describes software engineering methods, code efficiency, and software debugging.

When you first learned how to program, you were probably on your own schedule. You were free to do everything at the last minute if you wanted to, and you could radically change your design during implementation. When coding in the professional world, however, programmers rarely have such flexibility. Even the most liberal engineering managers admit that some amount of process is necessary. Knowing the software engineering process is as important these days as knowing how to code.

This chapter surveys various approaches to software engineering. It does not go into great depth on any one approach — there are plenty of excellent books on software engineering processes. The idea is to cover some different types of processes in broad strokes so you can compare and contrast them. I try not to advocate or discourage any particular methodology. Rather, I hope that by learning about the tradeoffs of several different approaches, you’ll be able to construct a process that works for you and the rest of your team. Whether you’re a contractor working alone on projects, or your team consists of hundreds of engineers on several continents, understanding different approaches to software development will help your job on a daily basis.

The last part of this chapter discusses Source Code Control solutions which make it easier to manage source code and keep track of its history. A Source Code Control solution is mandatory in every company to avoid a source code maintenance nightmare.

THE NEED FOR PROCESS

The history of software development is filled with tales of failed projects. From over-budget and poorly marketed consumer applications to grandiose mega-hyped operating systems, it seems that no area of software development is free from this trend.

Even when software successfully reaches users, bugs have become so commonplace that end users are forced to endure constant updates and patches. Sometimes the software does not accomplish the tasks it is supposed to or doesn’t work the way the user would expect. These issues all point to a common truism of software — writing software is hard.

One wonders why software engineering seems to differ from other forms of engineering in its frequency of failures. While cars do have their share of bugs, you rarely see them stop suddenly and demand a reboot due to a buffer overflow (though as more car components become software-driven, you just may.) Your TV may not be perfect, but you don’t have to upgrade to version 2.3 to get Channel 6 to work.

Is it the case that other engineering disciplines are just more advanced than software? Is a civil engineer able to construct a working bridge by drawing upon the long history of bridge building? Are chemical engineers able to build a compound successfully because most of the bugs were worked out in earlier generations?

NOTE Is software too new, or is it really a different type of discipline with inherent qualities contributing to the occurrence of bugs, unusable results, and doomed projects?

It certainly seems as if there’s something different about software. For one thing, technology changes rapidly in software, creating uncertainty in the software development process. Even if an earth-shattering breakthrough does not occur during your project, the pace of the industry leads to problems. Software often needs to be developed quickly because competition is fierce.

Software development schedules can also be unpredictable. Accurate scheduling is nearly impossible when a single gnarly bug can take days or even weeks to fix. Even when things seem to be going according to schedule, the widespread tendency of product definition changes (feature creep) can throw a wrench in the process.

Software is complex. There is no easy and accurate way to prove that a program is bug-free. Buggy or messy code can have an impact on software for years if it is maintained through several versions. Software systems are often so complex that when staff turnover occurs, nobody wants to get anywhere near the messy code that forgotten engineers have left behind. This leads to a cycle of endless patching, hacks, and workarounds.

Of course, standard business risks apply to software as well. Marketing pressures and miscommunication get in the way. Many programmers try to steer clear of corporate politics, but it’s not uncommon to have adversity between the development and product marketing groups.

All of these factors working against software engineering products indicate the need for some sort of process. Software projects are big, complicated, and fast-paced. To avoid failure, engineering groups need to adopt a system to control this unwieldy process.

SOFTWARE LIFE CYCLE MODELS

Complexity in software isn’t new. The need for a formalized process was recognized decades ago. Several approaches to modeling the software life cycle have attempted to bring some order to the chaos of software development by defining the software process in terms of steps from the initial idea to the final product. These models, refined over the years, guide much of software development today.

The Stagewise Model and Waterfall Model

A classic life cycle model for software is the Stagewise Model. This model is based on the idea that software can be built almost like following a recipe. There is a set of steps that, if followed correctly, will yield a mighty fine chocolate cake, or program as the case may be. Each stage must be completed before the next stage can begin, as shown in Figure 24-1.

image

FIGURE 24-1

The process starts with formal planning, including gathering an exhaustive list of requirements. This list defines feature completeness for the product. The more specific the requirements are, the more likely that the project will succeed. Next, the software is designed and fully specified. The design step, like the requirements step, needs to be as specific as possible to maximize the chance of success. All design decisions are made at this time, often including pseudocode and the definition of specific subsystems that will need to be written. Subsystem owners work out how their code will interact, and the team agrees on the specifics of the architecture. Implementation of the design occurs next. Because the design has been fully specified, the code needs to adhere strongly to the design or else the pieces won’t fit together. The final four stages are reserved for unit testing, subsystem testing, integration testing, and evaluation.

The main problem with the Stagewise Model is that, in practice, it is nearly impossible to complete one stage without at least exploring the next stage. A design cannot be set in stone without at least writing some code. Furthermore, what is the point of testing if the model doesn’t provide a way to go back to the coding phase?

A number of refinements to the Stagewise Model were formalized as the Waterfall Model in the early 1970s. The main advancement that the Waterfall Model brought was a notion of feedback between stages. While it still stresses a rigorous process of planning, designing, coding, and testing, successive stages can overlap in part. Figure 24-2 shows an example of the Waterfall Model, illustrating the feedback and overlapping refinements. Feedback allows lessons learned in one phase to result in changes to the previous phase. Overlap permits activity in two phases to occur simultaneously.

image

FIGURE 24-2

Various incarnations of the Waterfall Model have refined the process in different ways. For example, some plans include a “feasibility” step where experiments are performed before formal requirements are even gathered.

Benefits of the Waterfall Model

The value of the Waterfall Model lies in its simplicity. You, or your manager, may have followed this approach in past projects without formalizing it or recognizing it by name. The underlying assumption behind the Stagewise Model and Waterfall Model is that as long as each step is accomplished as completely and accurately as possible, subsequent steps will go smoothly. As long as all of the requirements are carefully specified in the first step, and all the design decisions and problems are hashed out in the second step, implementation in the third step should be a simple matter of translating the designs into code.

The simplicity of the Waterfall Model makes project plans based on this system organized and easy to manage. Every project is started the same way: by exhaustively listing all the features that are necessary. Managers using this approach can require that by the end of the design phase, for example, all engineers in charge of a subsystem submit their design as a formal design document or a functional subsystem specification. The benefit for the manager is that by having engineers specify requirements and design upfront, risks are, hopefully, minimized.

From the engineer’s point of view, the Waterfall Model forces resolution of major issues upfront. All engineers will need to understand their project and design their subsystem before writing a significant amount of code. Ideally, this means that code can be written once instead of hacked together or rewritten when the pieces don’t fit.

For small projects with very specific requirements, the Waterfall Model can work quite well. Particularly for consulting arrangements, it has the advantage of specifying specific metrics for success at the start of the project. Formalizing requirements helps the consultant to produce exactly what the client wants and forces the client to be specific about the goals for the project.

Drawbacks of the Waterfall Model

In many organizations, and almost all modern software engineering texts, the Waterfall Model has fallen out of favor. Critics disparage its fundamental premise that software development tasks happen in discrete linear steps. While the Waterfall Model allows for the overlapping of phases, it does not allow backward movement to a large degree. In many projects today, requirements come in throughout the development of the product. Often, a potential customer will request a feature that is necessary for the sale, or a competitor’s product will have a new feature that requires parity.

NOTE The upfront specification of all requirements makes the Waterfall Model unusable for many organizations because it is not dynamic enough.

Another drawback is that in an effort to minimize risk by making decisions as formally and early as possible, the Waterfall Model may actually be hiding risk. For example, a major design issue might be undiscovered, glossed over, forgotten, or purposely avoided in the design phase. By the time integration testing reveals the mismatch, it may be too late to save the project. A major design flaw has arisen but, according to the Waterfall Model, the product is one step away from shipping! A mistake anywhere in the waterfall process will likely lead to failure at the end of the process. Early detection is difficult and rare.

If you use the Waterfall Model, it is often necessary to make it more flexible by taking cues from other approaches.

The Spiral Model

The Spiral Model was proposed by Barry W. Boehm in 1986 as a risk-driven software development process. There are other spiral-like models. The one discussed in this section is part of a family of techniques known as iterative processes. The fundamental idea is that it’s okay if something goes wrong because you’ll fix it the next time around. A single spin through this Spiral Model is shown in Figure 24-3.

image

FIGURE 24-3

The phases of the Spiral Model are similar to the steps of the Waterfall Model. The discovery phase involves discovering requirements, determining objectives, determining alternatives (design alternatives, reuse, buying third party libraries, and so on), as well as determining any constraints. During the evaluation phase, implementation alternatives are evaluated, risks are analysed, and prototype options are considered. In the Spiral Model, particular attention is paid to evaluating and resolving risks in the evaluation phase. The tasks deemed most risky are the ones that are implemented in the current cycle of the spiral. The tasks in the development phase are determined by the risks identified in the evaluation phase. For example, if evaluation reveals a risky algorithm that may be impossible to implement, the main task for development in the current cycle will be modeling, building, and testing that algorithm. The fourth phase is reserved for analysis and planning. Based on the results of the current cycle, the plan for the subsequent cycle is formed.

Figure 24-4 shows an example of three cycles through the spiral in the development of an operating system. The first cycle yields a plan containing the major requirements for the product. The second cycle results in a prototype showing the user experience. The third cycle builds a component that is determined to be a high risk.

image

FIGURE 24-4

Benefits of the Spiral Model

The Spiral Model can be viewed as the application of an iterative approach to the best that the Waterfall Model has to offer. Figure 24-5 shows the Spiral Model as a waterfall process that has been modified to allow iteration. Hidden risks and a linear development path, the main drawbacks of the Waterfall Model, are resolved through iterative cycles.

image

FIGURE 24-5

Performing the riskiest tasks first is another benefit. By bringing risk to the forefront and acknowledging that new conditions can arise at any time, the Spiral Model avoids the hidden time bombs that can occur in the Waterfall Model. When unexpected problems arise, they can be dealt with by using the same four-stage approach that works for the rest of the process.

This iterative approach also allows for incorporating feedback from testers. For example, an early version of the product can be released for internal or even external evaluation. These testers could, for example, say that a certain feature is missing, or an existing feature is not working as expected. The Spiral Model has a built-in mechanism to react to such input.

Finally, by repeatedly analyzing after each cycle and building new designs, the practical difficulties with the design-then-implement approach are virtually eliminated. With each cycle, there is more knowledge of the system that can influence the design.

Drawbacks of the Spiral Model

The main drawback of the Spiral Model is that it can be difficult to scope each iteration small enough to gain real benefit. In a worst-case scenario, the Spiral Model can degenerate into the Waterfall Model because the iterations are too long. Unfortunately, the Spiral Model only models the software life cycle. It cannot prescribe a specific way to break down a project into single-cycle iterations because that division varies from project to project.

Other possible drawbacks are the overhead of repeating all four phases for each cycle and the difficulty of coordinating cycles. Logistically, it may be difficult to assemble all the group members for design discussions at the right time. If different teams are working on different parts of the product simultaneously, they are probably operating in parallel cycles, which can get out of synch. For example, the user interface group could be ready to start the discovery phase of the Window Manager cycle, but the core OS group could still be in the development phase of the memory subsystem.

The Rational Unified Process

The Rational Unified Process (RUP) is one of the best known refinements of the unified process. It is a disciplined and formal approach to managing the software development process. The most important characteristic of the RUP is that, unlike the Spiral Model or the Waterfall Model, RUP is more than just a theoretical process model. RUP is actually a software product, marketed and sold by Rational Software, a division of IBM. Treating the process as software is compelling for a number of reasons:

· The process itself can be updated and refined, just as software products periodically have updates.

· Rather than simply suggesting a development framework, RUP includes a set of software tools for working with that framework.

· As a product, RUP can be rolled out to the entire engineering team so that all members are using the exact same processes and tools.

· Like many software products, RUP can be customized to the needs of the users.

RUP as a Product

As a product, the RUP takes the form of a suite of software applications that guides developers through the software development process. The product also offers specific guidance for other Rational products, such as the Rational Rose visual modeling tool and the Rational ClearCase configuration management tool. Extensive groupware communication tools are included as part of the “marketplace of ideas” that allow developers to share knowledge.

One of the basic principles behind RUP is that each iteration on a development cycle should have a tangible result. During the Rational Unified Process, users will create numerous designs, requirement documents, reports, and plans. The RUP software provides visualization and development tools for the creation of these artifacts.

RUP as a Process

Defining an accurate model is the central principle of RUP. Models, according to RUP, help explain the complicated structures and relationships in the software development process. In RUP, models are often expressed in Unified Modeling Language (UML) format.

RUP defines each part of the process as an individual workflow. Workflows represent each step of a process in terms of who is responsible for it, what tasks are being performed, the artifacts or results of these tasks, and the sequence of events that drives the tasks. Almost everything about RUP is customizable, but several core process workflows are defined “out of the box” by RUP.

The core process workflows bear some resemblance to the stages of the Waterfall Model, but each one is iterative and more specific in definition. The business modeling workflow models business processes, usually with the goal of driving software requirements forward. The requirements workflow creates the requirements definition by analyzing the problems in the system and iterating on its assumptions. The analysis and design workflow deals with system architecture and subsystem design. The implementation workflow covers the modeling, coding, and integration of software subsystems. The test workflow models the planning, implementation, and evaluation of software quality tests. The deployment workflow is a high-level view on overall planning, releasing, supporting, and testing workflows. The configuration management workflow goes from new project conception to iteration and end-of-product scenarios. Finally, the environment workflow supports the engineering organization through the creation and maintenance of development tools.

RUP in Practice

RUP is aimed mainly at larger organizations and offers several advantages over the adoption of traditional life cycle models. Once the team has gotten over the learning curve to use the software, all members will be using a common platform for designing, communicating, and implementing their ideas. The process can be customized to the needs of the team, and each stage reveals a wealth of valuable artifacts that document each phase of the development.

A product like RUP can be too heavyweight for some organizations. Teams with diverse development environments or tight engineering budgets might not want to or be able to standardize on a software-based development system. The learning curve can also be a factor — new engineers that aren’t familiar with the process software will have to learn how to use it while getting up to speed on the product and the existing code base.

SOFTWARE ENGINEERING METHODOLOGIES

Software life cycle models provide a formal way of answering the question “What do we do next?” but are rarely (with the exception of formalized systems like RUP) able to contribute an answer to the logical follow-up question, “How do we do it?” To provide some answers to the “how” question, a number of methodologies have been developed that provide practical rules of thumb for professional software development. Books and articles on software methodologies abound, but a few innovations deserve particular attention:Agile, Scrum, Extreme Programming, and Software Triage.

Agile

To address the shortcomings of the Waterfall Model, the Agile Methodology was introduced in 2001 in the form of an Agile Manifesto. The entire manifesto reads as follows (http://agilemanifesto.org/):

MANIFESTO FOR AGILE SOFTWARE DEVELOPMENT

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

· Individuals and interactions over processes and tools

· Working software over comprehensive documentation

· Customer collaboration over contract negotiation

· Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

As can be understood from this manifesto, the term Agile is only a high-level description. Basically, it tells you to make the process flexible so that customers’ changes can easily be incorporated into the project during development.

Scrum

Agile is just a high-level foundation; it does not specify exactly how this should be implemented in real life. That’s where Scrum comes into play; it’s an Agile methodology with precise descriptions on how to use it on a daily base.

Just like the Spiral Model, Scrum is an iterative process. It is very popular as a means to manage software development projects. In Scrum, each iteration is called a sprint cycle. The sprint cycle is the central part of the Scrum process. The length of sprints, which should be decided at the beginning of the project, is typically between two and four weeks. At the end of each sprint, the goal is to have a version of the software available that is fully working and tested, and which represents a subset of the customers’ requirements. Scrum recognizes that customers will often change their minds during the development and thus allows the result of each sprint to be shipped to the customer. This gives customers the opportunity to see iterative versions of the software and allows them to give feedback to the development team about potential issues.

Roles

The Product Owner (PO) is the connection to the customer and to other people. The PO writes high-level user stories based on input from the customer, gives each user story a priority, and puts the stories in the Scrum product backlog. Actually, everyone on the team is allowed to write high-level user stories for the product backlog, but the Product Owner decides which user stories are kept and which are removed.

The Scrum Master (SM) is responsible for keeping the process running and can be part of the team, although not the team leader, because with Scrum the team leads itself. The SM is the contact person for the team so that the rest of the team members can concentrate on their tasks. The SM ensures that the Scrum process is followed correctly by the team; for example, by organizing the Daily Scrum meetings, which are discussed later. The Scrum Master and Product Owner should be two different persons.

The third and final role in the Scrum process is the Team itself. Teams, which develop the software, should be kept small, preferably fewer than 10 members.

The Process

The Scrum process enforces a daily meeting called the Daily Scrum or Standup. In this meeting, all team members stand together with the Scrum Master. According to the Scrum process, this meeting should start every day at exactly the same time and location, and should be no longer than 15 minutes. During this meeting, all team members get to answer three questions:

· What did you do since the last Daily Scrum?

· What are you planning to do after the current Daily Scrum?

· What problems are you facing to reach your goal?

Problems faced by team members should be noted by the Scrum Master who will try to solve them after the Daily Scrum meeting.

Before the start of each sprint cycle there is a Sprint Planning meeting in which team members must decide which product features they will implement in the new sprint. This is formalized in a sprint backlog. The features are selected from a product backlogcontaining prioritized user stories, which are high-level requirements of new features. User stories from the product backlog are broken down into smaller tasks with an effort estimation and are put in the sprint backlog. Once a sprint is in progress, the sprint backlog is frozen and cannot be changed during that sprint. The Sprint Planning should not take more than eight hours and is usually split into two parts: meeting with the Product Owner and the team to discuss the priority of product backlog items; and meeting with the team only to complete the sprint backlog.

In a Scrum team you will sometimes find a physical board with three columns: To Do, In Progress, and Done. Every task for the sprint is written on a small paper and stuck on the board in the correct column. Tasks are not assigned to people during a meeting; instead, every team member can go to the board, pick one of the To Do tasks, and move that paper to the In Progress column. When the team member is finished with that task, the paper is moved to the Done column. This method makes it easy for team members to quickly get an overview of the work that still needs to be done and what tasks are in progress or finished. Instead of a physical Scrum board, you can use a software solution to work with a virtual Scrum board.

Usually, a burn-down chart is also created every day that displays the days of the sprint on the horizontal axis and the remaining development hours on the vertical axis. This gives a quick overview of the progress made and can be used to determine whether all planned tasks can be completed during the sprint.

Once a sprint cycle is finished, there are two meetings: the Sprint Review and the Sprint Retrospective. The Sprint Review should be less than four hours and should discuss the results of the sprint cycle, including what tasks were completed and what tasks were not completed and why. This meeting also includes a Demo to demonstrate the implemented features. The Sprint Retrospective should have a three-hour maximum and should allow the team to think about how the last sprint cycle was executed. For example, the team can identify shortcomings in the process and adapt the process for the next sprint.

Benefits of Scrum

Scrum is resilient to unforeseen problems that come up during the development. When a problem pops up, it can be handled in one of the following sprints. The team is involved in every step of the project. It discusses user stories from the product backlog with the Product Owner and converts user stories into smaller tasks for inclusion in a sprint backlog. The team autonomously assigns work to its members with the aid of the Scrum tasks board. This board makes it easy to quickly see which team member is working on which task. And finally, the Daily Scrum meeting makes sure that everyone knows what is happening.

A huge benefit to the paying customer is a Demo that follows each sprint, which demonstrates the new iterative version of the project. The customer gets a quick sense of how the development is progressing and can make changes to requirements, which usually can be incorporated into a future sprint.

Drawbacks of Scrum

Some companies might find it difficult to accept that the team itself decides who is doing what. Tasks are not assigned to team members by a manager or a team leader. All members pick their own tasks from the Scrum tasks board.

The Scrum Master is a key person to make sure the team stays on track. It is very important that the SM trusts the team. Having too tight control over the team members will cause the Scrum process to fail.

A possible problem with Scrum is called feature creep. Scrum allows new user stories to be added to the product backlog during the development. The danger exists that project managers keep adding new features to the product backlog. This problem is best solved by deciding on a final release date, or the end date of the last sprint.

Extreme Programming (XP)

When a friend of mine arrived home from work a few years ago and told his wife that his company had adopted some of the principles of Extreme Programming, she joked, “I hope you wear a safety harness for that.” Despite the somewhat hokey name, Extreme Programming effectively bundles up the best of existing software development guidelines and new material into an increasingly popular methodology.

XP, popularized by Kent Beck in eXtreme Programming eXplained (Addison-Wesley, 1999), claims to take the best practices of good software development and turn them up a notch. For example, most programmers would agree that testing is a good thing. In XP, testing is deemed so good that you’re supposed to write the tests before you write the code.

XP in Theory

The Extreme Programming methodology is made up of 12 main guiding principles. These principles are manifested throughout all phases of the software development process and have a direct impact on the daily tasks of engineers.

Plan as You Go

In the Waterfall Model, planning happened once, at the beginning of the process. Under the Spiral Model, planning was the first phase of each iteration. In RUP, planning is an integral step in most of the workflows. Under XP, planning is more than just a step — it’s a never-ending task. XP teams start with a rough plan that captures the major points of the product being developed. Throughout the development process, the plan is refined and modified as necessary. The theory is that conditions are constantly changing and new information is obtained all the time.

Under XP, estimates for a given feature are always made by the person who will be implementing that particular feature. This helps to avoid situations where the implementer is forced to adhere to an unrealistic and artificial schedule. Initially, estimates are very rough, perhaps on the order of weeks for a feature. As the time horizon shortens, the estimates get more granular. Features are broken down into tasks taking no more than five days.

Build Small Releases

One of the theories of XP is that software projects grow risky and unwieldy when they try to accomplish too much at one time. Instead of massive software releases that involve core changes and several pages of release notes, XP advocates smaller releases with a timeframe closer to two months than 18 months. With such a short release cycle, only the most important features can make it into the product. This forces engineering and marketing to agree on what features are truly important.

Share a Common Metaphor

XP uses the term metaphor as other methodologies might use architecture. The idea is that all members of the team should share a common high-level view of the system. This isn’t necessarily the specifics of how objects will communicate, or the exact APIs that will be written. Rather, the metaphor is the mental model for the components of the system. Team members should use the metaphor to drive shared terminology when discussing the project.

Simplify Your Designs

A mantra frequently sung by XP-savvy engineers is “avoid speculative generality.” This goes against the natural inclinations of many programmers. If you are given the task of designing a file-based object store, you may start down the path of creating the be-all, end-all solution to all file-based storage problems. Your design might quickly evolve to cover multiple languages and any type of object. XP says you should lean towards the other end of the generality continuum. Instead of making the ideal object store that will win awards and be celebrated by your peers, design the simplest possible object store that gets the job done. You should understand the current requirements and write your code to those specifications to avoid overly complex code.

It may be hard to get used to simplicity in design. Depending on the type of work you do, your code may need to exist for years and be used by other parts of the code that you haven’t even dreamed of. As discussed in Chapter 6, the problem with building in functionality that may be useful in the future is that you don’t know what those hypothetical use cases are, and there is no way to craft a good design that is purely speculative. Instead, XP says you should build something that is useful today and leave open the opportunity to modify it later.

Test Constantly

According to eXtreme Programming eXplained, “Any program feature without an automated test simply doesn’t exist.” Extreme Programming is zealous about testing. Part of your responsibility as an XP engineer is to write the unit tests that accompany your code. A unit test is generally a small piece of code that makes sure that an individual piece of functionality works. For example, individual unit tests for a file-based object store may include testSaveObject, testLoadObject, and testDeleteObject.

XP takes unit testing one step further by suggesting that unit tests should be written before the actual code is written. Of course, the tests won’t pass because the code hasn’t been written yet. In theory, if your tests are thorough, you should know when your code is done because all the tests will complete successfully. I told you it was “extreme.”

Refactor When Necessary

Most programmers refactor their code from time to time. Refactoring is the process of redesigning existing working code to take into account new knowledge or alternate uses that have been discovered since the code was written. Refactoring is difficult to build into a traditional software engineering schedule because its results are not as tangible as implementing a new feature. Good managers, however, recognize its importance for long-term code maintainability.

The extreme way of refactoring is to recognize situations during development when refactoring is useful and to do the refactoring at that time. Instead of deciding at the start of a release which existing parts of the product need design work, XP programmers learn to recognize the signs of code that is ready to be refactored. While this practice will almost certainly result in unexpected and unscheduled tasks, restructuring the code when appropriate should make future development easier.

Code in Pairs

XP suggests that all production code should be written by two people working side-by-side simultaneously, called pair programming. Obviously, only one person can actually be in control of the keyboard. The other person takes a high-level approach, thinking about issues such as testing, necessary refactoring, and the overall model of the project.

As an example, if you are in charge of writing the user interface for a particular feature of your application, you might want to ask the original author of the feature to sit down with you. She can advise you about the correct use of the feature, warn you about any “gotchas” you should watch out for, and help oversee your efforts at a high level. Even if you can’t acquire the help of the original author, just grabbing another member of the team can help. The theory is that working in pairs builds shared knowledge, ensures proper design, and puts an informal system of checks and balances in place.

Share the Code

In many traditional development environments, code ownership is strongly defined and often enforced. A friend of mine worked previously in an environment where the manager explicitly forbade checking in changes to code written by any other member of the team. XP takes the extreme opposite approach by declaring that the code is collectively owned by everybody.

Collective ownership is practical for a number of reasons. From a management point of view, it is less detrimental when a single engineer leaves suddenly because there are others who understand that part of the code. From an engineer’s point of view, collective ownership builds a common view of how the system works. This helps design tasks and frees the individual programmer to make any change that will add value to the overall project.

One important note about collective ownership is that it is not necessary for every programmer to be familiar with every single line of code. It is more of a mindset that the project is a team effort, and there is no reason for any one person to hoard knowledge.

Integrate Continuously

All programmers are familiar with the dreaded chore of integrating code. This is the task when you discover that your view of the object store is a complete mismatch with the way it was actually written. When subsystems come together, problems are exposed. XP recognizes this phenomenon and advocates integrating code into the project frequently as it is being developed.

XP suggests a specific method for integration. Two programmers (the pair that developed the code) sit down at a designated “integration station” and merge the code in together. The code is not checked in until it passes 100 percent of the tests. By having a single station, conflicts are avoided and integration is clearly defined as a step that must occur before a check-in.

A similar approach can still work on an individual level. Engineers run tests individually or in pairs before checking code into the repository. A designated machine continually runs automated tests. When the automated tests fail, the team receives an email indicating the problem and listing the most recent check-ins.

Work Sane Hours

XP has a thing or two to say about the hours you’ve been putting in. The claim is that a well-rested programmer is a happy and productive programmer. XP advocates a work week of approximately 40 hours and warns against putting in overtime for more than two consecutive weeks.

Of course, different people need different amounts of rest. The main idea, though, is that if you sit down to write code without a clear head, you’re going to write poor code and abandon many of the XP principles.

Have a Customer on Site

Since an XP-savvy engineering group constantly refines its product plan and builds only what is currently necessary, having a customer contribute to the process is very valuable. Although it is not always possible to convince a customer to be physically present during development, the idea that there should be communication between engineering and the end user is clearly a valuable notion. In addition to assisting with the design of individual features, customers can help prioritize tasks by conveying their individual needs.

Share Common Coding Standards

Due to the collective ownership guideline and the practice of pair programming, coding in an extreme environment can be difficult if each engineer has her own naming and indenting conventions. XP doesn’t advocate any particular style, but supplies the guideline that if you can look at a piece of code and immediately identify the author, your group probably needs better definition of its coding standards.

For additional information on various approaches to coding style, see Chapter 3.

XP in Practice

XP purists claim that the 12 tenets of Extreme Programming are so intertwined that adopting some of them without others would largely ruin the methodology. For example, pair programming is vital to testing because if you can’t determine how to test a particular piece of code, your partner can help. Also, if you’re tired one day and decide to skip the testing, your partner will be there to evoke feelings of guilt.

Some of the XP guidelines, however, can prove difficult to implement. To some engineers, the idea of writing tests before code is too abstract. For those engineers, it may be sufficient to design the tests without actually writing them until there is code to test. Many of the XP principles are rigidly defined, but if you understand the theory behind it, you may be able to find ways to adapt the guidelines to the needs of your project.

The collaborative aspects of XP can be challenging as well. Pair programming has measurable benefits, but it may be difficult for a manager to rationalize having half as many people actually writing code each day. Some members of the team may even feel uncomfortable with such close collaboration, perhaps finding it difficult to type while others are watching. Pair programming also has obvious challenges if the team is physically spread out or if members tend to telecommute regularly.

For some organizations, Extreme Programming may be too radical. Large established companies with formal policies in place for engineering may be slow to adopt approaches like XP. However, even if your company is resistant to the implementation of XP, you can still improve your own productivity by understanding the theory behind it.

Software Triage

In the fatalistically-named book Death March (Prentice Hall, 1997) Edward Yourdon describes the frequent and scary condition of software that is behind schedule, short on staff, over budget, or poorly designed. Yourdon’s theory is that when software projects get into this state, even the best modern software development methodologies will no longer apply. As you have learned in this chapter, many approaches to software development are built around formalized documents or taking a user-centered approach to design. In a project that’s already in “death march” mode, there simply isn’t time for these approaches.

The idea behind software triage is that when a project is already in a bad state, resources are scarce. Time is scarce, engineers are scarce, and money may be scarce. The main mental obstacle that managers and developers need to overcome when a project is way behind schedule is that it will be impossible to satisfy the original requirements in the allotted time. The task then becomes organizing remaining functionality into “must-have,” “should-have,” and “nice-to-have” lists.

Software triage is a daunting and delicate process. It often requires the leadership of a seasoned veteran of “death march” projects to make the tough decisions. For the engineer, the most important point is that in certain conditions, it may be necessary to throw familiar processes out the window (along with some existing code, unfortunately) to finish a project on time.

BUILDING YOUR OWN PROCESS AND METHODOLOGY

It’s unlikely that any book or engineering theory will perfectly match the needs of your project or organization. I recommend that you learn from as many approaches as you can and design your own process. Combining concepts from different approaches may be easier than you think. For example, RUP optionally supports an XP-like approach. Here are some tips for building the software engineering process of your dreams.

Be Open to New Ideas

Some engineering techniques seem crazy at first or unlikely to work. Look at new innovations in software engineering methodologies as a way to refine your existing process. Try things out when you can. If XP sounds intriguing, but you’re not sure if it will work in your organization, see if you can work it in slowly, taking a few of the principles at a time or trying it out with a smaller pilot project.

Bring New Ideas to the Table

Most likely, your engineering team is made up of people from varying backgrounds. You may have people who are veterans of startups, long-time consultants, recent graduates, and PhDs on your team. You all have a different set of experiences and your own ideas of how a software project should be run. Sometimes the best processes turn out to be a combination of the way things are typically done in these very different environments.

Recognize What Works and What Doesn’t Work

At the end of a project (or better yet, during the project, like with the Sprint Retrospective of the Scrum methodology), get the team together to evaluate the process. Sometimes there’s a major problem that nobody notices until the whole team stops to think about it. Perhaps there’s a problem that everybody knows about but nobody has discussed.

Consider what isn’t working and see how those parts can be fixed. Some organizations require formal code reviews prior to any source code check-in. If code reviews are so long and boring that nobody does a good job, discuss code-reviewing techniques as a group. For example, consider reviewing only the interface files. Your group might even decide that changing an interface file after the interface is finalized with a review will be marked as a failure. This might help your team members to quickly learn how to write good interfaces.

Also consider what is going well and see how those parts can be extended. For example, if maintaining the feature tasks as a group-editable website is working, then maybe devote some time to making the website better.

Don’t Be a Renegade

Whether a process is mandated by your manager or custom-built by the team, it’s there for a reason. If your process involves writing formal design documents, make sure you write them. If you think that the process is broken or too complex, see if you can talk to your manager about it. Don’t just avoid the process — it will come back to haunt you.

SOURCE CODE CONTROL

Managing all source code is very important for any company, big or small, even for personal developers. In a company for example, it would be very impractical to store all the source code on the machines of the individual developers not managed by any Source Code Control software. This would result in a maintenance nightmare because not everyone will always have the latest code. All source code must be managed by Source Code Control software. There are three kinds of Source Code Control software solutions:

· Local: These solutions store all source code files and their history locally on your machine and are not really suitable for use in a team. These are solutions from the 70s and 80s and shouldn’t be used anymore. They are not discussed further.

· Client/Server: These solutions are split into a client component and a server component. For a personal developer, the client and server components can run on the same machine, but the separation makes it easy to move the server component to a dedicated physical server machine.

· Distributed: These solutions go one step further than the client/server model. There is no central place where everything is stored. Every developer has a copy of all the files including all the history. A peer-to-peer approach is used instead of a client/server approach. Code is synchronized between peers by exchanging patches.

The client/server solution consist of two parts. The first part is the server software, which is software running on the central server and which is responsible for keeping track of all source code files and their history. The second part is the client software. This client software should be installed on every developer’s machine and is responsible to communicate with the server software to get the latest version of a source file, get a previous version of a source file, commit local changes back to the server, rollback changes to a previous version and so on.

A distributed solution doesn’t use a central server. The client software uses peer-to-peer protocols to synchronize with other peers by exchanging patches. Common operations such as committing changes, rolling back changes and so on, are fast because no network access to a central server is involved. The disadvantage is that it requires more space on the client machine because it needs to store all the files, including the entire history.

Most Source Code Control systems have a special terminology, but unfortunately, not all systems use exactly the same terms. The following list explains a number of terms that are commonly used:

· Branch: The source code can be branched, which means that multiple versions can be developed side-by-side. For example, one branch could be created for every released version. On those branches, bug fixes could be implemented for those released versions, while new features are added to the main branch. Bug fixes created for released versions can also be merged back to the main branch.

· Checkout: This is the action of creating a local copy on the developer’s machine, either coming from a central server or from peers.

· Checkin, Commit, or Merge: A developer should make changes to the local copy of the source code. When everything works correctly on the local machine, the developer can checkin/commit/merge those local changes back to the central server, or exchange patches with peers.

· Conflict: When multiple developers make changes to the same source file, a conflict might occur during checkin of that source file. The Source Code Control software often will try to automatically resolve these conflicts. If that is not possible, the client software will ask the user to resolve any conflicts manually.

· Label or Tag: A label or tag can be attached to all files or to a specific commit at any given time. This makes it easy to jump back to the version of the source code at that time.

· Repository: The collection of all files managed by the Source Code Control software is called the repository. This also includes metadata about those files, such as checkin comments.

· Resolve: When checkin conflicts occur, the user will have to resolve them before checkin can continue.

· Revision or Version: A revision or version is a snapshot of the contents of a file at a specific point in time. Versions represent specific points to which the code can be reverted to, or compared against.

· Update or Sync: Updating or synchronizing means that the local copy on the developer’s machine is synchronized with a version on the central server or with peers. Note that this may require a merge, which may result in a conflict that needs to be resolved.

· Working Copy: The working copy is the local copy on the individual developer’s machine.

Several Source Code Control software solutions are available. Some of them are free, some of them are commercial. The following table lists a few available solutions:

FREE/OPEN-SOURCE

COMMERCIAL

LOCAL ONLY

SCCS, RCS

PVCS

CLIENT/SERVER

CVS, Subversion

IBM Rational ClearCase, Microsoft Team Foundation Server, Perforce

DISTRIBUTED

Git, Mercurial, Bazaar

TeamWare, BitKeeper, Plastic SCM

NOTE The preceding list is definitely not an exhaustive list. It’s just a small selection to give you an idea of what’s available.

This book does not recommend one or the other software solution. Most software companies these days have a Source Code Control solution already in place, which every developer needs to adopt. If this is not the case, the company should definitely invest some time for doing research on the available solutions, and to pick one that suits them. The bottom line is that it will be a maintenance nightmare without any Source Code Control solution in place. Even for your personal projects you might want to investigate the available solutions. If you find one that you like to work with, it will make your life easier. It will automatically keep track of different versions and a history of your changes. This makes it easy for you to change back to an older version if a change didn’t work out the way it was supposed to.

SUMMARY

This chapter introduced you to several models and methodologies for the software development process. There are certainly many other ways of building software, both formalized and informal. There probably isn’t a single correct method for developing software except the method that works for your team. The best way to find this method is to do your own research, learn what you can from various methods, talk to your peers about their experiences, and iterate on your process. Remember, the only metric that matters when examining a process methodology is how much it helps your team to write code.

The last part of this chapter briefly touched on the concept of Source Code Control. This should be an integral part of any software company, big or small, and can even be beneficial for personal projects at home. There are several Source Code Control software solutions available, so it is recommended that you try out a few and see which of them works for you.