Optimization in Practice with MATLAB for Engineering Students and Professionals (2015)

---

5.Introducing Linear and Nonlinear Programming

---

3

---

Welcome to the Fascinating World of Optimization

3.1 Overview

Welcome to the fascinating world of optimization. Indeed, you will find optimization to be a powerful addition to your education or to your toolkit in the workplace. This brief initial chapter provides you with a clear perspective as to how the book will play a key role in your understanding of optimization. As discussed in the Preface, this book takes a squarely practical perspective. As such, your learning will involve practicing optimization using many problems. By the end of your study of this book, you should expect to be able to work with others to optimize practically any design or system and improve its performance. If the design you would like to optimize is in your technical area of expertise, you might be able to do all of the work yourself. If the design or system you would like to optimize involves modeling issues with which you are not familiar, you will simply need to collaborate with someone (or a team) who is able to provide you with the computational performance models. In this latter case, as it often is in practice, you will provide your knowledge of optimization, and someone else may model the system. In other words, one person might do the structural analysis, another might do the financial analysis, and you might use your knowledge of this book to optimize the combined system to make it competitive in the market place.

3.2 What Is Optimization? What Is Its Relation to Analysis and Design?

What is optimization? Optimization can be defined in different ways. Some define it as “the art of making things the best.” Interestingly, many people do not like that definition as it may not be reasonable, or even possible, to do something in the very best possible way. In practice, doing something as well as possible within practical constraints is very desirable. Designing a product and doing all we can to increase profit as much as is practically possible is also very desirable. These comments may begin to give you an idea of what optimization is attempting to do in practice. It provides us with the means to make things happen in the best possible practical way.

Now, we may ask how this is different from what all engineers, all financial analysts, and most other professionals try to do? Well, the answer to this question is important indeed. Without optimization, we accomplish this by using experience, intuition, and just plain luck! With optimization, we do it in a systematic way, where we use the power of a computer to examine more possibilities than any human being could ever attempt. Furthermore, theoptimization approach makes sure that the search is done as efficiently as possible.

Since you have made the decision to educate yourself in the art and science of optimization (see Refs. [1, 2, 3, 4, 5]), it is important to keep in mind that you are making a reasonable investment that is intended to bring tangiblebenefits. Figure 3.1 provides an interesting way to clarify this point. Some books that take a somewhat different but useful approach include (Refs. [6, 7, 8]).

Figure 3.1. Motivation for Optimization: Creating New Possibilities

In other settings, you may be exposed to a very mathematical view of optimization, which, in my opinion, sometimes obscures the basic beauty, simplicity, and practical power of optimization. To avoid that pitfall, let us immediately start getting an operational idea of the optimization process. We can do this without any equation or complex terminology. Let us simply examine Fig. 3.2. This figure illustrates the design process using (i) traditional design approaches, and (ii) using optimal design approaches.

Figure 3.2. Traditional vs. Optimal Design Process

•Box A displays the input, which includes two basic issues. The first defines our dream design: the desired performance levels (e.g., maximize profit, minimize mass) and any constraints (e.g., deformation less than 5 mm, cost less than $7). The second item provides an initial design that we can obtain through any conventional means. That design is simply a starting point from which improvements can be made.

•Box B illustrates the analysis phase. Analysis is essentially what you do in almost all of your classes. Analysis usually tells you what the output result is for a given set of input conditions, whether you have a mechanical, electrical, or financial system under consideration. This is not design and this is not optimization; however, we need to be able to do analysis in order to design or optimize.

•Box C explains how the optimization process cycle begins. Using the initial design that is provided, it is very unlikely that it will satisfy all the constraints and maximize performance. Most likely, it will need to be improved by modifying it as intelligently as possible. This is where the power of optimization comes into play. This takes us to the next box.

•Box D is where the design is revised and improved in a very systematic way. This is not a trivial process. However, fortunately, the person who is mainly interested in applying optimization will not need to focus on most of what is in that box. In Parts I, II, and III of the book, we will learn what is needed to competently apply optimization, while Parts IV and V address more advanced topics as well as the details of Box D. In the process of optimization,Box D continually modifies the design with the expectation that the design will improve. Each modification is submitted to the analysis module. After the analysis is performed, the design performance is again evaluated to seewhether it meets our objectives. If it does, we are done. If it does not, we go through the loop one more time. Please note that the actual process is more complex than this simplified explanation.

•Box E illustrates the human element involved in making the required improvements in a traditional way. In other words, Box E replaces Box D. The question that person is trying to answer is: How can I change the design to make it perform better? The options are: ask a friend, use intuition, use experience, or just hope to be lucky! As you might guess, as we move into this not-so-new world of computers and of extreme competition, this is not necessarily the best way to proceed. Please note that human design decision-making is critically needed, but not at the level of Box D.

Observation: A few observations regarding the conventional vs. the optimal approaches are in order. Using the traditional approach, we can only perform a small number of improvement loops. Using computational optimization approaches, we can move through the improvement loop more effectively and efficiently. In addition, the improvements are based on rigorous thinking using the power of optimization, while the traditional approach is usually ad hoc, based on intuition and experience that can fail us, particularly in complex and innovative designs.

Generic Car Optimization: Today, the car industry makes powerful use of optimization. Optimization takes place at many levels, from individual parts to crash-performance to save lives. Other applications range from fuel efficiency to pollution minimization or noise reduction. In Fig. 3.3, we see the possible impact of optimization in optimizing for drag reduction and various other important performance attributes. The examination of Figs. 3.3(a)and 3.3(b) yields endless questions regarding the process, the effectiveness and the potential impact of optimization. Many of these questions will be addressed in this book, and many will be discovered and addressed during the course of your career regarding real-world designs and systems.

Figure 3.3. Generic Car Optimization

Role of Optimization in the Revolutionary Transformation of the Airplane: Over two centuries ago, Sir George Cayley was reported to have advanced the concept of the modern airplane. A century later, in 1903, the Wright brothers are credited for the first sustained flight with a powered, controlled airplane. The past few decades have seen revolutionary transformations of the airplane, while its basic shape has not drastically changed. In Fig. 3.4, weobserve how different attributes of interest can result in different layouts, sizes, fuel consumptions, cruise speeds for a business jet or a passenger jet. If you should gather a few experienced engineers in a room, you would have quite an engaging discussion about which of these transformation were or could have been influenced by computational optimization. The past three decades have experienced rapidly growing application of computational optimization in many critical aspects of airplane design, and the future is expected to bring us an acceleration of this trend, in part due to the exponential growth of computing power.

Figure 3.4. Evolutionary and Revolutionary Transformation of the Airplane

3.3 Why Should Junior and Senior College Students Study Optimization?

The next question that may come to mind is: as a Junior or Senior, why should I be studying optimization? Well, hopefully, what you have learned thus far should at least partially answer your question. In truth, this is a question that you will be able to answer in your own way after you will have had the opportunity to optimize designs yourself, and feel comfortable and confident that you could not have possibly gotten these optimal designs any other practical way.

In addition to these observations, you might also have the opportunity to be able to use optimization while you are still an undergraduate. There is no reason to wait. Applying optimization to your senior capstone design, forexample, is one significant possibility that comes to mind.

3.4 Why Should Graduate Students Study Optimization?

To address this question, many of my previous comments to the undergraduates also apply. However, in your case, there are significantly more opportunities. If you are doing research, as you most likely are, you should be able to use optimization to find better ways to proceed with your experiments or with your designs. Chances are that you would like to obtain a desired output from a system that you have modeled or are analyzing. This is one case where optimization should provide you with a way to obtain an optimal output.

3.5 Why Should Industry Practitioners Study Optimization?

As an industry practitioner, you will be able to use optimization to help you in any number of projects. And when you do, you may find optimal designs that others cannot realistically obtain without optimization. It is also important to keep in mind that when I use the word design, I mean any system for which you use computation to evaluate its performance. That involves a majority of the systems that we deal with in an engineering and/or financial environment. In addition, an increasing number of software packages now include an optimization module that allows its users to apply optimization. With solid practical knowledge of optimization, you are in a strong position to apply the optimization portion of these software packages effectively.

3.6 Why Use this Book, and What Should I Expect from It?

There are several popular books on optimization in the market, which focus on either theories or applications. This book, however, provides the following key advantages, making it distinct.

1.This book serves as a practical guide to the application of optimization. This book uses a special way to teach optimization that requires sufficient practice. Like other books on optimization, this book also provides a mathematical background of optimization. In addition, it provides a large number of examples to help readers understand optimization. This book helps readers to quickly learn how to solve practical optimization problems. Readers can follow the examples to learn how to solve real engineering design problems. One unique aspect of this book is that numerical and modeling issues involved in practical optimization are also discussed.

2.This book covers a broad range of knowledge on the topic of optimization. Various aspects of optimization are covered in this book, such as linear optimization, nonlinear optimization, multiobjective optimization, global optimization, and discrete optimization. This book also teaches students how to solve contemporary complex engineering problems. The problems after each chapter are a good exercise to help readers master practical optimization problems.

3.This book is suitable for different classes of readers including undergraduate students, graduate students, and industry practitioners. Starting with the fundamentals of MATLAB and optimization, this book moves on to exploreadvanced and more recent topics in optimization.

3.7 How this Book Is Organized

This book contains five parts comprising 19 chapters. They are organized as follows:

Part I. Helpful Preliminaries: This part includes Chapters 1 and 2. These chapters provide an introduction to MATLAB, and the necessary mathematical preliminaries for optimization. This part includes the following chapters:

•MATLAB as a Computation Tool

•Mathematical Preliminaries

Part II. Using Optimization—The Road Map: This part includes Chapters 3 to 5. These chapters illustrate the benefits of optimization, the modeling of optimization problems, and the classification of optimization problems. This part includes the following chapters:

•Welcome to the Fascinating World of Optimization

•Analysis, Design, Optimization, and Modeling

•Introducing Linear and Nonlinear Programming

Part III. Using Optimization—Practical Essentials: This part includes Chapters 6 to 10. These chapters examine how to solve multiobjective optimization, global optimization, and discrete optimization problems. Important practical numerical issues of optimization are addressed. The links between optimization theories and applications are studied. Practical optimization examples are provided. This part includes the following chapters.

•Multiobjective Optimization

•Numerical Essentials

•Global Optimization Basics

•Discrete Optimization Basics

•Practicing Optimization - Larger Examples

Part IV. Going Deeper: Inside the Codes and Theoretical Aspects: This part includes Chapters 11 to 13. Theorems and optimization algorithms for linear and nonlinear optimization are presented in this part. This part includes the following chapters:

•Linear Programming

•Nonlinear Programming with No Constraints

•Nonlinear Programming with Constraints

Part V. More Advanced Topics in Optimization: This part includes Chapters 14 to 19. Advanced topics, including discrete optimization, design optimization under uncertainty, Pareto frontier generation, physical programming, and evolutionary algorithms, are investigated in this part. This part includes the following chapters:

•Discrete Optimization

•Modeling Complex Systems: Surrogate Modeling and Design Space Reduction

•Design Optimization Under Uncertainty

•Methods for Pareto Frontier Generation/Representation

•Physical Programming for Multiobjective Optimization

•Evolutionary Algorithms

3.8 How to Read and Use this Book

This book covers a broad range of knowledge of optimization. Undergraduate students, graduate students, and industry practitioners can use this book in different ways for different purposes.

Undergraduate students in their Junior or Senior year, who have learned calculus and linear algebra, have an opportunity to explore their application in the study of optimization. Part I of this book provides a brief review of the mathematical preliminaries for learning optimization. MATLAB serves as the programming language and computational tool for solving the optimization problems in this book. For those uninitiated in the use of MATLAB, it is necessary to carefully study Chapter 1. Parts II and III present the basics for modeling and solving optimization problems using MATLAB. The knowledge in these two parts is sufficient for those students who wish to applyoptimization to practical engineering design problems.

To guide graduate students who would like to learn the theoretical aspects of the optimization algorithms behind the MATLAB functions, Part IV discusses the theorems and algorithms for linear and nonlinear programming.Advanced topics, including discrete optimization, optimization under uncertainty, Pareto frontier generation, physical programming, and evolutionary algorithms, are presented in Part V. These topics are important for research and development in engineering design. Doctorial students who conduct research on optimization are expected to learn these advanced topics.

This book teaches optimization with a practical approach, which favorably distinguishes it from other books on optimization. Industry practitioners, who are most concerned with how to apply optimization to practical problems, need only cover the chapters in Parts II and III. These chapters cover how to solve practical optimization problems, as well as the numerical and modeling issues encountered thereof.

3.9 Summary

This chapter provided a philosophical introduction to design optimization, its place in the world of science and engineering, and the importance of learning optimization to students, scholars, and industry practitioners. An illustration of how a generalized optimization process works is also provided. This chapter essentially serves as a gateway to the theory and practice of optimization taught in this book. To serve in that role, it provided an overview of what to expect from the upcoming chapters in this book, and how the overall content of this book is structured toward teaching optimization to undergraduate students, graduate students, and industry practitioners.

3.10 Problems

3.1Describe a design problem of your interest (in 400-500 words) where optimization can be applied to enrich the design. It could be a problem you are currently working on (e.g., Capstone design) or a problem you plan to workon. Clearly define the scope of applying optimization and the expected improvement in that context. Doctoral students are strongly recommended to identify a problem that is closely related to their principal area of research.

3.2Conceive a modern real life product (e.g., smartphone, solar PV, or PHEV) where optimization can be used to further improve its design. Describe the scope of applying optimization in that context (in 200-400 words). Specifically state (i) what objectives will need to be maximized and minimized, (ii) what features of the product could serve as design variables, and (iii) what practical constraints should be taken into consideration during optimization.

3.3Compare and contrast (i) quantitative optimization-based design, and (ii) experience-based design.

3.4From your own standpoint, explain the role of modern day computing (from portable ultrabooks to number-crunching supercomputers) in the application of optimization to real life design.

3.5Expand on the discussion associated with Fig. 3.4 where the evolutionary transformation of the airplane is briefly discussed. Let your discussion be guided by your initial understanding of the role of computational optimization in modern engineering and related fields, as well as finance and other quantitative areas. There is not specific good answer to this question, while some may be more thoughtful and imaginative than others. A bit of research and cursory exploration of this book may be helpful. Limit your discussion to no more than one to two pages.

BIBLIOGRAPHY OF CHAPTER 3

[1]G. N. Vanderplaats. Numerical Optimization Techniques for Engineering Design. Vanderplaats Research and Development Inc., 3rd edition, 2001.

[2]P. Y. Papalambros and D. J. Wilde. Principles of Optimal Design: Modeling and Computation. Cambridge University Press, 2nd edition, 2000.

[3]M. Avriel, M. J. Rijckaert, and D. J. Wilde. Optimization and Design. Prentice Hall, 1973.

[4]J. N. Siddall. Optimal Engineering Design: Principles and Applications. CRC Press, 2nd edition, 1982.

[5]S. S. Rao. Engineering Optimization: Theory and Practice. John Wiley and Sons, 4th edition, 2009.

[6]A. D. Belegundu and T. R. Chandrupatla. Optimization Concepts and Applications in Engineering. Cambridge University Press, 2011.

[7]K. Lange. Optimization. Springer, 2013.

[8]R. A. Sarker and C. S. Newton. Optimization Modeling: A Practical Approach. Taylor and Francis, 2007.