Suresh P. Sethi

Optimal

Control Theory

Applications to Management Science

and Economics

Third Edition

Optimal Control Theory

Suresh P. Sethi

Optimal Control Theory

Applications to Management Science

and Economics

Third Edition

123

Suresh P. Sethi

Jindal School of Management, SM30

University of Texas at Dallas

Richardson, TX, USA

ISBN 978-3-319-98236-6

ISBN 978-3-319-98237-3 (eBook)

https://doi.org/10.1007/978-3-319-98237-3

Library of Congress Control Number: 2018955904

2nd edition: © Springer-Verlag US 2000

© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of

the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,

broadcasting, reproduction on microfilms or in any other physical way, and transmission or information

storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology

now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication

does not imply, even in the absence of a specific statement, that such names are exempt from the relevant

protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book

are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the

editors give a warranty, express or implied, with respect to the material contained herein or for any errors or

omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in

published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

This book is dedicated to the memory of

my parents

Manak Bai and Gulab Chand Sethi

Preface to Third Edition

The third edition of this book will not see my co-author Gerald L.

Thompson, who very sadly passed away on November 9, 2009. Gerry

and I wrote the ﬁrst edition of the 1981 book sitting practically side by

side, and I learned a great deal about book writing in the process. He

was also my PhD supervisor and mentor and he is greatly missed.

After having used the second edition of the book in the classroom

for many years, the third edition arrives with new material and many

improvements. Examples and exercises related to the interpretation of

the adjoint variables and Lagrange multipliers are inserted in Chaps. 2–

4. Direct maximum principle is now discussed in detail in Chap. 4 along

with the existing indirect maximum principle from the second edition.

Chattering or relaxed controls leading to pulsing advertising policies are

introduced in Chap. 7. An application to information systems involving

chattering controls is added as an exercise.

The objective function in Sect. 11.1.3 is changed to the more popular

objective of maximizing the total discounted society’s utility of consumption. Further discussion leading to obtaining a saddle-point path on the

phase diagram leading to the long-run stationary equilibrium is provided

in Sect. 11.2. For this purpose, a global saddle-point theorem is stated

in Appendix D.7. Also inserted in Appendix D.8 is a discussion of the

Sethi-Skiba points which lead to nonunique stable equilibria. Finally,

a new Sect. 11.4 contains an adverse selection model with continuum of

the agent types in a principal-agent framework, which requires an application of the maximum principle.

Chapter 12 of the second edition is removed except for the material

on diﬀerential games and the distributed parameter maximum principle.

The diﬀerential game material joins new topics of stochastic Nash diﬀerential games and Stackelberg diﬀerential games via their applications to

marketing to form a new Chap. 13 titled Diﬀerential Games. As a result,

Chap. 13 of the second edition becomes Chap. 12. The material on the

distributed parameter maximum principle is now Appendix D.9.

The exposition is revised in some places for better reading. New

exercises are added and the list of references is updated. Needless to say,

the errors in the second edition are corrected, and the notation is made

consistent.

vii

viii

Preface to Third Edition

Thanks are due to Huseyin Cavusoglu, Andrei Dmitruk, Gustav Feichtinger, Richard Hartl, Yonghua Ji, Subodha Kumar, Sirong Lao, Helmut Maurer, Ernst Presman, Anyan Qi, Andrea Seidl, Atle Seierstad,

Xi Shan, Lingling Shi, Xiahong Yue, and the students in my Optimal

Control Theory and Applications course over the years for their suggestions for improvement. Special thanks go to Qi (Annabelle) Feng

for her dedication in updating and correcting the forthcoming solution

manual that went with the ﬁrst edition. I cannot thank Barbara Gordon

and Lindsay Wilson enough for their assistance in the preparation of

the text, solution manual, and presentation materials. In addition, the

meticulous copy editing of the entire book by Lindsay Wilson is much

appreciated. Anshuman Chutani, Pooja Kamble, and Shivani Thakkar

are also thanked for their assistance in drawing some of the ﬁgures in

the book.

Richardson, TX, USA

June 2018

Suresh P. Sethi

Preface to Second Edition

The ﬁrst edition of this book, which provided an introduction to optimal control theory and its applications to management science to many

students in management, industrial engineering, operations research and

economics, went out of print a number of years ago. Over the years we

have received feedback concerning its contents from a number of instructors who taught it, and students who studied from it. We have also kept

up with new results in the area as they were published in the literature.

For this reason we felt that now was a good time to come out with a

new edition. While some of the basic material remains, we have made

several big changes and many small changes which we feel will make the

use of the book easier.

The most visible change is that the book is written in Latex and the

ﬁgures are drawn in CorelDRAW, in contrast to the typewritten text

and hand-drawn ﬁgures of the ﬁrst edition. We have also included some

problems along with their numerical solutions obtained using Excel.

The most important change is the division of the material in the

old Chap. 3, into Chaps. 3 and 4 in the new edition. Chapter 3 now

contains models having mixed (control and state) constraints, current

value formulations, terminal conditions and model types, while Chap. 4

covers the more diﬃcult topic of pure state constraints, together with

mixed constraints. Each of these chapters contain new results that were

not available when the ﬁrst edition was published.

The second most important change is the expansion of the material in

the old Sect. 12.4 on stochastic optimal control theory and its becoming

the new Chap. 13. The new Chap. 12 now contains the following advanced topics on optimal control theory: diﬀerential games, distributed

parameter systems, and impulse control. The new Chap. 13 provides a

brief introduction to stochastic optimal control problems. It contains

formulations of simple stochastic models in production, marketing and

ﬁnance, and their solutions. We deleted the old Chap. 11 of the ﬁrst

edition on computational methods, since there are a number of excellent

references now available on this topic. Some of these references are listed

in Sect. 4.2 of Chap. 4 and Sect. 8.3 of Chap. 8.

ix

x

Preface to Second Edition

The emphasis of this book is not on mathematical rigor, but rather

on developing models of realistic situations faced in business and management. For that reason we have given, in Chaps. 2 and 8, proofs of the

continuous and discrete maximum principles by using dynamic programming and Kuhn-Tucker theory, respectively. More general maximum

principles are stated without proofs in Chaps. 3, 4 and 12.

One of the fascinating features of optimal control theory is its extraordinarily wide range of possible applications. We have covered some

of these as follows: Chap. 5 covers ﬁnance; Chap. 6 considers production

and inventory problems; Chap. 7 covers marketing problems; Chap. 9

treats machine maintenance and replacement; Chap. 10 deals with problems of optimal consumption of natural resources (renewable or exhaustible); and Chap. 11 discusses a number of applications of control

theory to economics. The contents of Chaps. 12 and 13 have been described earlier.

Finally, four appendices cover either elementary material, such as

the theory of diﬀerential equations, or very advanced material, whose

inclusion in the main text would interrupt its continuity. At the end

of the book is an extensive but not exhaustive bibliography of relevant

material on optimal control theory including surveys of material devoted

to speciﬁc applications.

We are deeply indebted to many people for their part in making this

edition possible. Onur Arugaslan, Gustav Feichtinger, Neil Geismar,

Richard Hartl, Steﬀen Jørgensen, Subodha Kumar, Helmut Maurer, Gerhard Sorger, and Denny Yeh made helpful comments and suggestions

about the ﬁrst edition or preliminary chapters of this revision. Many

students who used the ﬁrst edition, or preliminary chapters of this revision, also made suggestions for improvements. We would like to express

our gratitude to all of them for their help. In addition we express our

appreciation to Eleanor Balocik, Frank (Youhua) Chen, Feng Cheng,

Howard Chow, Barbara Gordon, Jiong Jiang, Kuntal Kotecha, Ming

Tam, and Srinivasa Yarrakonda for their typing of the various drafts of

the manuscript. They were advised by Dirk Beyer, Feng Cheng, Subodha Kumar, Young Ryu, Chelliah Sriskandarajah, Wulin Suo, Houmin

Yan, Hanqin Zhang, and Qing Zhang on the technical problems of using

LATEX.

We also thank our wives and children—Andrea, Chantal, Anjuli,

Dorothea, Allison, Emily, and Abigail—for their encouragement and understanding during the time-consuming task of preparing this revision.

Preface to Second Edition

xi

Finally, while we regret that lack of time and pressure of other duties prevented us from bringing out a second edition soon after the ﬁrst

edition went out of print, we sincerely hope that the wait has been worthwhile. In spite of the numerous applications of optimal control theory

which already have been made to areas of management science and economics, we continue to believe there is much more that remains to be

done. We hope the present revision will rekindle interest in furthering

such applications, and will enhance the continued development in the

ﬁeld.

Richardson, TX, USA

Pittsburgh, PA, USA

January 2000

Suresh P. Sethi

Gerald L. Thompson

Preface to First Edition

The purpose of this book is to exposit, as simply as possible, some

recent results obtained by a number of researchers in the application of

optimal control theory to management science. We believe that these results are very important and deserve to be widely known by management

scientists, mathematicians, engineers, economists, and others. Because

the mathematical background required to use this book is two or three

semesters of calculus plus some diﬀerential equations and linear algebra,

the book can easily be used to teach a course in the junior or senior

undergraduate years or in the early years of graduate work. For this

purpose, we have included numerous worked-out examples in the text,

as well as a fairly large number of exercises at the end of each chapter.

Answers to selected exercises are included in the back of the book. A

solutions manual containing completely worked-out solutions to all of

the 205 exercises is also available to instructors.

The emphasis of the book is not on mathematical rigor, but on modeling realistic situations faced in business and management. For that

reason, we have given in Chaps. 2 and 7 only heuristic proofs of the continuous and discrete maximum principles, respectively. In Chap. 3 we

have summarized, as succinctly as we can, the most important model

types and terminal conditions that have been used to model management problems. We found it convenient to put a summary of almost all

the important management science models on two pages: see Tables 3.1

and 3.3.

One of the fascinating features of optimal control theory is the extraordinarily wide range of its possible applications. We have tried to

cover a wide variety of applications as follows: Chap. 4 covers ﬁnance;

Chap. 5 considers production and inventory; Chap. 6 covers marketing;

Chap. 8 treats machine maintenance and replacement; Chap. 9 deals with

problems of optimal consumption of natural resources (renewable or exhaustible); and Chap. 10 discusses several economic applications.

In Chap. 11 we treat some computational algorithms for solving optimal control problems. This is a very large and important area that

needs more development.

xiii

xiv

Preface to First Edition

Chapter 12 treats several more advanced topics of optimal control: diﬀerential games, distributed parameter systems, optimal ﬁltering,

stochastic optimal control, and impulsive control. We believe that some

of these models are capable of wider applications and further theoretical

development.

Finally, four appendixes cover either elementary material, such as

diﬀerential equations, or advanced material, whose inclusion in the main

text would spoil its continuity. Also at the end of the book is a bibliography of works actually cited in the text. While it is extensive, it is by no

means an exhaustive bibliography of management science applications

of optimal control theory. Several surveys of such applications, which

contain many other important references, are cited.

We have beneﬁted greatly during the writing of this book by having discussions with and obtaining suggestions from various colleagues

and students. Our special thanks go to Gustav Feichtinger for his careful reading and suggestions for improvement of the entire book. Carl

Norstr¨

om contributed two examples to Chaps. 4 and 5 and made many

suggestions for improvement. Jim Bookbinder used the manuscript for

a course at the University of Toronto, and Tom Morton suggested some

improvements for Chap. 5. The book has also beneﬁted greatly from various coauthors with whom we have done research over the years. Both of

us also have received numerous suggestions for improvements from the

students in our applied control theory courses taught during the past

several years. We would like to express our gratitude to all these people

for their help.

The book has gone through several drafts, and we are greatly indebted to Eleanor Balocik and Rosilita Jones for their patience and

careful typing.

Although the applications of optimal control theory to management

science are recent and many fascinating applications have already been

made, we believe that much remains to be done. We hope that this book

will contribute to the popularity of the area and will enhance future

developments.

Toronto, ON, Canada

Pittsburgh, PA, USA

August 1981

Suresh P. Sethi

Gerald L. Thompson

Contents

1 What Is Optimal Control Theory?

1.1 Basic Concepts and Deﬁnitions . . . . . . . . . . . . . .

1.2 Formulation of Simple Control Models . . . . . . . . . .

1.3 History of Optimal Control Theory . . . . . . . . . . .

1.4 Notation and Concepts Used . . . . . . . . . . . . . . .

1.4.1 Diﬀerentiating Vectors and Matrices with Respect

To Scalars . . . . . . . . . . . . . . . . . . . . . .

1.4.2 Diﬀerentiating Scalars with Respect to Vectors .

1.4.3 Diﬀerentiating Vectors with Respect to Vectors .

1.4.4 Product Rule for Diﬀerentiation . . . . . . . . .

1.4.5 Miscellany . . . . . . . . . . . . . . . . . . . . . .

1.4.6 Convex Set and Convex Hull . . . . . . . . . . .

1.4.7 Concave and Convex Functions . . . . . . . . . .

1.4.8 Aﬃne Function and Homogeneous Function of

Degree k . . . . . . . . . . . . . . . . . . . . . . .

1.4.9 Saddle Point . . . . . . . . . . . . . . . . . . . .

1.4.10 Linear Independence and Rank of a Matrix . . .

1.5 Plan of the Book . . . . . . . . . . . . . . . . . . . . . .

2 The Maximum Principle: Continuous Time

2.1 Statement of the Problem . . . . . . . . . . . . . . .

2.1.1 The Mathematical Model . . . . . . . . . . .

2.1.2 Constraints . . . . . . . . . . . . . . . . . . .

2.1.3 The Objective Function . . . . . . . . . . . .

2.1.4 The Optimal Control Problem . . . . . . . .

2.2 Dynamic Programming and the Maximum Principle

2.2.1 The Hamilton-Jacobi-Bellman Equation . . .

2.2.2 Derivation of the Adjoint Equation . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

1

2

4

9

11

12

13

14

16

16

20

20

22

22

23

23

27

27

28

28

29

29

32

32

36

xv

xvi

CONTENTS

2.2.3

2.2.4

2.3

2.4

2.5

The Maximum Principle . .

Economic Interpretations of

Principle . . . . . . . . . .

Simple Examples . . . . . . . . . .

Suﬃciency Conditions . . . . . . .

Solving a TPBVP by Using Excel .

. .

the

. .

. .

. .

. .

. . . . . . .

Maximum

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . .

39

.

.

.

.

40

42

53

57

.

.

.

.

.

.

.

.

3 The Maximum Principle: Mixed Inequality

Constraints

3.1 A Maximum Principle for Problems with Mixed Inequality

Constraints . . . . . . . . . . . . . . . . . . . . . . . . .

3.2 Suﬃciency Conditions . . . . . . . . . . . . . . . . . . .

3.3 Current-Value Formulation . . . . . . . . . . . . . . . .

3.4 Transversality Conditions: Special Cases . . . . . . . . .

3.5 Free Terminal Time Problems . . . . . . . . . . . . . . .

3.6 Inﬁnite Horizon and Stationarity . . . . . . . . . . . . .

3.7 Model Types . . . . . . . . . . . . . . . . . . . . . . . .

4 The Maximum Principle: Pure State and Mixed

Inequality Constraints

4.1 Jumps in Marginal Valuations . . . . . . . . . . . . .

4.2 The Optimal Control Problem with Pure and Mixed

Constraints . . . . . . . . . . . . . . . . . . . . . . .

4.3 The Maximum Principle: Direct Method . . . . . . .

4.4 Suﬃciency Conditions: Direct Method . . . . . . . .

4.5 The Maximum Principle: Indirect Method . . . . . .

4.6 Current-Value Maximum Principle:

Indirect Method . . . . . . . . . . . . . . . . . . . .

5 Applications to Finance

5.1 The Simple Cash Balance Problem . . . . . . . .

5.1.1 The Model . . . . . . . . . . . . . . . . .

5.1.2 Solution by the Maximum Principle . . .

5.2 Optimal Financing Model . . . . . . . . . . . . .

5.2.1 The Model . . . . . . . . . . . . . . . . .

5.2.2 Application of the Maximum Principle . .

5.2.3 Synthesis of Optimal Control Paths . . .

5.2.4 Solution for the Inﬁnite Horizon Problem

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

69

70

79

80

86

93

103

109

. .

125

127

.

.

.

.

.

.

.

.

129

132

136

137

. .

147

.

.

.

.

.

.

.

.

159

160

160

161

164

165

167

170

180

.

.

.

.

.

.

.

.

CONTENTS

xvii

6 Applications to Production and Inventory

6.1 Production-Inventory Systems . . . . . . . . . . . . . . .

6.1.1 The Production-Inventory Model . . . . . . . . .

6.1.2 Solution by the Maximum Principle . . . . . . .

6.1.3 The Inﬁnite Horizon Solution . . . . . . . . . . .

6.1.4 Special Cases of Time Varying Demands . . . . .

6.1.5 Optimality of a Linear Decision Rule . . . . . . .

6.1.6 Analysis with a Nonnegative Production

Constraint . . . . . . . . . . . . . . . . . . . . . .

6.2 The Wheat Trading Model . . . . . . . . . . . . . . . .

6.2.1 The Model . . . . . . . . . . . . . . . . . . . . .

6.2.2 Solution by the Maximum Principle . . . . . . .

6.2.3 Solution of a Special Case . . . . . . . . . . . . .

6.2.4 The Wheat Trading Model with No Short-Selling

6.3 Decision Horizons and Forecast Horizons . . . . . . . . .

6.3.1 Horizons for the Wheat Trading Model with

No Short-Selling . . . . . . . . . . . . . . . . . .

6.3.2 Horizons for the Wheat Trading Model with No

Short-Selling and a Warehousing Constraint . . .

191

192

192

193

196

197

200

7 Applications to Marketing

7.1 The Nerlove-Arrow Advertising Model . . . . . . . . .

7.1.1 The Model . . . . . . . . . . . . . . . . . . . .

7.1.2 Solution by the Maximum Principle . . . . . .

7.1.3 Convex Advertising Cost and Relaxed Controls

7.2 The Vidale-Wolfe Advertising Model . . . . . . . . . .

7.2.1 Optimal Control Formulation for the

Vidale-Wolfe Model . . . . . . . . . . . . . . .

7.2.2 Solution Using Green’s Theorem When

Q Is Large . . . . . . . . . . . . . . . . . . . .

7.2.3 Solution When Q Is Small . . . . . . . . . . . .

7.2.4 Solution When T Is Inﬁnite . . . . . . . . . . .

.

.

.

.

.

225

226

226

228

232

235

.

236

.

.

.

237

245

247

.

.

.

.

.

259

259

260

262

267

268

8 The Maximum Principle: Discrete Time

8.1 Nonlinear Programming Problems . . . . . . .

8.1.1 Lagrange Multipliers . . . . . . . . . . .

8.1.2 Equality and Inequality Constraints . .

8.1.3 Constraint Qualiﬁcation . . . . . . . . .

8.1.4 Theorems from Nonlinear Programming

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

202

204

205

206

206

208

213

214

214

xviii

8.2

8.3

CONTENTS

A Discrete Maximum Principle . . . . . . . . . . .

8.2.1 A Discrete-Time Optimal Control Problem

8.2.2 A Discrete Maximum Principle . . . . . . .

8.2.3 Examples . . . . . . . . . . . . . . . . . . .

A General Discrete Maximum Principle . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

269

269

270

272

276

.

.

.

.

.

.

.

.

.

.

283

284

284

285

287

289

.

.

.

.

.

.

.

.

9 Maintenance and Replacement

9.1 A Simple Maintenance and Replacement Model . . .

9.1.1 The Model . . . . . . . . . . . . . . . . . . .

9.1.2 Solution by the Maximum Principle . . . . .

9.1.3 A Numerical Example . . . . . . . . . . . . .

9.1.4 An Extension . . . . . . . . . . . . . . . . . .

9.2 Maintenance and Replacement for

a Machine Subject to Failure . . . . . . . . . . . . .

9.2.1 The Model . . . . . . . . . . . . . . . . . . .

9.2.2 Optimal Policy . . . . . . . . . . . . . . . . .

9.2.3 Determination of the Sale Date . . . . . . . .

9.3 Chain of Machines . . . . . . . . . . . . . . . . . . .

9.3.1 The Model . . . . . . . . . . . . . . . . . . .

9.3.2 Solution by the Discrete Maximum Principle

9.3.3 Special Case of Bang-Bang Control . . . . . .

9.3.4 Incorporation into the Wagner-Whitin

Framework for a Complete Solution . . . . .

9.3.5 A Numerical Example . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

290

291

293

296

297

297

299

301

. .

. .

301

302

10 Applications to Natural Resources

10.1 The Sole-Owner Fishery Resource Model . .

10.1.1 The Dynamics of Fishery Models . .

10.1.2 The Sole Owner Model . . . . . . .

10.1.3 Solution by Green’s Theorem . . . .

10.2 An Optimal Forest Thinning Model . . . .

10.2.1 The Forestry Model . . . . . . . . .

10.2.2 Determination of Optimal Thinning

10.2.3 A Chain of Forests Model . . . . . .

10.3 An Exhaustible Resource Model . . . . . .

10.3.1 Formulation of the Model . . . . . .

10.3.2 Solution by the Maximum Principle

.

.

.

.

.

.

.

.

.

.

.

311

312

312

313

314

317

317

318

321

324

324

327

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

CONTENTS

11 Applications to Economics

11.1 Models of Optimal Economic Growth . . . . . . .

11.1.1 An Optimal Capital Accumulation Model

11.1.2 Solution by the Maximum Principle . . .

11.1.3 Introduction of a Growing Labor Force . .

11.1.4 Solution by the Maximum Principle . . .

11.2 A Model of Optimal Epidemic Control . . . . . .

11.2.1 Formulation of the Model . . . . . . . . .

11.2.2 Solution by Green’s Theorem . . . . . . .

11.3 A Pollution Control Model . . . . . . . . . . . .

11.3.1 Model Formulation . . . . . . . . . . . . .

11.3.2 Solution by the Maximum Principle . . .

11.3.3 Phase Diagram Analysis . . . . . . . . . .

11.4 An Adverse Selection Model . . . . . . . . . . . .

11.4.1 Model Formulation . . . . . . . . . . . . .

11.4.2 The Implementation Problem . . . . . . .

11.4.3 The Optimization Problem . . . . . . . .

11.5 Miscellaneous Applications . . . . . . . . . . . .

xix

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

335

335

336

336

338

339

343

343

344

346

347

348

349

352

352

353

354

360

.

.

.

.

.

.

365

366

370

372

375

377

383

.

.

.

.

385

386

387

388

388

. .

389

. .

of

. .

392

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

12 Stochastic Optimal Control

12.1 Stochastic Optimal Control . . . . . . . . . . . . . . .

12.2 A Stochastic Production Inventory Model . . . . . . .

12.2.1 Solution for the Production Planning Problem

12.3 The Sethi Advertising Model . . . . . . . . . . . . . .

12.4 An Optimal Consumption-Investment Problem . . . .

12.5 Concluding Remarks . . . . . . . . . . . . . . . . . . .

13 Diﬀerential Games

13.1 Two-Person Zero-Sum Diﬀerential Games . . . . . .

13.2 Nash Diﬀerential Games . . . . . . . . . . . . . . . .

13.2.1 Open-Loop Nash Solution . . . . . . . . . . .

13.2.2 Feedback Nash Solution . . . . . . . . . . . .

13.2.3 An Application to Common-Property Fishery

Resources . . . . . . . . . . . . . . . . . . . .

13.3 A Feedback Nash Stochastic Diﬀerential

Game in Advertising . . . . . . . . . . . . . . . . . .

13.4 A Feedback Stackelberg Stochastic Diﬀerential Game

Cooperative Advertising . . . . . . . . . . . . . . . .

.

.

.

.

395

xx

CONTENTS

A Solutions of Linear Diﬀerential Equations

A.1 First-Order Linear Equations . . . . . . . . . . . . . . .

A.2 Second-Order Linear Equations with

Constant Coeﬃcients . . . . . . . . . . . . . . . . . . . .

A.3 System of First-Order Linear Equations . . . . . . . . .

A.4 Solution of Linear Two-Point Boundary Value Problems

A.5 Solutions of Finite Diﬀerence Equations . . . . . . . . .

A.5.1 Changing Polynomials in Powers of k into

Factorial Powers of k . . . . . . . . . . . . . . . .

A.5.2 Changing Factorial Powers of k into Ordinary

Powers of k . . . . . . . . . . . . . . . . . . . . .

409

409

410

410

413

414

415

416

B Calculus of Variations and Optimal Control Theory

419

B.1 The Simplest Variational Problem . . . . . . . . . . . . 420

B.2 The Euler-Lagrange Equation . . . . . . . . . . . . . . . 421

B.3 The Shortest Distance Between Two Points on the Plane 424

B.4 The Brachistochrone Problem . . . . . . . . . . . . . . . 424

B.5 The Weierstrass-Erdmann Corner

Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 427

B.6 Legendre’s Conditions: The Second Variation . . . . . . 428

B.7 Necessary Condition for a Strong

Maximum . . . . . . . . . . . . . . . . . . . . . . . . . . 429

B.8 Relation to Optimal Control Theory . . . . . . . . . . . 430

C An Alternative Derivation of the Maximum Principle 433

C.1 Needle-Shaped Variation . . . . . . . . . . . . . . . . . . 434

C.2 Derivation of the Adjoint Equation and the Maximum

Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

D Special Topics in Optimal Control

D.1 The Kalman Filter . . . . . . . . . . . . . . . . . . .

D.2 Wiener Process and Stochastic Calculus . . . . . . .

D.3 The Kalman-Bucy Filter . . . . . . . . . . . . . . . .

D.4 Linear-Quadratic Problems . . . . . . . . . . . . . .

D.4.1 Certainty Equivalence or Separation Principle

D.5 Second-Order Variations . . . . . . . . . . . . . . . .

D.6 Singular Control . . . . . . . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

441

441

444

447

448

451

452

454

CONTENTS

D.7 Global Saddle Point Theorem . . . . . . . . . . . . . . .

D.8 The Sethi-Skiba Points . . . . . . . . . . . . . . . . . . .

D.9 Distributed Parameter Systems . . . . . . . . . . . . . .

xxi

456

458

460

E Answers to Selected Exercises

465

Bibliography

473

Index

547

List of Figures

1.1

1.2

1.3

1.4

The Brachistochrone problem . . .

Illustration of left and right limits

A concave function . . . . . . . . .

An illustration of a saddle point . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

9

18

21

23

2.1

2.2

2.3

2.4

2.5

2.6

2.7

2.8

An optimal path in the state-time space . . . . . . . .

Optimal state and adjoint trajectories for Example 2.2

Optimal state and adjoint trajectories for Example 2.3

Optimal trajectories for Examples 2.4 and 2.5 . . . . .

Optimal control for Example 2.6 . . . . . . . . . . . .

The ﬂowchart for Example 2.8 . . . . . . . . . . . . .

Solution of TPBVP by excel . . . . . . . . . . . . . . .

Water reservoir of Exercise 2.18 . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

34

44

46

48

53

58

60

63

3.1

3.2

State and adjoint trajectories in Example 3.4 . . . . . .

Minimum time optimal response for Example 3.6 . . . .

93

101

4.1

4.2

4.3

4.4

4.5

Feasible state space and optimal state trajectory

for Examples 4.1 and 4.4 . . . . . . . . . . . . . .

State and adjoint trajectories in Example 4.3 . .

Adjoint trajectory for Example 4.4 . . . . . . . .

Two-reservoir system of Exercise 4.8 . . . . . . .

Feasible space for Exercise 4.28 . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

128

143

147

151

157

5.1

5.2

5.3

5.4

5.5

5.6

Optimal policy shown in (λ1 , λ2 ) space .

Optimal policy shown in (t, λ2 /λ1 ) space

Case A: g ≤ r . . . . . . . . . . . . . . .

Case B: g > r . . . . . . . . . . . . . . .

Optimal path for case A: g ≤ r . . . . .

Optimal path for case B: g > r . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

163

164

169

170

174

179

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

xxiii

xxiv

5.7

5.8

LIST OF FIGURES

Solution for Exercise 5.4 . . . . . . . . . . . . . . . . . .

Adjoint trajectories for Exercise 5.5 . . . . . . . . . . .

186

187

Solution of Example 6.1 with I0 = 10 . . . . . . . . . . .

Solution of Example 6.1 with I0 = 50 . . . . . . . . . . .

Solution of Example 6.1 with I0 = 30 . . . . . . . . . . .

Optimal production rate and inventory level with diﬀerent

initial inventories . . . . . . . . . . . . . . . . . . . . . .

6.5 The price trajectory (6.56) . . . . . . . . . . . . . . . . .

6.6 Adjoint variable, optimal policy and inventory in the

wheat trading model . . . . . . . . . . . . . . . . . . . .

6.7 Adjoint trajectory and optimal policy for the wheat trading model . . . . . . . . . . . . . . . . . . . . . . . . . .

6.8 Decision horizon and optimal policy for the wheat trading

model . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6.9 Optimal policy and horizons for the wheat trading model

with no short-selling and a warehouse constraint . . . .

6.10 Optimal policy and horizons for Example 6.3 . . . . . .

6.11 Optimal policy and horizons for Example 6.4 . . . . . .

199

199

200

6.1

6.2

6.3

6.4

7.1

7.2

7.12

7.13

7.14

Optimal policies in the Nerlove-Arrow model . . . . . .

A case of a time-dependent turnpike and the nature of

optimal control . . . . . . . . . . . . . . . . . . . . . . .

A near-optimal control of problem (7.15) . . . . . . . . .

Feasible arcs in (t, x)-space . . . . . . . . . . . . . . . .

Optimal trajectory for Case 1: x0 ≤ xs and xT ≤ xs . .

Optimal trajectory for Case 2: x0 < xs and xT > xs . .

Optimal trajectory for Case 3: x0 > xs and xT < xs . .

Optimal trajectory for Case 4: x0 > xs and xT > xs . .

Optimal trajectory (solid lines) . . . . . . . . . . . . . .

Optimal trajectory when T is small in Case 1: x0 < xs

and xT > xs . . . . . . . . . . . . . . . . . . . . . . . . .

Optimal trajectory when T is small in Case 2: x0 > xs

and xT > xs . . . . . . . . . . . . . . . . . . . . . . . . .

Optimal trajectory for Case 2 of Theorem 7.1 for Q = ∞

Optimal trajectories for x(0) < x

ˆ . . . . . . . . . . . . .

Optimal trajectory for x(0) > x

ˆ . . . . . . . . . . . . . .

8.1

8.2

Shortest distance from point (2,2) to the semicircle . . .

Graph of Example 8.5 . . . . . . . . . . . . . . . . . . .

7.3

7.4

7.5

7.6

7.7

7.8

7.9

7.10

7.11

204

207

209

212

215

216

218

219

230

231

233

238

240

241

241

242

243

243

244

244

249

250

266

267

LIST OF FIGURES

xxv

8.3

8.4

Discrete-time conventions . . . . . . . . . . . . . . . . .

∗

Optimal state xk and adjoint λk . . . . . . . . . . . . .

270

275

9.1

9.2

Optimal maintenance and machine resale value . . . . .

Sat function optimal control . . . . . . . . . . . . . . . .

289

291

10.1 Optimal policy for the sole owner ﬁshery model . . . . .

10.2 Singular usable timber volume x

¯(t) . . . . . . . . . . . .

10.3 Optimal thinning u∗ (t) and timber volume x∗ (t) for the

forest thinning model when x0 < x

¯(t0 ) . . . . . . . . . .

10.4 Optimal thinning u∗ (t) and timber volume x∗ (t) for the

chain of forests model when T > tˆ . . . . . . . . . . . .

10.5 Optimal thinning and timber volume x∗ (t) for the chain

of forests model when T ≤ tˆ . . . . . . . . . . . . . . . .

10.6 The demand function . . . . . . . . . . . . . . . . . . . .

10.7 The proﬁt function . . . . . . . . . . . . . . . . . . . . .

10.8 Optimal price trajectory for T ≥ T¯ . . . . . . . . . . . .

10.9 Optimal price trajectory for T < T¯ . . . . . . . . . . . .

316

320

11.1

11.2

11.3

11.4

11.5

11.6

11.7

Phase diagram for the optimal growth model .

Optimal trajectory when xT > xs . . . . . . . .

Optimal trajectory when xT < xs . . . . . . .

Food output function . . . . . . . . . . . . . . .

Phase diagram for the pollution control model .

Violation of the monotonicity constraint . . . .

Bunching and ironing . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

320

322

323

324

326

329

330

.

.

.

.

.

.

.

340

346

347

348

351

358

359

12.1 A sample path of optimal production rate It∗ with

I0 = x0 > 0 and B > 0 . . . . . . . . . . . . . . . . . . .

374

13.1 A sample path of optimal market share trajectories . . .

13.2 Optimal subsidy rate vs. (a) Retailer’s margin and (b)

Manufacturer’s margin . . . . . . . . . . . . . . . . . . .

396

B.1 Examples of admissible functions for the problem . . . .

B.2 Variation about the solution function . . . . . . . . . . .

B.3 A broken extremal with corner at τ . . . . . . . . . . . .

420

421

428

404

xxvi

LIST OF FIGURES

C.1 Needle-shaped variation . . . . . . . . . . . . . . . . . .

C.2 Trajectories x∗ (t) and x(t) in a one-dimensional case . .

434

434

D.1 Phase diagram for system (D.73) . . . . . . . . . . . . .

D.2 Region D with boundaries Γ1 and Γ2 . . . . . . . . . . .

457

461

List of Tables

1.1

1.2

1.3

The production-inventory model of Example 1.1 . . . .

The advertising model of Example 1.2 . . . . . . . . . .

The consumption model of Example 1.3 . . . . . . . . .

3.1

3.2

3.3

Summary of the transversality conditions

State trajectories and switching curves . .

Objective, state, and adjoint equations for

types . . . . . . . . . . . . . . . . . . . . .

4

6

8

. . . . . . . .

. . . . . . . .

various model

. . . . . . . .

89

100

Characterization of optimal controls with c < 1 . . . . .

168

13.1 Optimal feedback Stackelberg solution . . . . . . . . . .

403

A.1 Homogeneous solution forms for Eq. (A.5) . . . . . . . .

A.2 Particular solutions for Eq. (A.5) . . . . . . . . . . . . .

411

411

5.1

111

xxvii

Chapter 1

What Is Optimal Control

Theory?

Many management science applications involve the control of dynamic

systems, i.e., systems that evolve over time. They are called continuoustime systems or discrete-time systems depending on whether time varies

continuously or discretely. We will deal with both kinds of systems in this

book, although the main emphasis will be on continuous-time systems.

Optimal control theory is a branch of mathematics developed to ﬁnd

optimal ways to control a dynamic system. The purpose of this book is

to give an elementary introduction to the mathematical theory, and then

apply it to a wide variety of diﬀerent situations arising in management

science. We have deliberately kept the level of mathematics as simple as

possible in order to make the book accessible to a large audience. The

only mathematical requirements for this book are elementary calculus,

including partial diﬀerentiation, some knowledge of vectors and matrices, and elementary ordinary and partial diﬀerential equations. The last

topic is brieﬂy covered in Appendix A. Chapter 12 on stochastic optimal control also requires some concepts in stochastic calculus, which are

introduced at the beginning of that chapter.

The principle management science applications discussed in this book

come from the following areas: ﬁnance, economics, production and inventory, marketing, maintenance and replacement, and the consumption

of natural resources. In each major area we have formulated one or more

simple models followed by a more complicated model. The reader may

© Springer Nature Switzerland AG 2019

S. P. Sethi, Optimal Control Theory,

https://doi.org/10.1007/978-3-319-98237-3 1

1

Optimal

Control Theory

Applications to Management Science

and Economics

Third Edition

Optimal Control Theory

Suresh P. Sethi

Optimal Control Theory

Applications to Management Science

and Economics

Third Edition

123

Suresh P. Sethi

Jindal School of Management, SM30

University of Texas at Dallas

Richardson, TX, USA

ISBN 978-3-319-98236-6

ISBN 978-3-319-98237-3 (eBook)

https://doi.org/10.1007/978-3-319-98237-3

Library of Congress Control Number: 2018955904

2nd edition: © Springer-Verlag US 2000

© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of

the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,

broadcasting, reproduction on microfilms or in any other physical way, and transmission or information

storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology

now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication

does not imply, even in the absence of a specific statement, that such names are exempt from the relevant

protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book

are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the

editors give a warranty, express or implied, with respect to the material contained herein or for any errors or

omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in

published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

This book is dedicated to the memory of

my parents

Manak Bai and Gulab Chand Sethi

Preface to Third Edition

The third edition of this book will not see my co-author Gerald L.

Thompson, who very sadly passed away on November 9, 2009. Gerry

and I wrote the ﬁrst edition of the 1981 book sitting practically side by

side, and I learned a great deal about book writing in the process. He

was also my PhD supervisor and mentor and he is greatly missed.

After having used the second edition of the book in the classroom

for many years, the third edition arrives with new material and many

improvements. Examples and exercises related to the interpretation of

the adjoint variables and Lagrange multipliers are inserted in Chaps. 2–

4. Direct maximum principle is now discussed in detail in Chap. 4 along

with the existing indirect maximum principle from the second edition.

Chattering or relaxed controls leading to pulsing advertising policies are

introduced in Chap. 7. An application to information systems involving

chattering controls is added as an exercise.

The objective function in Sect. 11.1.3 is changed to the more popular

objective of maximizing the total discounted society’s utility of consumption. Further discussion leading to obtaining a saddle-point path on the

phase diagram leading to the long-run stationary equilibrium is provided

in Sect. 11.2. For this purpose, a global saddle-point theorem is stated

in Appendix D.7. Also inserted in Appendix D.8 is a discussion of the

Sethi-Skiba points which lead to nonunique stable equilibria. Finally,

a new Sect. 11.4 contains an adverse selection model with continuum of

the agent types in a principal-agent framework, which requires an application of the maximum principle.

Chapter 12 of the second edition is removed except for the material

on diﬀerential games and the distributed parameter maximum principle.

The diﬀerential game material joins new topics of stochastic Nash diﬀerential games and Stackelberg diﬀerential games via their applications to

marketing to form a new Chap. 13 titled Diﬀerential Games. As a result,

Chap. 13 of the second edition becomes Chap. 12. The material on the

distributed parameter maximum principle is now Appendix D.9.

The exposition is revised in some places for better reading. New

exercises are added and the list of references is updated. Needless to say,

the errors in the second edition are corrected, and the notation is made

consistent.

vii

viii

Preface to Third Edition

Thanks are due to Huseyin Cavusoglu, Andrei Dmitruk, Gustav Feichtinger, Richard Hartl, Yonghua Ji, Subodha Kumar, Sirong Lao, Helmut Maurer, Ernst Presman, Anyan Qi, Andrea Seidl, Atle Seierstad,

Xi Shan, Lingling Shi, Xiahong Yue, and the students in my Optimal

Control Theory and Applications course over the years for their suggestions for improvement. Special thanks go to Qi (Annabelle) Feng

for her dedication in updating and correcting the forthcoming solution

manual that went with the ﬁrst edition. I cannot thank Barbara Gordon

and Lindsay Wilson enough for their assistance in the preparation of

the text, solution manual, and presentation materials. In addition, the

meticulous copy editing of the entire book by Lindsay Wilson is much

appreciated. Anshuman Chutani, Pooja Kamble, and Shivani Thakkar

are also thanked for their assistance in drawing some of the ﬁgures in

the book.

Richardson, TX, USA

June 2018

Suresh P. Sethi

Preface to Second Edition

The ﬁrst edition of this book, which provided an introduction to optimal control theory and its applications to management science to many

students in management, industrial engineering, operations research and

economics, went out of print a number of years ago. Over the years we

have received feedback concerning its contents from a number of instructors who taught it, and students who studied from it. We have also kept

up with new results in the area as they were published in the literature.

For this reason we felt that now was a good time to come out with a

new edition. While some of the basic material remains, we have made

several big changes and many small changes which we feel will make the

use of the book easier.

The most visible change is that the book is written in Latex and the

ﬁgures are drawn in CorelDRAW, in contrast to the typewritten text

and hand-drawn ﬁgures of the ﬁrst edition. We have also included some

problems along with their numerical solutions obtained using Excel.

The most important change is the division of the material in the

old Chap. 3, into Chaps. 3 and 4 in the new edition. Chapter 3 now

contains models having mixed (control and state) constraints, current

value formulations, terminal conditions and model types, while Chap. 4

covers the more diﬃcult topic of pure state constraints, together with

mixed constraints. Each of these chapters contain new results that were

not available when the ﬁrst edition was published.

The second most important change is the expansion of the material in

the old Sect. 12.4 on stochastic optimal control theory and its becoming

the new Chap. 13. The new Chap. 12 now contains the following advanced topics on optimal control theory: diﬀerential games, distributed

parameter systems, and impulse control. The new Chap. 13 provides a

brief introduction to stochastic optimal control problems. It contains

formulations of simple stochastic models in production, marketing and

ﬁnance, and their solutions. We deleted the old Chap. 11 of the ﬁrst

edition on computational methods, since there are a number of excellent

references now available on this topic. Some of these references are listed

in Sect. 4.2 of Chap. 4 and Sect. 8.3 of Chap. 8.

ix

x

Preface to Second Edition

The emphasis of this book is not on mathematical rigor, but rather

on developing models of realistic situations faced in business and management. For that reason we have given, in Chaps. 2 and 8, proofs of the

continuous and discrete maximum principles by using dynamic programming and Kuhn-Tucker theory, respectively. More general maximum

principles are stated without proofs in Chaps. 3, 4 and 12.

One of the fascinating features of optimal control theory is its extraordinarily wide range of possible applications. We have covered some

of these as follows: Chap. 5 covers ﬁnance; Chap. 6 considers production

and inventory problems; Chap. 7 covers marketing problems; Chap. 9

treats machine maintenance and replacement; Chap. 10 deals with problems of optimal consumption of natural resources (renewable or exhaustible); and Chap. 11 discusses a number of applications of control

theory to economics. The contents of Chaps. 12 and 13 have been described earlier.

Finally, four appendices cover either elementary material, such as

the theory of diﬀerential equations, or very advanced material, whose

inclusion in the main text would interrupt its continuity. At the end

of the book is an extensive but not exhaustive bibliography of relevant

material on optimal control theory including surveys of material devoted

to speciﬁc applications.

We are deeply indebted to many people for their part in making this

edition possible. Onur Arugaslan, Gustav Feichtinger, Neil Geismar,

Richard Hartl, Steﬀen Jørgensen, Subodha Kumar, Helmut Maurer, Gerhard Sorger, and Denny Yeh made helpful comments and suggestions

about the ﬁrst edition or preliminary chapters of this revision. Many

students who used the ﬁrst edition, or preliminary chapters of this revision, also made suggestions for improvements. We would like to express

our gratitude to all of them for their help. In addition we express our

appreciation to Eleanor Balocik, Frank (Youhua) Chen, Feng Cheng,

Howard Chow, Barbara Gordon, Jiong Jiang, Kuntal Kotecha, Ming

Tam, and Srinivasa Yarrakonda for their typing of the various drafts of

the manuscript. They were advised by Dirk Beyer, Feng Cheng, Subodha Kumar, Young Ryu, Chelliah Sriskandarajah, Wulin Suo, Houmin

Yan, Hanqin Zhang, and Qing Zhang on the technical problems of using

LATEX.

We also thank our wives and children—Andrea, Chantal, Anjuli,

Dorothea, Allison, Emily, and Abigail—for their encouragement and understanding during the time-consuming task of preparing this revision.

Preface to Second Edition

xi

Finally, while we regret that lack of time and pressure of other duties prevented us from bringing out a second edition soon after the ﬁrst

edition went out of print, we sincerely hope that the wait has been worthwhile. In spite of the numerous applications of optimal control theory

which already have been made to areas of management science and economics, we continue to believe there is much more that remains to be

done. We hope the present revision will rekindle interest in furthering

such applications, and will enhance the continued development in the

ﬁeld.

Richardson, TX, USA

Pittsburgh, PA, USA

January 2000

Suresh P. Sethi

Gerald L. Thompson

Preface to First Edition

The purpose of this book is to exposit, as simply as possible, some

recent results obtained by a number of researchers in the application of

optimal control theory to management science. We believe that these results are very important and deserve to be widely known by management

scientists, mathematicians, engineers, economists, and others. Because

the mathematical background required to use this book is two or three

semesters of calculus plus some diﬀerential equations and linear algebra,

the book can easily be used to teach a course in the junior or senior

undergraduate years or in the early years of graduate work. For this

purpose, we have included numerous worked-out examples in the text,

as well as a fairly large number of exercises at the end of each chapter.

Answers to selected exercises are included in the back of the book. A

solutions manual containing completely worked-out solutions to all of

the 205 exercises is also available to instructors.

The emphasis of the book is not on mathematical rigor, but on modeling realistic situations faced in business and management. For that

reason, we have given in Chaps. 2 and 7 only heuristic proofs of the continuous and discrete maximum principles, respectively. In Chap. 3 we

have summarized, as succinctly as we can, the most important model

types and terminal conditions that have been used to model management problems. We found it convenient to put a summary of almost all

the important management science models on two pages: see Tables 3.1

and 3.3.

One of the fascinating features of optimal control theory is the extraordinarily wide range of its possible applications. We have tried to

cover a wide variety of applications as follows: Chap. 4 covers ﬁnance;

Chap. 5 considers production and inventory; Chap. 6 covers marketing;

Chap. 8 treats machine maintenance and replacement; Chap. 9 deals with

problems of optimal consumption of natural resources (renewable or exhaustible); and Chap. 10 discusses several economic applications.

In Chap. 11 we treat some computational algorithms for solving optimal control problems. This is a very large and important area that

needs more development.

xiii

xiv

Preface to First Edition

Chapter 12 treats several more advanced topics of optimal control: diﬀerential games, distributed parameter systems, optimal ﬁltering,

stochastic optimal control, and impulsive control. We believe that some

of these models are capable of wider applications and further theoretical

development.

Finally, four appendixes cover either elementary material, such as

diﬀerential equations, or advanced material, whose inclusion in the main

text would spoil its continuity. Also at the end of the book is a bibliography of works actually cited in the text. While it is extensive, it is by no

means an exhaustive bibliography of management science applications

of optimal control theory. Several surveys of such applications, which

contain many other important references, are cited.

We have beneﬁted greatly during the writing of this book by having discussions with and obtaining suggestions from various colleagues

and students. Our special thanks go to Gustav Feichtinger for his careful reading and suggestions for improvement of the entire book. Carl

Norstr¨

om contributed two examples to Chaps. 4 and 5 and made many

suggestions for improvement. Jim Bookbinder used the manuscript for

a course at the University of Toronto, and Tom Morton suggested some

improvements for Chap. 5. The book has also beneﬁted greatly from various coauthors with whom we have done research over the years. Both of

us also have received numerous suggestions for improvements from the

students in our applied control theory courses taught during the past

several years. We would like to express our gratitude to all these people

for their help.

The book has gone through several drafts, and we are greatly indebted to Eleanor Balocik and Rosilita Jones for their patience and

careful typing.

Although the applications of optimal control theory to management

science are recent and many fascinating applications have already been

made, we believe that much remains to be done. We hope that this book

will contribute to the popularity of the area and will enhance future

developments.

Toronto, ON, Canada

Pittsburgh, PA, USA

August 1981

Suresh P. Sethi

Gerald L. Thompson

Contents

1 What Is Optimal Control Theory?

1.1 Basic Concepts and Deﬁnitions . . . . . . . . . . . . . .

1.2 Formulation of Simple Control Models . . . . . . . . . .

1.3 History of Optimal Control Theory . . . . . . . . . . .

1.4 Notation and Concepts Used . . . . . . . . . . . . . . .

1.4.1 Diﬀerentiating Vectors and Matrices with Respect

To Scalars . . . . . . . . . . . . . . . . . . . . . .

1.4.2 Diﬀerentiating Scalars with Respect to Vectors .

1.4.3 Diﬀerentiating Vectors with Respect to Vectors .

1.4.4 Product Rule for Diﬀerentiation . . . . . . . . .

1.4.5 Miscellany . . . . . . . . . . . . . . . . . . . . . .

1.4.6 Convex Set and Convex Hull . . . . . . . . . . .

1.4.7 Concave and Convex Functions . . . . . . . . . .

1.4.8 Aﬃne Function and Homogeneous Function of

Degree k . . . . . . . . . . . . . . . . . . . . . . .

1.4.9 Saddle Point . . . . . . . . . . . . . . . . . . . .

1.4.10 Linear Independence and Rank of a Matrix . . .

1.5 Plan of the Book . . . . . . . . . . . . . . . . . . . . . .

2 The Maximum Principle: Continuous Time

2.1 Statement of the Problem . . . . . . . . . . . . . . .

2.1.1 The Mathematical Model . . . . . . . . . . .

2.1.2 Constraints . . . . . . . . . . . . . . . . . . .

2.1.3 The Objective Function . . . . . . . . . . . .

2.1.4 The Optimal Control Problem . . . . . . . .

2.2 Dynamic Programming and the Maximum Principle

2.2.1 The Hamilton-Jacobi-Bellman Equation . . .

2.2.2 Derivation of the Adjoint Equation . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

1

2

4

9

11

12

13

14

16

16

20

20

22

22

23

23

27

27

28

28

29

29

32

32

36

xv

xvi

CONTENTS

2.2.3

2.2.4

2.3

2.4

2.5

The Maximum Principle . .

Economic Interpretations of

Principle . . . . . . . . . .

Simple Examples . . . . . . . . . .

Suﬃciency Conditions . . . . . . .

Solving a TPBVP by Using Excel .

. .

the

. .

. .

. .

. .

. . . . . . .

Maximum

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . .

39

.

.

.

.

40

42

53

57

.

.

.

.

.

.

.

.

3 The Maximum Principle: Mixed Inequality

Constraints

3.1 A Maximum Principle for Problems with Mixed Inequality

Constraints . . . . . . . . . . . . . . . . . . . . . . . . .

3.2 Suﬃciency Conditions . . . . . . . . . . . . . . . . . . .

3.3 Current-Value Formulation . . . . . . . . . . . . . . . .

3.4 Transversality Conditions: Special Cases . . . . . . . . .

3.5 Free Terminal Time Problems . . . . . . . . . . . . . . .

3.6 Inﬁnite Horizon and Stationarity . . . . . . . . . . . . .

3.7 Model Types . . . . . . . . . . . . . . . . . . . . . . . .

4 The Maximum Principle: Pure State and Mixed

Inequality Constraints

4.1 Jumps in Marginal Valuations . . . . . . . . . . . . .

4.2 The Optimal Control Problem with Pure and Mixed

Constraints . . . . . . . . . . . . . . . . . . . . . . .

4.3 The Maximum Principle: Direct Method . . . . . . .

4.4 Suﬃciency Conditions: Direct Method . . . . . . . .

4.5 The Maximum Principle: Indirect Method . . . . . .

4.6 Current-Value Maximum Principle:

Indirect Method . . . . . . . . . . . . . . . . . . . .

5 Applications to Finance

5.1 The Simple Cash Balance Problem . . . . . . . .

5.1.1 The Model . . . . . . . . . . . . . . . . .

5.1.2 Solution by the Maximum Principle . . .

5.2 Optimal Financing Model . . . . . . . . . . . . .

5.2.1 The Model . . . . . . . . . . . . . . . . .

5.2.2 Application of the Maximum Principle . .

5.2.3 Synthesis of Optimal Control Paths . . .

5.2.4 Solution for the Inﬁnite Horizon Problem

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

69

70

79

80

86

93

103

109

. .

125

127

.

.

.

.

.

.

.

.

129

132

136

137

. .

147

.

.

.

.

.

.

.

.

159

160

160

161

164

165

167

170

180

.

.

.

.

.

.

.

.

CONTENTS

xvii

6 Applications to Production and Inventory

6.1 Production-Inventory Systems . . . . . . . . . . . . . . .

6.1.1 The Production-Inventory Model . . . . . . . . .

6.1.2 Solution by the Maximum Principle . . . . . . .

6.1.3 The Inﬁnite Horizon Solution . . . . . . . . . . .

6.1.4 Special Cases of Time Varying Demands . . . . .

6.1.5 Optimality of a Linear Decision Rule . . . . . . .

6.1.6 Analysis with a Nonnegative Production

Constraint . . . . . . . . . . . . . . . . . . . . . .

6.2 The Wheat Trading Model . . . . . . . . . . . . . . . .

6.2.1 The Model . . . . . . . . . . . . . . . . . . . . .

6.2.2 Solution by the Maximum Principle . . . . . . .

6.2.3 Solution of a Special Case . . . . . . . . . . . . .

6.2.4 The Wheat Trading Model with No Short-Selling

6.3 Decision Horizons and Forecast Horizons . . . . . . . . .

6.3.1 Horizons for the Wheat Trading Model with

No Short-Selling . . . . . . . . . . . . . . . . . .

6.3.2 Horizons for the Wheat Trading Model with No

Short-Selling and a Warehousing Constraint . . .

191

192

192

193

196

197

200

7 Applications to Marketing

7.1 The Nerlove-Arrow Advertising Model . . . . . . . . .

7.1.1 The Model . . . . . . . . . . . . . . . . . . . .

7.1.2 Solution by the Maximum Principle . . . . . .

7.1.3 Convex Advertising Cost and Relaxed Controls

7.2 The Vidale-Wolfe Advertising Model . . . . . . . . . .

7.2.1 Optimal Control Formulation for the

Vidale-Wolfe Model . . . . . . . . . . . . . . .

7.2.2 Solution Using Green’s Theorem When

Q Is Large . . . . . . . . . . . . . . . . . . . .

7.2.3 Solution When Q Is Small . . . . . . . . . . . .

7.2.4 Solution When T Is Inﬁnite . . . . . . . . . . .

.

.

.

.

.

225

226

226

228

232

235

.

236

.

.

.

237

245

247

.

.

.

.

.

259

259

260

262

267

268

8 The Maximum Principle: Discrete Time

8.1 Nonlinear Programming Problems . . . . . . .

8.1.1 Lagrange Multipliers . . . . . . . . . . .

8.1.2 Equality and Inequality Constraints . .

8.1.3 Constraint Qualiﬁcation . . . . . . . . .

8.1.4 Theorems from Nonlinear Programming

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

202

204

205

206

206

208

213

214

214

xviii

8.2

8.3

CONTENTS

A Discrete Maximum Principle . . . . . . . . . . .

8.2.1 A Discrete-Time Optimal Control Problem

8.2.2 A Discrete Maximum Principle . . . . . . .

8.2.3 Examples . . . . . . . . . . . . . . . . . . .

A General Discrete Maximum Principle . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

269

269

270

272

276

.

.

.

.

.

.

.

.

.

.

283

284

284

285

287

289

.

.

.

.

.

.

.

.

9 Maintenance and Replacement

9.1 A Simple Maintenance and Replacement Model . . .

9.1.1 The Model . . . . . . . . . . . . . . . . . . .

9.1.2 Solution by the Maximum Principle . . . . .

9.1.3 A Numerical Example . . . . . . . . . . . . .

9.1.4 An Extension . . . . . . . . . . . . . . . . . .

9.2 Maintenance and Replacement for

a Machine Subject to Failure . . . . . . . . . . . . .

9.2.1 The Model . . . . . . . . . . . . . . . . . . .

9.2.2 Optimal Policy . . . . . . . . . . . . . . . . .

9.2.3 Determination of the Sale Date . . . . . . . .

9.3 Chain of Machines . . . . . . . . . . . . . . . . . . .

9.3.1 The Model . . . . . . . . . . . . . . . . . . .

9.3.2 Solution by the Discrete Maximum Principle

9.3.3 Special Case of Bang-Bang Control . . . . . .

9.3.4 Incorporation into the Wagner-Whitin

Framework for a Complete Solution . . . . .

9.3.5 A Numerical Example . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

290

291

293

296

297

297

299

301

. .

. .

301

302

10 Applications to Natural Resources

10.1 The Sole-Owner Fishery Resource Model . .

10.1.1 The Dynamics of Fishery Models . .

10.1.2 The Sole Owner Model . . . . . . .

10.1.3 Solution by Green’s Theorem . . . .

10.2 An Optimal Forest Thinning Model . . . .

10.2.1 The Forestry Model . . . . . . . . .

10.2.2 Determination of Optimal Thinning

10.2.3 A Chain of Forests Model . . . . . .

10.3 An Exhaustible Resource Model . . . . . .

10.3.1 Formulation of the Model . . . . . .

10.3.2 Solution by the Maximum Principle

.

.

.

.

.

.

.

.

.

.

.

311

312

312

313

314

317

317

318

321

324

324

327

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

CONTENTS

11 Applications to Economics

11.1 Models of Optimal Economic Growth . . . . . . .

11.1.1 An Optimal Capital Accumulation Model

11.1.2 Solution by the Maximum Principle . . .

11.1.3 Introduction of a Growing Labor Force . .

11.1.4 Solution by the Maximum Principle . . .

11.2 A Model of Optimal Epidemic Control . . . . . .

11.2.1 Formulation of the Model . . . . . . . . .

11.2.2 Solution by Green’s Theorem . . . . . . .

11.3 A Pollution Control Model . . . . . . . . . . . .

11.3.1 Model Formulation . . . . . . . . . . . . .

11.3.2 Solution by the Maximum Principle . . .

11.3.3 Phase Diagram Analysis . . . . . . . . . .

11.4 An Adverse Selection Model . . . . . . . . . . . .

11.4.1 Model Formulation . . . . . . . . . . . . .

11.4.2 The Implementation Problem . . . . . . .

11.4.3 The Optimization Problem . . . . . . . .

11.5 Miscellaneous Applications . . . . . . . . . . . .

xix

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

335

335

336

336

338

339

343

343

344

346

347

348

349

352

352

353

354

360

.

.

.

.

.

.

365

366

370

372

375

377

383

.

.

.

.

385

386

387

388

388

. .

389

. .

of

. .

392

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

12 Stochastic Optimal Control

12.1 Stochastic Optimal Control . . . . . . . . . . . . . . .

12.2 A Stochastic Production Inventory Model . . . . . . .

12.2.1 Solution for the Production Planning Problem

12.3 The Sethi Advertising Model . . . . . . . . . . . . . .

12.4 An Optimal Consumption-Investment Problem . . . .

12.5 Concluding Remarks . . . . . . . . . . . . . . . . . . .

13 Diﬀerential Games

13.1 Two-Person Zero-Sum Diﬀerential Games . . . . . .

13.2 Nash Diﬀerential Games . . . . . . . . . . . . . . . .

13.2.1 Open-Loop Nash Solution . . . . . . . . . . .

13.2.2 Feedback Nash Solution . . . . . . . . . . . .

13.2.3 An Application to Common-Property Fishery

Resources . . . . . . . . . . . . . . . . . . . .

13.3 A Feedback Nash Stochastic Diﬀerential

Game in Advertising . . . . . . . . . . . . . . . . . .

13.4 A Feedback Stackelberg Stochastic Diﬀerential Game

Cooperative Advertising . . . . . . . . . . . . . . . .

.

.

.

.

395

xx

CONTENTS

A Solutions of Linear Diﬀerential Equations

A.1 First-Order Linear Equations . . . . . . . . . . . . . . .

A.2 Second-Order Linear Equations with

Constant Coeﬃcients . . . . . . . . . . . . . . . . . . . .

A.3 System of First-Order Linear Equations . . . . . . . . .

A.4 Solution of Linear Two-Point Boundary Value Problems

A.5 Solutions of Finite Diﬀerence Equations . . . . . . . . .

A.5.1 Changing Polynomials in Powers of k into

Factorial Powers of k . . . . . . . . . . . . . . . .

A.5.2 Changing Factorial Powers of k into Ordinary

Powers of k . . . . . . . . . . . . . . . . . . . . .

409

409

410

410

413

414

415

416

B Calculus of Variations and Optimal Control Theory

419

B.1 The Simplest Variational Problem . . . . . . . . . . . . 420

B.2 The Euler-Lagrange Equation . . . . . . . . . . . . . . . 421

B.3 The Shortest Distance Between Two Points on the Plane 424

B.4 The Brachistochrone Problem . . . . . . . . . . . . . . . 424

B.5 The Weierstrass-Erdmann Corner

Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 427

B.6 Legendre’s Conditions: The Second Variation . . . . . . 428

B.7 Necessary Condition for a Strong

Maximum . . . . . . . . . . . . . . . . . . . . . . . . . . 429

B.8 Relation to Optimal Control Theory . . . . . . . . . . . 430

C An Alternative Derivation of the Maximum Principle 433

C.1 Needle-Shaped Variation . . . . . . . . . . . . . . . . . . 434

C.2 Derivation of the Adjoint Equation and the Maximum

Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

D Special Topics in Optimal Control

D.1 The Kalman Filter . . . . . . . . . . . . . . . . . . .

D.2 Wiener Process and Stochastic Calculus . . . . . . .

D.3 The Kalman-Bucy Filter . . . . . . . . . . . . . . . .

D.4 Linear-Quadratic Problems . . . . . . . . . . . . . .

D.4.1 Certainty Equivalence or Separation Principle

D.5 Second-Order Variations . . . . . . . . . . . . . . . .

D.6 Singular Control . . . . . . . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

441

441

444

447

448

451

452

454

CONTENTS

D.7 Global Saddle Point Theorem . . . . . . . . . . . . . . .

D.8 The Sethi-Skiba Points . . . . . . . . . . . . . . . . . . .

D.9 Distributed Parameter Systems . . . . . . . . . . . . . .

xxi

456

458

460

E Answers to Selected Exercises

465

Bibliography

473

Index

547

List of Figures

1.1

1.2

1.3

1.4

The Brachistochrone problem . . .

Illustration of left and right limits

A concave function . . . . . . . . .

An illustration of a saddle point . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

9

18

21

23

2.1

2.2

2.3

2.4

2.5

2.6

2.7

2.8

An optimal path in the state-time space . . . . . . . .

Optimal state and adjoint trajectories for Example 2.2

Optimal state and adjoint trajectories for Example 2.3

Optimal trajectories for Examples 2.4 and 2.5 . . . . .

Optimal control for Example 2.6 . . . . . . . . . . . .

The ﬂowchart for Example 2.8 . . . . . . . . . . . . .

Solution of TPBVP by excel . . . . . . . . . . . . . . .

Water reservoir of Exercise 2.18 . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

34

44

46

48

53

58

60

63

3.1

3.2

State and adjoint trajectories in Example 3.4 . . . . . .

Minimum time optimal response for Example 3.6 . . . .

93

101

4.1

4.2

4.3

4.4

4.5

Feasible state space and optimal state trajectory

for Examples 4.1 and 4.4 . . . . . . . . . . . . . .

State and adjoint trajectories in Example 4.3 . .

Adjoint trajectory for Example 4.4 . . . . . . . .

Two-reservoir system of Exercise 4.8 . . . . . . .

Feasible space for Exercise 4.28 . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

128

143

147

151

157

5.1

5.2

5.3

5.4

5.5

5.6

Optimal policy shown in (λ1 , λ2 ) space .

Optimal policy shown in (t, λ2 /λ1 ) space

Case A: g ≤ r . . . . . . . . . . . . . . .

Case B: g > r . . . . . . . . . . . . . . .

Optimal path for case A: g ≤ r . . . . .

Optimal path for case B: g > r . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

163

164

169

170

174

179

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

xxiii

xxiv

5.7

5.8

LIST OF FIGURES

Solution for Exercise 5.4 . . . . . . . . . . . . . . . . . .

Adjoint trajectories for Exercise 5.5 . . . . . . . . . . .

186

187

Solution of Example 6.1 with I0 = 10 . . . . . . . . . . .

Solution of Example 6.1 with I0 = 50 . . . . . . . . . . .

Solution of Example 6.1 with I0 = 30 . . . . . . . . . . .

Optimal production rate and inventory level with diﬀerent

initial inventories . . . . . . . . . . . . . . . . . . . . . .

6.5 The price trajectory (6.56) . . . . . . . . . . . . . . . . .

6.6 Adjoint variable, optimal policy and inventory in the

wheat trading model . . . . . . . . . . . . . . . . . . . .

6.7 Adjoint trajectory and optimal policy for the wheat trading model . . . . . . . . . . . . . . . . . . . . . . . . . .

6.8 Decision horizon and optimal policy for the wheat trading

model . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6.9 Optimal policy and horizons for the wheat trading model

with no short-selling and a warehouse constraint . . . .

6.10 Optimal policy and horizons for Example 6.3 . . . . . .

6.11 Optimal policy and horizons for Example 6.4 . . . . . .

199

199

200

6.1

6.2

6.3

6.4

7.1

7.2

7.12

7.13

7.14

Optimal policies in the Nerlove-Arrow model . . . . . .

A case of a time-dependent turnpike and the nature of

optimal control . . . . . . . . . . . . . . . . . . . . . . .

A near-optimal control of problem (7.15) . . . . . . . . .

Feasible arcs in (t, x)-space . . . . . . . . . . . . . . . .

Optimal trajectory for Case 1: x0 ≤ xs and xT ≤ xs . .

Optimal trajectory for Case 2: x0 < xs and xT > xs . .

Optimal trajectory for Case 3: x0 > xs and xT < xs . .

Optimal trajectory for Case 4: x0 > xs and xT > xs . .

Optimal trajectory (solid lines) . . . . . . . . . . . . . .

Optimal trajectory when T is small in Case 1: x0 < xs

and xT > xs . . . . . . . . . . . . . . . . . . . . . . . . .

Optimal trajectory when T is small in Case 2: x0 > xs

and xT > xs . . . . . . . . . . . . . . . . . . . . . . . . .

Optimal trajectory for Case 2 of Theorem 7.1 for Q = ∞

Optimal trajectories for x(0) < x

ˆ . . . . . . . . . . . . .

Optimal trajectory for x(0) > x

ˆ . . . . . . . . . . . . . .

8.1

8.2

Shortest distance from point (2,2) to the semicircle . . .

Graph of Example 8.5 . . . . . . . . . . . . . . . . . . .

7.3

7.4

7.5

7.6

7.7

7.8

7.9

7.10

7.11

204

207

209

212

215

216

218

219

230

231

233

238

240

241

241

242

243

243

244

244

249

250

266

267

LIST OF FIGURES

xxv

8.3

8.4

Discrete-time conventions . . . . . . . . . . . . . . . . .

∗

Optimal state xk and adjoint λk . . . . . . . . . . . . .

270

275

9.1

9.2

Optimal maintenance and machine resale value . . . . .

Sat function optimal control . . . . . . . . . . . . . . . .

289

291

10.1 Optimal policy for the sole owner ﬁshery model . . . . .

10.2 Singular usable timber volume x

¯(t) . . . . . . . . . . . .

10.3 Optimal thinning u∗ (t) and timber volume x∗ (t) for the

forest thinning model when x0 < x

¯(t0 ) . . . . . . . . . .

10.4 Optimal thinning u∗ (t) and timber volume x∗ (t) for the

chain of forests model when T > tˆ . . . . . . . . . . . .

10.5 Optimal thinning and timber volume x∗ (t) for the chain

of forests model when T ≤ tˆ . . . . . . . . . . . . . . . .

10.6 The demand function . . . . . . . . . . . . . . . . . . . .

10.7 The proﬁt function . . . . . . . . . . . . . . . . . . . . .

10.8 Optimal price trajectory for T ≥ T¯ . . . . . . . . . . . .

10.9 Optimal price trajectory for T < T¯ . . . . . . . . . . . .

316

320

11.1

11.2

11.3

11.4

11.5

11.6

11.7

Phase diagram for the optimal growth model .

Optimal trajectory when xT > xs . . . . . . . .

Optimal trajectory when xT < xs . . . . . . .

Food output function . . . . . . . . . . . . . . .

Phase diagram for the pollution control model .

Violation of the monotonicity constraint . . . .

Bunching and ironing . . . . . . . . . . . . . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

320

322

323

324

326

329

330

.

.

.

.

.

.

.

340

346

347

348

351

358

359

12.1 A sample path of optimal production rate It∗ with

I0 = x0 > 0 and B > 0 . . . . . . . . . . . . . . . . . . .

374

13.1 A sample path of optimal market share trajectories . . .

13.2 Optimal subsidy rate vs. (a) Retailer’s margin and (b)

Manufacturer’s margin . . . . . . . . . . . . . . . . . . .

396

B.1 Examples of admissible functions for the problem . . . .

B.2 Variation about the solution function . . . . . . . . . . .

B.3 A broken extremal with corner at τ . . . . . . . . . . . .

420

421

428

404

xxvi

LIST OF FIGURES

C.1 Needle-shaped variation . . . . . . . . . . . . . . . . . .

C.2 Trajectories x∗ (t) and x(t) in a one-dimensional case . .

434

434

D.1 Phase diagram for system (D.73) . . . . . . . . . . . . .

D.2 Region D with boundaries Γ1 and Γ2 . . . . . . . . . . .

457

461

List of Tables

1.1

1.2

1.3

The production-inventory model of Example 1.1 . . . .

The advertising model of Example 1.2 . . . . . . . . . .

The consumption model of Example 1.3 . . . . . . . . .

3.1

3.2

3.3

Summary of the transversality conditions

State trajectories and switching curves . .

Objective, state, and adjoint equations for

types . . . . . . . . . . . . . . . . . . . . .

4

6

8

. . . . . . . .

. . . . . . . .

various model

. . . . . . . .

89

100

Characterization of optimal controls with c < 1 . . . . .

168

13.1 Optimal feedback Stackelberg solution . . . . . . . . . .

403

A.1 Homogeneous solution forms for Eq. (A.5) . . . . . . . .

A.2 Particular solutions for Eq. (A.5) . . . . . . . . . . . . .

411

411

5.1

111

xxvii

Chapter 1

What Is Optimal Control

Theory?

Many management science applications involve the control of dynamic

systems, i.e., systems that evolve over time. They are called continuoustime systems or discrete-time systems depending on whether time varies

continuously or discretely. We will deal with both kinds of systems in this

book, although the main emphasis will be on continuous-time systems.

Optimal control theory is a branch of mathematics developed to ﬁnd

optimal ways to control a dynamic system. The purpose of this book is

to give an elementary introduction to the mathematical theory, and then

apply it to a wide variety of diﬀerent situations arising in management

science. We have deliberately kept the level of mathematics as simple as

possible in order to make the book accessible to a large audience. The

only mathematical requirements for this book are elementary calculus,

including partial diﬀerentiation, some knowledge of vectors and matrices, and elementary ordinary and partial diﬀerential equations. The last

topic is brieﬂy covered in Appendix A. Chapter 12 on stochastic optimal control also requires some concepts in stochastic calculus, which are

introduced at the beginning of that chapter.

The principle management science applications discussed in this book

come from the following areas: ﬁnance, economics, production and inventory, marketing, maintenance and replacement, and the consumption

of natural resources. In each major area we have formulated one or more

simple models followed by a more complicated model. The reader may

© Springer Nature Switzerland AG 2019

S. P. Sethi, Optimal Control Theory,

https://doi.org/10.1007/978-3-319-98237-3 1

1

## Ebook An introduction to environmental chemistry and pollution (3rd edition) Part 1

## Ebook An introduction to environmental chemistry and pollution (3rd edition) Part 2

## Spreadsheet modeling and decision analysis a practical introduction to management science revised book only

## A PRACTICAL GUIDE TO MANAGEMENT TRAINING AND DEVELOPMEN

## An introduction to management science 13th edition anderson test bank

## An introduction to management science quantitative approaches to decision making 14th edition anderson test bank

## An introduction to management science quantitative approaches to decision making revised 13th edition anderson test bank

## Introduction to management science 11th edition taylor test bank

## Introduction to management science 12th edition taylor test bank

## Introduction to management science a modeling and case studies approach with spreadsheets 5th edition hillier test bank

Tài liệu liên quan