[graycat]

Part III

In the end, it's super important to be the guy who OWNS a business and SELLS the results. E.g., for optimization, maybe develop the software for free, show the results and the savings, and then ask to get paid a fraction of the savings. Let's see, long ago one commercial airline was spending $89 million a month on jet fuel. I can believe $200 million a month now, also for FedEx. From a fast Google, get to

     http://www.transtats.bts.gov/fuel.asp

with data for US airlines

     2013 April 847.5 2,432.4

or 847.5 million gallons costing $2,432.4 million dollars in April, 2013.

Save 15% of $200 million a month and get paid 20% of the savings and get paid $6 million a month, from just one customer. And it's an easy sale: Take case A, what they are doing now, and cost it out. Then take case B, from optimization. Then compare costs. Simple. Compelling. Maybe not still compelling now, but would have been for much of the history of FedEx.

But, for my doing an in-house effort, Smith didn't take the work very seriously. Right, in the next year I might have burned off $50,000 in VM/CMS time sharing computer charges. Right. But jet fuel is expensive, too.

> it does appear that Simplex is covered

Yes, of course, the course will have to, but the introductory lecture didn't emphasize that.

In a sense, simplex is dirt simple -- just elementary row operations very much like in Gaussian elimination but, using the 'reduced costs', selected to make money at each iteration in, if you wish, a 'greedy' way. But there's more, e.g., some surprising points about moving along edges of a closed convex set, from an intersection of finitely many closed half spaces, from extreme point to extreme point. For the course, discrete and combinatorial optimization, really should know simplex quite well; it promises to be the core of the course. Also, again, simplex is worst case exponential!

For the career prospects of the course, only a tiny fraction of college courses have good, immediate, direct career contributions. So, I can't be offended that the OP's course does not have really good career contributions. But I am offended that the OP tried to claim that his optimization was so important that there would be good career prospects. Sure, Bixby (of C-PLEX) bought a nice house in Houston, but mostly I'm still looking for the yachts, or even the nice houses, of optimization experts. Heck, even job ads would be reassuring.

I have one of the best Ph.D. degrees in optimization, and it has been essentially useless for my career. The core reason is, the business guys with the projects and budgets don't understand optimization, have no respect for it, and don't want to bet part of their careers on it. There's usually little or no downside for ignoring optimization. For pushing a project in optimization and failing, there's a lot of downside. For being successful, there will likely be resentment, attacks from other managers who feel threatened, etc. and otherwise no great upside. So optimization projects are about as popular as a skunk at a garden party.

One final war story: Long the dean of engineering at MIT was T. Magnanti, an expert in optimization. Once he gave a Goldman Lecture at Hopkins on optimization of the design of large IP networks. From some old Bell Labs data (from some work likely close to the book with the cartoon), optimization should be able to save ballpark 15% of network capital expense; worthwhile money if can get it.

So, at one time there was a start up in Plano, TX attacking this problem. At the time, so was I. So, the TX people flew me down for an interview. They had some venture funding, and it may be that some of the people who met me were the venture partners. The company's main optimization guy from SMU had just bowed out. The CEO was a former IBM guy, and they flew me down partly because of my role at FedEx and also because I'd been at IBM's Watson lab.

So, I arrive. I'm met at the door by the CEO, the IBM guy. Right away he scowls, and I never see him again. Why? Because I didn't know his name (I'd had no communications with him), and my handshake was not impressive enough. He desperately needed some good expertise in optimization, e.g., in a back room had a high end PC with a copy of C-PLEX gathering dust while his people were trying total enumeration, but he wanted nothing to do with me. I'm not that hard to take, not even while tired from a plane trip and driving from the Dallas airport to Plano.

The point was, he was convinced that his IBM white shirt and IBM salesman handshake were what were crucial for his company and that my background in optimization was, well, whatever but likely not really good like a handshake. He had no respect for optimization. Soon he was 'promoted' to just a Board slot.

My background in optimization? Did I mention Goldman? He was the Chair of the committee that approved my Ph.D. research. On the committee was C. ReVelle, world expert in optimization for facility location (mentioned by the OP). Also on the committee was J. Cohon, world expert in multi-objective optimization and long President at CMU. My research was from a suggestion in three words by G. Nemhauser, world expert in optimization. One paper I'd published solved some long outstanding questions in the Kuhn-Tucker conditions in optimization and solved a problem stated in the famous paper by Arrow, Hurwicz, and Uzawa. I went through one of the best Ph.D. programs in optimization on the planet. Still, the CEO in TX wanted nothing to do with me. And, really, after my Ph.D., neither did FedEx.

When I left FedEx, I'd saved the company twice and for more had identified, formulated, done good first-cut progress on, and presented the three optimization problems, fleet scheduling, fuel buying, and vertical flight planning. All I needed was a little consulting money and a good VM/CMS time sharing account from my home in Maryland, but that was not enough to get Smith impressed. So, I lost, and so did Smith and FedEx.

In business, optimization is a Rodney Dangerfield field -- it "don't get no respect". So, if exploit optimization, then do it for your own company or sell just the results based on the savings obtained: Since the 'suits' are convinced that optimization can't save much, when the contract is signed they will believe that they won't have to pay much. When they have to pay $6 million a month, they will be surprised and pleased by how much they are saving but bitter and furious at the $6 million a month they are signed up to pay.

There is a secret in business: To get paid well without too much resentment, jealousy, etc. from others, get paid in ways that others don't really know how much you are making. So, if some big company has to pay you $6 million a month, even if you are saving them $30 million a month, they likely will be torqued; but if can get the $6 million a month from several ad networks from running many millions of ads, then no one will get torqued.

[amhey]

graycat - remember this is a 7-week course and the lecturer, Pascal Van Hentenryck, in the initial video is stirring up interest imitating Indiana Jones - he's on a quest, he's passionate. So it's not the Hillier and Lieberman textbook approach to Operations Research of old. The whole point was to make the difficult problem of combinatorial problem solving fun and attractive to a wide audience.

I agree with you that many managers haven't a clue what to do with an OR graduate. So it does depend on how you position yourself. One reason for taking the course is to be able to promote your skills passionately (try making a video of your skills in every day parlance like the introductory video to this lecture) using current terminology - which as Prof Van Hentenryck says are in demand at INFORMS conferences (go to the analytics conference - rather than the main conference which is more academic).

I agree with you that many execs haven't a clue what optimization can do - or care - in fact most managers satisfice. That's why it's important to find the right boss who does value your accomplishments. It's also important to update how you state your value to others - which is one reason I'm doing the course. I hope you stick with it - and I'd be interested to know what you think at the end of it.

Is the point of your argument that the leaderboard ranking forces people to try for an optimal solution, when in a business situation a solution at 99% of the value might be good enough?

[graycat]

The Indiana Jones take off is fine. Mentioning the knapsack problem is less good because it's not so important in practice. Saying that the knapsack problem is difficult to solve, e.g., encounters exponential algorithms because technically it's in NP-complete, is next to irrelevant for practice, misleading, and hype and not fine.

> it's also important to update how you state your value to others

On this, I outlined my suggestion: Own a little company and sell results based on how much money they save the customer. Make the sale about saving the customers money in ways that even an auditor can confirm are correct.

INFORMS is clearly an echo chamber, people in optimization looking for work and talking to themselves.

Broadly for optimization in business, there is a very serious problem: Optimization is not a 'profession' like law, medicine, or major parts of engineering. So, there is no licensing, certification such as the CPA, peer-review of practice, legal liability, etc. So, as I said, the field "don't get no respect". Also missing is a point the legal profession has: Any working lawyer must report only to a lawyer; the interface between the optimization guy and the business guy is nearly impossible.

> Is the point of your argument ...

I tried to make several points. One of the points was about 'optimal'. The mathematical definition is fine, but long that definition was taken as suggesting that what we should do in practice is look for such solutions, then strain to find them, etc. That turned out to be a grand mistake.

Why? Because maybe there is, compared with what the customers is doing now, $10 million to be saved with an optimal solution. But too commonly saving all $10 million is too difficult for the algorithms and computing. So, straining to save all the $10 million converts an important business problem into a much more difficult mathematical problem. It also turns out that commonly it's fairly easy to save, say, $9 million. The difficulty is saving the last of the money, and the most difficult money to save is the last, say, the last 10 cents.

'Optimal' was taken as a moral objective, as I said, as if saving the last 10 cents was worth much more than 10 cents.

Struggling over 'optimal' taken literally and, thus, making real problems much more difficult than necessary, was several torpedoes below the waterline of the ship of optimization.

Part of this mistake was the simplistic and excessive emphasis on NP-completeness -- for real problems the whole P versus NP question is next to irrelevant. One way to see this is the simplex algorithm -- it's the core of optimization and astoundingly fast in practice but worst case exponential. There is a polynomial algorithm for linear programming, but it's way too slow in practice. In practice, that an algorithm is worst case exponential is commonly just irrelevant.

I had to conclude that for business, optimization is a dead field. It got started due to WWII and US DoD funding, and maybe in places there is still some interest for US DoD problems.

Here is a little: A post above, in response to a post of mine, claimed that IBM had a good optimization group. If so, then good for IBM. But I was at IBM's Watson lab, published a paper on optimization, and off and on considered joining the optimization group there. Phil Wolfe, William Pulleyblank, Ellis Johnson. and others were in that group. At one point, Roger Wets was visiting. The group did the IBM Optimization Subroutine Library (OSL). Then in 3 years near 1994, IBM lost $16 billion. Johnson joined George Nemhauser at Georgia Tech. Pulleyblank became a professor at West Point. Basically the optimization group fell apart. Maybe they put a group back together, but losing Johnson and Pulleyblank were big mistakes.

E.g., again, with Pulleyblank at West Point, the US DoD remains interested in optimization.

Heck, I supported myself and my wife through our Ph.D. degrees by working in optimization for the US DoD.

In academics, the professors were to do research to get the field going, e.g., research in 'systems analysis', 'mathematical sciences', 'civil engineering', 'production', etc. Yes, if optimization problems were easy to solve, then optimization would have central roles in those fields. Alas, mostly important practical optimization problems are not so easy to solve, even approximately. So, the professors are still doing research -- maybe in some decades or centuries they will have something of serious importance for those fields. I doubt that the research is very well supported.

I tried to give a summary of essentially a 'cultural contradiction' expecting optimization to be a popular field in business: By the time computing is ready to make optimization easy enough, there are other things to do with the computing making much more money than with optimization.

It's not that optimization can't save money in business; there is money to be saved; in a lot of stable businesses, optimization can provide some of the highest ROI available to the business. So, there is some ground available there, what is in principle some fertile ground. So, there can be some optimization groups here and there. If the course prof has such a group in Australia, good for him. With some really impressive 'cases' published in, say, INFORMS, maybe mainline business will try optimization again. I doubt it, but maybe. Don't hold your breath waiting; there are lots of impressive cases long since published in INFORMS, and ORSA, Mangement Science, etc. The optimization literature is huge going back to the late 1940s, e.g., for Dantzig at Rand and Berkeley.

Here's a little on the difficulty: In the US there are college accrediting groups, and for some years they said that an undergraduate degree in business should have courses in optimization and statistics. So, for years each business school student, undergraduate or MBA, got a course in optimization. For some years, I taught such courses. Still the field didn't take off.

I can't recommend that anyone try to have a career in optimization in business. You stand to have an easier time supporting a family with a career as a plumber, literally. With software, do an information technology start up, sell out, and pocket, say, $10 million -- knocks the socks off optimization. With irony, if interested in 'optimization' of your career and financial security, then avoid optimization!

Optimization is like some item at Dunkin Donuts that doesn't sell. Lots of other stuff at Dunkin Donuts sells really well, but that one item just doesn't. They can do a good job getting the item ready to eat, put it out in the display cases, and wait, and what happens is the item just sits there and goes stale. Then they throw away the stale, unsold items. It was a waste. Finally, Dunkin Donuts just quits offering the item.

Dunkin Donuts doesn't go on and on about why the item really should sell. Instead, they just listen to the clear message they've gotten from the market and, thus, save having to figure out solid reasons it doesn't sell.

Similarly all across business -- some stuff doesn't sell or doesn't sell very well or sells only a little and then only into a tiny market. Optimization in business is like that -- at best, it's a super tough sale; usually it just doesn't sell.

Optimization, as a field, in business, is a dead duck. F'get about it and pursue something else.

[tadworthington]

Narcissistic babble.

[graycat]

It's not about me, not at all.

It's about the course and optimization as a field, especially in business.

So, I contributed.

Since I've been there, done that, got the T-shirt, I'm able to make some contributions few others can, but to make these contributions I have to draw on some of my personal experiences. It's not about me or "narcissistic".

And, it's not "babble": Instead, in some situations it's darned valuable information, that I very much wish I'd had long ago. With that information, I would have avoided trying to have a career in optimization. Much of my Ph.D. is in optimization; some of the rest of my Ph.D. may yet prove to be valuable, but the optimization part was essentially a waste. Yes, we expect some useless chaff in with the good wheat, but still we don't like it. Optimization cost me a lot.

Broadly optimization is a siren song, especially for people with at least one foot in computing. Since in principle and sometimes in pracitice the field can save money, enough to give quite high ROI, the song sounds good. Alas, mostly the song is not good but an invitation to disaster, to taking a career into a swamp.

I gave you some rare and good insight that could be quite valuable; ignore it if you wish.

"Experience is the great teacher, and some will learn from no other".

I learned about optimization from experience; it's the very expensive way to learn; if you throw away what I reported from my experience, then you get to take the expensive way, also.