[clausok]

I know what that voodoo may be. Of the many failed "replace Excel" projects I've seen in the investment banking \ hedge fund world over 20 years, the usual mistake is that the focus was on the specific Excel model itself and not the generic building capabilities that Excel provides. An Excel model, i.e., the specific map from inputs to outputs, is typically fairly easy to replace with another tool. But what you've done is replace just the pants, not the sewing machine. Excel is the pants AND the sewing machine. And that sewing machine part is very hard to replace. So you can end up with the same model in C++, the very same map from inputs to outputs, but the user's ability to make changes and understand the model is greatly diminished. So they say "no thanks" and stick with the Excel model.

[stuntkite]

You summed it up perfectly. Jobs I successfully finished after a few bites of tragically not understanding involved working closely with the Excel department and making sure they could function as usual but extracting data to go into the systems and providing a net gain for those people.

At the end of the day, the people that do the magic that keeps everyones job paying aren't always (or nearly never are) the people that make the contract hire decisions. If you can give those people a tool that lets them 2x their work and be excited about it there's a big net win. If you want them to move to a statically modeled web interface with dynamic tables and they lose all their ability to do everything they know in Excel, everyone loses.

Portability and timing is key if you have to take on that sort of gig. Team buy in might be worth more than that. Never take somebodies sewing machine if you still want that biz to make enough money to pay you.

[clausok]

Agree. If you're selective about which bits and pieces you move away from Excel, and not religious about getting rid of Excel entirely, you can often have a great result.

What trips people up is that Excel models can be deceptively inviting of replacement ideas. They may have many workflows within them that are obviously much easier to accomplish with a different tool. E.g., grouping and applying aggregate functions; filtering; joining; filling missing data, especially with constraints such as filling forward EPS estimates but only into the same fiscal quarter; etc. You could look at an ocean of INDEX(,MATCH(...)) functions in Excel and discover that the same manipulation in Pandas would require just two lines of code. But it is easy to overlook the importance of seemingly simple calculation flows, in the same model, that are uniquely suited for being expressed in Excel, the universal language for describing calculations.

For example, I have seen data analysts move from Excel to Python, replacing their models as they go -- after being wowed by Pandas' data manipulation capabilities -- and then get bogged down for months trying to recreate what Excel's =RTD("BLOOMBERG.RTD","",...) already did for them. And when the portfolio manager tells them, "this number looks wrong", which happens often, they spend half a day dumping data into Excel and building a sheet that's illustrative for the PM.

[stuntkite]

>Excel models can be deceptively inviting of replacement ideas.

In my mind that's actually what is the root at any talk of "Why isn't there an open source alternative for X amazing program?"

As a wiser person than me once said, "It's easy to imagine so I imagine it's easy." Deceptively simple operations in things that have a long legacy of supremacy can just be near impossible to clone even if you can churn out a POC that looks reasonable in 20 minutes. That trap is a big hill a lot of us die on. At one point in my life I swore off ever doing ETL, and now I do it for a living. Seems like that's actually the world now. Sheesh. With MS embracing Python for big data, maybe the next big thing is just what it's always been... Excel.

[zmj]

Excel is the code and its interpreter. You don't want to lose the REPL flow for development, but maybe you can add an AOT compiler for production?

[turkeydonkey]

I worked six years in the IT department of a small domestic manufacturer, and deciding when a model had outgrown Excel and needed to be created as a tool (sometimes just a pretty interface for our ERP system) was one of our main tasks. I hate the hell that users can bring on a company when they misuse Excel but it will never go away.