Hacker News new | ask | show | jobs
by sheetjs 3131 days ago
> my career in business software has still mostly been about converting spreadsheets into more "proper" applications.

Still true for many people in 2017. @patio11 probably said it best:

> Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built.

https://twitter.com/patio11/status/655674551615942657

FWIW we ended up in business software after one of our open source libraries to read and write spreadsheets (https://github.com/sheetjs/js-xlsx, demo http://oss.sheetjs.com/) ended up garnering lots of demand from businesses looking to build those "proper" applications

6 comments

> Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built.

It's a pithy quote, you'll get a very different reaction from a finance or small to med-sized business professional.

Excel is the ultimate maker studio that actually lives up to the promise of "build your own mini-app". I'm not saying it can't be improved on. It's just that the core model of flexibility is so compelling, pushing everyone into a SaaS app is invariably going to take away substantial power from the user.

Excel is an incredible swiss army knife when dealing with one person in one location on one computer, but the warts start showing up when any of those assumptions break.

One computer: once you decide you want to use your smartphone, for all but the most basic sheets there are problems with Excel mobile preview, even with Microsoft's official apps.

One location: Excel's lack of a universal time (showing different times in for users in different timezones) have led to gnarly workarounds. There was an attempt to correct for it in the XLSX format (cell type 'd') but it has its own problems.

One person: Excel has no inherent sync strategy. You are forced to farm out to email (and the inherent filename versioning nonsense) or use a system like SharePoint (which has its own problems you notice when dealing with people in different timezones across the world, like accidental file locking and data loss)

I hear you. I think One Computer and One Person issues have generally been solved by Office Online or Google Spreadsheets. I can't speak to difficulty around lack of universal time nor do I know how important of a requirement that is to most companies.

Any solution, SaaS or otherwise, will have flaws. It might solve Excel's weaknesses really well, but will likely lack its strengths.

Excel is still going strong in 2017 to a large degree because its strengths continue outweigh its weaknesses and that people have found reasonable ways, or add-on tools, to work around it's weaknesses.

It will be interesting to see in the coming decade if niche industry solutions or an even more incredible swiss army knife were to come around and dethrone the incumbent.

Add another 'one': one point in time.

What I mean is that Excel is essentially a write-only language. It's very quick to prototype something for a semi-technical person, but auditing / code-reviewing an Excel sheet later is essentially impossible. The ubiquitous manual loop unrolling doesn't help readability either.

From my experience, this is more dependent on the team/process than Excel. I've worked in investments for several years and auditing Excel sheets was a core, well-run process at several companies I worked for. It was quite easy for smart, non-technical folks to follow formulas and equations to track down why their checks weren't passing.

It also scaled fine to thousands of employees.

What's with all talk wrt Office 365 and web versions and syncing that goes on with that? Is none of that actually the case? I wouldn't be surprised, just don't know.
True but there is a threshold. If the problem is small enough, you can't beat an Excel spreadsheet's flexibility and low startup cost.

Having worked with a number of finance teams, the biggest problem is not when Excel is used as an analytics tool but as a financial database. Excel as a source of truth for financial data is truly terrifying. Even financial audits are still often handled with custom data pulls exchanged over Excel spreadsheets.

Shameless plug: that's why we ended up developing one of those SaaS tools (http://modfin.io). It lets finance teams map their source data to a proper, easily auditable accounting ledger so that they don't need to do their "magic" in Excel.

Why is it terrifying?

I am a programmer and I use csv/Excel as the single-entry financial database for a small healthcare business ($500k revenue). The accountant is fine with it. Filtering and pivot tables are a joy in Excel, and I can still run Python scripts whenever I need (mainly to auto-categorize bank transactions). Most other businesses around here use QuickBooks, which has a proper system of accounts, but then I can't play with the data.

I'm guessing because of 1) lack of built in logging, 2) lack of change control and 2) difficult to automate testing and verification (because of it's relatively unstructured nature).

Excel as the source of truth for a financial system is particularly scary to me b/c it's so easy for someone/anyone to change entries or miscalculate so it's very difficult for me to get confidence in its completeness and correctness.

Completely agree about analytics and scripting, but I'd utilize the CSV/Excel reports as point-in-time analytics that can be tied back a structured source of truth (or a source of truth I can hold liable, e.g. a bank or credit card statement).

Logging: I just save a copy of the spreadsheet every time I make a tax filing. Do I need anything more fine-grained?

Validation: I'm confident that I can catch duplicated rows and mistyped amounts because Excel formulas verify that the sum of all transactions for each account equals the account balance, and that transfers between accounts add up to $0. Do I need anything more?

Point-in-time doesn't work because I need to keep track of info that doesn't show up in the bank statements. I run a Python script that adds pretty-prints names and adds categories, and I also manually enter check details, category adjustments, business purpose of meals and flight tickets, etc.

I don't use a proper database because to do so I'd have to give up Excel's nice features (or worse, switch back and forth between Excel and the database).

> Validation: I'm confident that I can catch duplicated rows and mistyped amounts because Excel formulas verify that the sum of all transactions for each account equals the account balance, and that transfers between accounts add up to $0. Do I need anything more?

I'm sure that the people who made these billion dollar mistakes were pretty confident too... https://www.cio.com/article/2438188/enterprise-software/eigh...

"The error occurred when the accountant omitted the minus sign on a net capital loss of $1.3 billion" (Fidelity)

"There were honest mistakes made in a spreadsheet used in the implementation of a new accounting standard." (Fannie Mae, $1.36 billion loss)

There's a bunch of lower (million dollar) losses listed in the linked page too.

I don't doubt that people have made mistakes involving spreadsheets. But these mistakes are not unique to spreadsheets. Switching to an enterprise system doesn't magically prevent data entry problems.[1] Similarly, you're as likely to implement an accounting function incorrectly in Excel as in SQL or Java.

I think that you need to show that Excel requires more data entry, or riskier data entry, or that Excel can't catch as many errors, etc. For now I'm not convinced.

[1] https://www.accountingweb.com/aa/auditing/human-errors-the-t...

This doesn't scale well. If only one person edits the spreadsheet and that person knows how to save copies and make sure there is only one latest version it may work. If you give this to 10 people they will start making copies, saving those with different names ans save the copies on some server or on their own machines and after a while you don't have a single source of truth anymore. And you almost never have logging when many people edit copies of Excel documents.
That's true. Currently I handle scaling by automating nearly all the work. Most routine transactions are automatically categorized without manual input; I just skim the report. We modify 0-2 transactions/day, which is low enough for Dropbox to work well. This wouldn't work for a Fortune 500.
How do you know the account balance total they you are validating your line items against is correct? Is that coming from another accounting system?
I validate account totals against the balance reported by the bank.
Apart from the generic question of the pros and cons of using Excel as a data store and how you solve issues around access control, concurrent access, backups, etc., financial databases have some additional requirements to be properly auditable. For example, one of the biggest differences is that the data should be immutable with only additions allowed. E.g. if you made an error and need to correct it (say you sent the wrong amount of money to someone or simply just had a bug), you should keep the record of the erroneous transaction and reverse it or book an adjustment.

You can try to institute the same controls over Excel, either directly in code or just implicitly in terms of how it's used, but they tend to break down pretty quickly with scale.

No. An domain-specific SaaS app takes all the power away from the people writing their spreadsheet programs. Only one app is needed, a hosted spreadsheet with backup and versioning an replication. The only domain-specific business is in writing plugins.
I agree that purpose-specific tools aren't the answer, unless the question was, "can I have a straight jacket?"

Flip side, my feeling is that, if an analysis was worth doing, it's probably worth doing right. And to do it right, it should probably be done in a way that one can show their work. I like where the R community in particular has gone in dropping the barrier to reproducible analysis to roughly the same place as Excel.

My friend's startup has a project that matches up with a lot of these requirements, called Coda. It's easy to query data and build visualizations, has a spreadsheet data model, and has an easier programming model. Excel did a great job getting people to "program" their spreadsheets though.

I also wonder about google sheets capabilities. It does work so well for many people working on one spreadsheet, except for undo. The major flaw I've seen is undo, because in google sheets it was global undo, not personal undo.

I wrote this in another thread, but I think the next move should be to create an Excel-compatible spreadsheet program with a few additional constraints inspired by programming. Make primary keys required, each column needs a type, etc. These are concepts I’ve successfully explained to non-technical coworkers to great effect. By enforcing them in a tool, I think you would get about 80% of the benefits of true software without all of the overhead that entails.
This actually exists inside excel since 2010, it is called get & transform (known before as power query) and lives in the data tab. It allows the user to mash data from different sources (databases, csv, excel files, current workbook) and perform sql-like opérations : filter, join, etc. In a wisiwyg-record-macro fashion. I find it very intuitive and the steps are easily reproducible as each transformation step is translated into a language derived from F#. It is a game-changer for business users that dare using it, as it is rigorous and precise like a classical SQL flavor, but also usable with zero training.

Since it was a very powerful tool but struggled getting traction, they are using it as the core query tool for power BI (alongside DAX for great dashboarding).

Well done Microsoft.

Airtable does something like this. I don’t know how excel-compatible it is, but the core of data types and something similar to primary keys are enforced in a pretty intuitive way, and basic formulas work fine. Haven’t done much advanced stuff with it.
There's a reason chefs don't use multi-tools; purpose-built tools always beat generic tools when you need to maximize efficiency on a task.
I would say a chef's knife is the ultimate kitchen multi-tool. Having one great knife you really know how to use for cutting everything is much more efficient than a bunch of different knives.
Of course the multitool is the single point of failure. If the chef isn't careful, that one great knife can be the vector that means ALL the customers get salmonella, not just the ones who ordered the chicken!
A nice quote but getting data between SaaS apps is much harder than between Excel sheets, and modifying a SaaS app with that one little thing you need for your particular case is of course impossible.
Getting data between SaaS apps is not only easier, but formats like JSONAPI are far more powerful than pushing csv binaries that may or may not adhere to a spec.
One can see parallels between Excel and Unix command line scripting here.
> Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built.

With new (or prospective) clients that are SMBs, one of the first things I ask is, "so, what are you still using spreadsheets for?"

> Every spreadsheet shared in a business is an angel announcing another SaaS app still needs to be built.

Really? I am working in company which is not handling super-sensitive data (i.e. not a Bank, for example) but I can assure you that if went around proposing to adopt a SAAS for what Excel is used internally I would not get very far.

No matter how good you are as a SAAS provider, our data stay in-house. End of story.

Many people on HN make the jump from "I would not buy that" to "There is not a market for that", but there is, in fact, a market for SaaSifying lots of things.

It's no value judgement if you want to continue using Excel for privacy, compliance, or security reasons. You do you. But SaaS companies get very good at developing privacy, compliance, and security stories, particularly as they move into enterprise. (Below enterprise it's less of a big deal, partially because customers care about them less and partially because small businesses are, as a judgement-free statement of engineering reality, abominably bad at data security.)

Yes - I agree, the point is that most of the time an Excel "app" is solving a very specific problem using data coming from some other (internally managed, usually legacy) system. Like: I run a report, transform the result in .csv with some ultraedit macro, slurp it up in Excel and off I go.

(off I go can often being something like: upload the result to the same or a different legacy system, again in some custom format).

Maybe I can pester my own IT dept. to add an extra option to the report so that it spouts off a .csv directly. That's all, the rest still lives in Excel.

Then I am free to play with data as much as I like (take also in account that in some cases you want these in a Excel just to be able to manipulate them better, while the old app works record by record, you can make changes across a thousand records using the Excel interface... and keep also in mind that this is maybe something you need to do once or twice a year, when you renew your catalog prices or whatever).

Now, what is the cost of:

- going out on the market looking for a SAAS that can get my data exactly in the format I use internally, applies the required transformations and send the result back in a format I can use

- assess that the SAAS vendor is indeed trustable for my desired level of security, SOX compliance, etc.

- add one more vendor to my portfolio of vendors/licenses/purchasing orders

So the reason not to go to SAAS is actually a combination of security/external resource dependency/cost/bureacracy.

Where "cost" is mostly inertia+sunk cost fallacy. I am sure that a SAAS may cost less than the hours spent manually doing all the stuff I mentioned above (including mantaining the Excel spreadsheet) but the latter is a "hidden cost" because it something that happens infrequently and is part of the normal chores of whoever is using the Excel sheet itself.

While introducing a new SAAS app will be an IT cost (to identify/approve it, add a recurring subscription etc.)