[shoo]

> Tool usage: I guess some programmers are evaluated the same way; basically, you develop a tool for co-workers to perform analyses with

> often difficult to develop a data science tool covering many use-cases

yes ; yes. if you find the right niche a tool can be very valuable, and it should be possible to estimate or compare the value provided by the tool to the existing process without using the tool.

I worked in a domain where software was used to automate or optimise business decisions as part of a large, expensive construction project. Some components of the work that my colleagues & I did could arguably be framed as data-science (more accurately operations research), although a lot of the work was just software development. Occasionally there were small consulting projects for clients where the output of the project was a report summarising some modelling/simulation with recommendations.

The bulk of the work was building software tools used by the client to automate and optimise business decisions. The value of such tools could be evaluated in a few of obvious ways: How much labour cost did the tool save the client by automating away previously manual processes? How much value did the tool provide the client by making better business decisions than the previous process? How much incidental value did the tool provide by forcing standarisation of previously ad-hoc processes? (e.g. capturing data required as inputs, data quality...) How much did the client pay for the tool? (the above points would inform this one!)

The tools I worked on were used as part of the planning / design process. When they were effective, these tools directly identified designs that would be cheaper to construct than designs produced by the previous process. The value of these construction savings could be estimated and was much larger than the value from automating previous manual work. In at least one case, prior to a sale to a very major client, there was a benchmark & comparison done between the client's existing process and the new process using the proposed tool as part of the business case to fund the sale of the product & related integration work.