Hacker News new | ask | show | jobs
by Jugurtha 2078 days ago
>The only friction I see with data analysis roles is that data analysts typically don't need (or even outright refuse) to do collaborative work where there are multiple people working on the same code and data.

In your experience, what do you attribute this to?

One way to look at it is that programmers tend to see notebooks as code, and think it makes sense to use the tooling used for code.

Another way to look at it is as a document, and build around that.

Would fiction writers benefit from learning Git ? Sure. Could it be more productive to reduce the activation energy to accomplish the job of tracking changes with another implementation ? I think yes. There are more writers who use Google Docs than there are who use Git.

For example, we added collaborative editing on notebooks[0]. Multiple people can work simultaneously on the same notebook, see each other cursors and changes live. It's a much better experience. We also added multiple checkpoints behind the scenes[1].

I believe that as programmers, we have the opportunity to create better abstractions around workflows to simplify what we take for granted and do as second nature for others.

One issue is cognitive load. People having to remember to do things. We strive to make it automatic. One example is for experiment tracking: sure, they 'just' have to track experiments and add the necessary code, but we do it automatically so it happens without them remembering. Models are detected are saved, parameters and metrics are logged, etc. This considerably reduces the barrier to entry as even a beginner who knows nothing about tracking gets their experiments tracked.

This is what we're doing. Handle complexity on behalf of a user, and reduce the skills required to get things right. Same for deployment. Can they learn Docker? Sure. Why do they have to, though. So we automatically build model images for them, and they can deploy them themselves clicking a button. As I said, these are problems we've had in the past, for example a 'data scientist' having to sollicit the help of à colleague to deploy their model. This was a bottleneck for us.

[0]: https://iko.ai/docs/notebook/#collaboration

[1]: https://iko.ai/docs/notebook/#multiple-checkpoints