Hacker News new | ask | show | jobs
by headcanon 3191 days ago
I've seen this demo at a meetup and its actually quite cool. You use a graphical interface to connect boxes of data transformations, play with the settings, and it will generate a python script that will perform the exact operations you specify to the data in question.

What you're describing are tools for data engineering - sanitizing real-world datasets to be fed into models. Researchers do not have to deal with this task as much because they work with well-defined datasets to provide fair comparisons of the algorithms they develop. Industry is the opposite - the algorithms are usually formulaic and well-defined, but the data itself is not.