Hacker News new | ask | show | jobs
by thecity2 3537 days ago
Curious since you say you work on deep learning/optimization. I'm a data scientist who does most everything in Python, but I've been interested in learning Rust. What is the state of "data science" toolkit for Rust at this point? For reference, I regularly use sklearn, Pandas, and Spark and would be looking for similar tools.
1 comments

I'm not sure that there's a mature Rust alternative to "all of the above" quite yet. Some things that I've had to build for my own work (not always "built," sometimes I just wrote FFI bindings) include:

* N-D dense arrays, fashionably simply called tensors these days ;) (this I wrote from scratch to suit my own needs, but there are several implementations of this in Rust, as well as in C++)

* linear algebra (bindings to BLAS, LAPACK etc.)

* numerical integration (bindings to GSL or QUADPACK)

* optimization and machine learning (this I work on day-to-day)

* neural networks (I also work on this)

* multicore/manycore/GPU/distributed support (and this)

There's also the more practical aspects of using Pandas/iPython (Jupyter)/Spark for data science, including ETL, interactive work, and plotting, which IMO are not quite there yet in the Rust ecosystem. But I think that a lot of the pieces are there, e.g. bleeding fast CSV parsing libraries (https://github.com/BurntSushi/rust-csv), and what's left is the important last mile of bringing these things together in ergonomic libraries or frameworks.