[minimaxir]

> WASM-based dimension reduction viz

> Implement a dimension reduction algorithm in Rust and compile to WASM and integrate the WASM code with Qdrant Web UI.

Easy, just use a Rust crate to fit a PCA (https://crates.io/crates/pca), then at runtime do a matmul between the fitted matrix and the embeddings to get it reduced. :P

Speaking of which, there's a surprising spike in downloads for that crate on the date this blog post was made.

It's not as simple in practice, and even popular dimensionality reduction techniques like UMAP require you to reference the original dataset which is infeasible for large datasets. The hacky approach that would be good for production use (maybe not "just want to visualize 2D embeddings because they look cool") would be to train a small Parametric UMAP model (with likely a non-Rust implementation: https://umap-learn.readthedocs.io/en/latest/parametric_umap....), then convert the trained model to ONNX for WASM.

[olliej]

“Easy, just…” lol I do like the joke (and I appreciate you pointing out that in reality things aren’t necessarily _that_ simple), but as I understand it these programs (internships?) are targeted mostly at young folk/undergrads who are still learning the core skills, in which case projects that are conceptually easy but require you learn the skills to actually implement the conceptually easy task within the confines of real world constraints feels reasonably appropriate?

I could be wrong, because it’s not really a field I do anything in but based on what you said it’s sounds about right for a first “real project” task an intern would do at a big tech summer internship (this is as someone who has helped summer interns over the years, but never did one myself so it’s based on seeing what tasks are given to someone in their first ever internship vs people who have interned previously)