Hacker News new | ask | show | jobs
by markhalonen 459 days ago
I suggest https://perspective.finos.org/ for data viz to be built in. We use DuckDB paired with Perspective for client-side BI use case, and it's been great.
5 comments

+1

we're using Perspective in crabwalk[0] (it's like dbt specifically built for duckdb and written in rust) and it's amazing paired with duckdb. Near instant loads for hundreds of thousands of rows and you can keep everything in arrow.

0 - https://github.com/definite-app/crabwalk

Where are you using/advocating crabwalk?

It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of SQL scripts. Presumably the all-local case needs less complexity. Unless the idea is that this will eventually support more connectors/backends and work as a full dbt replacement?

A few features:

* Built-in column level lineage (i.e. dump in 20 .sql files and crabwalk automatically figures out lineage)

* Visualize the lineage

* Clean handling of input / output (e.g. simply specify @config output and you can export results to parquet, csv, etc.)

* Tests are not yet implemented, but crabwalk will have built-in support for tests (e.g. uniqueness, joins, etc.)

we're using it in our product (https://www.definite.app/), but only for lineage right now.

Glad you dig it! Check out our pro version to - it also support DuckDB, Python/Pyodide and more! https://prospective.co
Wow that's really cool! Part of my PhD thesis was about writing stable treemapping algorithms for temporal data. The idea being that you want your treemap cells not to fly around like what I'm seeing in your demo, but to remain more or less in the same position without sacrificing too much on the cells aspect ratios. We've come up with a pretty effective and fast method to do that, check out the paper and a demo down below. Maybe we could even do a collaboration to get this implemented in perspective.

https://github.com/EduardoVernier/eduardovernier.github.io/b...

https://youtu.be/Bf-MRxhNMdI?list=PLy5Y4CMtJ7mKaUBrSZ3YgwrFY... (see the GIT method)

That looks much better, thanks I will read up.
Have a look at https://sql-workbench.com eventually, as it's using DuckDB WASM & Perspective to render the query results. Let me know what you think!
This is actually how I discovered Perspective!
Hahaha, nice. It's a small world.
The online demo looks great and promising, too bad it's unusable for me. I've tried installing it with conda from conda-forge and no luck. I've tried installing it with pip, the same. I've also cloned the repository from github, tried to build it and failed, but I don't remember the details.

Why is some software so difficult to install beats me.

Have you ever reported an issue? I use perspective heavily on a variety of platforms both conda and pypi without any problems.
Not yet, because I wanted to give it one more try while documenting all the steps.
Why Perspective? If going for a D3 wrapper, Plot would offer more flexibility.
We've built a nice integration for Plot + DuckDB, found here: https://www.duckplot.com/!