Hacker News new | ask | show | jobs
by chambers 1037 days ago
Love the idea of this. Is there a feature to support views or linked records from other CSV files?

Basically, my team is curious if there's a smaller, lighter, local-alternative to Airtable, which we can collaborate on with Git. It's a big ask and maybe not on your roadmap, but something that is on our minds.

4 comments

Have you tried the trick where you put CSV into SQLite?

I do that. Many, maybe linked, CSVs. Load all into SQLIte, query, filter, sort, etc.

In the repos to process we have only scripts to load and filter. The SQLite things are short lived.

Also, memory only SQLite FTW!

Duckdb does this straight from csv. You can treat them pretty much just like un-indexed tables, including queries, joins and output back to csv or parquet. its

Really nice.

That has not been on my roadmap, but I'll think about it. It sounds like it may have to be a separate product, not that that's a bad thing.
Grist was one of the first tools we looked at but it wasn't the desktop-first app my team was hoping for. Self-hosting a service felt like a little too much upkeep, on top of all the other systems we own. I recall the data was also not git diff-able.
Desktop first made me think of Superintendant[0] which I’ve enjoyed using

0: https://superintendent.app/

There's an electron app that doesn't require hosting at all. It's a desktop solution. https://github.com/gristlabs/grist-electron
Git based database? Sounds super interesting. Are you guys actively looking for something or is it a nice to have?
Many moons and a few jobs ago, I rolled an in-house Wiki in python, that was basically a very thin web UI over mercurial. Each page was a checked-in plain text file (I think some sort of markdown flavor, but this was circa 2010, a totally different era!), each edit was a revision/commit. Things like article history just shelled out to mercurial. "Querying" was down via filesystem operations. Rolled the whole thing in a day and a half or so - less than I'd already spent trying to get a MediaWiki instance stood up!
More like a git-based log than database. We use it for requests and questions that come up on our slack. We figured out how to extract threads from slack, transform them into CSV files and then load them into Airtable.

Airtable has a great UI but the data becomes isolated from our ETL script and it means we have to re-download and export to CSV to use in other contexts.

It'd be great if we could load a CSV into a desktop editor, and then have it store certain views/linked records to other CSV files in a local "db.json" file.

If you want a proper relational database, sqlite would suit better. There are GUI tools to work with sqlite database files and sqlite has csv import and export. But that won't help much if you already have a large CSV you just need to edit a few rows on.

As for versioning sqlite data, just don't ever update or delete any rows but instead only ever insert the updates as a new row. Potentially you could add a 'deleted' boolean and 'inserted' date columns to the tables you want to version. That way you can use '... and deleted = false' to filter out old data and you also know when updates occurred.

The problem with SQLite is that you have binary blobs and you don't have useful text diffs. So you lose a lot easy history reviewing tools based on git and also easy merge/pull reviews.

The closest to a best of both worlds is to use something like JSON to store the data in git and then a tool like Datasette to build an SQLite-powered view on top of that repo.