|
|
|
|
|
|
by robhawkes
130 days ago
|
|
|
The most time and energy has been getting my head around the source data [0] and industry-specific nuances. In terms of stack I have a self-hosted Dagster [1] data pipeline that periodically dumps the data onto Cloudflare R2 as parquet files. I then have a self-hosted NodeJS API that uses DuckDB to crunch the raw data and output everything you see on the map. [0] Mostly from https://bmrs.elexon.co.uk/
[1] https://dagster.io/ |
|
|