The one critique I have of D3 is that it is becoming increasingly closely paired with Observable, such that when I'm searching for examples of how to make something in d3, I can only find Observable examples that require the Observable runtime in order to function properly.
I think the examples should have a JS native version in addition to the Observable implementation.
That's part of the reason I started my book "Visualisation with D3.js"[1]... it's early access and I'm trying to find more time to work on it, but it's all JS native
It's a real pain. I work with lots of "data science" types who are often trying to concurrently learn JavaScript and d3 - often with JS as their first or second programming language. It's difficult enough for an experienced programmer coming back to d3 after a few versions to pick out what's d3 vs. what's Observable - and very difficult to help a relative newbie through the process of turning what's shown in an Observable notebook into something that can run in an ordinary web page.
Indeed - I don't like the pairing with Observable. I don't like Observable in general because the full experience can't be self-hosted. However, both projects belong to Mike, and D3.js is free - so I can't complain.
Didn't they have an export function that allowed self-hosting something you developped on Observable? (Or is it the development part that you want to self-host for "full experience"?)
You can self-host the runtime, and use code from notebook cells in whatever context you like. The closed part is the “notebook chrome” (editor, history view, social features, etc.).
What’s the difference between the two? As someone who only dips into D3 every few years and just discovered Observable, I wasn’t aware that they were different. I thought Observable was just a way of sharing D3 visualizations.
Observable is like Jupyter for Javascript. A general JS notebook environment, with the added twist that it uses reactive dataflow ideas to evaluate cells in a "right" and deterministic order (a common footgun in Jupyter where cells might be reevaluated manually, resulting in unexpected state).
D3 is a JS library for data visualisations.
Both are the brainchild of Mike Bostock. That's the connection. :)
Someone should make an open source version of the Observable ecosystem or integrate it into Jupyter tooling better. The notebook format is not encrypted or anything.
The difference is that examples in Codepen are easily reproducable in another environment.
d3, on the other hand, has converted most of their examples to Observable, which requires extensive research to convert back to JS (in the best case) and in the worst case requires reverse engineering the runtime.
I'm not the only one with this opinion, there are plenty of others who have struggled through similar issues and discussed it.
"The process of converting an Observable notebook to standalone html and/or js is clearly not as simple and copying and pasting. The use of Observable notebooks for all D3 examples has made the introduction to the library much more challenging for this reason." [0]
The claim that one has to "reverse engineer the runtime" is simply absurd.
The people that complain about D3 on observable always turn out to have taken neither the time to understand Observable nor D3 and are looking for a way to just copy paste something into their codebase, without understanding it, and without any willingness to really learn the tools they are using. Which would have never worked in the first place, because D3 is not a plotting library like vega or observables plot, it's a library library, building material from which you can build your own bespoke visualisations.
Observable is in essence reevaluating the entire code with each "requestAnimationFrame", no fancy diffing a la react or other magic, if it runs in observable, it will also run in your code as a static thing. Copy paste the cells out of observable, prepend them with a "const" and you're done,
not more work than removing and rewriting the boilerplate that's in every codepen.
> taken neither the time to understand Observable nor D3 and are looking for a way to just copy paste something into their codebase, without understanding it, and without any willingness to really learn the tools they are using.
That's how many of us learn, by dipping our toes into a new SDK by copying and pasting into our bare HTML/JS page and tweaking things to see what happens. Eventually we will get around to reading the full documentation.
I have made d3 examples and played around with the toolkit since back when it was in its infancy (called protovis or something). It has definitely become more and more difficult over the years for a beginner to jump into.
(Background: I have met Mike since I got my PhD from the hci/graphics lab at Stanford, but we did not overlap.)
This is my experience as well - when deciding whether to use a certain library or framework, I don't start reading the reference manual. Rather, I highly prefer looking at and tweaking existing examples, mostly to understand what the typical usage looks like and most importantly where the limitations lie.
I feel like the last point is often overlooked by documentation authors: When evaluating whether to use a library, I'm not just interested in what it can do, but also what it can't do! That's why I really enjoy the "you should use X if you want to do Y"/"you shouldn't use X if you want to do Z" sections present in some manuals.
Yeah I encountered this problem recently as someone who was once reasonably familiar with d3 and came back to it after several years away.
I find myself wanting to start with a working example I can riff off of more in d3 than with many other tools I've used, so I totally understand the appeal of interactive examples in Observable but find it super frustrating and cumbersome when it comes to actually rolling what I'm working on into an existing application.
So, is d3 tree-shakeable yet? If I import { scaleLinear } from 'd3' in a webpack-built project, will I get only the scaleLinear code? As far as I remember, in v6 this will import a lot of (or perhaps the rest of) d3 as well.
I am probably being unreasonable; but it's kind of nice to be able to have a single third-party dependency out of which you can pick what you need at the moment, rather than a host of small third-party dependencies. Kind of like lodash, from which you can at least import individual functions (import merge from 'lodash/merge') rather than installing all those functions as individual packages.
Exciting to see ESM everywhere. It works so well with Observable. I'm trying to put live playgrounds at observable for my packages as part of docs [0] - just starting, nothing exciting to see there yet, but I think the concept of playground as part of docs is quite awesome.
Used this in a data viz class and I’d say it’s overkill for most visualizations. Also very difficult to learn and frustrating to implement with all the “enter, exit” stuff. Excel, Tableau, etc so much easier and most of what you need to do visualizations in the corporate setting. Obviously it’s excellent for very specialized unusual visualizations but it’s rare those are really needed.
All the "enter, exit" stuff is what makes d3 beautiful. From a core handful of ideas, Mike is able to express a vast world of visualizations. I think it is the most amazing framework I have had the pleasure of using in my 30+yrs of software development. Ya, you could grab bar chart library and just give it data and get a graph out, or, you could spend a little time with d3 and actually learn something so powerful you can make bar charts - or virtually any other kind of visualization you will ever need. Well worth working past the frustration, imo.
Why can't we have both? -- I had a similar frustration and got to work on https://hal9.ai, you get prebuilt blocks to easily import/transform/visualize data, while also being able modify the D3/Plot sources to your hearts contempt.
> Excel, Tableau, etc so much easier and most of what you need to do visualizations in the corporate setting.
D3 is the best way to make visualizations online. Yes, it requires learning the library and building things from scratch, but it's light years better than the dog shit modules Tableau offers.
D3 v6 was released on 27th Aug 2020, so your feeling is right. We seem to get a major release about every year now, compared to every two years before v5.
I think the examples should have a JS native version in addition to the Observable implementation.