[smj-edison]

I have mixed feelings about this. On the one hand, I agree: text is infinitely versatile, indexable, durable, etc. But, after discovering Bret Victor's work[1], and thinking about how I learned piano, I've also started to see a lot of the limitations of text. When I learned piano, I always had a live feedback loop: play a note, and hear how it sounds, and every week I had a teacher coach me. This is a completely different way to learn a skill, and something that doesn't work well with text.

Bret Victor's point is why is this not also the approach we use for other topics, like engineering? There are many people who do not have a strong symbolic intuition, and so being able to tap into their (and our) other intuitions is a very powerful tool to increase efficiency of communication. More and more, I have found myself in this alternate philosophy of education and knowledge transmission. There are certainly limits—and text isn't going anywhere, but I think there's still a lot more to discover and try.

[1] https://dynamicland.org/2014/The_Humane_Representation_of_Th...

[dkarl]

I think the downside, at least near-term, or maybe challenge would be the better word, is that anything richer than text requires a lot more engineering to make it useful. B♭ is text. Most of the applications on your computer, including but not limited to your browser, know how to render B♭ and C♯, and your brain does the rest.

Bret Victor's work involves a ton of really challenging heavy lifting. You walk away from a Bret Victor presentation inspired, but also intimidated by the work put in, and the work required to do anything similar. When you separate his ideas from the work he puts in to perfect the implementation and presentation, the ideas by themselves don't seem to do much.

Which doesn't mean they're bad ideas, but it might mean that anybody hoping to get the most out of them should understand the investment that is required to bring them to fruition, and people with less to invest should stick with other approaches.

[smj-edison]

> You walk away from a Bret Victor presentation inspired, but also intimidated by the work put in, and the work required to do anything similar. When you separate his ideas from the work he puts in to perfect the implementation and presentation, the ideas by themselves don't seem to do much.

Amen to that. Even dynamic land has some major issues with GC pauses and performance issues.

I do try to put my money where my mouth is, so I've been contributing a lot to folk computer[1], but yeah, there's still a ton of open questions, and it's not as easy as he sometimes makes it look.

[1] https://folk.computer/

[assimpleaspossi]

Folk computer looks interesting. I wonder what it is. You'll never find that out by looking at that link.

[smj-edison]

That's fair. It's still pre-alpha, and under heavy development, but it's working on taking the best of dynamicland[1] and trying to take it a lot further.

In terms of technical details, we just landed support for multithreaded task scheduling in the reactive database, so you can do something like When /someone/ wishes $::thisNode uses display /display/ with /...displayOpts/ { and have your rendering loop block the thread. Folk will automatically spin up a new thread when it detects that a thread is blocking, in order to keep processing the queue. Making everything multithreaded has made synchronizing rendering frames a lot tricker, but recently Omar (one of the head devs) made statements atomic, so there is atomic querying for statements that need it.

In terms of philosophy, Folk is much more focused on integration, and comes from the Unix philosophy of everything as text (which I still find amusingly ironic when the focus is also a new medium). The main scripting language is Tcl, which is sort of a child of Lisp and Bash. We intermix html, regex, js, C, and even some Haskell to get stuff done. Whatever happens to be the most effective ends up being what we use.

I'm glad that you mention that the main page is unhelpful, because I hadn't considered that. Do you have any suggestions on what would explain the project better?

[1] https://dynamicland.org/

[tomjakubowski]

> B♭ is text.

Yes, but musical notation is far superior to text for conveying the information needed to play a song.

[satvikpendem]

I don't understand, musical notation is text though so how can it be superior to itself?

[BrenBarn]

I think they mean staff notation, not a textual notation like "B♭".

[dhosek]

Although, one could make the argument that staff notation is itself a form of text, albeit one with a different notation than a single stream of Unicode symbols. Certainly, without musical notation, a lot of music is lost (although, one can argue that musical notation is not able to adequately preserve some aspects of musical performance which is part of why when European composers tried to adopt jazz idioms into their compositions in the early twentieth century working from sheet music, they missed the whole concept of swing which is essential to jazz).

[thaumasiotes]

> one could make the argument that staff notation is itself a form of text, albeit one with a different notation than a single stream of Unicode symbols

Mostly this is straightforwardly correct. Notes on a staff are a textual representation of music.

There are some features of musical notation that aren't usually part of linguistic writing:

- Musical notation is always done in tabular form - things that happen at the same time are vertically aligned. This is not unknown in writing, though it requires an unusual context.

- Relatedly, sometimes musical notation does the equivalent of modifying the value of a global variable - a new key signature or a dynamic notation ("pianissimo") takes effect everywhere and remains in effect until something else displaces it. In writing, I guess quotation marks have similar behavior.

- Musical notation sometimes relates two things that may be arbitrarily far apart from each other. (Consider a slur.) This is difficult to do in a 1-D stream of symbols.

> although, one can argue that musical notation is not able to adequately preserve some aspects of musical performance

Nothing new there; that's equally true of writing in relation to speech.

[xorcist]

How is that not text? Surely if we consider Arabic to be text (lots of ligatures, grouping, right-to-left notation) then music notes must be, too?

[tremon]

"I cannot read A, and I cannot read B. Therefore, A and B must be identical".

[nimih]

For complex music, sure, but if I'm looking up a folk tune on, say, thesession.org, I personally think a plain-text format like ABC notation is easier to sight-read (since for some instruments, namely the fiddle and mandolin, I mainly learn songs by ear and am rather slow and unpracticed at reading standard notation).

[codebaobab]

Yes. And I create and manage the musical notation for over 100 songs in text, specifically Lilypond.

[donkyrf]

If we accepted the validity of this argument, then literally everything that can be represented by a computer can be referred to as text.

It renders the term "text" effectively meaningless.

[smj-edison]

To be fair, in Lilypond's case, it is an ASCII interface that renders to sheet music (kind of like openSCAD).

[godelski]

Working in any science should also make this argument clearer. Data as text is hard to read and communicate. Even explanations of results. But graphs? Those are worth a thousand words. They communicate so much so fast. There's also a lot of skill to doing this accurately and well, just as one can say about writing. A whole subfield of computer graphics is dedicated to data visualization because it's so useful. Including things like colors. Things people often ignore because it feels so natural and obvious but actually isn't.

I think it's naïve to claim there's a singular best method to communicate. Text is great, especially since it is asynchronous. But even the OP works off of bad assumptions that are made about verbal language being natural and not being taught. But there's a simple fact, when near another person we strongly prefer to speak than write. And when we can mix modes we like to. There's an art to all this and I think wanting to have a singular mode is more a desire of simplicity than a desire to be optimal

[casey2]

Data that can be visualized is rarely useful. Better to create a language to talk about it.

Often you need a language in the first place to even be interested in the graph at all. Graphs are worth a thousand words if you are willing to throw out any data that

Is higher than 3D

Requires control flow or recursion to explain

Of course you can have diagrams systems that are languages e.g. Feynman Diagrams (a sort of DSL for quickly reading QM math). I would hold this up as a much greater achievement of human ingenuity than r/dataisbeautiful spam. But the differentiation here isn't between text and graphs, but between languages and glyphs.

[inciampati]

It is true that graphs communicate very well. But they do come from text... And in the end we need to be able to describe what we see in them in text.

[godelski]

I think you're reaching. Justifying the answer you want rather than the answer that is.

No, graphs do not need come from text. I've frequently hand generated graphs as my means of recording experimental output. This is a common method when high precision is not needed (because your uncertainty level is the size of your markers). But that's true for graphs in general anyways.

Importantly, graphs are better at conveying the relationship between data, rather than information about a single point. (something something - Poincaré ;)

Besides, plots aren't the only types of graphs. Try network graphs.

Besides, graphs aren't the only visual communication of data.

I'll give you an even more obvious one: CAD. Sure, you can do that in text... but it takes much more room to do and magnitudes more time to interpret. So much so that everyone is going to retranslate it into a picture. Hell, I'll draw on paper before even pulling up the software and that's not uncommon.

[inciampati]

> CAD. Sure, you can do that in text... but it takes much more room to do and magnitudes more time to interpret.

Fascinating example for me. I do CAD... using text! My only experience with it is programmatic in openscad. We check the visualization, but only on output of the final product. For me it's dramatically easier to work with. That may be a personal defect but it's also consistent. Underneath the rendering is always data, which is text, markup, but strings of fundamental data.

And in science it's not a stretch at all that numbers come first. I'll argue you're reaching. Today no one is drawing their numbers from experiments directly on a graph. They record them digitally. In textual form typically, and then render them visually to obtain generic understanding. But also there, in the end, your conclusions (per tradition) need to be point estimates with error bounds expressible in concise textual terms. You may obtain them from looking at images but the hard truth is numerical, digital, textual.

[VorpalWay]

I have tried OpenSCAD, but found it extremely limited and awkward. I much prefer parametric CAD like Fusion 360, OnShape (which I'm currently using) or FreeCAD (which has a really bad UX). And my day job is as a C++/Rust developer, so you would think that I would have good chances to prefer a textual representation.

Part of this might be OpenSCAD specifically. It is CSG based, which is really not ideal, making it hard to add things like chamfers and fillets to your model. Most OpenSCAD models I come across for 3D printing have a crude look probably because this is so hard.

But part of it is just that text for most people just isn't the right representation in this case. (If you look at the relative usage of parametric CAD to textual CAD on sites for 3D models you will see that I'm right. Also, look at what approach commercial packages offer.)

[CasperH2O]

You may want to have a look at build123d. Its a Python library with an active and accessible community.

[tremon]

I do CAD... using text! My only experience with it is programmatic in openscad

That does not mean that the CAD drawing itself is text. It is an artifact, produced from text. Using your argument you could just as easily argue that all computer code is text, and I don't think that's a useful redefinition of the word "text".

[godelski]

I'm absolutely fascinated by your answer!

Can you tell me more about the pipeline? Are you really starting from scratch by programming? You don't do any sketching first? I'm really having a hard time imagining doing anything reasonably complicated with this method. I'll admit that there are some advantages like guaranteeing bounds but there's so much that seems actually harder to do that way.

  > They record them digitally

Like I said, it is contextually dependent. If you're recording with digital equipment to a computer, then yeah, it's just easier to record that way and dump into a plot. But if you don't have that then no. And again, even recording by hand it is still dependent.

But some data is naturally image data (pictures?). Some data is naturally in other modalities (chemical reactions? Smell? Texture? Taste?). Yes, with digital recording equipment you can argue that this is all text but at that point I'd argue you're being facetious as everything is text by that definition.

  > You may obtain them from looking at images but the hard truth is numerical, digital, textual.
 

Here I think you have a fundamental misunderstanding and are likely limiting yourself based on your experience.

First off, not every measuring device is digital. So just that alone makes it down right false. And pretending all measurements are digital is just deceptive or naive.

Second, and I cannot stress this enough: *every single measurement is a proxy* to the thing you intend to measure.

You can't even measure a damn meter directly. You can measure distance through reference length that is an approximation of a standard distance (aka a ruler). You can measure distance through reference to an approximation of time and through the use of some known velocity, such as the speed of light through a given medium (approximating time, approximating c in the medium, approximating the medium). And so on.

What you cannot do is measure a meter directly.

And most of the things we're trying to measure, model, and approximate in modern science are far more abstract than a standard unit!

The idea that the ground truth is textual is ridiculous. That would only be true on the condition that the universe itself is running on a digital computer. Despite the universe being able to do computation, I see little reason to believe it is digital.

[tsimionescu]

No, you do not need to, and will not generally be able to, describe everything that a graph conveys in text. Graphs can give you an intuitive understanding of the data that text would not be able to, simply by virtue of using other parts of the brain and requiring less short term memory. If a graph can be replaced with 5 pages of text, that doesn't mean that you get the same information from both - you're likely much more able to keep one image in your short term memory than 5 pages of text.

[godelski]

A word is worth a thousand images. Wait...

[inciampati]

But a graph, which provides a view at a certain level of resolution, can often be described in a few consise statements. That's why we make them, to get a view we can condense.

[tsimionescu]

No, if we can condense something in a few short statements, we don't generally bother making a graph. We exactly make graphs when something is not easily explained in words, but instead requires visualization.

Of course, not all graphs are equally information dense, and some are only used for decorative purposes more than actually conveying information. But in the general case, and especially when used well, graphs convey much more information at a glance than a short text description could.

[smj-edison]

I feel like it's more that we have statements that are "pointers" to the graph. "According to Figure 1, we see that temperature rises do to pressure." So we can summarise with words, but the intuition and proof comes from the visual medium.

[PopAlongKid]

Many years ago, in college, I used to volunteer for Recording For The Blind, reading various math texts aloud. I had to verbally describe every illustration in the textbook, including graphs, using a few concise statements. Not perfect, but possible.

[tsimionescu]

You can describe any graph to some low level of detail, sure. But does it actually help anyone? Do people with complete blindness, for example, gain anything from hearing a description of the graph of f(x) = x as "a straight line at a 45° angle crossing the graph at 0", compared to what seeing people gain from viewing that graph?

[smj-edison]

But they are multiple different "views" into data, and I would posit that a textual view of data is no different than a graphical view, no? If you import data from a parquet file, you go straight from numbers to graphs, so I disagree that it comes from text. Both graphs and text come from information. Circles on surveys, Arduino temperature readings, counter clickers when doing surveys. Those are not just text.

[tomjakubowski]

Take a problem like untangling a pile of cords. Writing out how to do that in text would be a drag, and reading those directions probably wouldn't be helpful either. But a kid can learn how to untangle just by observation.

Physical intuition is an enormous part of our intelligence, and is hard to convey in text: you could read millions of words about how to ride a bike, and you would learn nothing compared to spending a few hours trying it out and falling over until it clicks.

[bmicraft]

I think the bicycle argument doesn't work; you don't learn to ride a bicycle, you train to do it. Knowing how to do it isn't good enough, your conscious brain isn't fast enough to calculate and achieve balance. You need to train your reflexes to keep the balance for you.

[agnishom]

this is the "is all knowledge propositional knowledge" question

[bavell]

... training IS learning.

[JohnLocke4]

I think the obvious thing to do here is to say "Always bet on symbolics".

What separates text from images is that text is symbolic while images are visceral or feelings based. In the same way, text comes in short when it comes to the feeling you get when seeing an image. Try to put in to text what you feel when you look at Norman Rockwell's Freedom of Speech or a crappy 0.5MB picture of your daughter taken on an iPhone 3. Hard isn't it? Visual and symbolic are not isomorphic systems.

Examples of symbolic systems like text are sheet music and Feynman diagrams. You would be hard pressed if you tried to convey even 2KB of sheet music in a book

[safety1st]

I mean, this very discussion is a case study in the supremacy of text. I skimmed the OP's blog post in thirty seconds and absorbed his key ideas. Your link is to a 54 minute video on an interesting topic which I unfortunately don't have time to watch. While I have no doubt that there are interesting ideas in it, video's inferior to text for communicating ideas efficiently, so most people reading this thread will never learn those ideas.

Text is certainly not the best at all things and I especially get the idea that in pedagogy you might want other things in a feedback loop. The strength of text however is its versatility, especially in an age where text transformers are going through a renaissance. I think 90%+ of the time you want to default to text, use text as your source of truth, and then other mediums can be brought into play (perhaps as things you transform your text into) as the circumstances warrant.

[smj-edison]

Actually, you might want to check the video again, it has sections and a full transcript on the right side, precisely to make skimming easy!

> video's inferior to text for communicating ideas efficiently

Depends on the topic tbh. For example, YouTube has had an absolute explosion of car repair videos, precisely because video format works so well for visual operations. But yes, text is currently the best way to skim/revisit material. That's one reason I find Bret's website so intriguing, since he tries to introduce those navigation affordances into a video medium.

> The strength of text however is its versatility, especially in an age where text transformers are going through a renaissance. I think 90%+ of the time you want to default to text, use text as your source of truth, and then other mediums can be brought into play (perhaps as things you transform your text into) as the circumstances warrant.

Agree, though not because of text's intrinsic ability, but because its ecosystem stretches thousands of years. It's certainly the most pragmatic choice of 2025. But, I want to see just how far other mediums can go, and I think there's a lot of untapped potential!

[whattheheckheck]

The fidelity and encoding strength of the "idea" you got the gist of from skimming might be less than the "idea" you receive when you spend the time to watch the 54 minute video

[makeitdouble]

I came back here after the video (btw he speak very deliberately, watching it at 1.5 or 2x while digesting the message is fine)

I'd compare it's message to a "warning !" sign. It's there to make you stop and think about our computing space, after that it's up to you to act or not on how you perceive it.

That's totally wishy-washy, so it might not resonate, but after that I went to check more of what dynamicland is doing and sure enough they're doing things that are completely outside of the usual paradigm.

A more recent video explaining the concept in a more practical and down to earth framing: https://youtu.be/PixPSNRDNMU

(here again, reading the transcript won't nearly convey the point. Highly recommend watching it, even sped up if needed)

[einpoklum]

Can you explain what you mean by "This is... something that doesn't work well with text"? Text as opposed to what? If you were to "play" music by typing notes, then you would compare your typed note against the string of correct notes. Of course that sounds a bit silly, and probably not what you meant, so, please elaborate.

[smj-edison]

Sorry if that wasn't clear! I meant text as opposed to having verbal and physical coaching. My teacher would often demonstrate a technique by playing it on her piano, which was adjacent to mine. I even had a masterclass with one teacher who would grab my hand and guide it as she demonstrated what I needed to do.

An example of where text falls short: if I said "be sure to rainbow your wrist when jumping in that passage," it wouldn't make any sense unless someone had seen an explanation. I suppose I could try to explain "when moving higher, make an upwards arc, and loop around at the end, to prevent jerking your wrist around when going back and forth," but even then that's still way too ambigious, since there's also a certain way you need to pivot your wrist so you can hold onto the upper chord as long as possible. It's just much easier to demonstrate and see if the student did it correctly.

[groundzeros2015]

The missing ingredient you mentioned is the coach. You can pay a private math tutor to watch you solve math and engineering problems and give you direction a long the way. Few families do that.

[smj-edison]

For now, in most cases, yes. I think Khan Academy is a great example of moving in the right direction. They have a lot of interactive lessons in early math, where you drag and drop for counting and grouping. Another good example is the DragonBox series of apps where they make math more intuitive by providing immediate feedback and a new representation.

Dynamicland is pushing the state-of-the-art here too. I think you'd really like their essay "The Library"[1].

[1] https://dynamicland.org/2019/The_Library.pdf

[TimByte]

Where I keep coming back, though, is that text still seems to be the backbone that lets those richer systems scale and persist

[ignoramous]

Especially today when UI & tools built around LLMs, specifically code gen & image gen, demonstrate a seismic shift in just how far text will go.

[fercircularbuf]

Thank you so much for introducing me to this talk. Changed my way of thinking.