[chearon]

Paste the following into https://chearon.github.io/dropflow/ to see that it _is_ possible to style individual Arabic characters in canvas:

<div style="font-size: 10em;">ع<span style="color: blue;">ر</span>ب<span style="color: red;">ي</span></div>

That uses harfbuzzjs to do shaping and, for the segments that it has to, it paints paths instead of using fillText. There is an even better method which Mozilla's pdfjs uses: for all the glyphs that you want to draw, build a font (easy with HarfBuzz) that maps sequential characters to those glyphs. Then use fillText with that font and the character that corresponds to the glyph that you want for each glyph. That's nearly as fast as fillText on the whole string.

The points you make are really important. I rant about how even Google Sheets doesn't do rich text correctly because of fillText's simplicity here: [1]. But I think many of your points could be solved by using HarfBuzz. I dream of having shapeText and fillGlyphs methods on the canvas as an alternate to HarfBuzz because it would be less wasteful. Leave the high-level APIs up to client-side libraries like dropflow and scrawl.

Google has proposed a placeElement method [2] that allows you to render HTML and CSS into a canvas, but that destroys what's so great about canvas, which is that it's crazy fast. DOM is very heavy-weight.

[1] https://github.com/chearon/dropflow#harfbuzz [2] https://github.com/WICG/canvas-place-element

[chrismorgan]

Sadly that doesn’t work with Indic text. Take for example, my name:

  <div style=font-size:10em><span style=color:#e01b24>క</span><span style=color:#5e5c64>్</span><span style=color:#e66100>ర</span><span style=color:#e5a50a>ి</span><span style=color:#26a269>స</span><span style=color:#1a5fb4>్</span></div>

Ideally it’d render about the same as my manual splitting/colouring: https://temp.chrismorgan.info/%E0%B0%95%E0%B1%8D%E0%B0%B0%E0....

Now sometimes you can colour parts differently: if you stick with the inherent vowel on a conjunct, here making it LETTER KA, SIGN VIRAMA, LETTER RA, ditching the VOWEL SIGN I, then essentially the K from KA and the A from RA will be coloured LETTER KA, and the R from LETTER RA will be coloured SIGN VIRAMA. I haven’t decided yet if that’s an improvement! But this split colouring I only see in Dropflow—I’m not experiencing it in Firefox or Chromium, both of which do split Arabic colouring.

[chearon]

That's because Noto Sans Telugu (the font dropflow automatically downloaded based on the text; doesn't seem like a great pick but not wrong?) is returning ligatures. Not all fonts will be able to support styling individual characters. I get the same results in the dropflow playground as I get in Firefox and Chrome [1]. Maybe you were using different fonts in those browsers?

You might be able to turn OpenType features off in those browsers to make it look like your manual coloring, I'm not sure.

> if you stick with the inherent vowel on a conjunct, here making it LETTER KA, SIGN VIRAMA, LETTER RA, ditching the VOWEL SIGN I, then essentially the K from KA and the A from RA will be coloured LETTER KA, and the R from LETTER RA will be coloured SIGN VIRAMA. I haven’t decided yet if that’s an improvement!

Maybe because it changes the shaping results? I don't know enough about the writing system to understand this yet :)

[1] https://jsfiddle.net/fz15xu20/

[rikroots]

> Google has proposed a placeElement method that allows you to render HTML and CSS into a canvas

I've just read it. Oh, dear ... what an awful proposal!

My initial thoughts, reacting to the README at https://github.com/WICG/canvas-place-element

> There’s a strong need for better text support on Canvas. [...] This includes not only visual features but also the possibility of supporting the same level of user interaction as the rest of the web

Agreed, but ... adding HTML/CSS to a raster (or WebGL etc) image is not the way to do it. I much prefer your idea of incorporating HarfBuzz-like functionality into the canvas/text APIs - especially given that HarfBuzz is included as part of most browsers' code base.

I've played with mixing HTML/CSS with canvas in my canvas library. The results are ... interesting[1][4], but (probably) not the ideal solution. Making it easy for developers to build canvas interactions with HTML/events anywhere on the page is much more productive and useful[2].

> There is currently no guarantee that the canvas fallback content currently used for accessibility always matches the rendered content, and such fallback content can be hard to generate.

My thinking is that directly reflecting the canvas text back into the DOM is often not useful for people using screen readers. They don't need to hear every number on the chart axis when instead they could be presented with just the measure and range of the axis. If the HTML element rapidly/repeatedly updates its content, it's going to be a very unpleasant experience for the user. I've experimented with this sort of thing in [2], but need feedback from real screenreader users to understand if the solution meets their needs.

> Access to live interactive forms, links, editable content with the same quality as the web. This will help to close the app gap with Flash.

Flash is dead. Please leave its bones in the crypt.

> A limited set of CSS shaders, such as filter effects, are already available, but there is a desire to use general WebGL shaders with HTML

I only work with 2D canvas, but I can understand the desire here. A different approach might be to convince browser devs to work on improving SVG (and CSS) filters to support WebGL shaders, which can then be used by the canvas? Though Safari still doesn't support using SVG filters in the canvas so maybe convince them first?

Playing with filter effects is one of the joys I get from working on my canvas library[3] - but that's got nothing to do with text layout ... except when applying the filter to text, of course![4]

[1] - Use stacked DOM artefact corners as pivot points https://scrawl-v8.rikweb.org.uk/demo/dom-015.html

[2] - London crime charts https://scrawl-v8.rikweb.org.uk/demo/modules-001.html

[3] - A gallery of compound filter effects https://scrawl-v8.rikweb.org.uk/demo/filters-103.html

[4] - Editable header text colorizer and animation effect snippets https://scrawl-v8.rikweb.org.uk/demo/snippets-006.html

[chearon]

Interesting take on representing charts in an accessible way. I feel like, as web developers, we were fed a myth that lots of markup and attributes automatically makes your content accessible. But it takes more thinking than that.

I've taken a similar approach to layering canvases with normal HTML (typically the HTML is on top). I don't have a problem logically representing what's painted on the canvas and doing my own hit detection either. Shaders and text shaping in canvas sound a lot more attractive to me than placeElement, but I guess we'll see. I should get around to campaigning for ctx.shapeText and ctx.fillGlyphs but I don't know how much folks care about it.