[kepano]

Whoa! Didn't expect this to bubble up to the top of HN. Some context about why we created JSON Canvas:

https://obsidian.md/blog/json-canvas/

We just released it today, so this is still a very nascent project. A little over a year ago we released Obsidian Canvas. The .canvas file format has felt stable enough to give it a name and resources that other apps can freely use. See the original Show HN:

https://news.ycombinator.com/item?id=34066824

The spec is conservative, and definitely does not support many features canvas apps will want to implement (yet).

The purpose of giving JSON Canvas a name and site is to encourage an interoperable ecosystem to grow around this format. We're definitely looking for feedback of all kinds!

It's great to see all the suggestions already shared in this thread because it starts to provide a roadmap for how this could become a more useful format for other apps.

[remram]

Who was involved in this open spec? Or was it built by Obsidian with a hope that it works for everyone else?

Which existing formats were considered before building your own, e.g. SVG/Excalidraw/draw.io/...?

[eichin]

Yeah, didn't see any discussion in the other thread, do you have any notes on why this isn't SVG?

[rootlocus]

Looking at the JSON, I assume this is a higher level format that may output SVG. For instance, the arrows between nodes are defined by reference to node names and node sides. In SVG that would be a hardcoded curve and shape for the arrow which loses the semantic meaning "there's a relation between two nodes", where nodes may also have semantic meaning such as "idea" or "book".

A tool may parse the json canvas file and extract information such as "there are three ideas linked to this book". Another tool may output an SVG.

[npunt]

It makes sense if you see Obsidian as the starting point - it's a document store. While other canvas products may be more graphics-oriented, Obsidian's is about laying out documents and objects and providing simple relationships between them. For this purpose, JSON's probably a lot easier to work with than XML/SVG.

[remram]

Obsidian's own file format might have merit, but posting it as an "open file format", "created to provide (...) interoperability, and extensibility to data created with infinite canvas apps"... and then publishing 1.0 without consulting with anybody else... is rather egregious, I think.

I agree with you that posting Obsidian's spec to start the conversation would have been welcome, but this is not what they did.

[npunt]

Obsidian's philosophy is file over app [1] and releasing a spec for their Canvas feature is fulfilling that promise. It's a strictly positive 'today is better than yesterday' thing for them to have done so.

You shouldn't need to ask permission or consult anyone to do this. That's a silly bar, 'egregious' even. When markdown was created, Gruber wasn't asking all his friends making text editors whether they'd support it. He just released it, and others chose to adopt it on its merits.

I don't meant this to sound harsh, but just to call it as I see it - your comment is basically creating a 'damned if you do, damned if you don't' scenario and an example of why people building things shouldn't worry much about comments in forums.

[1]: https://stephango.com/file-over-app

[gcr]

I’m glad Obsidian is publishing this, even though the spec clearly isn’t ready yet.

If I had the final say for how this format were to be publicized, I would probably just include the JSON schema “as-is” in the Obsidian developer docs. Then, it’s just as usable by third parties without making promises about interoperability.

But Obsidian is thinking ahead here. They’ve intentionally given us a duck to poke at[1], knowing we’d offer feedback and critique. If the goal is to get people to imagine a better implementation, preying on HN’s “I could implement this so much better” sense is one hecking smart way to get the community to pitch in.

1: https://bwiggs.com/notebook/queens-duck/

[keybored]

However, you should ask Gruber if he is willing to bless your your name if you include a hint towards “markdown” in it (referencing Commonmark), or else there’ll be a big kerfuffle. (funny example to use)

[rat9988]

I'm not sure I agree with you here at all. They can start a format that serves their own purpose, and make it open for interoperability and extensibility sake. Nothing wrong with it.

[remram]

"egregious" was a strong word, apologies. Still the silence is deafening here.

[haswell]

This is an odd bit of gatekeeping and I really don’t understand your position here. Calling this egregious seems odder still. It suggests that companies that choose to publish how they do things have some obligation to do so under a specific framework or shouldn’t do so unless they’re willing to get public buyin.

Many of the best projects that become defacto standards start as a solution to a very specific and real problem that had no public input.

If JSON canvas isn’t a good solution for broad adoption/interoperability, it won’t become one. The world is not harmed by its release, and at worst, it’s now far easier for people building software to understand and make tools that can interact with Obsidian. At best, it becomes a solid foundation and option for tools going forward.

Obsidian didn’t have to do anything here. I’m glad they did.

[blowski]

And indeed, there are many projects that spend so long trying to gather and process feedback that it becomes a talking shop with no useful output. Better to share something early and then iterate.

[naikrovek]

SVG is great and you are the first person I have ever seen who would (and did) imply that SVG might be or would be better than JSON for this kind of thing.

SVG really stretches the “human readable” thing to its limit, and I personally would not adopt an SVG-like format for anything that isn’t SVG.

[eichin]

Looking at the samples, this isn't human readable either (long opaque id strings for linking dominate the syntax) and both syntaxes have vast amounts of punctuation and whitespace. And neither of them are particularly human writable. But something svg-based would have lots of existing options for additional processing in consistent ways, that would be ad-hoc for a json-based syntax.

(Given that this is coming from obsidian, I kind of expected something actually markdown-like - which would certainly be a challenge, but not an impossible one.)

Granted, this notation is a little higher level than SVG, so the model mismatch might be a bigger problem anyway?

[ornel]

I opened a JSON created with the Obsidian canvas thing and renamed the long ids with human readable names and it keeps working without re-renaming. So they or whoever uses this format could maybe come up with a way of using some form of human-readable ids[0] to make it easier to modify the JSON by hand

[0] https://pypi.org/project/hrid/

[littlestymaar]

SVG is a very poor format by all measures so if you can start from scratch for your use case you should definitely avoid it. I'm still glad SVG exists because it's available on the web for free, but this file format is really a mess (very hard to parse, the feature set is so broad it's never entirely implemented and the implantations diverge making it surprisingly hard to support multiple browsers).

[Idiot211]

Relevant XKCD:

https://xkcd.com/927

[kepano]

I can't believe it took this long for someone to post this haha

[altairprime]

For those having difficulty viewing the spec at the linked jsoncanvas.org site, the spec/1.0 pane on the right has the scrollbar hidden; there's more content if you scroll that pane down.

[dolmen]

Direct link viewed on GitHub: https://github.com/obsidianmd/jsoncanvas/blob/main/spec/1.0....

[hiccuphippo]

Click the spec link, it shows the same content in a full page.

[tylerdurden91]

Great work & thanks for doing this.

I built a library at AWS for a general canvas editor called Diagram Maker. It recently got archived so I stood up a fork here: https://github.com/sameergoyal/diagrammer and the data format we use is strikingly similar. Check it out here: https://sameergoyal.github.io/diagrammer/?path=/docs/docs-us.... The key differences are panels, workspace & editor.

I dont actively work on the project outside of bugs, but maybe there are ways to collaborate here, like moving my project to use & extend the JSON canvas spec.

[desmondl]

I agree with a lot of comments that it's minimal, but in my opinion that is a good thing. I'm a big fan of Obsidian, and of the things I like about it is the data source is all markdown files. Markdown is meant to be very lightweight and portable, and overcomplicating it will limit adoption and extensibility (imagine markdown vs pdf).

JSON Canvas seems to follow in that spirit by being very lightweight, so a lot of implementation details (i.e. how are files rendered, what file formats are supported, etc), edit tags, etc) are left open to implementation.

Markdown and JSON are meant to be non-opague file formats that prioritizes portability and human readability over other features. An application format like Sqlite has a lot of benefits over markdown, but it loses the benefits of text based formats like being compatible with git and is less portable.

What I would like to see is a convention for extending the node and edge definitions, similar to frontmatter in markdown files- something that is not required for basic rendering but is a nice-to-have for applications to consume) - that way portability between apps of varying complexity can be maximized while still allowing for more complex features that some apps might implement. Markdown has the benefit of supporting extensions (for example like tables in GFM) - apps that are not compatible can still render the unsupported markup. But there should be an explicit way to extend open JSON formats.

Some feedback off the top of my head and from reading the comments:

1. *Specifying node ordering*. Obsidian seems to just send whatever is last touched to the top, but this makes a common feature in many apps (move up/down/back/front) more difficult to impement

2.*More explicit definition for line shape*. Adding a way to "bend" a line in a specific way. Useful for creating more complex diagrams.

3. *Relations between nodes*. Group nodes contain child nodes, but the spec doesn't specify how the child nodes are defined. I would expect it to have a `children` property to nest nodes. Obsidian seems to implicitly link nodes to groups based on whether their bounds intersect. This makes it difficult to implement some common features:

a. nodes that exist outside of the bounds of its group, for example a node that "floats" just outside of the edge of the group's borders.

b. nodes that are not part of a group even though it exists within the bounds of that group.

There are many different ways for a canvas app to extend the spec to implement those features, but it seems like something that should be defined in the spec to maximize portability

4. *Extensibility.* Either explicitly support or provide a standard for defining more styles for nodes and edges, such as stroke width, stroke style, rotation, etc. It seems like "color" should be a part of this as well, rather than being an explicit property of a node.

5. *Embeds.* Supporting "embeds" as a node type. I even think the "file" node should be redefined as `embed` with a `uri` property to support different schemes (`file://`, `oembed://`, `https://`) and maybe a `mime-type` (`text/markdown`, `image/webp`). The file node's "subpath" property seems to be only relevant for markdown files, so I think that should be an extension rather than an explicitly defined.

6.*YAML* :) (Should just seemlessly convert from json, but yaml is more readable than json)

Being able to design standards that evolve over time and making tough decisions about what what to make explicit and what to leave implicit is a skill I want to improve on as a developer this year. Does anyone have any resource recommendations or best practices to recommend for me to research?

[exceptione]

> 6. YAML

Please don't, it has one of the most confusing syntax out there with lists and maps, and it won't do well for parsing.

[desmondl]

I haven’t had any issues with yaml in markdown frontmatter or openapi specs. What kind of issues do you see with list and maps that make you against yaml? I agree that for computers and consistency json is preferred. I already use a linter for my markdown files so I would do the same with yaml to keep lists and maps consistent

[cstrahan]

YAML is kind of like C++:

> You like C++ because you're only using 20% of it. And that's fine, everyone only uses 20% of C++, the problem is that everyone uses a different 20% :)

https://eli.thegreenplace.net/2009/10/17/the-c-bashing-seaso...

The YAML footguns are too numerous to reproduce here, so here are some sources:

https://stackoverflow.com/questions/3790454/how-do-i-break-a...

https://www.arp242.net/yaml-config.html

https://noyaml.com/

YAML isn't terrible if you only ever have to read what you wrote. Now consider that there are 63 different ways to write multi-line strings in YAML -- how many of those have you committed to memory? Yeah... now throw 10-100 developers into the mix, each with their own favorite alternative syntaxes -- good luck making sense of your YAML.

[desmondl]

Yeah true, I'm starting to remember the headaches with yaml when I was using kubernetes or cloudformation....

[bbkane]

Point taken, but you can mitigate a lot of this with yamllint.

[8n4vidtmkvmk]

That's pretty sad that you need to lint your config lang.

[exceptione]

In the past I had to craft yaml files. Sometimes I needed quotes for a string, sometimes I had to put in a dash in front of a key, or just not. You basically needed to have the whole schema in your head.

There can only be so much nesting before you lose track of what item belong to which parent. Copying some yaml structures over to another level requires care, as the result might look correct, but the white space parser thinks otherwise.

I have lost hours of debugging yaml files when a dash was missing somewhere or when I needed one more leading space. The parser accepts it happily, but half of the typical javascript programs will only detect things are wrong when it has already executed on half of your spec. The other half will just run with input that wasn't intended that way.

I remember writing artillery.io test specs where all those problems pop up.

Now the good thing from JSON is JSON Schema. The latest spec allows you to specify quite advanced validations. Yaml has no such thing.

As to your remark: Yaml for front matter is defensible, as you dont have deeply nested structures. Though, as an obsidian user you want to make sure your front-matter is conforming to your own schema. That would require writing a json spec and then have your yaml internally converted to json before handing it over to the validator.

A spec is worthless if you cannot validate against it. Json and xml have a good story there. I concede that yaml is more human-readable than json without an editor. Correctness is the holy grail though.

[gjvc]

JSON5 has comments. This is the major thing. A configuration which does not allow comments is not a configuration for humans, it's a serialization for programs.

CMakePresets.json is an offender here...

[exceptione]

Agreed. Json seems to be designed with machine interpretation as first concern. Having to wrap object keys into quotes eases parsing I guess, but for humans it is a nuisance.

[chatmasta]

> Markdown and JSON are meant to be non-opague file formats that prioritizes portability and human readability over other features

I don't think human readability is a critical feature of JSON at this point. If that's your priority, you can use YAML. Readable JSON is nice because for small files you can read or edit small sections of it, and it's easy to debug when manipulating it with machine code. But there are plenty of cases where a huge JSON file is still useful even if it's barely human readable.

My heuristic has always been: use YAML if you expect humans to create the file (or maintain large chunks of it), otherwise use JSON. For example, Kubernetes config is YAML because humans create it from scratch, and it would suck to do that with JSON. Whereas package.json is JSON because machine code initializes it and humans only make minor edits to specific fields.

In the case of this canvas format, I wouldn't expect humans to create the file from scratch, so use JSON over YAML. Then the question is, will humans even care about reading the raw JSON? Probably not. So why not use something like SQLite or Protobuf? The most compelling reason would be that humans writing code to interface with the format can use parsing tools from their language's standard library.

[naikrovek]

> I don't think human readability is a critical feature of JSON at this point. If that's your priority, you can use YAML.

Wow you have kinda lost the plot on a few things.

JSON was designed to be human readable and writable. YAML was designed to be a human readable format for the automated interchange of data between automated systems. Human writability was neither a goal for YAML nor its intended use. Like everyone else on the frakking planet, you’ve misunderstood what YAML was intended and designed for. YAML was never intended for human-written configuration storage, which is what everyone used it for the instant after they became aware of it.

YAML can bite you very hard if you misunderstand it. JSON is simply invalid if you misunderstand it when writing it.

If you don’t need human readability, use a binary format. Binary formats are so freaking fast compared to literally any structured text format, past, present, or future. High speed and low latency matter and binary formats make both of those easier.

If you need to inspect the binary data, write a viewer using the code you use to read it. It’s a lot simpler than people believe it to be. I find Protobuf to be more of a hassle than writing the code myself, and protobuf is very easy to use, and I’m quite a moron. Binary stuff is not hard.

[desmondl]

Yep, I think the compelling reason of humans writing code is key here. SQLite would make it less accessible for people to write external tooling to integrate with an obsidian vault. There are lots of existing and open that support diffing/parsing/syncing/manipulating json, while with sqlite you have to not only know sql but support another application’s database schema, which third party developers are less likely to do

[louthy]

> I agree with a lot of comments that it's minimal, but in my opinion that is a good thing

The purpose of a spec is to specify, and if you don’t specify and leave things open to interpretation, then that completely defeats the purpose.

Anybody who’s worked with a poorly defined spec knows exactly how bad this can be. A good example would be the shambles that is the HL7 spec used in healthcare.

A former colleague had a phrase for this: “once you’ve seen one HL7 message… you’ve seen one HL7 message”. Which really highlights the issue of a standard that’s open to interpretation.

The issues raised (in the comments here) seem to hint at a lack of specificity. That is something that they should really look at improving.

I think overall any group that tries to come up with a standard that can unify a field should be lauded and supported. But perhaps calling this a 0.1 release, and taking the feedback on board, would be the best way forward.

[keybored]

> I agree with a lot of comments that it's minimal, but in my opinion that is a good thing. I'm a big fan of Obsidian, and of the things I like about it is the data source is all markdown files. Markdown is meant to be very lightweight and portable, and overcomplicating it will limit adoption and extensibility (imagine markdown vs pdf).

What Markdown got right was creating a nicely readable lightweight markup syntax.

And Markdown also demonstrated how to create a bad precedence for future consolidation by being so loosey goosey and underspecified (and with a bad reference implementation). That there is a Commonmark at all is solely because of others picking up the slack and doing the unthankful gruntwork of creating 100 if-then-else statements in a semi-formal prose format.

[dolmen]

I wonder why this document has been labelled "1.0" while it is so poorly specified (many ambiguities).